Introduction
Objective
This document describes the facilities and support provided to software development teams (SDTs) in the GN Project. Most of this support is provided by the project's Software Governance and Support function, which is Task 2 of Work Package 9 (WP9T2). It is intended to provide an overview of the available resources, services, processes and opportunities, and to suggest how they support the engineering efforts.
The main focus of this guide is on the operationalization of software development and maintenance by a software development team. The second part of the guide lists a few related pursuits that indirectly support and help the enhancement of the team's work: the PLM, GLAD training events, and the framework of Common Best Practices.
Intended audience
The document is designated for software development teams in GÉANT working on software projects. In particular, it should help newly created teams to learn the context of GÉANT as the organization, and the services it offers to facilitate software development.
Software development in GÉANT
Regardless of the software lifecycle adopted by the SDT, it usually involves several typical activities. Below, we outline these activities and describe how they are supported by WP9T2 or other dedicated GÉANT teams.
Nowadays, software development process is no longer limited to delivering software with useful features and impressive user-friendly interface. Software development has become a much more holistic process with the increase of complexity, the increase of development teams and the introduction of agile methodologies, which leverages requirements to software development environments to providing more code resilience to issues introduced from multi-participant collaboration and ability to automate the build and delivery processes, thus the platforms themselves grow and increase complexity of maintenance,
Software development, build&deployment
In order to better support software development in GÉANT and address the needs of SDTs, WP9T2 supports and maintains a number of software development platforms, which can also be easily integrated to provide even more effective and user-friendly environment.
- BitBucket is a Git-based code repository developed by Atlassian, which not only secures typical storage and versioning capabilities but also provides several extensions.
- Jira is an issue and ticket tracker, also commonly used for defining and managing requirements and general software project management. Service Desk and other extensions (like issue collectors) allow the development teams to support their users and customers to get feedback and raise issues directly via the dedicated form.
- GitLab is an open source end-to-end software development platform, that spans the entire software development lifecycle from repository management through team communication to task coordination and Continuous Integration/Continuous Delivery (CI/CD), including a simplified form of an issue and bug management.
- Artifactory is a repository for managing software packages and components in various versions that helps to share released software and libraries.
- Bamboo is a CI/CD service well integrated with other Atlassian services like BitBucket and Jira. It allows for automating several activities in the software development process. The instance provided by WP9T2 helps is automating CI/CD pipelines which could be used for software integration, testing and deployment.
- SonarQube is a static code analysis tool that supports quality assurance.
In addition, virtual machines for testing and software integration are provided within QALab and available through a dedicated service desk operated by WP9T2.
Communication
Due to its distributed organizational structure, GÉANT offers a variety of tools supporting communication within teams and between teams.
There are no project- or team specific mailboxes for team members that would be managed by GÉANT. Instead, they are provided and managed by the home NRENs of the team members.
For instant communication, GÉANT offers and administers a Slack space. Several topic-oriented channels already exist, and new ones can be created on demand. The channels could be left publicly open or restricted to specific members. Slack allows for archiving messages, but due to the license restrictions, there is a limit on the number of messages in the archive.
Confluence (available at https://wiki.geant.org) is another commonly used communication tool. It allows for documenting the work, online collaboration and exchange of data. Most projects have dedicated spaces, with clearly separated public and private content. Additionally, Confluence includes several plugins that extend its capabilities for sketching, planning, scheduling events, tracking comments etc.
Quality assurance
Quality assurance and management (QA) usually relies on tests conducted at different levels. In GÉANT it is the SDTs who are responsible for organizing and executing tests, although there are some best practices for SDTs (elaborated by WP9T2) that provide recommendations to the teams with that respect. Another pillar of quality is static code analysis and review, which is as important as software testing and is necessary to deliver high-quality software products. WP9T2 offers a variety of code review services: from the assessment performed solely by a tool (SonarQube and Mend) and interpreted by WP9T2's experts, up to a comprehensive review conducted by human subject-matter experts. The services provide a high level of security and maintainability related protection for the GÉANT products and can be customized to fit the SDTs' needs.
Also, Jira (managed by WP9T2) is essential for effective QA-related work. It allows for tracking the progress of issues and tasks, starting from requirements, through development, up to fixing and verifying defects.
Project analytics and reporting
The primary source of holistic information about GÉANT software projects is Software Catalogue (SC). It is a GÉANT-wide system (hosted and developed by WP9T2) that collects and delivers a variety of information about GÉANT-created software. The SC provides a rich and consistent profile for a single software project, no matter if its resources are hosted in a single place or are distributed over the Internet (issue trackers, Git repositories, etc.). That way, SC provides a big picture from two perspectives: the individual project, and the entire organization. Project records cover a broad spectrum of information including (but not limited to) automatically discovered data, such as community members, technology stack, analytics and reports (statistics, visualisations for: code, issues tracker, member's activity), as well as manually maintained information (e.g., basic description, specific practices applied by the team). The SC integrates smoothly with external analytical tools such as SonarQube and Mend, and provides the access to data in GitHub and GitLab (which is coming soon!) to learn more about the project offer than what these tools can tell about the source code and its evolution.
Management of external software libraries, their licences and their mutual compatibility is also a crucial concern for GÉANT. WP9T2 offers a service based on the Mend tool, which scans the software files and identifies the licenses of various product components. It can be also used to support the interpretation of mutual compatibility of present licences and identification of components with licenses that are not compatible with the product's licence. In addition, it identified obsolete versions of libraries and those that ate known to be vulnerable.
Help and Knowledge Base
For technical information, GÉANT provides a dedicated Confluence space with Knowledge Base, which provides answers to several recurring issues. It is a good starting point for collecting information, and is constantly extended and updated.
General and long-term support
In GÉANT, SDTs are given a high level of freedom and independence in organizing their internal work, which means that they can adopt, modify or customize the work process in the most effective way, depending on actual needs and established patterns that work for them. However, some requirements and constraints are project-wide and need to be addressed by all teams.
In the area of product licensing and IPR governance, the IPR coordinator provides guidance and support in specific cases that require human judgement or advice.
Product Lifecycle Management
The process-oriented framework that is common to all teams is Product Lifecycle Management (PLM), which identifies stages of the software development process and defines criteria for transiting between them. The criteria are called gates; each gate has several requirements associated with them. For example, a team willing to transfer and deploy their product in production needs to successfully perform a series of checks that will further be reviewed and approved by the Project Management Board. Only then the product will be made available to the users (either internally within GÉANT or publicly). While some of the checks can be conducted directly by the team, some others are offered and performed by dedicated teams in GÉANT, which provide such services. In the case of transferring the project to production, a comprehensive code review is conducted by WP9T2's Test&Analysis subteam.
PLM is managed by GÉANT Project Management Office, but WP9T2 helps the SDTs in passing the PLM gates with Common Best Practices.
Supporting the process management within teams
WP9T2's Software Management subteam works and provides support focused on the optimization of software processes. Specifically, two frameworks have been created: the Software Maturity Model, which sets objectives that should be addressed by the teams in respective process areas, and the Catalogue of Common Best Practices provide directional advice on how to address the objectives from the maturity model. These frameworks are complementary and can be implemented and adopted incrementally, depending on the subject team's needs and context. In addition to that, the Software Management subteam helps also in implementing the best practices in the teams by recognizing their expectations and collaborating with the teams to adapt the practices to their needs. Best practices have been extracted from the knowledge and experience developed by all GÉANT teams.
Training
Effective software development heavily relies on continuous improvement and learning. For that purpose, the GÉANT Learning and Development (GLAD) team organizes and offers training to all GÉANT members. Specifically, two annual events are organized by WP9T2 (under the auspices of GLAD), designated for software developers: School of Software Engineering (SSE) and Secure Code Training (SCT). SSE usually involves a 3-day long workshop focused on a single topic that is both relevant and interesting to the participants. It is not technology-intensive but rather emphasizes developing analytical skills, modelling, collaboration and creating thinking. SCT is a more technical workshop and is intended to increase the skills of GÉANT's software developers in writing secure code.
To present the tools that are transferred to production, WP9T2 provides workshops, organized also by GLAD. They have the form of infoshares or webinars.
There are also other pieces of training delivered directly by GLAD. For the details of planned and forthcoming events, please refer to the GLAD calendar.