General SA4 Testing Framework
Tailoring Approach to Testing
Testing as a cognitive, critical and often creative process.
Wise application of standards, tools, best practices, adapted to a specific case, goal, available resources, and local culture. This document as a support for establishing the common ground and customisable toolbox, and supporting guide. Relationship to the existing GN documents, testing standards, and software testing practices. Acknowledge that the recent standardization efforts try to build upon practices and experiences from different fields of engineering and industries. Do not reinvent the wheel, use the existing expertise, tools, practices, and processes. Actual testing, even with automated execution is designed and performed by humans.
No guarantee that any specification can ensure fully repeatable testing practice that is applicable in all possible contexts. Often, inherent fluidity of the environment and goals and complexity of tested system is likely to preclude any exhaustive formalization.
Avoid over-administrative approach; no one-size-fits-all. Schematic and formal application and implementation of the standards, or procedures templates “by the book”, this document included, endangers to petrify, bureaucratise, and over-proscribe validation and testing process and impede innovation. What matters is the quality. However, running production-level services may require formalized processes, traceability or even auditability of validation, and particularly of check procedures.
General Testing and Validation Governance
This document provides GN with a generalised technical specification for testing and validation. It can be seen as a a contribution to the general specification of GN approach to validation and testing, on the base of its the overall testing policy (for implementation and management of services - any sources/refs?), and as an (tentative?) technical expression of the shared testing strategy.
The overall policy and strategy, as well as this technical document, must be further discussed and agreed by stakeholders and published.
Should be periodically reviewed and updated on the base the experience from the actual performed tests, organizational and environmental changes and evolution of overall policies and goals. It is (one of / input for) general GN-level testing related policy and strategy documents.
Evaluation Planning
Need to familiarize with the context in order to define the scope, organize development of the test plan, identify and estimate risks, establish approach towards risks, design test strategy, determine staffing profile and schedule, draft test plan, establish consensus/get approval for the plan, publish the plan.
Traceability is the key characteristic of both Evaluation and Test Enactment related artifacts. Repetability for tests related to core requirements and functionality. No need to repeat and maintain repeatability of tests that were already done unless possible to fully automate or required for compliance or periodic audits.
Test Enactment
Test design including test specification and design requirements, test environment set-up, test execution and related reporting.
In parallel: Monitor test progress on the base of test measures (metrics/reports), report test status to stakeholders, issue control/correction directives and perform corrective/adaptive actions for/on the test design, environment, execution, and perhaps even evaluation level test plan. Can be done in a single flow or iteratively.
Possibly provide suggestions or inputs for the update of the general Testing and Validation Process and Strategy.