...
- Organizational numeric identifier of the document, may be ommitted if the document name is used as the identifier
- Descriptive and identifying name of the document
- Name of testing (sub)project or use case, if not clear from document name
- Document date
- Version number
- Author or authors and their contacts
- Version history (table with version numbers, dates, contributors and descriptions of changes)
...
- Test Design Specification
- Test Case Specification
- Test Data Requirements
- Test Data Report
- Test Environment Requirements
- Test Environment Report
- Detailed test results (base for status and completion reports on the base of individual tests execution logs and anomalies/incidents reports)
Test Plan
Test Plan The test plan provides an overview of what the system needs to meet in order to satisfy its intended use, user needs, or scope of the intended testing effort, and how the actual validation is to be conducted. It details out the operational aspects of executing the test strategy for the particular testing. It outlines the objectives, scope, approach, resources (including people, equipment, facilities and tools) and amount of their allocation, methodologies and schedule of intended the testing activitieseffort. It usually also describes the team composition, training needs, entry and exit criteria, risks, contingencies, test cycle details, quality expectations and tracking and reporting and tracking process. The Test Plan test plan may account for one or several test suites, but it does not detail individual test cases.
It As a pivotal document, the test plan is an instrument mutual understanding between the testing team, development team and management. In case of major impediments or changes, it should be updated as needed and communicated to all concerned. Such updates may lead to further changes documents specifying test design, test cases, and data and environment requirements. Given the multifaceted and overarching nature of the test plan and In order to avoid unnecessary backpropagation of changes, in should not over-specify the implementation details precised in subordinate test level documents.
The recommended structure of the test plan is as follows.
...
- Project plan
- Product plan
- Related test plans
- Requirements specifications
- High level design document
- Detailed design document
- Development and testing standards
- Methodology guidelines and examples
- Organisational standards and guidelines
- Source code, documentation, user guides, implementation records
Glossary
Terms Key terms and acronyms used in the document, target domain , and testing as needed are described here. The glossary facilitates communication and helps in eliminating confusion.
...
An additional list of features that will not be tested may be included, along with the reasons. For example, it may be explained that a feature will not be available or completely implemented at the time of testing. This map prevent possible misunderstandings and waste of effort in tracking the defects that are not related to the plan.
Together with the list of test items, this section describes the scope of testing.
Approach
This section describes the strategy of the test plan that is appropriate for the plan level of the plan and in agreement with other related plans. It may extend the background and contextual information provided in the introduction.
...
This section describes what is produced by the testing process. These deliverables my be the subject of quality assessment before their final approval or acceptance, and besides all the elements of test documentation that are described here, may also include test data used during testing, test scripts, code for execution of tests in testing frameworks and outputs from test tools.
Activities/Tasks
This section details outlines the testing activities and tasks, along with their dependencies and estimates of their duration and the resource required resources.
Staffing and Training Needs
...
The schedule of phases should be detailed to the level which ensures certainty that is attainable with information available at the moment of planning. It should define the timing of individual testing phases, milestones, activities and deliverables and be based on realistic and validated estimates, particularly as the testing is often interweaved with development. Testing is the most likely victim of slippage in other the upstream activities, so it is a good idea to tie all test dates directly to the completion dates of their related developments.This section may also describe the approach for dealing with specific slippages, which may include simplification or reduction of some non-crucial activities, relaxation of the scope or coverage, elimination of some test cases, engagement of additional resources, extension of testing duration.
Risks and Contingencies
This sections, which complements "Suspension Criteria and Resumption Requirements", defines all other risk events, their likelihood, impact and counter measures to overcome them. Some risks may be testing related manifestations of the overall project risks. Some risk examples are lack or loss of personnel at the beginning or during testing, unavailability or late delivery of required hardware, software, data or tools, delays in training, or changes to the original requirements or designs.
...
Although some contingency events may actually be opportunities, it is, due to the limited scope of testing within the wider project or service delivery context, quite unlikely that the opportunities related to the very testing process will occur. However, the outcome of testing or some of its results may offer opportunities related to the subject of testing that may be enhanced, exploited or shared.
...
The approach for dealing with schedule slippages should be described in responses to associated risks. Possible actions include simplification or reduction of non-crucial activities, relaxation of the scope or coverage, elimination of some test cases, engagement of additional resources, or extension of testing duration.
Test Status Report
Test Status Report is a one-time interim summary of the results of the execution of testing activities. It may describe the status of the all testing activities or be limited on a single test suite. This report needs to be highly informative and concise and , as well as the Test Summary Report, is sublimes the information obtained during test execution and recorded in test logs and incident reports. It needs to be highly informative and concise and should not elaborate minor operational details.
...
On the top of the report that contains standard metadata in the condensed form shuld should be provided, but without version history, since the document itself is considered to be a quick snapshot.
Summary
The document can start or end with totals of passed, failed and pending test cases, scenarios or tests, and identified defects (if individually tracked). It may also deliver show the coverage of test cases or code, consumption of resources and other established progress metrics, in numbers and, possibly, charts. Close to this summary, a comprehensive assessment should be provided.
If needed, it provides evaluations and recommendations based on the interim results and incidents encountered during the testing. It may also signal the red flags that deserve recipients’ attention. It should report the resolution of issues that were highlighted in the previous status report, but there is no need to report the issues that were already reported as solved.
The summary of activities conduced since the previous status report is also optional. If present, is should exist in all status reports.
The details of the progress are expressed in a form of a table describing the outcome of execution of individual test cases or scenarios, cumulatively since the start of testing. Typical columns to be used are:
- Test case ID
- Test case name
- Last execution date
- The last execution status (not run, failed, partial, passed) or counts of failed and passed test executions
- Number of associated defects
- Brief comment
The listed tests cases, scenarios or procedures, can be grouped by test suites (if the report addresses several suites), test types or areas.
Test Completion Report
The test completion or summary report is a management report that brings together the key information uncovered by the accomplished tests. It recapitulates the results of the testing activities and indicates whether the tested system is fit for purpose according to whether or not it has met acceptance criteria defined in the project plan. This is the key document in deciding whether the quality of the system and the performed are sufficient for to allow taking of the step following the testing. Although it provides a working assessment on success or failure of the system under test, the final decision is made by the evaluation team.
This document reports all pertinent information about the testing, including an assessment about how well the testing has been done, the number of incidents raised and outstanding, and crucially an assessment about the quality of the system. Also recorded for use in future planning are details of what was done, and how long it took.
The test summary report can have a structure similar to the test status report, but some details can be omitted from its tables. If there are if more than dozen items, consider reducing the number them at the higher level of
On the other hand, the narrative can be more elaborate, and it should also bring together all significant issues that were reported in individual status reports.
The details of the progress are expressed in a form of a table describing the outcome of execution of individual test cases or scenarios, cumulatively since the start of testing. Typical columns to be used are:
- Test case ID
- Test case name
- Last execution date
- The last execution status (not run, failed, partial, passed) or counts of failed and passed test executions
- Number of associated defects
- Brief comment
If there are if more than dozen items, consider grouping and subtotaling the table rows according to the test suites, test items or scenarios (as listed in the project plan), test types or areas. If a single report addresses several suites, each should have a separate test status report, or at least its own totals and details tables.
Observations and Highlights
If needed, the document provides evaluations and recommendations based on the interim results and incidents encountered during the testing. It may also signal the red flags that deserve recipients’ attention. It should report the resolution of issues that were highlighted in the previous status report, but there is no need to report the issues that were already reported as solved.
Activities
The summary of activities conduced since the previous status report is optional. If present, is should exist in all status reports.
Test Completion Report
The test completion or summary report is a management report that brings together the key information uncovered by the accomplished tests. It recapitulates the results of the testing activities and indicates whether the tested system is fit for purpose according to whether it has met the acceptance criteria defined in the project plan. This is the key document in deciding whether the quality of the system and the performed are sufficient for to allow taking of the decision or step that was the reason for the testing effort. Although the completion report provides a working assessment on success or failure of the system under test, the final decision is made by the evaluation team.
This document reports all pertinent information about the testing, including an assessment about how well the testing has been done, the number of incidents raised and outstanding events. It must describe all deviations from the original test plan, their justifications for the deviations and impacts. Provided data should be sufficient for the assessments of the quality of the testing effort.
The provided narrative should be more elaborate than in the status reports.
Summary
It recapitulates the evaluation of the test items. Although the test completion report can reflect the structure of the test status report, the details that were only temporarily significant can be omitted from it. The table rows or subsections should correspond to the test items or scenarios listed in the project plan. The summary should indicate the versions of the items that were tested, as well as the used testing environment. For each item it should be briefly explained what was tested and what was the outcome.
Test Assessment
This is a comprehensive assessment of the conducted testing. It should also point at the areas that may require further investigation and testing.
Variances
All variations and discrepancies from the original test plan should be noted here. This section can also provide an assessment of differences between the test environment and the operational environment and their effect on the test results.
Test Results
This is a comprehensive interpretation of the test results. Include description of issues or defects discovered during the testing. Also describe the unexpected results and problems that occurred during the testing. For resolved incidents, their resolutions should be summarised. For unresolved test incidents, an approach their resolution should be proposed.
Evaluation and Recommendations
Propose the decisions regarding the tested system and suggest further actions on the base of the acceptance criteria, quality of the test process, test results and outcomes for individual test items. Provide recommendations for improvement of the system or testing that result from the reported testing.
Activities
The summary of activities conduced during testing should record what testing was done and how long it took. In order to facilitate improvement of future test planning, it should be in a form that does not require later thorough investigation of the plans and records of the conducted testingProvided data should be also sufficient for the assessments of the quality of the testing effort. In order to improve future test planning, it should records what testing was done and how long it took.
Test Design Specification
...