...
- Test Design Specification
- Test Case Specification
- Test Data ReportData Report
- Test Environment Report
- Test Execution Log
- Detailed Test Results
- Test Incident Report
...
First, the data requirements that are implied by the test plan and test design are put together. At the same time On the base of use cases, the tools for preparation or generation of test data are selected. Since the test case specifications provide more elaborate details , and the on the base of.They include requirements related to type, range, representativeness, quality, amount, validity, consistency and coherency of test data. Additional concerns may related to sharing of test data with the development team or even end users. If test data are not entirely fabricated, but is extracted from the existing databases or services and can be associated with the real services, processes, business entities or persons, the policies and technical procedures for its anonymisation, obfuscation or protection may need to be established. If data are generated, then the approach for creation of adequate data should be established, and validity of tests results obtained with such data elaborated. On the base of the set test levels and use cases, the tools or means for or collection, generation preparation of test data are defined.
Test case specifications provide more elaborate details about data needed and should be sufficient to support the actual collection or generation of adequate test data. The selected tools are then used to prepare these data for execution of all use cases, including their injection into the system or interactions during the test procedure steps. At the same time, the corresponding expected test outputs are defined, and, if possible, automated methods for comparing the baseline test data against anticipated results. The limitations of test data and supporting tools are identified and explained what to do in order mitigate these limitations during test execution and interpretation and evaluation of the results. Finally, the measures that ensure usability and relevancy of test data throughout testing process need to be conducted. They include data maintenance, for example to support the changes in the system, but also data versioning and backup. The decisions and knowledge produced during the preparation of the test data should be captured in the second stage of writing of this report.
Test Environment Test Environment Report
This document describes the data used in testing. It is typically produced in two stages. First, the requirements for the test environment that are implied by the test plan, test design specification and possibly even the test cases are put together, and the initial
...
The test execution log is the record of test cases executions and obtained results, in the order of their running. Along with test incident reports, the test execution log is a base for test status and completion reports. If the testing is organised around scenarios instead of test cases, the general structure of the log is unchanged, except that inputs, states and outputs are replaced with interpretation of the interactions and results for each segment of the scenario, supported by key excerpts or snapshots of characteristic inputs and outputs.XXX It allows checking These documents allow direct checking of the progress of the testing and provides valuable information for finding out what caused an incident.
This log provides a chronological record of relevant details about the execution of tests, by recording which tests cases were run, who ran them, in what order, and whether the test were passed or failed. The test passed if the actual and expected results were identical; it failed if there was a discrepancy. For each test execution, the versions of the system under test, its components, test environment, and specifics of input data are recorded.If the test design specification permits to classify the execution status as "partial", it must also clarify how to treat such outcomes within the feature pass/fail criteria and acceptance criteria.
Each test execution Each Test Execution should start with a standardised header for each execution . The executions should be ordered from the oldest to the newest. Optionally, the recorded executions may grouped by test cases, but, if this is done, it is important to maintain ability to trace the actual execution sequence of all test cases in order to detect possible interference between them.. For each test execution, the versions of the system under test, its components, test environment, and specifics of input data must be recorded. These are the recommended descriptors:
- Test case ID or short identifying name - may be placed at the top of the group
- Order of execution number - useful for cross-referencing
- Date and time
- Testers - person who run the test, may also include observers
- Test bed/facility - if several test beds are used in testing
- Environment information - versions of test items, configuration(s) or other specifics
- Specific presets - initial states or persistent data, if any
- Specific inputs - inputs/test parameters or data that are varied across executions, if any
- Specific results - outputs, postconditions or final states, if different from the expected; may refer to detailed test results
- Execution status - passed, failed, partial (if permitted)
- Incident reports - if one or several test incident reports are associated with this execution, they is referenced here
- Comments - notes about the significant test procedure steps, impressions, suspicions, and other observations, if any
...
In case of a deviation or partial success or failure, a more detailed description of the execution status should be provided. If the test design specification permits to classify the execution status as "partial", it must also clarify how to treat such outcomes within the feature pass/fail criteria and acceptance criteria.
The data captured in the test execution log, along with other test level specifications and reports, should be sufficient to reproduce individual tests, that is, to recreate the needed setup, execute the test and produce same or similar results.
If the testing is organised around scenarios instead of test cases, the general structure of the log is unchanged, except that inputs, states and outputs are replaced with interpretation of the interactions and results for each segment of the scenario, supported by key excerpts or snapshots of characteristic inputs and outputs.
Detailed Test Results
Detailed Test Results, which a tester gets after performing the test, is always documented along with the test case during the test execution phase. After performing the tests, the actual outcome is compared with the expected outcome and the deviations are noted. The deviation, if any, is known as defect.
...