Testing strategies
Quality assurance
Testing strategies
And now we are moving from the practice of testing to exploring the new testing technologies (which, which, however, are vulnerable to the risks of thought processes). You came up with the idea that seems to promise improvement and is confirmed by your intuition. Testing is a complex process, and after the objective analysis not all reasonable ideas prove useful. A typical example is a random testing. Intuitively, it seems intuitively that every strategy, using the knowledge about programs, must be better than inputting random data. However, the objective indicators, such as a number of errors found, show that random testing often exceeds the ideas that seemed reasonable.
The empirical assessments of testing strategies. Evaluate every testing strategy, but however interesting it may seem, make an objective assessment, using the precise criteria in the reproduced testing process.
Criterion
Applying the principle, we wonder what criterion we should use. The literature on testing offers such indicators as the number of tests which have initially caused the program failures”. The practitioners find this indicator not so useful. We want to find all the mistakes but not just one. Let’s say that the use of the criterion “first failure” will be correct, and this criterion is used repeatedly. But the future mistakes can have an entirely different nature; but the automatic process must identify the maximum possible number of errors, but not dwell on the first one.
The number of testing is not useful to the managers (that have to decide whether to stop testing and release the product) and the users (that need to assess the density of errors). More adequate indicator is the time of testing, necessary for detecting errors.
Criterion. The most important feature of the testing strategy is the number of bugs found as a function of time.
The function of detection, i.e. the number of errors depending on the time fc (t), is useful for two reasons — using the software base with the known bugs, you can assess the strategy, looking how many mistakes the base can detect for the given time. The test automation companies can add fc (t) in the model of reliability to estimate how many mistakes have left, and answer the old question “When to finish the testing?”
