The principles of program testing
The principles of program testing
Despite that we’re all aware of theoretical restrictions of the software, in fact we waste a lot of efforts and costs to perform this task. Although it is considered unreasonable and extremely dangerous to refuse from testing.
The other techniques of verification, such as static analysis, model validation and tests have great potential, but none of them is perfect enough to replace the tests as the dominant technology. Therefore you must understand, what is the scope and limitation area of testing and how to do it right.
Definition. In order to test the program, you should try to make it work wrong.
Based on this principle, the testing process gains its purpose: its only task is to find errors by initiating its failure. Any conclusion concerning its quality belongs to the field of quality assurance, but it has nothing to do with testing. This definition also reminds us that testing (in contrast to debugging), is not about correcting the failures, but only about their search.
Tests or specifications. Testing doesn’t replace specification.
The risk of misconception that the tests can serve as specifications was demonstrated by a range of program accidents that have happened just because no one thought about exceptional case. The specifications can be used for generating tests (even automatically, as in the case of model validation); however, the contrary process is impossible without human intervention.
Regression testing
Judging by the practice of software creation, the feature of this testing is the sad tendency to the return of the bugs fixed. This phenomenon is known as regression and makes it necessary to use regression testing, i.e., the checking that the fixed program areas still work properly. Therefore, you should always remember the mistake you have once found.
Predictions
Testing is useful only if you can clearly define that it has been carried out. This criterion is called “test prediction”. If you have a several dozen or even hundred of tests, you can check their results separately, but as their number increases, it is becoming less likely. This task requires automation.
Test cases, checked manually and automatically
Many test cases are checked manually. The testers create interesting scenarios for completion and prepare the corresponding tests. The process, limited to the manual tests, can’t take full advantage of the facilities of the modern computers.
All the approaches just complement each other.
The main advantage of the manual tests is their profundity: they reflect the developer’s understanding of the existing range of problems and data structure. The benefit of professional services automation is their breadth of coverage: they check a wide range of values (including the extreme ones) people can miss.