January 2011 Edition
Computer Science Department, University of Cape Town
| MIT Notes Home | Edition Home |
| MSc-IT Study Material January 2011 Edition Computer Science Department, University of Cape Town | MIT Notes Home | Edition Home | |
Software testing never completes. It is an ongoing process that begins at the project's inception and continues until the project is no longer supported. During the software's lifetime the burden of testing slowly shifts: from the developers during design and programming, to independent testing teams, and finally to the customers. Every action that the customer performs with the software can be considered a test of the software itself.
Unfortunately, time and money constraints often intervene: it may not be worth the developer's time or money to fix particular software errors. This trade-off between resources spent vs potential benefit can easily occur for small errors, and for errors which are not often encountered by the software's users.
It is also possible (although difficult) to be statistically rigorous when discussing software errors. For instance, it is possible to develop a statistical model of the number of expected software failures with respect to execution time. Error rates over a given period can then be specified given a particular probability. When that probability is low enough, testing could be considered “complete”.
An important aspect of testing is to ensure that the written software is written in a way that it can easily be tested. As the software becomes more difficult to test, so the software will be tested less often.
There are a number of guidelines that software engineers can follow in order to write software that can be easily tested. The design principles mentioned in Chapter 7, Design, are a good place to start. In addition to this, here are some further guidelines:
Operability: This is partly a self-fulfilling quality: the fewer bugs that a software system has, the easier the software will be to test, since the testing will not progress erratically while time is taken to repair bugs. Clearly, the more care is taken during software development to produce bug-free code, the easier the testing that follows will be.
Observability: Software tests examine the outputs produced by the software for particular inputs. This means that software is easier to test if the inputs produce distinct, predictable outputs. Further, it can be helpful to be able to examine the software's internal state, especially where there may be no predictable outputs, and especially when an error has been discovered.
Controllability: As just mentioned, software testing involves examining outputs for given inputs. This means that the more easily we can provide inputs to the software, the more easily we can test the software. This implies that software is more testable when the tester has the ability to easily control the software in order to provide the test inputs. This controllability applies to the tests as well: tests should be easily specifiable, automated, and reproducible.
Decomposability: When software can be decomposed into independent modules, these modules can be tested individually. When an error occurs in an individual module, the error is less likely to require changes to be made in other modules, or for the tester to even examine multiple modules.
Simplicity: Clearly, the simpler the software, the fewer errors it will have, and the easier it will be to test. There are three forms of simplicity that software can demonstrate: functional simplicity, in which the software does no more than is needed of it; structural simplicity, in which the software is decomposed into small, simple units; and code simplicity, in which the coding standards used be the software team allows for the easy understanding of the code.
Stability: If changes need to be made to the software, then testing becomes easier if these changes are always contained within independent modules (via, for instance, decomposability), meaning that the code that needs to be tested remains small.
Understandability: Clearly, the more the testers understand the software, the easier it is to test. Much of this relates to good software design, but also to the engineering culture of the developers: communication between designers, developers and testers whenever changes occur in the software is important, as is the ability for the testers and developers to easily access good technical documentation related to the software (such as APIs for the libraries being used and the software itself).
Test cases are controlled tests of a particular aspect of the software. The objective of a test case is to uncover a particular error. Testing software, then, is the development of a suite of such test cases, and then their application to the software.
Tests should be kept simple, testing for specific errors (or specific classes of errors) rather than testing whole branches of the software's functionality at a time. In this way, if a test fails the failure will point to a particular area of the software that is at fault.