Unit testing is concerned with testing the smallest modules of the software. For each module, the module's interface is examined to ensure that information properly flows to and from the module. The data structures which are internal to the module should be examined to ensure that they remain in a consistent state as the module is used. The independent paths (see the section called “ Flow graphs, cyclomatic complexity and white-box testing ” in this chapter) through the module should each be tested. Boundary conditions should also be closely examined to ensure that they are properly handled (such as, for example, not reading past the end of an array). Importantly, remember to test the error handling code and ensure that they handle and report errors correctly.

Unit tests can easily be incorporated into the development process itself: unit tests can be written while each module is written. Indeed, some software design methods (such as extreme programming, see Chapter 2, Process and Model) ask for unit tests to be developed before the software itself. Regression tests (a form of integration testing), too, can be incorporated into the development process — one way of doing so is to have them automatically run each night, after all code developed that day has been submitted to a central repository.

Once unit testing is complete, the next important step is to combine the separate modules and ensure that they function correctly together. One may think that because each module functions correctly by itself that they will function correctly together. This is not always the case. For example, module A might have certain expectations about the behaviour of module B that are not being met, causing A to function incorrectly.

It may seem that integration testing can be carried out by combining all modules at once and testing how they function. Unfortunately, this “big bang” approach can make it difficult to track an error down to any one particular module. A better approach is to combine modules together incrementally, testing their behaviour at every step. Each increment brings us closer to having the complete software, but each increment remains constrained enough for us to properly test.

There are two general methods for performing module integration: the top-down and bottom-up approaches. Top-down integration testing begins by creating the overall software where much of its functionality is provided by empty stub modules. These modules perform no function other than to allow the software to compile, and to provide a way to exercise the modules which are being tested. The stubs are then replaced with the actual modules, one at a time, beginning with the modules involved with user-interaction and ending with the modules which perform the software's functionality. As each module passes its tests, a new module is integrated.

Clearly, top-down integration is difficult because of the need to create stub modules. The proper testing of some modules may rely upon a wide range of behaviour from the sub-modules, and so either the testing must be delayed until the stubs have been replaced, or the stubs must offer much functionality themselves (and may themselves be buggy).

The alternative to the top-down approach is bottom-up integration testing: here, modules which do not rely on other modules are combined together to create the software. Because we begin with modules that do not rely on other modules, no stub code is needed at all. At each step we test the combined software. When all tests have passed we add another module together. Ultimately, all the modules will be combined into the functioning software.

Whether a top-down approach, a bottom-up approach, or a mixture of both approaches, is used, whenever a new module is added to the software in order to be tested, the whole software has changed. Modules which rely on the new modules may behave differently, and so once again allthe modules have to be tested. Regression testing is the testing of the previously tested modules when a new module is added. Regression testing should occur at every incremental step of the integration testing process.

Unit and integration testing asks the question, “Are we developing the software correctly?” Validation testing, on the other hand, asks, “Are we developing the correct software?” In other words, validation testing is concerned with whether the software meets its requirements.

Validation testing focuses on the user-visible portions of the software, such as the user-visible inputs and outputs, and the software's actions. The tests examine these user-visible portions to ensure that they meet the software requirements. While not part of the software itself, the documentation should also be examined to ensure that they meet any requirements concerning them.

We have previously mentioned beta testing as the process of the customers themselves testing the software. This can be a useful tool in the validation testing process, since the developers cannot foresee exactly how the customers may use the software.

Software is always employed within some larger context, such as all the systems and processes which a business customer may have in place. System testing is concerned with how the software behaves as it integrates into the broader system in which it will be used.

For example, when the software fails or suffers from an error it must not cause the whole system that is using it to fail. Recovery testing examines how the software behaves when it fails. Security testing examines how well the software protects sensitive information and functionality from unauthorised access. Stress testing examines how the software functions under an abnormal load. While software may perform well by itself, its behaviour can be quite different when its used in a larger setting; performance testing examines the software's performance within the context of the system as a whole.