January 2011 Edition
Computer Science Department, University of Cape Town
| MIT Notes Home | Edition Home |
| MSc-IT Study Material January 2011 Edition Computer Science Department, University of Cape Town | MIT Notes Home | Edition Home | |
Complete:
Testing is the [________] process of running software in with the intent of [______________] in the software. It is important to realise that testing is a [__________] process, and is not the accidental discovery of software bugs.
A discussion of this question can be found at the end of this chapter.
Who are the different parties involved in software testing, and how does the testing shift from one party to another?
A discussion of this question can be found at the end of this chapter.
What guidelines would you give for developing software that is easily testable?
A discussion of this question can be found at the end of this chapter.
Complete:
Test cases are controlled tests of [__________________] of the software. The objective of a test case is [_____________]. Testing software, then, is the development of [___________________], and then their application to the software.
A discussion of this question can be found at the end of this chapter.
What are the different stages of software testing?
A discussion of this question can be found at the end of this chapter.
What is the difference between white-box and black-box testing?
A discussion of this question can be found at the end of this chapter.
Why is it not practically possible to test every logical path through a piece of software? What alternatives are there?
A discussion of this question can be found at the end of this chapter.
Debugging begins with the software [_________] a test case. The software engineer debugging the software will then [_____________] a possible cause and [___________________] in the software. The failed test is then rerun. This might not always provide an exact reason for the bug, and so further test cases may also be written to [_______________________]. This all occurs iteratively, with each step hopefully providing more information to the developer as to the root cause of the error.
A discussion of this question can be found at the end of this chapter.
What are some common debugging techniques?
A discussion of this question can be found at the end of this chapter.
Testing is the planned process of running software in with the intent of discovering errors in the software. It is important to realise that testing is a purposeful process, and is not the accidental discovery of software bugs.
There are three parties involved in software testing:
The developers
Independent testers
The customers / users
Testing originally begins with the developers, who need to ensure that the software they have coded works as they intend. Testing is then taken over by independent testers, who are able to use more specialised testing frameworks than the developers themselves. They can also perform user-based testing, which most developers would not be able to do.
Finally, as the software is delivered to the customer and the software begins to be used, its users become the final group of testers.
Operability. New code developed for relatively bug-free software will have fewer bugs than code developed for relatively buggy software.
Observability. Software whose internal state can easily be examined, and that produces well defined outputs for particular inputs, is easier to test.
Controllability. If the tester has the ability to easily control the software, testing becomes easier: controllability allows for the easier inputting of data and examining of output.
Decomposability. Independent modules can more easily be tested, and changes made to a module are less likely to affect other modules.
Simplicity. The simpler the software, the fewer errors it will have.
Stability. Testing is easier if changes made to correct bugs are limited to independent modules.
Understandability. The better the testers understand the software — through good software design, documentation, and so on — the better they can test the software.
Test cases are controlled tests of particular aspects of the software. The objective of a test case is uncover a particular error. Testing software, then, is the development of a collection of test cases, and then their application to the software.
Unit testing, which tests the smallest modules of the software.
Integration testing, which tests the software as the modules are brought together.
Validation testing, which tests the software to ensure that it meets its requirements specification.
System testing, which examines how well the software integrates with the various systems and processes used by the customer.
Black-box testing tests the module as an object whose inner-workings are unknowable, except for how it handles its inputs and outputs. Black-box testing therefor does not examine a module's inner state, only the interfaces to the module. It assumes that if it handles its inputs and outputs correctly, then the module itself behaves correctly.
White-box testing examines the software's internal state; it does not assume that because a module has handles its inputs and outputs correctly that the module has behaved correctly. For example, after correctly outputting some data, a module may leave its internal state with invalid values, and so any further action will fail.
The number of possible logical paths through a piece of software can grow exponentially as if statements and loops are added. This would make the amount of time required to test a large software package prohibitive.
An alternative testing strategy is to determine the software's cyclomatic complexity, which tells us how many independent paths there are through the software. Each of these independent paths can then be tested; this will provide good code coverage (it will execute every line of code) without exhaustively testing every logical path through the software, and so greatly reduces the amount of time needed to execute the software.
Debugging begins with the software failing a test case. The software engineer debugging the software will then hypothesise a possible cause and implement the needed changes in the software. The failed test is then rerun. This might not always provide an exact reason for the bug, and so further test cases may also be written to narrow down the cause. This all occurs iteratively, with each step hopefully providing more information to the developer as to the root cause of the error.
Brute force debugging; the developer searches through stack-traces, memory-dumps, log files, and other data generated by the program to locate the error.
Backtracking; the developer examines the code that immediately produces the observable bug, and then moves backwards through the execution path until the cause of the bug has been found.
Cause elimination; the developer hypothesises reasons why the bug has occurred, and then either directly examines the code to see if these reasons exist, or produces further tests to narrow down the choice between various hypotheses.
Bisect; the developer examines previous versions of the software until one without the bug is located. The difference between that version of the source code and the next (in which the bug does not exist) is where the bug will be located.