CSE 15L, Fall 2007

Software Tools and Techniques Laboratory

The Scientific Debugging Process

We can outline the scientific debugging process in a number of steps:

  1. Characterize the intended behavior of the code. Focus on the behavior important to a user of the code; you don't need to explain how it works internally. Consider questions like: What input does it take? What output should it give?

For each deviation from the intended behavior:

  1. Characterize the failure. Again, take the perspective of someone using the code. Is it a compile-time, run-time, or semantic error? What behavior do you observe that leads you to believe there is a defect in the code?
  2. Form a hypothesis. Now you need to think like a programmer. What could be wrong with the code that would cause the failure you observed? Record each hypothesis even if you discard it later; invalidating hypotheses is an important part of the process.
  3. Make a prediction. Consider changing the conditions under which you observed the failure: use different input, add debugging output, or alter the code in some way. Assuming your hypothesis is correct, what do you expect to observe? Often an even better question is the reverse: assuming your hypothesis is incorrect, what do you expect?
  4. Perform an experiment. Change the conditions according to your prediction and compile or run the program again.
  5. Observe the results. What is the relevant behavior now?
  6. Reach a conclusion. Did the results match your prediction? Is the bug fixed? Is your hypothesis invalidated? Do the results suggest a new hypothesis?

Ideally you'll repeat the necessary steps until you're convinced the bug is fixed. Keep in mind it's a good idea to consider positive propositions such as "the bug is fixed," "this change fixed the bug," and "the program behaves correctly" to be hypotheses in themselves, and test those too.

Note that the process isn't as linear as it may initially appear in the steps above. Often you will have multiple hypotheses and predictions simultaneously. This is one reason it's a good idea to make notes and try to focus on one specific problem at a time.

Valid XHTML 1.0 Strict Valid CSS!