CSE 15L, Winter 2010

Scientific Debugging

Hypotheses in Debugging


A debugging hypothesis isn't just a guess about which part of the code has a defect, and it doesn't address how a defect got there or what code "should" be there instead. A hypothesis is not concerned with possible changes or what effects they might have. A debugging hypothesis is, first and foremost, a detailed explanation how the current code produces the behavior you've already observed.

A medical analogy

Just as a doctor needs a good diagnosis before prescribing a treatment, a debugger needs a good hypothesis before attempting to fix a program. Like a diagnosis, a hypothesis does not tell you what to do; it helps you understand what may be happening so you can try to predict the results of experiments.

Just as a medical diagnosis can be supported or invalidated by the results of examinations or lab tests, debugging hypothesis can be supported or invalidated by experimental results. Failed attempts to invalidate a hypothesis lend us more confidence it may be correct, and non-invasive tests are usually the best way to gain confidence. An attempt to treat an illness or fix a bug is an experiment too, it can be harmful if the diagnosis is incorrect, or the hypothesis false, so it's best to gather support for the hypothesis first.


Here is a pretty good hypothesis:

The code failed to compile because the variable Foo (capitalized) was declared on line 4, and later the variable foo (uncapitalized) was used on lines 8 and 12. Because Java variable names are case sensitive, in effect foo had been used but not declared, so the compiler failed to find the symbol and printed an error message to that effect.

Note this hypothesis includes concrete details that account for the behavior already observed, in this case a compile-time failure. One can imagine various experiments based on this hypothesis. One experiment would be to change capitalization in the use of another variable to verify the prediction that it will cause a similar failure. Another experiment would be to change the capitalization in the declaration of the variable on line 5, or in the uses on lines 10 and 12, but not both, of course!


Now that we've seen an example of a good hypothesis and some predictions and experiments that might be based on it, let's take a look at some inadequate hypotheses that exhibit some common mistakes:

Our hypothesis was that the variable should not be capitalized.

This might be called a "prescription," but not a hypothesis. It doesn't explain how the current code produces the behavior already observed; instead it jumps ahead to suggesting an experiment, although without saying so.

We hypothesized that changing the capitalization would make it work.

This is not a hypothesis, but it is almost a prediction. The biggest problem with this as a prediction is that the phrase "would make it work" does not clearly indicate what specific results are expected from the experiment.

Hypothesis: If we look at the code, we'll find a defect.

I might call this a "procedural forecast." It's not even a prediction in the sense of stating the expected results of an experiment; instead it seems to express hope for finding a hypothesis.

Hypothesis: The mistake is in the loop.

This might be a good hunch, but isn't specific enough to be a hypothesis. The right thing to do is pursue this hunch and look for a specific defect in the loop, then make a hypothesis that explains how the postulated defect produced the failure already observed.

It's useful to mention hunches in your report, whether or not they turn out to be good hunches, but don't make the mistake of calling a hunch a hypothesis!


Now that we've seen some good and bad examples of hypotheses, it may help clarify the concept to consider this definition of "hypothesis":

  1. A statement that suggests an explanation for an observation or an answer to a scientific problem and can be tested experimentally;
  2. a proposed explanation of a phenomenon.

This definition (from the South Dakota Science Standards Glossary) captures quite well the way a hypothesis functions in scientific debugging. It's important that each hypothesis explains the phenomenon you've observed—a failure, usually—and allows you to make predictions about the results you expect to observe when you perform experiments.


Valid XHTML 1.0 Strict Valid CSS!