CSE 15L, Spring 2010

Scientific Debugging

Scoring Guide for Notes and Reports

Graders will use the following scoring guide to structure their evaluation of student work and give appropriate, constructive feedback. The guide is subject to ongoing revision.

Students are advised to pay attention to the directions on the notes and reports pages. Work carefully following those directions should receive high scores.

Notes

record keeping: 50%

shows exactly how program was invoked, including commands and input
shows actual behavior: output, error messages
(with very long output, may be reasonable to show just parts inspected)
shows every experiment, especially experiments that don't fix failures (whether intended to or not)
includes final code (unless code in multiple files has changed)

process: 50%

documents actual failures even if code defects seen first
only one (experimental) condition is changed at a time
- for changes intended to fix a failure, the same test(s) as before are repeated
at end, shows final comprehensive testing; include tests from earlier steps to ensure they all work
does not contain too much written explanation (such as hypotheses, predictions, and conclusions)

Reports

program behavior: 10%

describes intended behavior precisely and correctly
makes clear why the failure chosen was the most interesting
describes how the actual behavior deviated from what was intended

hypotheses: 30%

states hypotheses in reasonable detail, not just hunches, not confused with predictions or experiments
includes falsified hypotheses (when results didn't match predictions)

experiments: 30%

each prediction and experiment involve changing only one condition (not changing code and tests at the same time)
predictions stated and experiments explained clearly and in sufficient detail for the reader to understand what was done
does not claim results prove a hypothesis true (unless they really do)

good form: 30%

uses terms correctly (especially hypothesis and prediction)
- for instance, anything labeled "hypothesis" is strictly part of the hypothesis
is concise and unambiguous
- does not include blocks of code, input, or output from the notes; instead, incorporates examples and important details into sentences
tells the story in chronological order (failure before defect, unless it is explained the defect was seen first)
avoids false claims and self-contradictions
uses correct spelling and grammar (occasional mistakes forgiven)
is written in complete sentences, with appropriate paragraph or section breaks