The page limit is 10 pages. This is firm. The paper must be in 12 point font, with at least 1 inch of margin on all edges.
Papers will be evaluated in two parts: a set of numeric scores and a written commentary. Scores will be on a 1 to 7 scale (with 4 as an "average" in all cases 1 is bad, 7 is best). Do not assign a zero score. Also, please use integer values.
Scoring will be in four categories:
Import -- is the work (both its area and results) important to the OS community? Low scores might be given for evaluations of marketing literature and papers on inappropriate or dead topics. High scores are for papers that nearly all attendees will want to read and understand carefully. (This score is sometimes more a measure of how ``hot'' the area is and not a measure of the paper's quality; it will not affect the grade.)
Novelty -- are the observations novel / germane? Low scores should be given for papers that re-hash obvious results or known observations about works in the topic area. High scores are for papers that point out new research areas (portions of design space not explored that ought to be), new fields, or demonstrate new ways to view / attack a problem.
Quality -- are the observations / criticisms sound? A low score might go to a paper whose observations are incorrect or whose critiques are biased or not well supported in your opinion. High scores are for papers with enough justification to convince you that the opinions are correct and viable.
Overall -- should we accept this paper or not? This is by far the most important number. It need not be an average of the other numbers, but it should reflect them. This number can also reflect issues in addition to those described above (e.g., poor presentation or lack of knowledge of related work).
Note that the conference evaluation contains criteria for novelty and importance to the OS community; when I grade these, these will be less important -- I will pay more attention to the quality of the reasoning and the soundness of the observations / criticisms, so having selected what's currently ``hot'' (or what's not) wouldn't be so important.
The review should be in the following (best if you cut-and-paste or used the review template). The reviews will be machine parsed to generate statistics.
Paper 99 -------- 8< -------- scores -------- 8< -------- scores -------- Import Novelty Quality Overall 7 1 5 4 -------- 8< -------- comments -------- 8< -------- comments -------- Your comments on the paper. This is public comments that the authors of the papers will see. Provide feedback to improve their paper, etc.
|[AFW]||cophenhafer guan savla gillen lee ghose sair||here|
|[AGHW]||abbott hamerly kuhl tower bellardo jain sinanoglu||here|
|[BCH]||fei boyer li kondo hickson gopal schneider||here|
|[BCGL]||williams ghose liu mysore wurster guan savla||here|
|[GJG]||anderson chen koletsou leong cophenhafer kuhl tower||here|
|[GKLL]||gillen gray nightingale abbott hamerly koletsou kondo||here|
|[KNSS]||hickson lee schneider fei boyer li mysore||here|
|[SS]||wurster jain sair williams chen liu leong||here|
|[TKML]||bellardo gopal sinanoglu anderson gray nightingale||here|
You should write up your reviews for Dec 2. You should also email me the entire review. The authors will get the comments. I will sum up / average the numeric scores, and provide those for the authors.
On Dec 7 and Dec 9, the four ``accepted'' papers' groups will give an oral presentation in front of the entire class. The conference will be held in AP&M 4882. Each group will have about 1/2 hour total, so you should plan on 20-25 minutes for the presentation and 5-10 minutes for questions and answers from the audience.
firstname.lastname@example.org, last updated