Reviews For Paper
Paper ID 853
Title Dance Dance Convolution
Masked Reviewer ID: Assigned_Reviewer_1
Review:
Question 
Summary of the paper (Summarize the main claims/contributions of the paper.) DDR is popular rhythm based video game, where players step on a platform in sync with music as directed via on-screen charts. The paper proposes a task called "learning to choreograph", where given a raw audio track, the goal is producing a new step chart. The authors propose a framework for this based on a generative model, showing that it outperform existing literature.
Clarity (Assess the clarity of the presentation and reproducibility of the results.) Below Average
Clarity - Justification The paper is extremely unclear regarding the motivating data sets. Much more detail needs to be in the paper regarding the data in section 3.
Significance (Does the paper contribute a major breakthrough or an incremental advance?) Below Average
Significance - Justification The motivation of the paper is clear, however the experiments are not convincing that this methodology can readily be applied to the aforementioned application.

Correctness (Is the paper technically correct?) Paper is technically correct
Correctness - Justification We checked all details of the paper.
Overall Rating Weak accept
Detailed comments. (Explain the basis for your ratings while providing constructive feedback.) 1. How were the 90 data sets collected? How might they be biased? How are we to know that they are representative of songs of users for DDR?
2. Why was not more data collected and analyzed regarding more users to understanding how the model works/fails and compares to other methods? Such comparisons are critical for understanding the weaknesses and successes for any model.
3. What happens when the training/test is changed. Sensitivity needs to be addressed here.
4. Please address the computational complexity/run time of all methods (and scalability).
5. What is the robustness of the method to any fixed parameters. Please describe this in detail. For example, when one parameter changes in the model, how does this then change the generated choreography in Figure 8?
Reviewer confidence Reviewer is knowledgeable
Masked Reviewer ID: Assigned_Reviewer_2
Review:
Question 
Summary of the paper (Summarize the main claims/contributions of the paper.) This paper presents (1) a new dataset of musical audio paired with simple choreographic annotation from the game of Dance Dance Revolution, and (2) a recurrent neural model for predicting choreographic moves directly from audio. The presented method compares favorable against several strong baselines evaluated using both perplexity and event accuracy.
Clarity (Assess the clarity of the presentation and reproducibility of the results.) Excellent (Easy to follow)
Clarity - Justification Very clearly and carefully written -- an enjoyable read.
Significance (Does the paper contribute a major breakthrough or an incremental advance?) Above Average
Significance - Justification The models presented here are relatively straightforward, but they well-reasoned and positioned nicely in the context of prior work on musical audio analysis. The dataset and specific task are new and may -- as the authors point out -- open up interesting future work on music analysis.
Correctness (Is the paper technically correct?) Paper is technically correct
Correctness - Justification Modeling decisions are reasonable and clearly described, the experiments are thorough, the baselines are strong.
Overall Rating Strong accept
Detailed comments. (Explain the basis for your ratings while providing constructive feedback.) I like this paper and think it should be accepted. The models presented here aren't exactly ground-breaking, but they are carefully reasoned and work well -- hard to fault. The identification of the new task and dataset represents a potentially further-reaching contribution: leveraging naturally annotated musical data from online gaming communities opens up news possibility for musical analysis research, a domain where datasets are traditionally quite small. Finally, this paper is very well written and provides a thorough survey of recent neural architectures for music recognition and beat detection.
Reviewer confidence Reviewer is knowledgeable
Masked Reviewer ID: Assigned_Reviewer_3
Review:
Question 
Summary of the paper (Summarize the main claims/contributions of the paper.) The paper proposes an algorithm that generates step charts for DDR, a rhythm-based video game. The algorithm tries to learn from a set of existing step charts produced by human experts and generate charts for arbitrary songs. The problem is divided into two independent task: step placement (determining timing of steps) and step selection (determining the action expected by the players). For both tasks the authors propose to use LSTM algorithms. The experiments were performed to evaluate the performance of the proposed algorithm.
Clarity (Assess the clarity of the presentation and reproducibility of the results.) Above Average
Clarity - Justification The paper is well-written and is easy to follow. The experimental design could have been described in more detail.
Significance (Does the paper contribute a major breakthrough or an incremental advance?) Above Average
Significance - Justification While the paper does not propose any novel machine learning method, the novelty of this paper is in its application to a new domain and demonstration that state of the art neural networks could result in good quality step charts.
Correctness (Is the paper technically correct?) Paper is technically correct
Correctness - Justification The paper appears to be technically correct.
Overall Rating Weak accept
Detailed comments. (Explain the basis for your ratings while providing constructive feedback.) This is an application paper that focuses on generation of step charts for rhythm-based video games. This is a challenging task that typically requires human experts and some level of creativity. It was really interesting to see that neural networks could be trained to perform this task. The paper is mostly well written and enjoyable to read. The work done is impressive. A slight downside to the paper is that the experiments were not described in sufficient detail. So, this reviewer was not able to understand exactly how were the resulting step charts evaluated. The attempts to evaluate the performance both quantitatively and visually are admirable. However, the evaluation is still not going far enough -- it would be great to see how the game players perceive the generated step charts (do they find them exciting and engaging) and how well can they complete them. An additional philosophical question is are there any indications that the produced step charts have creative elements or are they purely rehashing the patterns observed in the training set.
Reviewer confidence Reviewer is knowledgeable