Conference Management Toolkit

View Reviews

Paper ID

187

Paper Title

MusPy: A Toolkit for Symbolic Music Generation

Track Name

ISMIR2020

Reviewer #1

Questions

2. The title and abstract reflect the content of the paper.
- Strongly Agree
3. The paper discusses, cites and compares with all relevant related work.
- Strongly Agree
4. The writing and language are clear and structured in a logical manner.
- Strongly Agree
5. The references are well formatted.
- Strongly Agree
6. The topic of the paper is relevant to the ISMIR community.
- Strongly Agree
7. The content is scientifically correct.
- Strongly Agree
8. The paper provides novel methods, findings or results.
- Agree
9. The paper provides all the necessary details or material to reproduce the results described in the paper.
- Strongly Agree
10. The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
- Agree
11. Please explain your assessment of reusable insights in the paper.
- This paper present a tool for symbolic music generation (MusPy). Moreover, the dataset analysis capability of MusPy can also be exploited for musicology studies. Figures 5 to 10 may illustrate what I mean. Distribution of musical keys for different dataset, or cross-dataset generalizability results can give a precious insight also for musicology studies.
15. The paper will have a large influence/impact on the future of the ISMIR community.
- Agree
16. Overall evaluation
- Strong Accept
17. Main review and comments for the authors
- In this paper, a tool for symbolic music generation is presented.
  This tool permits the easy integration of ML tools for dataset analysis like PyTorch or TensorFlow. In the same way it integrates tools for audio rendering, graphical representation, format conversion that are made possible by third party software like music21, pretty_midi and Pypianoroll.
  The authors demonstrate that MusPy is 1) a tool for developing music generation systems and 2) statistical analysis of symbolic music datasets.

Reviewer #2

Questions

2. The title and abstract reflect the content of the paper.
- Strongly Agree
3. The paper discusses, cites and compares with all relevant related work.
- Disagree
4. The writing and language are clear and structured in a logical manner.
- Strongly Agree
5. The references are well formatted.
- Agree
6. The topic of the paper is relevant to the ISMIR community.
- Agree
7. The content is scientifically correct.
- Disagree
8. The paper provides novel methods, findings or results.
- Disagree
9. The paper provides all the necessary details or material to reproduce the results described in the paper.
- Disagree
10. The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
- Strongly Disagree
11. Please explain your assessment of reusable insights in the paper.
- The author does not present any idea about the way someone can reuse the library, even though it is open source, as far as I can see, so we cannot go further into the experience with the resources provided.
15. The paper will have a large influence/impact on the future of the ISMIR community.
- Strongly Agree
16. Overall evaluation
- Weak Accept
17. Main review and comments for the authors
- The paper presents a great library and I wish you can continue working on this for a long time in order to help the community. The related works should be better discussed, as we can find many libraries in Python for dealing with music and they are not presented or even compared with yours. The paper has a great focus on the datasets and I am pretty sure that this could be the main topic in the title and abstract, but the authors didn't notice it, maybe. The MusPy solution seems to be a great tool, but requires more comparison with other solutions in order to get more attention.

Reviewer #3

Questions

2. The title and abstract reflect the content of the paper.
- Strongly Agree
3. The paper discusses, cites and compares with all relevant related work.
- Strongly Agree
4. The writing and language are clear and structured in a logical manner.
- Agree
5. The references are well formatted.
- Strongly Agree
6. The topic of the paper is relevant to the ISMIR community.
- Agree
7. The content is scientifically correct.
- Agree
8. The paper provides novel methods, findings or results.
- Agree
9. The paper provides all the necessary details or material to reproduce the results described in the paper.
- Strongly Agree
10. The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
- Disagree
11. Please explain your assessment of reusable insights in the paper.
- This paper proposes a library for symbolic music data management, but nothing about their system design seems particularly novel or interesting beyond the particular problem it solves.
15. The paper will have a large influence/impact on the future of the ISMIR community.
- Disagree
16. Overall evaluation
- Weak Accept
17. Main review and comments for the authors
- This paper describes the development of a data management library for symbolic music data, then explores some potential applications of the tool to compare symbolic music datasets. While I don't see much value in this paper from a theoretical perspective, the tool they developed addresses an important development bottleneck in deep learning-based symbolic music generation and would be of great value to the researchers working on this problem in the ISMIR community.
  
  The comparative analysis of different common symbolic music datasets is also interesting, and I wish the authors had spent more time analyzing the results there. For instance, why does the hymnal dataset generalize worse than the other multipitch datasets? Would a different preprocessing scheme cause the Bach dataset to generalize better? It seems like there are several potential insights here which may inform the development of future symbolic music datasets.
  
  It would also be interesting to study the design of such a library from a UX perspective. A user study with a few students interested in symbolic music generation could shed light on what design features are important, or what additional uses (besides training a neural network) potential users might try to do with your library. While deep learning approaches are popular now, a successful library design should anticipate potential alternative use cases which may have different needs.

Reviewer #4

Questions

2. The title and abstract reflect the content of the paper.
- Strongly Agree
3. The paper discusses, cites and compares with all relevant related work.
- Strongly Agree
4. The writing and language are clear and structured in a logical manner.
- Strongly Agree
5. The references are well formatted.
- Strongly Agree
6. The topic of the paper is relevant to the ISMIR community.
- Strongly Agree
7. The content is scientifically correct.
- Strongly Agree
8. The paper provides novel methods, findings or results.
- Strongly Agree
9. The paper provides all the necessary details or material to reproduce the results described in the paper.
- Strongly Agree
10. The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
- Strongly Agree
11. Please explain your assessment of reusable insights in the paper.
- The manner the software is exposed and presend can be understood as an reusable insight, in my opinion.
15. The paper will have a large influence/impact on the future of the ISMIR community.
- Agree
16. Overall evaluation
- Strong Accept
17. Main review and comments for the authors
- Overall:
  * The paper is very well-writen, the ideas are well developed, and the figures are beautiful and easily readable. For me it was a pure pleasure to read this paper!
  * The subject is VERY important within the MIR community, and the contribution of the software is amazing!
  * The documentation and codes available are very welcome and wasy to follow. However I was not able to run and test it by myself, because I'm having some problems with Python on my new machine, unfortunately...
  * I think that a more detailed explanation of the experiment reported in the paper with the neural nets would be good. I am aware of the limited space, but the documentation of the software could contain a better description of it, instead of needing to read the associated .py file. Another examples of usage of the package would be good also, and I hope they will be availabel when the software is officialy published.
  
  It was quite hard to find points to improve the paper, but here I comment about some stuff:
  
  Introduction:
  * Suppor the claim in lines 37-40 about datasets, representations and metrics using a reference;
  * Include a summary of the paper at the end of the section.
  
  Related work:
  * At the end of the first paragraph (lines 102~105), it is mentioned that MusPy deals with more than one Machine Learning framework, but additionaly to TensorFlow, only PyTorch is mentioned. However, the text, up to this point, indicates that more frameworks are also compatible. It would be interesting to clarify this point here.
  
  Dataset Analysis:
  * At line 244 you want to mention "Figure 7".
  
  Experiments and results:
  * The explanation of the large log-perplexity for JSBach Chorale Dataset in lines 287~293 is not clear.
  * In Figure 10, both plots could follow the same color scheme: note that the top plot associates white with 160 and the bottom associates white with 40. It is very quick to note this and undestand both plots, but for comparison reasons between both, sharing the same scale could be better.
  * It is mentioned in line 340 that the information about the stratified and unified datasets are present in Figures 8 and 10, but it is only in Figure 10!

Change Password

User Profile

Change Email

Reviewer #1

Questions

Reviewer #2

Questions

Reviewer #3

Questions

Reviewer #4

Questions

Old Review

Reviewer # () | Older | Newer

.