Reviewer #1
Questions
2. Importance/Relevance
4. Of broad interest
5. Originality/Novelty
3. Moderately original; provides limited new insights or understanding
6. Justification of Originality/Novelty Score (required)
This paper presents a new large-scale dataset for MusicXML scores. The dataset is fully in the public domain, allowing practitioners in industry and academia to use it without any copyright restrictions.
7. Theoretical Development
3. Probably correct; provides limited new insights or understanding
9. Experimental Validation
3. Limited but convincing
11. Clarity of Presentation
4. Very clear
13. Reference to Prior Work
3. References adequate
15. Overall evaluation of this paper
4. Definite accept
16. Justification of Overall evaluation of this paper (required)
The dataset in this paper is a good contribution towards ensuring that large-scale copyright-free data is available to all researchers. The paper is well-written and the dataset has been properly documented and analyzed. The empirical validation is largely convincing, however I have some concerns regarding the interpretation of the results in Fig 3 (which is why I gave an empirical validation score of 3 and not 4).
First, it is not explained what the two dashed lines are (besides the mean) in the violin plot. Are they quartiles? Standard deviation?
Second, for the interpretation of Fig 3:
1. "Regarding fine-tuning, we find that this process increases richness in all models" -> this does not follow from Fig 3. For A, R, R&D the distributions before and after finetuning are fairly similar, for R there is a difference but it might not be significant.
2. "and improves quality in three (D, R∩D, Random)" -> this also does not follow. The only significant increase is in R.
The authors should rephrase some of their interpretations to accurately reflect what Fig 3 is showing. Since this change will not alter the direction of the conclusions given in the paper, it does not take away from the overall quality of the work.
Reviewer #2
Questions
2. Importance/Relevance
3. Of sufficient interest
5. Originality/Novelty
3. Moderately original; provides limited new insights or understanding
6. Justification of Originality/Novelty Score (required)
The primary contribution of this paper is the introduction of the PDMX dataset - claimed to be the largest publicly available collection of public domain MusicXML files. This is a valuable addition, as most existing symbolic music datasets have licensing issues or are significantly smaller in scale. The use of the more expressive MusicXML format, rather than the more common MIDI, is a notable aspect of the dataset.
This paper also presents MusicRender extension to MusPy, which enables parsing of the performance-related information present in MusicXML. This can enable richer symbolic music representations for downstream tasks.
While the dataset's filtering approaches based on user ratings and deduplication are relatively straightforward, the paper's observation that "we show promising results for unconditional multitrack generation that indicate improved performance when filtering for high-quality scores" is a common-sense finding that aligns with existing knowledge in the field. The paper does not present a groundbreaking methodological innovation in this regard.
7. Theoretical Development
3. Probably correct; provides limited new insights or understanding
9. Experimental Validation
3. Limited but convincing
10. Justification of Experimental Validation Score (required if score is 1 or 2).
The paper points out that the MusicXML format used in the PDMX dataset contains much more information than the MIDI format used in other datasets. The authors also introduce MusicRender for parsing MusicXML files to reflect the real perceptual rendering of the notes. However, the paper does not provide any experiments to demonstrate that parsing the MusicXML format and extracting the additional performance information is more useful for MIR tasks compared to parsing MIDI files. Without this comparative analysis, it is difficult to assess the practical benefits of MusicRender and the MusicXML format over the more commonly used MIDI format.
11. Clarity of Presentation
3. Clear enough
13. Reference to Prior Work
3. References adequate
15. Overall evaluation of this paper
3. Marginal accept
16. Justification of Overall evaluation of this paper (required)
The paper presents PDMX, which is claimed to be the largest publicly available dataset of public domain MusicXML files. While this dataset could be a valuable resource for symbolic music research, there are several aspects that raise concerns and confusion:

1. Data Quality: The paper indicates that assessing the "very limited works exist to assess the 'quality' of symbolic music, and existing high-quality datasets are much smaller." However, the analysis only compares the harmonic properties of songs with different user ratings within the PDMX dataset. They do not provide a comparative evaluation against other prominent symbolic music datasets to demonstrate that PDMX contains sufficient high-quality content.

2. Multitrack: The paper states that PDMX is a multitrack dataset and suggests that few large datasets have diverse multitrack music. Yet, the statistics provided show that over 90% of the songs have fewer than five tracks, and over half are solo works. This raises questions about the true extent of the dataset's multitrack diversity and instrument statistics.

3. Metadata: While the paper highlights the rich metadata available in PDMX, many important metadata are missing. From the description "genre tags are absent from 67% of songs", and "a large amount of unrated classical music." This raises doubts about the usefulness of the metadata in this dataset. Furthermore, the paper does not compare the metadata in PDMX to that of other symbolic music datasets, such as EMOPIA which includes emotion-related tags and accurate key signature, which are important tags but not included in PDMX. The lack of a more comprehensive description of the metadata makes it difficult to assess its usefulness for downstream MIR tasks.

Despite these concerns, the paper does present some valuable contributions. The dataset's public domain licensing and support for MusicXML format, compared to the more commonly used MIDI, could make PDMX a useful dataset. Additionally, the introduction of the MusicRender for parsing and integrating performance-related information from MusicXML files is a technical contribution that may enable richer symbolic music representations.

Overall, while PDMX has the potential to be a valuable dataset, the paper does not provide a sufficiently robust evaluation of its quality, and metadata to fully establish its significance and utility for the music research community. A more comprehensive assessment, particularly in comparison to existing datasets, would be necessary to support the claims made in the paper.
Reviewer #3
Questions
2. Importance/Relevance
4. Of broad interest
5. Originality/Novelty
3. Moderately original; provides limited new insights or understanding
6. Justification of Originality/Novelty Score (required)
From the aspect of collecting music from MuseScore, it is a regular idea; the data processing pipeline and analysis itself makes the paper valuable.
7. Theoretical Development
3. Probably correct; provides limited new insights or understanding
9. Experimental Validation
4. Theoretical paper: sufficient validation; Empirical paper: rigorous validation
11. Clarity of Presentation
4. Very clear
13. Reference to Prior Work
4. Excellent references
15. Overall evaluation of this paper
4. Definite accept
16. Justification of Overall evaluation of this paper (required)
This paper contributes a novel symbolic music dataset, contributing a lot to MIR community; besides, the paper analyses the music quality and limitations, making this paper insightful.

There is a concern that the instrument distribution imbalance in is dataset is not-ignorable. Most music data only contains popyphonic piano music data, and this imbalancement affects the dataset application to multi-instrument model training; MuseScore comes from sheet music, which requires a quantization in music data, which limits its application to performance modelling. It would be better if paper can have a discussion on relevant topics.