Reviewer #1
Questions
2. I am an expert on the topic of the paper.
Agree
3. The title and abstract reflect the content of the paper.
Agree
4. The paper discusses, cites and compares with all relevant related work
Agree
5. Please justify the previous choice (Required if “Strongly Disagree” or “Disagree” is chosen, otherwise write "n/a")
n/a
6. Readability and paper organization: The writing and language are clear and structured in a logical manner.
Agree
7. The paper adheres to ISMIR 2026 submission guidelines (uses the ISMIR 2026 template, has at most 6 pages of technical content followed by “n” pages of references, AI usage declaration or ethical considerations, references are well formatted). If you selected “No”, please explain the issue in your comments.
Yes
8. Relevance of the topic to ISMIR: The topic of the paper is relevant to the ISMIR community. Note that submissions of novel music-related topics, tasks, and applications are highly encouraged. If you think that the paper has merit but does not exactly match the topics of ISMIR, please do not simply reject the paper but instead communicate this to the Program Committee Chairs. Please do not penalize the paper when the proposed method can also be applied to non-music domains if it is shown to be useful in music domains.
Strongly agree
9. Scholarly/scientific quality: The content is scientifically correct.
Agree
10. Please justify the previous choice (Required if "Strongly Disagree" or "Disagree" is chosen, otherwise write "n/a")
n/a
11. Novelty of the paper: The paper provides novel methods, applications, findings or results. Please do not narrowly view "novelty" as only new methods or theories. Papers proposing novel musical applications of existing methods from other research fields are considered novel at ISMIR conferences.
Agree
12. The paper provides all the necessary details or material to reproduce the results described in the paper. Keep in mind that ISMIR respects the diversity of academic disciplines, backgrounds, and approaches. Although ISMIR has a tradition of publishing open datasets and open-source projects to enhance the scientific reproducibility, ISMIR accepts submissions using proprietary datasets and implementations that are not sharable. Please do not simply reject the paper when proprietary datasets or implementations are used.
Agree
13. Pioneering proposals: This paper proposes a novel topic, task or application. Since this is intended to encourage brave new ideas and challenges, papers rated "Strongly Agree" and "Agree" can be highlighted, but please do not penalize papers rated "Disagree" or "Strongly Disagree". Keep in mind that it is often difficult to provide baseline comparisons for novel topics, tasks, or applications. If you think that the novelty is high but the evaluation is weak, please do not simply reject the paper but carefully assess the value of the paper for the community.
Disagree (Standard topic, task, or application)
14. Reusable insights: The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
Agree
15. Please explain your assessment of reusable insights in the paper.
The following ideas may generalize to other generative recommendation and tokenization settings:

I1 - The number of tokens per item is a key factor driving both sequence length and effective vocabulary size. Independent tokenization allocates one token to each modality, thereby tying the token budget to the number of modalities; fusing modalities prior to tokenization breaks this dependency, enabling flexible selection of n.

I2 - Variance-covariance regularization loss (adapted from VICReg) is useful to encourage balanced and decorrelated quantization subspaces. Product quantization is most effective when each subspace is equally informative and captures complementary information.

I3 - If two items collapse to the same ID, no downstream decoder can distinguish between them. Consequently, evaluating ID collisions and codebook utilization through the conflict rate before training the recommender provides a low-cost indicator of downstream performance.
16. AI Usage Policy: The paper complies with the ISMIR 2026 AI Usage Policy for Authors. (Authors are welcome to use any tool they wish for preparing and writing the paper but they must ensure that all content is correct and original. Authors must declare any use of LLMs and AI tools if it's a part of core methodology (including literature review, related work, generated figures, tables or other illustrations). Using AI tools for editing purposes (e.g,. checking grammar and fixing typos) need not be declared. Please pay attention to any hallucinated text, figures, or references and flag them in the comments below)
Agree
17. Please justify the previous choice (Required if "Disagree" is chosen, otherwise write "n/a")
n/a
18. Write ONE line (in your own words) with the main take-home message from the paper.
By fusing multiple modalities into a joint embedding prior to product quantization, FusID obtains compact, conflict-free semantic IDs that achieve performance comparable to or better than per-modality baselines on next-song recommendation at substantially lower generative cost.
21. Potential to generate discourse: The paper will generate discourse at the ISMIR conference or have a large influence/impact on the future of the ISMIR community.
Agree
22. Overall evaluation: Keep in mind that minor flaws can be corrected, and should not be a reason to reject a paper. Please familiarize yourself with the reviewer guidelines at https://ismir.net/reviewer-guidelines
Weak accept
23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.
# Summary

In this paper, the authors propose FusID, a pipeline for generating multimodal semantic IDs for music recommendation. Rather than tokenizing each modality independently (as in TalkPlay and related work), FusID concatenates five pre-trained modality features (tags, metadata, lyrics, audio, collaborative signal), passes them through a single small projection network to produce sub-embeddings. The network is trained with a contrastive loss (MSE or InfoNCE over playlist co-occurrence) plus a VICReg-style variance/covariance regularizer. FuseID then applies product quantization (independent k-means, K=1024 per subspace) to yield an n-token ID.

# Strength

S1 - The research is grounded in a compelling motivation. The observation that independently tokenizing each modality, as done in existing approaches, can limit both efficiency and the ability to capture inter-modal interactions is well founded. Learning a unified representation through joint multimodal fusion is therefore a sensible and well-motivated direction.

S2 - The scope of evaluation.

The authors evaluate their approach from multiple perspectives, including semantic ID quality, recommendation performance, and generative recommendation efficiency.
- Semantic ID quality is evaluated directly rather than solely through downstream recommendation metrics. While many tokenization studies report only end-task performance, the authors assess token quality using CUR, Conflict Rate, and Cardinality (Table 1). This analysis reveals important intermediate properties, most notably the zero-conflict behavior of MSE-5, that help explain the observed downstream improvements. Furthermore, the inverse relationship between conflict rate and recommendation performance within the MSE family is clear and highly interpretable.
- The efficiency results are well motivated and practically relevant. The relative cost analysis presented in Table 3 is particularly compelling, demonstrating that MSE-3 outperforms TalkPlay while requiring only one-third of the generative cost. This represents a meaningful improvement in the performance-efficiency trade-off and is a genuinely useful finding.

# Weakness

W1 - The claims of the abstract are not realized by explicit mechanisms

The paper claims to address three limitations of the existing methods: cross-modal redundancy, missing inter-modal interaction, and disregard for varying modality importance. But:
- "Varying modality importance" is never explicitly modeled. The fusion network is a single deep neural network over a concatenation of all modalities. There is no gating, attention, or learned per-modality weighting. Whatever importance weighting occurs is implicit in the first linear layer's weights and is never measured. The claim that "prioritizing salient features while ensuring balanced, disentangled information load can be effectively addressed through the objective function alone" (Section 7) is asserted, not demonstrated. There is no experiment showing that less important modalities are down-weighted.
- "Inter-modal interaction": a single projection over concatenated features captures only cross-modal combinations. The paper offers no evidence that anything beyond what independent tokenization captures is being learned in the fusion variant.
- "Redundancy": the covariance loss reduces redundancy across the n sub-embeddings, but the abstract frames redundancy as a cross-modality problem. The mechanism reduces redundancy among output tokens, not demonstrably among modalities.

W2 - The breadth of baselines is limitted

The paper situates itself in the semantic-ID tokenization literature, yet the only generative baselines are TalkPlay and TalkPlay+TT. The most natural and directly comparable methods are absent:
- TIGER [1] appears only inside the ablation (Table 4), built on the authors' own fused embeddings. It's not as a standalone, independently-tuned baseline. That is not the same as comparing to TIGER as a competing tokenizer.
- LETTER (Wang et al., CIKM'24) is highly relevant: it is a learnable tokenizer that also injects collaborative signals via contrastive alignment and adds a diversity loss to mitigate code-assignment bias. It is conceptually the closest prior work to FusID's contrastive + regularization recipe.
- PSRQ+MCCA (Wang et al., CIKM'25) is a multimodal music semantic-ID method that explicitly targets inter-modal modeling and modal-joint IDs.

Wang, W., Bao, H., Lin, X., Zhang, J., Li, Y., Feng, F., ... & Chua, T. S. (2024). Learnable item tokenization for generative recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (pp. 2400-2409).

Wang, S., Ouyang, T., Xiao, Q., Wang, D., Ren, Y., Xu, S., ... & Luo, C. (2025). Progressive Semantic Residual Quantization for Multimodal-Joint Interest Modeling in Music Recommendation. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management (pp. 6119-6127).

W3 - No long-tail / cold-start breakdown, and no non-accuracy metrics, despite this being the natural payoff of multimodality

The entire motivation for multimodal content features is to help items that collaborative signals cannot, i.e., long-tail and cold-start songs. Yet:
- All results are aggregate MRR/Hit@k. There is no head/tail split (e.g., top-20% popular vs. rest). Without it, we cannot tell whether FusID's gains come from better tail modeling (the interesting case) or simply from re-ranking popular items.
- No beyond-accuracy metrics (coverage, diversity, Average Recommendation Popularity).

This connects to a methodological bias risk in dataset construction (next point), which makes the missing popularity-bias metric especially important.

W4 - Positive-pair construction likely amplifies popularity bias, and this is never measured

Labeling the top-5 co-occurring songs per anchor as positives, will systematically over-select popular co-occurrences: popular songs co-occur with almost everything, so they will appear in many anchors' top-5 positive sets. The contrastive objective then pulls popular items toward many anchors, plausibly inflating their centrality in the embedding space. In the inference time, the rule "when a semantic ID was associated with multiple distinct songs, we ranked the recommendations by popularity" compounds this directly.

W5 - No dataset statistics

Genre/decade/language distribution, popularity, and sparsity information are absent. Especially, for a paper whose thesis is about modality importance and redundancy, the modality-level statistics of the corpus are exactly what's needed to interpret results.

W6 - No handling of missing modalities

Concatenation-based fusion implicitly assumes all five modalities are present. In real catalogues, lyrics (instrumentals), audio, or metadata are routinely missing. The paper does not describe imputation, masking, or a missing-modality ablation. Given that the filtering step already discarded ~76% of MPD songs for incomplete features, missing-modality robustness is not an exception here. This is a real-world deployment concern.

W7 - No analysis of what each sub-embedding/token encodes, or of per-modality contribution. Measuring mutual information between each sub-embedding and each modality would directly support the disentanglement and "balanced load" claims. Per-modality ablation (leave-one-modality-out) could be helpful to show modality importance.

W8 - The off-diagonal block-wise vs. full-covariance choice is unjustified empirically

The paper states it uses only off-diagonal block-wise cross-covariance between sub-embeddings (rather than VICReg's full off-diagonal penalty) but gives only a one-line rationale ("our goal is to avoid feature overlap within each embedding"). The logic is also slightly confusingly stated: the prose says "avoid feature overlap within each embedding E_i = {e_1,...,e_n}", but the formula penalizes covariance "between" e_i and e_j.
Reviewer #2
Questions
2. I am an expert on the topic of the paper.
Strongly agree
3. The title and abstract reflect the content of the paper.
Strongly agree
4. The paper discusses, cites and compares with all relevant related work
Strongly agree
5. Please justify the previous choice (Required if “Strongly Disagree” or “Disagree” is chosen, otherwise write "n/a")
n/a
6. Readability and paper organization: The writing and language are clear and structured in a logical manner.
Strongly agree
7. The paper adheres to ISMIR 2026 submission guidelines (uses the ISMIR 2026 template, has at most 6 pages of technical content followed by “n” pages of references, AI usage declaration or ethical considerations, references are well formatted). If you selected “No”, please explain the issue in your comments.
Yes
8. Relevance of the topic to ISMIR: The topic of the paper is relevant to the ISMIR community. Note that submissions of novel music-related topics, tasks, and applications are highly encouraged. If you think that the paper has merit but does not exactly match the topics of ISMIR, please do not simply reject the paper but instead communicate this to the Program Committee Chairs. Please do not penalize the paper when the proposed method can also be applied to non-music domains if it is shown to be useful in music domains.
Agree
9. Scholarly/scientific quality: The content is scientifically correct.
Agree
10. Please justify the previous choice (Required if "Strongly Disagree" or "Disagree" is chosen, otherwise write "n/a")
n/a
11. Novelty of the paper: The paper provides novel methods, applications, findings or results. Please do not narrowly view "novelty" as only new methods or theories. Papers proposing novel musical applications of existing methods from other research fields are considered novel at ISMIR conferences.
Agree
12. The paper provides all the necessary details or material to reproduce the results described in the paper. Keep in mind that ISMIR respects the diversity of academic disciplines, backgrounds, and approaches. Although ISMIR has a tradition of publishing open datasets and open-source projects to enhance the scientific reproducibility, ISMIR accepts submissions using proprietary datasets and implementations that are not sharable. Please do not simply reject the paper when proprietary datasets or implementations are used.
Strongly disagree
13. Pioneering proposals: This paper proposes a novel topic, task or application. Since this is intended to encourage brave new ideas and challenges, papers rated "Strongly Agree" and "Agree" can be highlighted, but please do not penalize papers rated "Disagree" or "Strongly Disagree". Keep in mind that it is often difficult to provide baseline comparisons for novel topics, tasks, or applications. If you think that the novelty is high but the evaluation is weak, please do not simply reject the paper but carefully assess the value of the paper for the community.
Agree (Novel topic, task, or application)
14. Reusable insights: The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
Agree
15. Please explain your assessment of reusable insights in the paper.
The approach proposed could be generalized to other recommendation domains, the first stage (semantic ID generation from multimodal data) could be generalized to other MIR tasks.
16. AI Usage Policy: The paper complies with the ISMIR 2026 AI Usage Policy for Authors. (Authors are welcome to use any tool they wish for preparing and writing the paper but they must ensure that all content is correct and original. Authors must declare any use of LLMs and AI tools if it's a part of core methodology (including literature review, related work, generated figures, tables or other illustrations). Using AI tools for editing purposes (e.g,. checking grammar and fixing typos) need not be declared. Please pay attention to any hallucinated text, figures, or references and flag them in the comments below)
Agree
17. Please justify the previous choice (Required if "Disagree" is chosen, otherwise write "n/a")
n/a
18. Write ONE line (in your own words) with the main take-home message from the paper.
The paper proposes a novel two-stage architecture for multimodal generative recommendation, based on multimodal semantic ID generation.
21. Potential to generate discourse: The paper will generate discourse at the ISMIR conference or have a large influence/impact on the future of the ISMIR community.
Agree
22. Overall evaluation: Keep in mind that minor flaws can be corrected, and should not be a reason to reject a paper. Please familiarize yourself with the reviewer guidelines at https://ismir.net/reviewer-guidelines
Weak accept
23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.
The paper proposes a novel two-stage architecture for multimodal generative recommendation, based on multimodal semantic ID generation. The approach is novel, likely to generalize to other domains or tasks. However there are some aspects to be clarified to ensure that the experimental methodology is solid and that results are reliable.

The hyperparameter selection, e.g., tau in the contrastive loss alpha, and gamma in the regularization loss, is not motivated. Furthermore, comparison baselines (for both semantic ID generation and recommendation) should be optimized with a separate HPO, for fairness of comparison; the fact that the HP are shared with the other architectures does not guarantee that the comparison is fair. Furthermore for the comparison with TIGER, the analysis should not use the HP of the original TIGER paper, since that paper bases experiments on another dataset.

In addition, for the proposed approach results are reported for all n's , whereas only for one n for the comparison baseline, which renders the comparison unfair

The comparison with generative models (section 5.1) Should also be carried out with respect to TalkPlay TT.
Reproducibility is limited: In addition to the lack of code, several aspects are not described in enough detail for the reader to reproduce the setup. In particular regarding the embeddings of tags, metadata, and lyrics.

Finally, the fact that the proposed model outperforms baselines somewhat comes at no surprise, since the pre-training phase already gave access to the interaction data (user--item interactions are used as one of the modalities).
Reviewer #3
Questions
2. I am an expert on the topic of the paper.
Strongly agree
3. The title and abstract reflect the content of the paper.
Agree
4. The paper discusses, cites and compares with all relevant related work
Agree
5. Please justify the previous choice (Required if “Strongly Disagree” or “Disagree” is chosen, otherwise write "n/a")
n/a
6. Readability and paper organization: The writing and language are clear and structured in a logical manner.
Strongly agree
7. The paper adheres to ISMIR 2026 submission guidelines (uses the ISMIR 2026 template, has at most 6 pages of technical content followed by “n” pages of references, AI usage declaration or ethical considerations, references are well formatted). If you selected “No”, please explain the issue in your comments.
Yes
8. Relevance of the topic to ISMIR: The topic of the paper is relevant to the ISMIR community. Note that submissions of novel music-related topics, tasks, and applications are highly encouraged. If you think that the paper has merit but does not exactly match the topics of ISMIR, please do not simply reject the paper but instead communicate this to the Program Committee Chairs. Please do not penalize the paper when the proposed method can also be applied to non-music domains if it is shown to be useful in music domains.
Strongly agree
9. Scholarly/scientific quality: The content is scientifically correct.
Agree
10. Please justify the previous choice (Required if "Strongly Disagree" or "Disagree" is chosen, otherwise write "n/a")
n/a
11. Novelty of the paper: The paper provides novel methods, applications, findings or results. Please do not narrowly view "novelty" as only new methods or theories. Papers proposing novel musical applications of existing methods from other research fields are considered novel at ISMIR conferences.
Agree
12. The paper provides all the necessary details or material to reproduce the results described in the paper. Keep in mind that ISMIR respects the diversity of academic disciplines, backgrounds, and approaches. Although ISMIR has a tradition of publishing open datasets and open-source projects to enhance the scientific reproducibility, ISMIR accepts submissions using proprietary datasets and implementations that are not sharable. Please do not simply reject the paper when proprietary datasets or implementations are used.
Agree
13. Pioneering proposals: This paper proposes a novel topic, task or application. Since this is intended to encourage brave new ideas and challenges, papers rated "Strongly Agree" and "Agree" can be highlighted, but please do not penalize papers rated "Disagree" or "Strongly Disagree". Keep in mind that it is often difficult to provide baseline comparisons for novel topics, tasks, or applications. If you think that the novelty is high but the evaluation is weak, please do not simply reject the paper but carefully assess the value of the paper for the community.
Agree (Novel topic, task, or application)
14. Reusable insights: The paper provides reusable insights (i.e. the capacity to gain an accurate and deep understanding). Such insights may go beyond the scope of the paper, domain or application, in order to build up consistent knowledge across the MIR community.
Agree
15. Please explain your assessment of reusable insights in the paper.
Three findings are likely to be reused beyond this submission.
1. The empirical decoupling between fused-embedding quality and quantization strategy via the TIGER (legacy/matched) ablation gives the community a reusable template for separating "what does the encoder learn" from "what does the tokenizer preserve" in any semantic-ID pipeline.
2. The conflict-rate / cardinality / CUR triplet, computed both at the dataset and test-set level, is a clean diagnostic battery that I would expect other generative-recommendation papers to adopt.
16. AI Usage Policy: The paper complies with the ISMIR 2026 AI Usage Policy for Authors. (Authors are welcome to use any tool they wish for preparing and writing the paper but they must ensure that all content is correct and original. Authors must declare any use of LLMs and AI tools if it's a part of core methodology (including literature review, related work, generated figures, tables or other illustrations). Using AI tools for editing purposes (e.g,. checking grammar and fixing typos) need not be declared. Please pay attention to any hallucinated text, figures, or references and flag them in the comments below)
Agree
17. Please justify the previous choice (Required if "Disagree" is chosen, otherwise write "n/a")
n/a
18. Write ONE line (in your own words) with the main take-home message from the paper.
Replacing per-modality tokenization with a jointly-fused, contrastively trained, regularized embedding before product quantization yields better-utilized codebooks and stronger next-song recommendation while remaining computationally cheaper than coarse-to-fine multimodal baselines.
21. Potential to generate discourse: The paper will generate discourse at the ISMIR conference or have a large influence/impact on the future of the ISMIR community.
Agree
22. Overall evaluation: Keep in mind that minor flaws can be corrected, and should not be a reason to reject a paper. Please familiarize yourself with the reviewer guidelines at https://ismir.net/reviewer-guidelines
Weak accept
23. Main review and comments for the authors. Please summarize strengths and weaknesses of the paper. It is essential that you justify the reason for the overall evaluation score in detail. Keep in mind that belittling or sarcastic comments are not appropriate.
STRENGTHS

The motivation is clear. The paper identifies real limitations of per-modality tokenization: redundant codes across overlapping modalities, limited interaction between modalities, and fixed capacity allocation across modalities.

The evaluation looks at more than recommendation accuracy. The semantic-ID diagnostics, especially conflict rate, cardinality, and codebook underutilization, are useful and make the empirical story easier to interpret.

The ablation with TIGER variants is helpful. Since these variants operate on the learned FusID embedding, they give some evidence about what comes from the representation and what comes from the quantization scheme. The comparison is not perfect, but it is still informative.

The efficiency result is a strong practical point. MSE-3 doing better than TalkPlay is valuable for settings where longer semantic-ID sequences are expensive.

The paper is easy to read, and most of the relevant implementation details are included. The promised code release is also important for reproducibility.

WEAKNESSES

I'm not sure that the paper's claim about handling varying modality importance is totally grounded. FusID avoids assigning one codebook to each original modality, but it still gives every learned sub-embedding the same 1024-cluster capacity. The variance loss encourages balanced use of the sub-embeddings, but this is not the same as learning that some modalities or subspaces deserve more capacity than others.

The fusion module is quite simple. It is a Linear -> BatchNorm -> ReLU -> Linear -> LayerNorm projection over concatenated features. This may be enough empirically, but I feel it makes the claim about capturing inter-modal interactions weaker than the paper suggests. An ablation against a simpler concatenation/projection baseline, or against a stronger multimodal fusion module, would help.

The experiments are sound but limited to MPD. Some other multimodal item domain would make the generalization claims much stronger.

Some choices are not completely justified. e.g. tau=0.7 is used but we don't see at least some mention to whether a sweep was used to establish this as the value.

There may be a confound in the way ID conflicts are handled. When several songs share a semantic ID, recommendations are ranked by popularity. This can favor models whose conflicts occur among popular songs.

The paper describes FusID tokens as essentially orthogonal, but this is not directly tested. If order should not matter in the same way it matters for coarse-to-fine RQ tokens, then a token-order permutation experiment would be useful.

There are some typos: "multiodal", "feaures", "contatenation"...

DETAILED COMMENTS AND QUESTIONS

1. How sensitive is FusID to the contrastive-pair construction choices, especially top-5 positives, bottom-90% negatives, and the C(s)>=3 filtering rule?

2. Was tau=0.7 selected on validation data for InfoNCE? It would be useful to know whether lower temperatures were tried.

3. For TIGER (matched), were the RQ-VAE hyperparameters tuned on the same validation set as FusID, or were the original TIGER defaults used? If only defaults were used, FusID may have received more tuning attention.

4. Please report the regularization weight alpha in L_total and clarify how it was selected. The training and model-selection procedure for the representation model could use a few more details.

CONCLUSION

This is a focused and useful contribution to multimodal semantic-ID design for music recommendation. The evaluation is sound and the empirical results are good. The main limitations are that the novelty is moderate and the fusion architecture is minimal.