------------------------- METAREVIEW ------------------------ This paper focuses on an important problem, has a good/novel design in the proposed approach. The experiments are also extensive. The presentation is clear. Reviewers also concern about the motivation of the work, the lack of complexity analysis and some related work. Some presentation could be further improved to make the paper more understandable. ----------------------- REVIEW 1 --------------------- SUBMISSION: 6494 TITLE: Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders AUTHORS: Yupeng Hou, Zhankui He, Julian McAuley and Wayne Xin Zhao ----------- Summary ----------- In this paper, this paper proposes VQ-Rec, a novel approach to learn vector-quantized item representations for transferable sequential recommender systems. I consider the idea of the paper is very interesting but the description of the motivations of the paper is a disaster. ----------- Strengths ----------- S1: The authors study an interesting problem of jointly learning item text and item features (e.g., item ids). S2: The authors have given extensive experiments to verify the performance of the proposed methods on the pre-training task. ----------- Weaknesses ----------- W1: The motivations are not clear to the readers. I highly recommend the authors to re-organize or even re-write the paper. W2: There are no details for pre-training settings for text-based baseline methods. W3: There is no time complexity analysis and the evaluation regarding the complexity. ----------- Ethical issues ----------- NA ----------- Technical quality ----------- SCORE: 1 (Good - Generally solid work, any claims needing adjustment are unlikely to impact the main contributions.) ----------- Presentation quality ----------- SCORE: -2 (Very poor - Heavy rewriting is required to make the paper more readable.) ----------- Impact ----------- SCORE: 2 (Excellent - Potential to bring new ideas and/or open up new research directions.) ----------- Suggestions for improvements ----------- My concerns mainly lie in the motivation and experiments in this paper. The motivation of this work, as introduced, is on how to use the item text in the pre-training settings. However, there are no experimental results or illustrated examples to show the consequence of “binding between item text and item representations might be too tight, leading to over-emphasizing the effect of text features and exaggerating the negative impact of domain gap”. I do not think it is a good description of the motivation. After reading the paper multiple times, in my opinion, the proposed technique is a unified framework to model both item text and item features. More specifically, the authors encode the item text into the item code, to make it in a similar distribution of item id. Therefore, I highly recommend the authors re-organize and re-write the paper to make its motivation clear and friendly for the readers. Also, I want to ask the authors why not implement VQ-Rec_{ID+t}, as I thin k one of the main advantages of this work is a unified framework of jointly exploiting the item id and item text information. As the proposed method benefited from pre-training, I wonder whether the text-based models also benefited from some pre-training of NLP. As the authors mention the efficiency in Section 2.5, I highly expect the authors to provide the time complexity analysis and give some experimental results on the time complexity, as the pre-training process is usually very costly. ----------- Overall evaluation ----------- SCORE: 2 (WEAK ACCEPT - I half-champion and would accept if someone else is also at least half-championing.) ----------------------- REVIEW 2 --------------------- SUBMISSION: 6494 TITLE: Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders AUTHORS: Yupeng Hou, Zhankui He, Julian McAuley and Wayne Xin Zhao ----------- Summary ----------- This work bridges PLM and Recsys via proposing VQ-Rec, which perform PQ on pretrained embedding on item text to encode the item presentation as discrete code for downstream recommendation tasks. Lots of technical issues have been carefully addressed by OPQ, contrastive learning, PLM, negative sampling methods, etc. This is an important step towards removing the conventional one-hot ID representations (which cannot generalize), which bring several benefits like better generalization for cold-start items, better cross-domain and cross-platform recommendations. ----------- Strengths ----------- - Very well-motivated method, with lots of detailed designs to improve - Crystal-clear presentation - Extensive experiments with detailed analyses like ablation study, the effect of capacity, training data, transferability, cold-start analysis, etc. Outperforms several baselines including UniSRec ----------- Weaknesses ----------- - In ablation study, random code performs slightly better, and the authors assume it's due to the shorter text length. However, it's also the only dataset in a different platform. Would this make the pretraining less effective? - In figure 2(b), Unisec performs better when there are <60% of training data. Does this mean Unisec generalizes better for sparse datasets? - It'd be also useful to include #param for each method, and the training time analysis. ----------- Technical quality ----------- SCORE: 2 (Excellent - The approach is well-justified and all the claims are convincingly supported.) ----------- Presentation quality ----------- SCORE: 2 (Excellent - Very well written, a pleasure to read, easy to follow.) ----------- Impact ----------- SCORE: 2 (Excellent - Potential to bring new ideas and/or open up new research directions.) ----------- Suggestions for improvements ----------- see weakness. ----------- Overall evaluation ----------- SCORE: 2 (WEAK ACCEPT - I half-champion and would accept if someone else is also at least half-championing.) ----------------------- REVIEW 3 --------------------- SUBMISSION: 6494 TITLE: Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders AUTHORS: Yupeng Hou, Zhankui He, Julian McAuley and Wayne Xin Zhao ----------- Summary ----------- The paper proposes a approach to learning Vector-Quantized item representations for transferable sequential recommender by contrastive learning and aligning Code-Embedding in cross-domain via differentiable permutation matrices. Two sampling strategies are proposed to generate negative samples. Experiments are conducted on six benchmarks, including cross-domain and cross-platform scenarios. ----------- Strengths ----------- S1: The research problems are important and may have many practical applications. It is difficult to reuse an existing well-trained recommender for servicing new recommendation scenarios. Improving the transferability of learning models trained is important. Existing methods mainly leverage the generality of natural language texts for bridging the domain gap in recommender systems, but the binding between item text and item representations is "too tight" which over-emphasizing the effect of text features and exaggerating the negative impact of domain gap. S2: The proposed method is well-motivated. Relaxing the strong binding between item text and item representations is important for new scenerioes item representations generation. ----------- Weaknesses ----------- W1: Although learning transferable models by discrete code is new, there are already many Vector Quantized based methods which is the backbone of VQ-rec. The authors overclaimed the contributions of this work while missing a lot of closely related works that have already generated discrete code to form item embedding for downstream tasks. First of all, baseline only includes full-precision recommender, but lacks discrete code based method. The improvement of VQ-rec is low compared with other approaches such as UniSRec. From Table 2, the recall and NDCG of VQ-rec is even lower than UniSRec in many cases. In Table 2, dataset setting (ID, t, or ID+t) is different for baselines and the result of RecGURU(t) is missing. BTW, compared with zhan et al. [56], what's the advantage exiting in VQ-rec. A lot of the important and closely related works of discrete-code based learning item representations are missing, such as [1-4]. I would suggest the authors to check the Related Work section for more reference. [1] Reformer: The Efficient Transformer [2] Linear-Time Self Attention with Codeword Histogram for Efficient Recommendation [3] Towards Robust Blind Face Restoration with Codebook Lookup Transformer [4] Diffusion bridges vector quantized Variational AutoEncoders W2: In Table3, the case study results of "Random Code" and "w/o Code-Emb Alignment" performance better than VQ-Rec in many cases. W3: The formula for index generation (Equ 2) does not match the description (OPQ). (minor) ----------- Technical quality ----------- SCORE: 1 (Good - Generally solid work, any claims needing adjustment are unlikely to impact the main contributions.) ----------- Presentation quality ----------- SCORE: 1 (Good - Understandable to a large extent, but parts of the paper need some work.) ----------- Impact ----------- SCORE: 1 (Broad - Could help ongoing research in a broader research community.) ----------- Suggestions for improvements ----------- more demonstration about questions proposed above can make this work more complete. ----------- Overall evaluation ----------- SCORE: 2 (WEAK ACCEPT - I half-champion and would accept if someone else is also at least half-championing.)