------------------------- METAREVIEW ------------------------ In this paper, authors present a technique for using 2D CNNs for sequential recommendations. While the reviewers highlighted concerns about the depth of the novelty of the proposed approach and the lack of hyperparameter studies and sensitivity tests in the evaluation, they also agreed that the method appears to achieve better performance than the considered baselines on two public datasets. Therefore, the paper is suitable for publication as a short paper. ----------------------- REVIEW 1 --------------------- SUBMISSION: 1055 TITLE: CosRec: 2D Convolutional Neural Networks for Sequential Recommendation AUTHORS: An Yan, Shuo Cheng, Wang-Cheng Kang, Mengting Wan and Julian McAuley ----------- Three positive aspects of the paper: ----------- o They introduce a novel pattern (‘skip’ pattern) and provide a running example. o They propose 2D CNN-based framework, which is novel in the sequential recommendations, to catch the ‘skip’ pattern. Furthermore, they visually show their model capture the ‘skip’ pattern. o The performance of their model outperforms those of the competitors. ----------- Three negative aspects of the paper: ----------- o There are so many parameters (e.g., 2D filters) to be trained in the model. Thus, sensitivity tests for the hyperparameters are necessary since the model seems to overfit easily. o They need to formulate the forward process in Section 3.3. o They need to explain in more detail about Section 4.3 and Figure 3. For instance, they have to give where the kernels (a) and (b) are from. ----------- Overall evaluation ----------- SCORE: 1 (weak accept) ----- TEXT: The paper proposed a method for sequential recommendation. They first construct a pair-wise encoding tensor using the input sequence. In order to capture high-level sequential patterns (including ‘skip’ pattern), they feed the pair-wise encoding tensor to a 2D convolutional neural network, which yields a vector that represents a sequence. To catch the user’s global preference, they concatenate the user vector and the sequence vector, then feed into a fully connected neural network which results in a probability distribution of next interaction items. Comments: - They introduce a ‘skip’ pattern that is probably in real-world datasets and provide a running example for explanation. A novel approach is proposed to capture the ‘skip’ pattern. - The performance of their model outperforms those of the competitors. - It would be better to analyze how many the ‘skip’ behaviors exist in real-world datasets. - Sensitivity tests are necessary since the model seems to overfit easily. ----------------------- REVIEW 2 --------------------- SUBMISSION: 1055 TITLE: CosRec: 2D Convolutional Neural Networks for Sequential Recommendation AUTHORS: An Yan, Shuo Cheng, Wang-Cheng Kang, Mengting Wan and Julian McAuley ----------- Three positive aspects of the paper: ----------- 1. The research task is important. 2. Two real-world datasets in the experiments. 3. Comparisons with existing CNN-based approaches. ----------- Three negative aspects of the paper: ----------- 1. The idea is not novel. 2. The intuition is partially equivalent to bidirectional modeling and co-attention. 3. Hyperparameter studies and important studies are missing. ----------- Overall evaluation ----------- SCORE: -1 (weak reject) ----- TEXT: In this paper, the authors propose to pair-wisely encode item embeddings in a session for sequential recommender systems. The embeddings of items in the session are constructed as 2D data, thereby being fed into 2D CNN for recommendations. Experiments are conducted on two real-world datasets with some analysis. Detailed comments are as follows. * The idea using a higher-dimensional convolutional network for sequential recommenders is not novel [1]. The authors should mention and compare with the existing related work. [1] Tuan, Trinh Xuan, and Tu Minh Phuong. "3D convolutional networks for session-based recommendation with content features." Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 2017. * The intuition behind the idea of pairwise modeling is to consider interactions between items. This concept can also be incorporated by some existing studies, such as co-attention and bidirectional modeling like [2]. The authors should emphasize the benefit of modeling with 2D CNN. [2] Sun, Fei, et al. "BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer." arXiv preprint arXiv:1904.06690 (2019). * Based on the idea, the structure is not essential to be limited in two dimensions, I am interested in if more complicated relations between items can be captured by a high-dimensional structure. * The authors should conduct the significance test, such as t-test and permutation test, to verify if the improvements are significant. ----------------------- REVIEW 3 --------------------- SUBMISSION: 1055 TITLE: CosRec: 2D Convolutional Neural Networks for Sequential Recommendation AUTHORS: An Yan, Shuo Cheng, Wang-Cheng Kang, Mengting Wan and Julian McAuley ----------- Three positive aspects of the paper: ----------- 1. Given the reported results, the new method achieves better performance as compared with other baselines. 2. Case study on convolutional filter weights visualization is provided. 3. Two public datasets are used for performance validation. ----------- Three negative aspects of the paper: ----------- 1. Lack of clarifications on how hyperparameters and training/test ratios are set in the experiments. 2. The novelty of this model is limited to some extent. But considering to solve the specific task, it is reasonable. 3. More explanations are needed to show how the better performance is achieved. ----------- Overall evaluation ----------- SCORE: 1 (weak accept) ----- TEXT: Some key details in the experimental settings are missing, such as how to tuning hyperparameters for the developed approach and what is the training/test data split in both evaluated datasets. More discussions are needed for clarifying why perform convolutions on user behavior sequences when compared with other mechanisms (attention for recurrent neural network argument).