Reviewer #1 Questions 1. {Summary} Please briefly summarize the main claims/contributions of the paper in your own words. (Please do not include your evaluation of the paper here). This work designed a factual and informative review generation model for explainable recommendation. It models the review generation as a QA process, and a personalized retriever produces the latent query, which will retrieve several reviews as evidence. The tf-idf keywords are generated according to the query and guide the generation of answer. 2. {Strengths and Weaknesses} Please provide a thorough assessment of the strengths and weaknesses of the paper, touching on each of the following dimensions: novelty, quality, clarity, and significance. Strengths: 1. The proposed review generation model can consider both factual and informative aspects for better explanation effect. 2. The QA process of the model can intuitively guarantee the quality of generated review, based on the important latent query. Weaknesses: 1. The authors should analyze the situation when PRAG wins or fails against other approaches in comparison, and provide some analysis, so as to give the audience some guided conclusions. It is not enough to simply list the numbers. 3. {Questions for the Authors} Please carefully describe questions that you would like the authors to answer during the author feedback period. Think of the things where a response from the author may change your opinion, clarify a confusion or address a limitation. Please number your questions. NA 4. {Evaluation: Novelty} How novel are the concepts, problems addressed, or methods introduced in the paper? Good: The paper makes non-trivial advances over the current state-of-the-art. 5. {Evaluation: Quality} Is the paper technically sound? Good: The paper appears to be technically sound. The proofs, if applicable, appear to be correct, but I have not carefully checked the details. The experimental evaluation, if applicable, is adequate, and the results convincingly support the main claims. 6. {Evaluation: Significance} How do you rate the likely impact of the paper on the AI research community? Good: The paper is likely to have high impact within a subfield of AI OR modest impact across more than one subfield of AI. 7. {Evaluation: Clarity} Is the paper well-organized and clearly written? Good: The paper is well organized but the presentation has minor details that could be improved. 8. (Evaluation: Reproducibility) Are the results (e.g., theorems, experimental results) in the paper easily reproducible? (It may help to consult the paper’s reproducibility checklist.)checklist.) Good: key resources (e.g., proofs, code, data) are available and sufficient details (e.g., proofs, experimental setup) are described such that an expert should be able to reproduce the main results. 9. {Evaluation: Resources} If applicable, how would you rate the new resources (code, data sets) the paper contributes? (It might help to consult the paper’s reproducibility checklist) Not applicable: For instance, the primary contributions of the paper are theoretical. 10. {Evaluation: Ethical considerations} Does the paper adequately address the applicable ethical considerations, e.g., responsible data collection and use (e.g., informed consent, privacy), possible societal harm (e.g., exacerbating injustice or discrimination due to algorithmic bias), etc.? Not Applicable: The paper does not have any ethical considerations to address. 11. (OVERALL EVALUATION) Please provide your overall evaluation of the paper, carefully weighing the reasons to accept and the reasons to reject the paper. Accept: Technically solid paper, with high impact on at least one sub-area of AI or modest-to-high impact on more than one area of AI, with good to excellent quality, reproducibility, and if applicable, resources, and no unaddressed ethical considerations. Top 60% of accepted papers. 13. (CONFIDENCE) How confident are you in your evaluation? Quite confident. I tried to check the important points carefully. It is unlikely, though conceivable, that I missed some aspects that could otherwise have impacted my evaluation. 14. (EXPERTISE) How well does this paper align with your expertise? Very Knowledgeable: This paper significantly overlaps with my current work and I am very knowledgeable about most of the topics covered by the paper. Reviewer #2 Questions 1. {Summary} Please briefly summarize the main claims/contributions of the paper in your own words. (Please do not include your evaluation of the paper here). This paper proposed a personalized retriever-reader model to generate factual and informative reviews, which act as explanations for recommendation. The generation process is formalized as a question-answering scheme. 2. {Strengths and Weaknesses} Please provide a thorough assessment of the strengths and weaknesses of the paper, touching on each of the following dimensions: novelty, quality, clarity, and significance. Strengths: 1. This paper proposes a novel review generation framework containing a retriever and a reader module. 2. The authors conduct extensive experiments on three datasets and many aspects of metrics to verify the effectiveness of their model. Weaknesses: 1. Some details are missing (see questions). 3. {Questions for the Authors} Please carefully describe questions that you would like the authors to answer during the author feedback period. Think of the things where a response from the author may change your opinion, clarify a confusion or address a limitation. Please number your questions. During my review of this paper, I posed some questions to answer: 1. Why do you call the retriever as a "Personalized Retriever"? Which model designs or motivations can reflect the "personalized" characteristics? You should clarify this important concept in your paper. 2. How to obtain the final input review embedding? The equation is not clear. According to my understanding, the user embedding v_u, item embedding v_i, and type embedding c will add to each review (sentence) embedding. The summation will be taken as the token embedding for BERT. Is that right? This part is written difficult to read clearly and lacks many details. 3. The personalized attention part is also not clear about how to implement the weighted attention. On what embedding? How multiply? 4. In human evaluation, why don't you evaluate the factuality metric for reviews besides fluency and informativeness? 5. Can you explain why PRAG does not perform well on USR metric? Any insights? 4. {Evaluation: Novelty} How novel are the concepts, problems addressed, or methods introduced in the paper? Good: The paper makes non-trivial advances over the current state-of-the-art. 5. {Evaluation: Quality} Is the paper technically sound? Good: The paper appears to be technically sound. The proofs, if applicable, appear to be correct, but I have not carefully checked the details. The experimental evaluation, if applicable, is adequate, and the results convincingly support the main claims. 6. {Evaluation: Significance} How do you rate the likely impact of the paper on the AI research community? Fair: The paper is likely to have modest impact within a subfield of AI. 7. {Evaluation: Clarity} Is the paper well-organized and clearly written? Fair: The paper is somewhat clear, but some important details are missing or unclear. 8. (Evaluation: Reproducibility) Are the results (e.g., theorems, experimental results) in the paper easily reproducible? (It may help to consult the paper’s reproducibility checklist.)checklist.) Fair: key resources (e.g., proofs, code, data) are unavailable and/or some key details (e.g., proof sketches, experimental setup) are unavailable which make it difficult to reproduce the main results. 9. {Evaluation: Resources} If applicable, how would you rate the new resources (code, data sets) the paper contributes? (It might help to consult the paper’s reproducibility checklist) Fair: The shared resources are likely to be of some use to other AI researchers. 10. {Evaluation: Ethical considerations} Does the paper adequately address the applicable ethical considerations, e.g., responsible data collection and use (e.g., informed consent, privacy), possible societal harm (e.g., exacerbating injustice or discrimination due to algorithmic bias), etc.? Poor: The paper fails to address obvious ethical considerations. 11. (OVERALL EVALUATION) Please provide your overall evaluation of the paper, carefully weighing the reasons to accept and the reasons to reject the paper. Borderline accept: Technically solid paper where reasons to accept, e.g., good novelty, outweigh reasons to reject, e.g., fair quality. Please use sparingly. 13. (CONFIDENCE) How confident are you in your evaluation? Very confident. I have checked all points of the paper carefully. I am certain I did not miss any aspects that could otherwise have impacted my evaluation. 14. (EXPERTISE) How well does this paper align with your expertise? Expert: This paper is within my current core research focus and I am deeply knowledgeable about all of the topics covered by the paper. Reviewer #7 Questions 1. {Summary} Please briefly summarize the main claims/contributions of the paper in your own words. (Please do not include your evaluation of the paper here). This paper proposes approach to address the factual and informativeness of explaination in explainable recommendation. The approach adopts a retrieve-reader framework. For the retriever, a query embedding is created from user's words and the item's reviews. The relevant reviews are retrieved with the query. For the reader module, a keyword guided question-answering approach is adopted to extract the snippet from retrieved reviews. Keyword is generated with a traned GPT model, common question such as "what was great?" is adopted as prompt fo, and UnifiedQA, a text-to-text QA model is used in the approach. Experiments are conducted with baselines including PETER and PEPLER, while ablation study is also given to give insights on the performance of the proposed approach. Mauve is used to compare final result explaination text and ground truth. 2. {Strengths and Weaknesses} Please provide a thorough assessment of the strengths and weaknesses of the paper, touching on each of the following dimensions: novelty, quality, clarity, and significance. Strengths: (1) extensive experiments on major benchmark datasets. Weakness: (1) proposed approach tries to address the research problem, still wish to see more advanced approach to address the problem. For example, keywords can control the topics in explaination, but any way to control the how informative in that topic? And it looks that the approach to address "factual" and non-factural problem is relatively weaker. (2) Evaluation: will "entailment" be the best way to evaluate factuality? Mauve is adopted, but still wish to see BLEU and Rouge, which will be easy to make comparison. (3) In writing, wish to see more details on the QA model. Readers may or maynot know well on the UnifiedQA model, then reader might wonder is this extractive QA or abstractive QA? 3. {Questions for the Authors} Please carefully describe questions that you would like the authors to answer during the author feedback period. Think of the things where a response from the author may change your opinion, clarify a confusion or address a limitation. Please number your questions. Wish to see the BLEU and ROUGE score 4. {Evaluation: Novelty} How novel are the concepts, problems addressed, or methods introduced in the paper? Fair: The paper contributes some new ideas or represents incremental advances. 5. {Evaluation: Quality} Is the paper technically sound? Good: The paper appears to be technically sound. The proofs, if applicable, appear to be correct, but I have not carefully checked the details. The experimental evaluation, if applicable, is adequate, and the results convincingly support the main claims. 6. {Evaluation: Significance} How do you rate the likely impact of the paper on the AI research community? Fair: The paper is likely to have modest impact within a subfield of AI. 7. {Evaluation: Clarity} Is the paper well-organized and clearly written? Good: The paper is well organized but the presentation has minor details that could be improved. 8. (Evaluation: Reproducibility) Are the results (e.g., theorems, experimental results) in the paper easily reproducible? (It may help to consult the paper’s reproducibility checklist.)checklist.) Fair: key resources (e.g., proofs, code, data) are unavailable and/or some key details (e.g., proof sketches, experimental setup) are unavailable which make it difficult to reproduce the main results. 9. {Evaluation: Resources} If applicable, how would you rate the new resources (code, data sets) the paper contributes? (It might help to consult the paper’s reproducibility checklist) Fair: The shared resources are likely to be of some use to other AI researchers. 10. {Evaluation: Ethical considerations} Does the paper adequately address the applicable ethical considerations, e.g., responsible data collection and use (e.g., informed consent, privacy), possible societal harm (e.g., exacerbating injustice or discrimination due to algorithmic bias), etc.? Good: The paper adequately addresses most, but not all, of the applicable ethical considerations. 11. (OVERALL EVALUATION) Please provide your overall evaluation of the paper, carefully weighing the reasons to accept and the reasons to reject the paper. Borderline accept: Technically solid paper where reasons to accept, e.g., good novelty, outweigh reasons to reject, e.g., fair quality. Please use sparingly. 13. (CONFIDENCE) How confident are you in your evaluation? Quite confident. I tried to check the important points carefully. It is unlikely, though conceivable, that I missed some aspects that could otherwise have impacted my evaluation. 14. (EXPERTISE) How well does this paper align with your expertise? Knowledgeable: This paper has some overlap with my current work. My recent work was focused on closely related topics and I am knowledgeable about most of the topics covered by the paper. Reviewer #8 Questions 1. {Summary} Please briefly summarize the main claims/contributions of the paper in your own words. (Please do not include your evaluation of the paper here). The paper proposes a novel framework to generate factual and informative explanations for recommendations. The framework first generates a latent query for review predictions, and then formulates the explanation generation as the question answering task. The view is pretty interesting and sounds reasonable. The generated explanations are also evaluated from multiple aspects, such as factuality, informativeness, quality, etc. They also conduct human evaluation to inspect the fluency and informativeness of the generated explanations. The framework also contributes to more accurate rating prediction. 2. {Strengths and Weaknesses} Please provide a thorough assessment of the strengths and weaknesses of the paper, touching on each of the following dimensions: novelty, quality, clarity, and significance. Strengths: 1. The paper is well-written. 2. The proposed retriever-reader framework is interesting and reasonable for generating factual and informative explanations. 3. Both automatic and human evaluations are conducted. Specifically, metrics regarding factuality and informativeness are involved for evaluation, which support their claim. Weakness: 1. It would be better to have more ablation studies to inspect the effect of each component. 2. Metric on factuality is not reported in human evaluations, while this will be more convincing than the pretrained entailment models. 3. {Questions for the Authors} Please carefully describe questions that you would like the authors to answer during the author feedback period. Think of the things where a response from the author may change your opinion, clarify a confusion or address a limitation. Please number your questions. Why was the metric on factuality not reported in human evaluations? In my view, this will be more convincing than the pretrained entailment models. 4. {Evaluation: Novelty} How novel are the concepts, problems addressed, or methods introduced in the paper? Good: The paper makes non-trivial advances over the current state-of-the-art. 5. {Evaluation: Quality} Is the paper technically sound? Good: The paper appears to be technically sound. The proofs, if applicable, appear to be correct, but I have not carefully checked the details. The experimental evaluation, if applicable, is adequate, and the results convincingly support the main claims. 6. {Evaluation: Significance} How do you rate the likely impact of the paper on the AI research community? Good: The paper is likely to have high impact within a subfield of AI OR modest impact across more than one subfield of AI. 7. {Evaluation: Clarity} Is the paper well-organized and clearly written? Excellent: The paper is well-organized and clearly written. 8. (Evaluation: Reproducibility) Are the results (e.g., theorems, experimental results) in the paper easily reproducible? (It may help to consult the paper’s reproducibility checklist.)checklist.) Excellent: key resources (e.g., proofs, code, data) are available and key details (e.g., proof sketches, experimental setup) are comprehensively described such that competent researchers will be able to easily reproduce the main results. 9. {Evaluation: Resources} If applicable, how would you rate the new resources (code, data sets) the paper contributes? (It might help to consult the paper’s reproducibility checklist) Good: The shared resources are likely to be very useful to other AI researchers. 10. {Evaluation: Ethical considerations} Does the paper adequately address the applicable ethical considerations, e.g., responsible data collection and use (e.g., informed consent, privacy), possible societal harm (e.g., exacerbating injustice or discrimination due to algorithmic bias), etc.? Not Applicable: The paper does not have any ethical considerations to address. 11. (OVERALL EVALUATION) Please provide your overall evaluation of the paper, carefully weighing the reasons to accept and the reasons to reject the paper. Accept: Technically solid paper, with high impact on at least one sub-area of AI or modest-to-high impact on more than one area of AI, with good to excellent quality, reproducibility, and if applicable, resources, and no unaddressed ethical considerations. Top 60% of accepted papers. 13. (CONFIDENCE) How confident are you in your evaluation? Somewhat confident, but there's a chance I missed some aspects. I did not carefully check some of the details, e.g., novelty, proof of a theorem, experimental design, or statistical validity of conclusions. 14. (EXPERTISE) How well does this paper align with your expertise? Somewhat Knowledgeable: This paper has little overlap with my current work. I am somewhat knowledgeable about some of the topics covered by the paper.