----------------------- REVIEW 1 --------------------- PAPER: 299 TITLE: Detecting Cohesive and 2-mode Communities in Directed and Undirected Networks AUTHORS: Jaewon Yang, Julian McAuley and Jure Leskovec OVERALL EVALUATION: 2 (accept) REVIEWER'S CONFIDENCE: 4 (high) Presentation Suggestion: 1 (The contribution is of general interest to most WSDM attendees and should be presented as a standard talk.) Nominate for best paper award?: 2 (No) ----------- REVIEW ----------- * Summary of the contribution. The authors define a new algorithm for finding a particular type of community structure in a network. They call these "2-mode" communities and they can either be directed or undirected. They define a new "block-model" for the graphs, this has a variety of features not captured by existing block-models. They fit a given network to this model which outputs different types of communities. These new communities are evaluated on * Main strengths of the paper - What are the arguments that could be given for accepting this paper, number as S1, S2, S3,... Place the most important first. S1. The results are better than existing methods and they test on networks with ground-truth. S2. They handle a variety of interesting community cases that most other methods do not handle. S3. They spend the time to interpret their communities. * Main weaknesses of the paper - What are the arguments that could be given for rejecting this paper, number as W1, W2, W3, ... Place the most important first. W1 There are no timing results shown for the large networks, and it isn't exactly clear how this method differs from the BigCLAM W2 Only "weak" convergence results the block-coordinate descent method (although these are standard), and no discussion of final objective values or how many iterations were performed for each of the networks. W3. Given their model, they really ought to compare with methods that compute sparse non-negative matrix factorizations of the adjacency matrix, which is closely related to what they are doing. * Concrete suggestions for improvements - In order of importance, what are concrete suggestions for improving this paper, either in the camera-ready version or on a resubmission elsewhere. - List the time taken for your method on the large networks. - Add some comparison against non-negative matrix factorization. - It'd be nice to see algorithmic perturbation results to understand what exactly matters, e.g. does the algorithmic initialization have a profound impact on runtime or quality? ----------------------- REVIEW 2 --------------------- PAPER: 299 TITLE: Detecting Cohesive and 2-mode Communities in Directed and Undirected Networks AUTHORS: Jaewon Yang, Julian McAuley and Jure Leskovec OVERALL EVALUATION: 2 (accept) REVIEWER'S CONFIDENCE: 4 (high) Presentation Suggestion: 1 (The contribution is of general interest to most WSDM attendees and should be presented as a standard talk.) Nominate for best paper award?: 2 (No) ----------- REVIEW ----------- This paper focuses on detecting communities in directed and undirected networks. Their algorithm, CODA, can not only detect cohesive communities, which is the traditional definition of dense communities, and 2-mode communities, which nodes have similar out-link nodes or in-link nodes. CODA is based on the directed affiliation network model, and the model and the algorithm are both the variation of a work from J. Yang and J. Leskovec. In the experiment they show that CODA can outperform other algorithm in both directed networks and undirected networks since it can detect both types of communities. Moreover, they show that CoDA can be parallelized and it is scalable to a network with millions of nodes. Pros: 1. The experiment adopts comprehensive types of networks, showing that their algorithm can outperform other algorithms significantly. Additionally, they show many case studies to strongly demonstrate the usage of 2-mode communities, in which other algorithm they compare cannot detect. 2. The method simply modifies a previous algorithm but the intuition is very clear and effective. 3. Section 5.2 is novel analysis. Although there are several algorithms can detect both 2-mode and cohesive communities, few of them showing the percentage of each type of communities. Cons: Among the compared algorithm, only MMSB is designed for directed networks. Thus, although it outperforms all other algorithms they compared, it is not clear whether CoDA can outperform other algorithm designed to detect both cohesive and 2-mode communities in directed networks. A recent survey from Malliaors and Vazirgiannis categorizes the community detection algorithm in directed networks and shows the most of recent algorithms which can also detect cohesive communities and 2-mode communities at the same time. Among them, a previous work from Satuluri and Parthasarathy can symmetrize a directed graph while conserving both the property of these two types of communities, so after the symmetrization, any network clustering can be applied and detect cohesive and 2-mode communities. Although the experimental data is very comprehensive, it is better to also compare comprehensive algorithms which contribute to detect the same kinds of communities as SoDA. In summary this is a good paper. With a more careful review of the literature and a comparison with a more competitive strawman this paper would be even stronger. ----------------------- REVIEW 3 --------------------- PAPER: 299 TITLE: Detecting Cohesive and 2-mode Communities in Directed and Undirected Networks AUTHORS: Jaewon Yang, Julian McAuley and Jure Leskovec OVERALL EVALUATION: 1 (weak accept) REVIEWER'S CONFIDENCE: 3 (medium) Presentation Suggestion: 2 (The contribution is of interest to a specific subset of attendees and is best presented as a poster.) Nominate for best paper award?: 2 (No) ----------- REVIEW ----------- Strong Points - Delineation of two types of communities: cohesive and 2-mode communities - Special treatment of directed edges, and notions of communities based on relationships to members outside the community (as opposed to connections among one an other) - A detailed experimental evaluation on real data. Weak Points: - The notion of affiliation network models has been used in the past, thus reducing the novelty of this paper - Related Work: While Section 1 lists a lot of related papers, the comparison with these pieces of work is inadequate. For example, it would be good to know concretely what are the key new ideas and differences compared with work on block models, affiliation networks, etc. There is only a very brief description of this distinction. ------------------------- METAREVIEW ------------------------ PAPER: 299 TITLE: Detecting Cohesive and 2-mode Communities in Directed and Undirected Networks The paper addresses an important problem, and expands upon Yang and J. Leskovec's WSDM'13 work. All reviewers liked the paper, and I am joining their opinion.