=========================================================================== BuildSys Review #25A Updated 25 Aug 2013 2:21:24pm EDT --------------------------------------------------------------------------- Paper #25: Towards Automatic Spatial Verification of Sensor Placement in Buildings --------------------------------------------------------------------------- Overall merit: 3. Weak accept Reviewer expertise: 2. Some familiarity ===== Paper summary ===== This paper presents an experimental demonstration of the use of empirical mode decomposition (EMD) as a means to cluster sensors in a building, based on the the sensor measurements (e.g., temperature, CO2 and humidity). The main contribution is the empirical demonstration that the medium frequency intrinsic mode function (IMF) can be used to determine whether or not any two sensors are located in the same room . ===== Comments for author ===== This paper presents an experimental demonstration of the use of empirical mode decomposition (EMD) as a means to cluster sensors in a building, based on the the sensor measurements (e.g., temperature, CO2 and humidity). The main contribution is the empirical demonstration that the medium frequency intrinsic mode function (IMF) can be used to determine whether or not any two sensors are located in the same room . This result is worth continued effort for investigations. The presentation of the paper should be improved. The paper contains certain obvious typos and grammatical mistakes. For example, "cypher-physical systems" should be "cyber-physical systems". The sentence "Buildings have become…, are poorly understood, …" contains two verbs. In addition, there is no explicit description, either in words or in mathematical expressions, of what "spatial verification" means. The meaning can be inferred only after the reviewer's repeated effort to go through the paper. Also, the relationship between the sensor measurements and the spatial verification problem has never been explicitly described. There are also some imprecise statements: "An IMF is a function with the same number of extrema and zero crossings (or differ by one at most) as the raw signal…" This is obviously not the case for the original signal and IMF6 in Figure 1(a). The original signal has no zero crossing, and it seems to have much fewer extrema than IMF6. While the authors demonstrate through an experiment that the medium frequency IMF is useful, there remain a lot of questions and concern. For example, 1. No rationale is provided to explain why the medium frequency IMF would work as an indicator of sensor location. 2. What happens when there are two separate rooms with similar temperature, CO2, humidity and so on? 3. What happens when the sizes of the rooms are different? Intuitively the measurements in different parts of a large room can be different. 4. Without significant evidence it is difficult to convince other people that the medium frequency IMF and the suggested correlation coefficient boundary would work in general. 5. How does the proposed method work to verify whether or not the position of a sensor is correct, instead of simply identifying which room it resides in? =========================================================================== BuildSys Review #25B Updated 28 Aug 2013 5:07:18pm EDT --------------------------------------------------------------------------- Paper #25: Towards Automatic Spatial Verification of Sensor Placement in Buildings --------------------------------------------------------------------------- Overall merit: 2. Weak reject Reviewer expertise: 2. Some familiarity ===== Paper summary ===== This paper presents a method to detect if a number of sensors are in the same room through classification. The motivation for this work comes from the fact that in large buildings there are a large number of sensors that may be mislabeled if maintained manually, so an automated method is preferable. The method itself is an application of existing classification methods. The training data set is obtained from a set of sensors deployed in a few rooms of a building. Each room had the same set of sensors. The results presented show that for the parameter values chosen, the method is able to detect that two sensors are in the same room correctly 93% of the time. This is higher than some competing methods. ===== Comments for author ===== The paper is well written, but I am not convinced about the motivation of the paper and the effectiveness of the method, or even the feasibility of a purely data driven approach to such a problem. Sure sensors are frequently mislabeled in buildings, anyone who has worked with buildings will agree with that. However, the big question is how large is the fraction of sensors that are mislabeled? I'd suspect it is small in general, that is what I have observed in the field. As a result, an algorithm to determine if two sensors are in the same room, for it to be preferred over manual verification, must be extremely accurate. Otherwise applying such an algorithm will - in the best case - lead to no improvement in sensor labeling over leaving things as they are, and in the worst case introduce more mislabeling. The authors should first start with an assessment of what fraction of sensors are mislabeled typically, and what is the required degree of accuracy is. In terms of presented results, there are some claims that sound grandiose compared to the meagre data and limited scenarios from which those conclusions are obtained. Take this for example: "This suggests that it generalizes across rooms in a building - perhaps even across buildings, since physical boundaries (i.e. walls, doors) are one of the primary reasons why modulation in the traces have a "locality of phenomena" property. These results suggest that a two-room study should be enough learn the discriminator value for the entire building." It is a risky conclusion to make. If walls and doors are primarily responsible for locality of phenomenon (which i don't believe is true, occupants play a big role), won't the difference in walls and doors and exposure of south facing walls vs. north facing walls etc. cause variations in the required discriminator value? =========================================================================== BuildSys Review #25C Updated 1 Sep 2013 8:57:28pm EDT --------------------------------------------------------------------------- Paper #25: Towards Automatic Spatial Verification of Sensor Placement in Buildings --------------------------------------------------------------------------- Overall merit: 4. Accept Reviewer expertise: 3. Knowledgeable ===== Paper summary ===== This paper describes a research study done to evaluate whether it is possible to identify sensors located in the same rooms and in different rooms from the sensor data. The authors have used a spatial clustering algorithm that was built on a prior research study. The results show that the clustering algorithm classified the sensors correctly for 93% of the time - which is a significant improvement over a raw time series based approach. ===== Comments for author ===== The paper is very well written and the results are very promising in terms of how accurately the sensors were classified. This should be of interest to the BuildSys community. It would be great if the authors discuss the following in some detail: - how sensitive is this approach to the size of a room? Would it still achieve high accuracy results if the rooms are large and sensors are located at different parts of the room? - How sensitive are the results to the changes in the environment over time? would it likely classify the sensors correctly as a facility changes over time? - how sensitive is this approach of the usage of and occupants of different rooms? =========================================================================== Comment Paper #25: Towards Automatic Spatial Verification of Sensor Placement in Buildings --------------------------------------------------------------------------- This paper was discussed at the PC meeting. The paper has been conditionally accepted with shepherding. The Pc would like the authors to make writing improvements as discussed in the reviews.