Chris Barngrover

The goal of this research is to use advances in machine learning and computer vision to utilize optical and acoustic sensors in concert for object detection and classification in underwater applications. A specific goal of the research will be object detection and classification of mines found on the sea floor.

The new and interesting element of this research will be applying existing computer vision techniques to imaging sonar sensors and combining this output with the usual computer vision results on images from optical sensors. The combination of acoustic and optical sensor data allows for more robust classification by using the optical sensor in close proximity to an object or in clear water conditions, while maintaining the same level of capability via the sonar sensor when operating further from an object or in poor water clarity.

The other interesting focus of the research will be in adaptive and active learning techniques intended to alter the classification algorithms in real time in order to improve results in changing environments.

This project will involve multiple phases of development with overlap in efforts across the phases. Only the phases intended for FY11 are listed here:

Phase 1: Simple Mine Data Collection – This phase has two branches to it. The first is to gather existing data, side scan sonar and optical, from previous tests. In addition, it is necessary to gather data in a controlled environment such as TRANSDEC. For this data capture, I will need access to a side scan sonar and a camera system in some capacity. Options include outfitting my research vehicle with a side scan sonar, attaching an imaging system to the REMUS100 vehicle, or creating a static test system with a sonar and a camera. I will also need access to mine examples and the TRANSDEC facility.

Phase 2: Simple Mine Optical Classifiers – This phase involves experimentation with potential classification features for the mines in optical sensor data. This will be a natural progression from my previous research. The goal from this phase is an understanding of which features best represent the simple mines in optical sensor data. The data sets will include existing data where available, and data captured from the TRANSDEC setup.

Phase 3: Simple Mine Sonar Classifiers – This phase involves experimentation with potential classification features for the mines in sonar sensor data. The goal from this phase is an understanding of which features best represent the simple mines in sonar sensor data. The data sets will include existing sonar data and data captured from the TRANSDEC setup.

Phase 4: Adaptive and Active Learning – This phase involves experimentation with existing adaptive and active learning techniques for real time improvement of classification algorithms for the simple mines. From a theoretic standpoint, the data sets do not matter to the research into learning techniques, however in application, it is useful to show the benefits with real data.

Phase 5: Simple Mine Hybrid Classifiers – This phase will be a combination of phases 2 and 3 with potential input from phase 4 based on its results. The strengths of the classification algorithms from the optical and sonar sensors respectively will be used to combine the algorithms for improved classification. Considerations will be scoring based on the individual algorithm outputs versus a cascading approach, most likely with the sonar classification leading to follow-up by the optical classification.

The target AUV for this research is the Hydroid REMUS 100, 600, 6000. This vehicle is currently used by the Navy for missions to find mines on the ocean floor. All of the data captured from these missions is analyzed post mission by trained Marines. Some of the data sets for the research come from this platform.

The vehicle that will often be the testbed for this research will be the Stingray. The original Stingray, shown here, will be utilized until the new version is available. This platform is open source and developed by SD iBotics with support from UCSD. The level of control I have over this vehicle will allow me to pursue interesting research avenues including combining sonar and camera algorithms, adaptive learning techniques, and active learning techniques.