Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterization

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward, in large part, by breakthroughs in deep convolutional neural networks (CNNs). These state-of-the-art methods are poised to become widely used in environmental monitoring applications due to the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data, etc.) that can be analyzed to observe, model, and understand environmental conditions. As part of this project we developed new tools and algorithms to process imagery collected with sediment profile imaging (SPI) camera technology which was developed as a reconnaissance tool for characterizing physical, chemical, and biological seafloor processes in near-surface sediments. For SPI applications we used CNNs to characterize grain size and the location of the sediment-water interface and to develop object detectors for infauna (e.g., worms) and key biological features (e.g., feeding voids). These novel image processing tools have been consolidated into a stand-alone SPI image processing platform (iSPI v1.2) that streamlines and standardizes the generation of data from SPI imagery and makes this type of data extraction more cost-effective and repeatable. While these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for CNN model training and validation and optimized hardware and software to ensure that CNNs can be trained effectively and in a reasonable amount of time. In the case of SPI image analysis we have overcome many of these rate limiting challenges by utilizing a diverse image library built across multiple projects coupled with staff expertise and a combination of both cloud and onsite computing resources.