First Drain, Then Train: Unleash the Full Potential of AI on Large Underwater Datasets (After Removing the Water from Them)

Large image datasets like ImageNet ignited the artificial intelligence boom of the last decade, fueling groundbreaking discoveries in science and industry. The underwater domain has no shortage of large image datasets, but it has not yet benefited from the full power of computer vision and machine learning algorithms that enabled these discoveries. A major reason for this delay is water itself: water masks many computationally valuable features of a scene, and does so with high variability across the global ocean, significantly complicating the training process for learning-based algorithms. For example, networks trained on images taken under a specific set of conditions (location, season, time of day, turbidity, depth, etc.) generally perform poorly when used on imagery acquired under different conditions. A high-quality, globally representative training dataset is not within reach in the near future, as collecting sufficiently varied calibrated photographs requires a gargantuan multi-year, multi-nation effort.

An alternative approach would be to turn underwater images into their equivalents in air, through the consistent reconstruction of colors that were lost. Then, the full potential of time-tested AI optimized for photos taken on land could be realized, and new algorithms can be developed specific to the contents of “drained” underwater images. Historically, methods aiming to restore lost colors and contrast in underwater images, however, have yielded unstable results. A recent computer vision algorithm broke new ground not only by demonstrating why these methods performed inconsistently, but also by showing that near-perfect color reconstruction is possible using RGBD images. In this talk, we will introduce Sea-thru, a physics-based computer vision algorithm that consistently removes water from underwater RGBD images, creating exciting opportunities for the future of underwater research, exploration, and conservation.