An Open-Source System for Do-It-Yourself AI in the Marine Environment

Anthony Hoogs1, Matthew David Dawkins2, Benjamin Richards3, George Cutter4, Deborah Hart5, M. Elizabeth Clarke6, William Michaels7, Jon Crall8, Linus Sherrill8, Neil Siekierski9, Matthew Woehlke9 and Kyle Edwards9, (1)Kitware, Clifton Park, United States, (2)Kitware, Saratoga Springs, NY, United States, (3)NOAA, Honolulu, HI, United States, (4)NOAA Southwest Fisheries Science Center, Antarctic Ecosystem Research Division, La Jolla, CA, United States, (5)NOAA Fisheries Woods Hole Laboratory, Woods Hole, United States, (6)NOAA NWFSC, Seattle, WA, United States, (7)NOAA. Fisheries, US DOC, Silver Spring, MD, United States, (8)Kitware, NY, United States, (9)Kitware Inc., Clifton Park, United States
Abstract:
The decreasing cost and increasing capabilities of cameras has greatly expanded the amount of image data collected by marine scientists. In many cases, however, there is far more data than manual analysis could ever examine, and training analysts with the required expertise can be costly and error-prone. Computer vision and machine learning can help, but creating an operationally viable solution usually requires deep learning expertise, large amounts of correctly labeled training data, and software engineering skills.

To address these challenges, NOAA National Marine Fisheries has funded the development of an open-source toolkit, Video and Imagery Analysis for the Marine Environment (VIAME; github.com/VIAME). VIAME includes a wide range of image and video analysis capabilities such as fish detection and classification, tracking, image classification and stereo matching. Built upon an established open-source video analytics toolkit, Kitware Image and Video Exploitation and Retrieval (KWIVER; kwiver.org), VIAME enables the rapid integration of new visual analytics, and includes user interfaces, databases and evaluation/scoring capabilities. Algorithms from multiple NOAA Fisheries Science Centers (FSCs) are integrated, such as length measurement from the Alaska FSC.

More significantly, VIAME includes the capability for scientists to create new analytics, specific to their problems, through user interfaces without any programming or knowledge of deep learning. Through image search and interactive query refinement, users can quickly build a complete detection and classification capability for a novel problem by providing only positive/negative feedback on examples suggested by the system, and then run it on any amount of imagery or video. For more challenging problems, users can manually annotate images and then train a deep learning detection and classification capability.

VIAME is installed at all six NOAA FSCs and a variety of labs around the world. It has been successfully applied to many problems including scallop detection, plankton classification, fish classification, seal detection and image registration. It is freely available with permissive open-source licensing.

Back to: Artificial Intelligence Systems for Advancing the Study of Aquatic Ecosystems II Posters