Recording, Streaming and Displaying Live Digital Holograms of Ocean Life

How can we make the ocean feel present and relevant in our daily lives? This is the great problem underlying efforts to communicate and visualize new discoveries and stories from the ocean. There has been a great deal of effort put into capturing 3D images and video of the ocean, but these techniques have the limitations of only capturing non-moving objects in 3D, or requiring a viewer to wear headgear in order to feel the sense of presence, both of which are obstacles for public visualization of ocean life.

We present a novel camera design for filming design for filming and streaming video of aquatic life, paired with a digital holographic display that can show the three-dimensional content to multiple simultaneous viewers without requiring headgear. This camera, which is a first of its kind, is an array of 32 cameras that can capture wide-angle, coherent lightfield imagery at recreational scuba depth, and is suitable for filming wildlife at scales from megafauna down to 1m. A second, non-submersible camera is used to capture tank footage of smaller ocean organisms down to a few cm in size. Custom software detects the pose of each camera and aligns the images to produce a coherent lightfield, which can then be played back in a holographic display.

We performed a series of film experiments with various partners to film lightfield video of ocean life ranging from 20m down to 2cm. We will present several case studies detailing the planning of a lightfield shoot, experiences during the film sessions, processing the data, and methods of display and interaction. Finally, we will show a series of videos of the lightfield footage being presented in a lightfield display.

Underlying this work is our belief that advances in imaging and display technology can change the way we connect to stories of life around us. Our goal is to create imaging techniques that can non-destructively capture digital holographic video of the most fragile and remote ecosystems on our planet, and bring that sense of presence to the public, fostering a connection to those environs. We believe that the best way to view ocean life is in situ, with the nuance and complex interactions that only occur in nature, and we strive to create a medium that allows the general public to view the complex stories of life playing out all at every moment, all around us, in our oceans.