The ocean is a vibrant and bustling realm. Yet, for many, the underwater world remains largely hidden from view. Water acts like a cloak, distorting the light that penetrates its depths, causing colors to scatter and fade. This makes capturing the true hues of marine life incredibly challenging without getting up close.
Now, researchers from MIT and the Woods Hole Oceanographic Institution (WHOI) have designed an innovative image-analysis tool that penetrates these underwater optical effects. This tool generates images of marine environments that appear as though the water has been removed, showcasing the true colors of oceanic scenes. By combining color correction with a computational model, they can transform captured images into an immersive three-dimensional underwater “world” that can be explored virtually.
The team named their creation “SeaSplat,” a nod to its underwater focus and a technique called 3D gaussian splatting (3DGS). This method stitches together images to construct a comprehensive 3D representation of the underwater environment, allowing for detailed exploration from any angle.
“With SeaSplat, we can explicitly model the effects of water, enabling a clearer view and producing superior 3D models of underwater scenes,” explains MIT graduate student Daniel Yang.
Using SeaSplat, the researchers created true-color 3D representations from images captured by divers and underwater vehicles at various sites, including the U.S. Virgin Islands. This method produced more vibrant and accurate color representations than previous techniques.
This tool could be invaluable for marine biologists, assisting them in monitoring the health of ocean ecosystems. For example, as an underwater robot photographs a coral reef, SeaSplat simultaneously processes these images, rendering a true-color 3D environment. Scientists can then virtually navigate through this representation, examining potential signs of coral bleaching and other ecological changes.
“Bleaching may appear white when viewed up close, but from a distance, it might look blue and hazy, making it difficult to detect,” notes Yogesh Girdhar, an associate scientist at WHOI. “Using SeaSplat imagery could make it easier to identify coral bleaching and differentiate between coral species by revealing their true colors.”
Girdhar and Yang are set to present their findings on SeaSplat at the IEEE International Conference on Robotics and Automation (ICRA). Their study also includes contributions from John Leonard, a professor of mechanical engineering at MIT.
Aquatic Optics
Underwater, the color and clarity of objects are adversely affected by light distortion as it travels through water. In recent years, researchers have developed several color-correcting tools aimed at restoring true colors in aquatic settings. While one promising method, dubbed “Sea-Thru,” accurately reproduces underwater colors, it requires substantial computational resources, complicating its application for 3D scene models.
Parallel advancements in 3D gaussian splatting have produced tools capable of seamlessly integrating images to create a complete 3D representation of a scene. These 3D worlds enable “novel view synthesis,” allowing users to view generated environments not just from the original perspective, but from multiple angles.
However, applying 3D reconstruction to underwater imagery has proven challenging due to two major optical effects: backscatter and attenuation. Backscatter occurs when light reflects off minuscule particles, creating a haze, while attenuation refers to how certain light wavelengths fade over distance—leading red objects to appear more subdued compared to blues when viewed from afar.
While the color of objects remains fairly consistent in air regardless of angle or distance, underwater visuals can shift dramatically based on perspective. This variation complicates the stitching process of underwater images and often results in lost details due to backscatter and attenuation effects.
“Imagine if you could drain the ocean—what stunning sights might you uncover?” imagines Leonard.
A Model Swim
In this groundbreaking work, Yang and his colleagues devised a color-correcting algorithm that considers the effects of both backscatter and attenuation. This algorithm determines how much distortion each pixel experiences due to these underwater optical phenomena and effectively restores the pixel’s true color.
Integrating this color-correcting algorithm with a 3D gaussian splatting model resulted in SeaSplat, enabling rapid analysis of underwater images to produce true-color 3D virtual environments that can be explored from any perspective.
The team tested SeaSplat on various underwater scenes, including images from the Red Sea, the Caribbean near Curaçao, and the Pacific Ocean close to Panama. They also utilized images captured by a remote-controlled underwater robot in the U.S. Virgin Islands, covering a range of marine conditions. Remarkably, SeaSplat enabled them to generate vivid 3D worlds that maintained true colors, regardless of angle or distance.
“Once we create a 3D model, scientists can ‘swim’ through it, exploring intricate details with genuine colors,” Yang explains.
Currently, the method necessitates significant computational power typically found in a desktop computer, which might be too cumbersome for underwater robots. However, SeaSplat is well-suited for tethered operations, whereby a vehicle linked to a ship captures images transmitted back to a ship’s computer.
“This represents the first successful approach for swiftly creating high-quality 3D models underwater with accurate colors,” Girdhar asserts. “This advancement will prove essential in quantifying biodiversity and assessing the health of coral reefs and other marine communities.”
This research received support from the Investment in Science Fund at WHOI and the United States National Science Foundation.
Photo credit & article inspired by: Massachusetts Institute of Technology
