Meet Sara Beery, a new assistant professor in MIT’s renowned Department of Electrical Engineering and Computer Science (EECS), driven by a passion for addressing ecological challenges. Beery has dedicated her research career to leveraging her skills in computer vision, machine learning, and data science to face pressing issues in conservation and sustainability. Drawn to MIT’s commitment to “computing for the planet,” she aims to revolutionize environmental and biodiversity monitoring at a global scale.
In the lush Pacific Northwest, salmon play a pivotal role in maintaining the ecosystem’s health, particularly given their intricate reproductive lifecycle. Each year, millions of salmon undertake a remarkable migration to spawn. Their journey begins in freshwater streams, where eggs hatch and young fry swim toward the ocean. After spending several years growing into adults, these salmon return to their birthplace streams to deposit their eggs and ensure the survival of their species. Unfortunately, both male and female salmon typically die shortly after spawning, but their contribution to the ecosystem is far-reaching.
Salmon migration not only aids in species continuation but also supports a diverse array of organisms within the ecosystems they traverse. They transport vital nutrients such as carbon and nitrogen from the ocean upstream, enriching the terrestrial habitats. Additionally, salmon form a crucial part of various predator-prey dynamics, serving as nourishment for species like bears and birds, while also helping regulate better prey populations. Moreover, decomposed salmon provide essential nutrients back to the ecosystem, emphasizing their vital role in the overall health of rivers and oceans.
Beyond their ecological importance, salmon also hold significant economic and cultural value in the region. Both commercial and recreational salmon fishing bolster local economies, while Indigenous communities in the Pacific Northwest view salmon as a central element of their diets, traditions, and ceremonies.
Salmon Migration Monitoring
Unfortunately, salmon populations face serious threats from human activities like overfishing, hydropower development, habitat loss, and climate change. Therefore, effective monitoring and management of salmon fisheries are crucial for balancing ecological, cultural, and human interests. Accurately assessing salmon numbers during their migration is vital for tracking endangered populations, evaluating recovery strategies, regulating fishing seasons, and managing commercial and recreational fisheries. However, traditional monitoring methods are labor-intensive and inefficient.
Beery is spearheading a pioneering research initiative to enhance salmon monitoring through advanced computer vision techniques. This project is part of her broader focus on the intersection of artificial intelligence, nature, and sustainability—an ideal match for funding from MIT’s Abdul Latif Jameel Water and Food Systems Lab (J-WAFS). The seed grant she received in 2023 marked her first research funding since joining MIT.
Historically, monitoring salmon populations involved human observers counting fish from riverbanks—a process that has evolved to include underwater sonar systems, which utilize acoustics instead of light sensors. This method requires setting up tents near the river to analyze sonar camera data on a laptop. Despite being an upgrade, it still relies heavily on human effort, making it time-consuming.
To address this urgent need for automated monitoring, Beery emphasizes, “We need these technological tools. We can’t keep up with the demand of monitoring and understanding and studying these really complex ecosystems without some automation.”
To automate salmon counts in the Pacific Northwest, the research team, which includes PhD student Justin Kay, has been gathering video data from sonar cameras stationed at various rivers. They meticulously annotate a portion of this data to develop a computer vision system capable of detecting and counting migrating fish without human intervention. Kay explains, “The computer vision algorithm identifies a fish, encloses it in a box, and tracks it. If a fish enters the frame from one side and exits from the other, it is counted as moving upstream.” The system has yielded remarkable results, achieving counting errors between 3 to 5 percent—well below the target error threshold of 10 percent.
Testing and Deployment: Merging Human Effort with Automation
This innovative technology is currently being deployed to observe salmon migration on the newly revitalized Klamath River, where the largest dam removal project in U.S. history recently took place. After two decades of efforts from Klamath tribes, scientists, and environmental groups, four dams were removed, restoring 240 miles of river and opening up nearly 800 square miles of salmon habitat. Remarkably, Beery notes, salmon began their upriver migration within just eight days post-dam removal. The research team is collaborating with California Trout to process new data to adapt their model for counting salmon in this freshly accessible environment.
Nonetheless, training the model poses challenges as varied riverbed features, lighting conditions, and water clarity can confuse the algorithm. Counting accuracy diminishes in unfamiliar conditions, significantly increasing the margin of error to 15-20 percent. To counter this, researchers have incorporated an automatic adaptation algorithm, able to calibrate the counting model to new conditions independently. In testing, this self-initializing technology has successfully reduced the counting error to a range of 10 to 15 percent, making it closer to deployment in novel locations without heavy reliance on manual intervention.
Real-Time Management through the “Fishbox”
Another challenge the team faced was establishing an efficient data infrastructure. To operate the computer vision system, video from sonar cameras needed to be sent to the cloud or manually mailed from river sites to the lab—a process fraught with limitations. While cloud solutions suffer from limited internet access in remote areas, shipping data can cause significant delays.
To overcome these hurdles, the team developed an energy-efficient computer dubbed the “Fishbox.” This compact, lightweight device allows fishery managers to process data on-site by connecting the Fishbox to existing laptops and sonar systems. Consequently, managers gain the power to make real-time, hour-by-hour decisions, enhancing the responsive management of salmon populations.
Building Community Connections
Moreover, the research team is focused on fostering community collaboration around salmon fisheries management in the Pacific Northwest. Beery shares her enthusiasm, stating, “We’re excited to engage stakeholders who are eager to access our technology and enhance our integration with them.” Recognizing the importance of direct collaboration in the realm of food and water systems, the team organized a workshop in Seattle this past June, uniting NGOs, tribes, and fish and wildlife departments to explore automated methods for salmon monitoring. Kay called it an “awesome opportunity” for mutual sharing of ideas and practices regarding sonar use. Discussions continue through a shared Slack channel, bringing together over 50 participants from various organizations that might not have collaborated otherwise.
Looking Ahead
As the team focuses on refining their computer vision system and engaging diverse stakeholders—from Indigenous communities to fishery managers—the project stands poised to significantly enhance the efficiency and accuracy of salmon monitoring and management in the region. Beery acknowledges the importance of the J-WAFS seed grant in sustaining momentum for this impactful project, stating, “The J-WAFS seed grant has allowed us to broaden the scope of our project and maintain active collaboration on what I believe is vital work.”
This year marks the 10th anniversary of J-WAFS, which continues to support MIT faculty as they endeavor to develop innovative solutions tackling pressing global issues related to water and food systems.
Photo credit & article inspired by: Massachusetts Institute of Technology
