AI will protect seals and beluga whales

In the cold waters of Alaska, artificial intelligence (AI) helps researchers rescue endangered animals. Scientists are teaming up with environmentalists and the local population to fight for the survival of whales and seals amid rising shipping and expanding port construction.

When Erin Moreland just dreamed of becoming a zoologist, she imagined how long she would sit on the rocks, making sketches of seals and other marine animals. This was supposed to help her better understand their life and protect their environment.

Instead, she sat in front of a computer screen all the time, forced to view thousands of aerial photographs of sea ice in an attempt to find signs of life in the waters of Alaska. A huge amount of time was spent with her and her team to classify the received data, so by the time of publication this information was already outdated.

“There must be some more efficient way of working with information,” she recalls her thoughts at that moment. - Scientists should be freed from this routine so that they can focus on studying animals and the challenges their populations may face. The need to spend a huge amount of time on such labor-intensive tasks makes it difficult to do the main thing. ”


Erin Moreland. NOAA Photos

A more effective way was found. The idea was born in a completely unexpected way - it appeared due to the view from the window of the Seattle office of Moreland and its responsibilities in a jury. This spring, she, along with other scientists from the National Oceanic and Atmospheric Administration (NOAA), will use artificial intelligence to monitor the endangered species of beluga whales, walruses, polar bears and other animals. Now it will save years of work on the analysis of information and focus on ensuring that this information falls into the right hands to protect animals.

Research teams train AI tools to distinguish seals from rocks and whale whistles from machine noise. This will provide an opportunity to understand the behavior of marine mammals, which means it will increase their chance of survival among the melting ice and the increasing activity of people.

Moreland’s project combines AI technology, advanced cameras on NOAA turboprops flying around the Beaufort Sea in northern Alaska in April and May, scanning and classification of photographs to count individuals in seal and polar bear populations. Now the data can be adjusted within a few hours, rather than months as before. Her colleague Manuel Castellot, an associate at NOAA, will be able to apply similar algorithms to records from equipment scattered across the bottom of the Cook Bay. This will help to quickly understand how the declining population of beluga whales spent the winter.

Data will be validated by scientists, statistically analyzed, and sent to specialists such as John Courland, Assistant Regional Administrator of NOAA Alaska's Resource Protection.


The scientist Manuel Castellot (right) goes to Cook Bay in Alaska every spring and autumn to collect microphones from the bottom of the sea. With their team, they were the first to test the equipment, adjusting it accordingly so that the microphone would float to the surface. Then they take him aboard, download data from him and send him back to the bottom of the ocean, where the microphone will listen for the next six months. (Photo by Daniela Huson with Ocean Conservation Research).

Courland's Juneau office oversees marine mammal conservation and restoration programs throughout the state and in surrounding waters. It also helps instruct federal agencies that issue various permits or take any action that could affect the livelihoods of endangered mammals.

Of the four species of seals that exist in the Bering Sea - lahtak, ringed seal, spotted seal (larga) and striped seal (lionfish) - the first two are species that are declining in number, that is, in the near future they can fall into the category of animals under the threat of extinction. The whale whale that lives in the Cook Strait is already an endangered species; its population totals only 279 individuals, while 30 years ago there were about a thousand of them.

The whale whale lives in separate groups that are isolated from each other; these groups do not interbreed and do not leave their habitats. Therefore, as Castellot warns, the chance that another population will later replace the extinct is void. “Other belugas will not survive there, because they are not familiar with the area. We can lose this biodiversity forever, ”he says.

The Kurland Office has developed recommendations to reduce the negative impact of human activity on populations, for example, building up territories or transport activity. They consist in the refusal to carry out such work in certain places during the periods of crossing these mammals and nursing offspring, but these requirements often lack timely data.

“We sometimes don’t even have basic information, so obtaining such data will give us a much clearer picture of the possible types of responses to protect these populations,” said Kurland. “In both cases, both with whales and seals, this kind of information is represented by the most advanced science, which is able to fill the gaps in our knowledge.”


Erin Moreland carried out her first exploration in 2007, flying around in a helicopter from an icebreaker. Scientists collected 90,000 photographs and spent months deciphering them just to get pictures of 200 seals. It was an extremely tedious and laborious task, giving inaccurate data. (Photo courtesy of NOAA).

The AI ​​project began many years ago when Moreland sat in front of a computer screen at the NOAA Seattle Marine Mammal Laboratory and across Lake Washington, contemplated Microsoft headquarters in Redmond, Washington. She felt that there must be some technological solution to her problem, but she did not know who to turn to for help.

Good luck came when as a member of the jury in 2018, she accidentally heard two jury judges talking about AI. She told them about her work and thus came out to Dan Morris from Microsoft AI for Earth. He suggested putting this task on the summer hackathon, when for a week software developers and engineers compete with teams working on specific projects. Fourteen Microsoft engineers have signed up to work on this issue.

“In the field of wildlife conservation, thousands of scientists are engaged in a routine, looking at pictures and listening to recordings,” Morris said. - Remote devices allow you to collect a variety of data, but scientists still have to puzzle over how to use them. The years spent annotating the images are not only a waste of time, but also a matter of losing the relevance of this data. ”

Moreland’s idea turned out to be more complicated than it first appeared. While there are many models for recognizing people in the pictures, none of them could find seals, especially when real-time aerial photography. Nevertheless, hundreds of thousands of samples that have been classified by NOAA experts previously helped train the model to recognize which photos and audio recordings relate to mammals.

“Part of the problem was that there were already 20 terabytes of accumulated ice photograph data and it was not possible to work with such an array on a laptop,” says Morris. “We transferred the hard drives daily to Seattle and Redmond.” But the cloud allowed us to work with all this array at once and train AI models. So with the help of Azure technology, we coped with the data problem. ”


Can you distinguish on this aerial photograph of seals (left)? Look at the thermal image (on the right) and then go back to the first photo - do you see them now? AI helps NOAA scientists sort out the images (photo provided by NOAA, based on observations in Kotzebue Bay on the Alaskan coast during ice melting, which forces seals to group together more than usual).

The first helicopter reconnaissance for seals was carried out by helicopter in 2007. Scientists have collected about 90,000 images and spent months scanning them to discover only 200 seals. The process was very laborious, and the results were inaccurate.

Seals live apart, which makes them difficult to recognize in the pictures. Conducting reconnaissance is also not an easy task, since the plane must fly high enough so that the animals do not get scared and dive into the water, but on the other hand, low enough to get high-resolution shots that allow scientists, for example, to distinguish ringed seals from common seals. Alaska's rainy and cloudy weather makes data collection even more difficult.

Subsequent observations were significantly improved by the simultaneous use of thermal and color cameras on modified aircraft at higher altitudes. But even in this situation, the influence of dirty ice and reflections from it made it difficult to distinguish individual animals in the picture.

Another problem was human resources. As a result of monitoring in 2016, a million pairs of thermal and color images were obtained, which the previous software reduced to 316,000 active sites. Scientists had to manually sort them and classify them. For three employees working on this, it took six months.


The striped seal, or lionfish, rarely comes ashore. When the summer melting of ice begins, they float into the Arctic Ocean until an ice field forms again in autumn (photo provided by NOAA).


Lahtaks get their livelihood on the bottom of the sea, and deposits of iron stain their faces (photo provided by NOAA).


The ringed seal builds ice caves to hide its cubs from polar bears; access to such shelters is possible only from the water (photo provided by NOAA).


Larga, or the motley seal, is similar to an island seal, but unlike the latter, they produce offspring and spend time molting on floating ice floes, and not on the ground (photo provided by NOAA).

When Moreland shared with her colleagues information about the capabilities of the project to use AI to solve its problems, it turned out that Castellot faced similar problems.


, . , : , , . , : , -. ( Daniela Huson with Ocean Conservation Research).

Every spring and autumn, Castellot flies from Seattle to Anchorage, takes a boat to the sea, takes out microphones mounted at 15 points on the bottom of Cook Bay, unloads data from them and throws this equipment back into the sea. For the rest of the year, they and the team spend trying to classify each sound, determining which whistles and screams belong to belugas, which humpback whales, and which killer whales, whether the roar came from a flying airplane or from a sailing ship, and crackling from breaking ice or from construction work.

Naturally, there is almost no time left for the analysis and decoding of whale communication itself.
Whales are guided by sound, using echolocation to move, especially in Cook Bay, where most of the year is dark, and melting glaciers make the water even more cloudy. The sound, which is amplified several times under water, can disorient animals and lead to an inability to determine where the bottom is, to follow the flock, to catch prey or to hear a predator, for example, a killer whale. If the cub does not hear the clicks and whistles emitted by the mother, he may lag behind and die.

“Sounds resulting from human activities mask the key signals that whales use to search for food or each other. It also negatively affects reproduction, and if they cannot eat, mate and produce offspring, then it is impossible to increase the population, ”says Castellot.

The problem is that Anchorage is an important hub for cargo ships, as well as military and commercial flights. Port expansion also means driving into the bottom of piles for new docks. There are now seventeen ring oil pipelines in Cook Bay; In addition, glacier exits require cleaning so that the shipping channels are operational. The noise drowns out whistles and screams of whales.


Belugas are sometimes called sea canaries because of the large vocal range of the sounds they make. (Photo by David Merron Photography / Getty Images).

In collaboration with Microsoft, Castellot managed to develop a reliable set of algorithms that showed 99% correspondence to manual classification in record logs. The group is developing a new technology that should make this process even more efficient.
As climate change allows human activities to expand farther north, according to Castellot, such work will be necessary to implement protective measures for other whale populations that are also likely to encounter noise pollution in the coming decades.

“Cook's Belugas make up a very small population, which is concentrated close to where people live in Alaska. Therefore, activities related to development and development can pose a great threat to them, says Kurland. “Seals live in wider open spaces where the impact of human activity is less, so climate change poses the main danger for them, which, of course, is more difficult to reduce.” 

“The studied species of seals feed and care for their offspring on ice floes, where they spend molting every spring there, so these animals are highly dependent on the state of the ice. Lionfish, or striped seals, rarely go ashore at all, and when ice melts in summer, they swim into the ocean until ice forms again in autumn. Thus, global warming has the most negative impact on the Arctic and subarctic regions, ”says Morris.

In her opinion, an analysis of data on the seal population and its distribution will make it possible to take the first step in understanding how they are trying to cope with environmental changes and what they need to survive.


The whale whale lives only in the Arctic Ocean and nearby seas, for example, in the Bering Sea (photo by David Merron Photography / Getty Images).

“We live in a time of change, and we need all our strength to protect our natural resources,” says Tom Gray, a representative of a local tribe that catches beluga whales in the eastern Bering Sea through nets. With his knowledge and experience, he provided substantial assistance to the Castellot team in catching whales in Cook's Bay and installing suction cup sensors on them.

When Gray was small and lived in Nome (Alaska), he said, “there were no ships sailing past us in the Northwest Passage. “Hundreds of ships are passing by in a year, and most likely there will be thousands in the near future.”

“In Alaska, we have beluga whale, moose, Canadian deer, all these unique animals that we often talk about. But I'm not sure that our people have everything necessary to maintain the health of the populations of these animals. Now we are losing the battle, and it is likely that these species will disappear forever. We need scientists and innovators to protect these animals and prevent them from disappearing from the face of the earth. ”