Mitch Aide, a tropical ecologist based in Puerto Rico, thinks we should listen to the earth a lot more than we do now — and not just listen to it, but record and store its sounds on a massive scale. His aims are not spiritual, but scientific: He, his colleagues, and other experts are developing and deploying audio recorders, data transmission systems, and new artificial intelligence software that together are rapidly expanding scientists’ ability to understand ecosystems by listening to them.
Today, Aide can nail a cheap digital audio recorder to a tree in Puerto Rico’s Luquillo Forest and transmit its recordings to a computer running prototype software, which indicates almost in real time whether any of 25 species of frogs and birds are vocalizing in the forest. The system’s apparent simplicity belies its power – Aide thinks that it and similar systems will allow scientists to monitor ecosystems in ways we can’t yet imagine.
He dreams that one day soon, audio recordings of natural soundscapes will be like rainfall and temperature data, collected from a worldwide network of permanent stations, widely available for analysis, and permanently archived. Each clip will be “like a museum specimen,” he said, “but containing many species.” Aide says scientists will be able to efficiently determine how species are moving or changing in response to global warming, habitat destruction, or human disturbance, and chart population shifts over large areas.
Audio recorders are better than cameras because they can record species over far larger areas.
A recent, steep drop in the price of recording equipment and the rapidly expanding capabilities of user-friendly artificial intelligence algorithms are heralding an era of big natural audio data. One key use of biological acoustic monitoring is tracking what is known as “defaunation,” the hard-to-detect decline of animals like birds and monkeys from habitat that appears intact — for example, animals shot and trapped by poachers in an intact forest.
To fulfill his dream of advancing biological acoustic monitoring, Aide co-founded a company, Sieve Analytics, in 2014. At the company’s small apartment office in San Juan, he recently told me that audio technology promises to push ecological science forward just as satellite imagery has dramatically increased scientists’ ability to track change in tropical forests.
“We now can find fires and logging as they happen, in vast, remote areas, from satellites,” said Aide. “We can even tell which crops are planted after the trees are cut down.” But the problem with satellite images, he says, is that “we can’t see the fauna.”
Because of this, biologists must generally fall back on traditional methods — expensive and time-consuming field surveys by highly trained specialists — to confirm animal species in an area. But the mere presence of fieldworkers can scare animals, and surveys don’t always produce primary records of species’ presence, like specimens or recordings. Cheap digital trail cameras are now routinely used in wildlife surveys to compensate for this.
The AudioMoth recording device in New Forest National Park, in the U.K., where it is searching for sounds of the New Forest cicada. Courtesy of Alex Rogers
But cameras, although useful, can only photograph what’s directly in front of them and don’t effectively detect small animals or those high up in trees. So animal declines within apparently intact habitats can go unnoticed for long periods. Disease or poaching could be wiping species out without researchers being aware of it and climate change could be forcing species out of one area and into another.
Biologists have long recognized the value of recording sound to identify animals and learn about their behavior. For example, the first recordings of marine mammals – of beluga whales in Canada in 1949 – led to an ongoing explosion of whale research. The transition from analog tape to digital recording has shrunk recorders’ size and increased their quality, helping to establish the research fields of bioacoustics and ecoacoustics over the past several decades. Scientists are learning enormous amounts about how species interact with each other and the environment through sound, such as how urban birds are evolving to sing louder and at a higher pitch than their rural counterparts.
Audio recorders are in a sense better than cameras because they can record species over far larger areas, said Aide. But the use of autonomous environmental audio recorders has been constrained because of cost. Commercial models can run $500 to $1,000 because there’s not been a large, non-specialist market for them. Manually identifying bird, frog, or insect species’ songs on a recording from an area that may host hundreds of species is also skilled, time-consuming work. This has limited most acoustic research to single species or small areas.
Recent technical breakthroughs are changing that. First, cheap audio recorders, designed specifically to monitor animals, are now available. Aide showed me a small recorder about the size of a credit card and the thickness of a pack of cigarettes, an AudioMoth. Developed by a British research group called Open Acoustic Devices and first made available in late 2017, it’s only $70, power-efficient, and open source, meaning the hardware design and associated software are freely available.
A chorus of frogs, captured by new audio recording technology, in a Puerto Rican forest. Credit: Sieve Analytics
The AudioMoth can record everything from extremely low-frequency gunshots to extremely high-frequency bat vocalizations that are far beyond the range of human hearing. Users can build their own weatherproof enclosures or, if they’re prepared to accept a slight loss in audio quality, simply place the device in a disposable ziploc bag and tie it to a tree. About 9,000 AudioMoths have already been sold, sales divided about equally between researchers, conservation organizations, and private individuals. A miniaturized version, the μMoth, has just been announced; it weighs a mere 5 grams and could be mounted on a living bird.
AudioMoths have been used to map the foraging habitat of the Cuban greater funnel-eared bat, an endemic species that roosts only in a single cave, and are being used to search for the New Forest cicada, an insect that’s thought to be extinct in Britain. (The cicada has not been heard there since 2000.) It’s also been used to detect poachers’ gunshots in jungles in Belize.
Aide sees the AudioMoth as a huge boost for acoustics research around the world, especially for developing-world researchers on small budgets. Recorders could ideally be like modern weather stations, transmitting data in near-real time for immediate analysis. He’s now working with a group of Costa Rican electronic engineers to make a cheap transmitter that uses cellphone data networks.
Aide said the technology is now sufficiently affordable and mature to roll out a large, global acoustic monitoring program. His rough estimate is that a five-year monitoring program in 200 ecoregions around the world will cost about $10 million — a fraction of the cost of old-school faunal surveys over that scale and time.
Computer algorithms can rapidly analyze thousands of hours of audio recordings to identify specific species’ calls.
The second big challenge standing in the way of large-scale bioacoustics monitoring, that of analysis, is also being solved. In recent years, Aide and his colleagues have developed two user-friendly algorithms (available via an online portal called ARBIMON) that can rapidly analyze thousands of hours of audio recordings to identify specific species’ calls. However, these algorithms, although useful for certain types of research such as analyzing how a frog species’ breeding seasons are changing with climate, can’t do the “holy grail” work of simultaneously identifying all the species in a recording.
To this end Aide’s company has begun a collaboration with Microsoft’s AI [Artificial Intelligence] for Earth initiative, and they’ve built a prototype convolutional neural network (CNN) — a type of deep learning algorithm — that can already identify 25 species of Puerto Rican frogs and birds simultaneously to a very high degree of accuracy. The CNN needs to be trained to identify additional species by supplying it with numerous examples of each species’ songs. Crucially, the CNN is very user-friendly. “You don’t have to be a computer nerd or a programmer to use it,” said Aide.
Tracking multiple species’ range expansions or contractions via a grid of listening stations could help scientists understand how environmental change disrupts the ecological connections among species and better plan nature preserves in a warmer future. By monitoring a decline in frog calls, listening stations could track the spread of diseases like the deadly amphibian ailment chytridiomycosis. Poaching could be tracked by listening for gunshots, animal alarm calls, and human voices. Listening stations could figure out how species’ reproductive patterns change in response to weather and climate by locating and identifying breeding calls and songs.
A 20-second spectrogram, showing various audio frequencies, from Puerto Rico includes the calls of these six species. Courtesy of Sieve Analytics
Scientists could also learn how human-generated sound and natural sounds interact to shape the ecosystem — for example, do certain birds or animals quit calling and move away as human noise increases? If certain frequencies are silent, perhaps the animals that use those frequencies – like flying insects or bats – are in trouble.
There are still significant challenges to fully automating bird song recognition, according to Tom Stephenson, one of the architects of a cellphone app called Bird Genie that works like a “Shazam for birds” — a reference to the smartphone app that listens to songs and identifies them. Bird songs are often less stereotypical than speech or music. Numerous species have many more than one song each, there are local dialects of those songs, and individual birds often sing them in their own individual ways. There is a shortage of reliable recordings to train the algorithms. Birds often sing over each other, too: Picking species out of a noisy dawn chorus is challenging for in-situ human experts, and often impossible for current computer algorithms.
But James Greenfield, vice president of Amazon’s Elastic Compute Cloud (EC2) — and who is not involved with this bioacoustic research — said that “the rate of progress of AI is mindblowing” and that “it’s like we’re going through a Cambrian [evolutionary] explosion of these technologies.” He added that “roughly every month someone does something with AI that was thought to be impossible,” and that automatically identifying large numbers of species within vast numbers of audio recordings will doubtless be possible sooner rather than later.
Australian researchers are installing 100 sets of recorders in different ecosystems across the country.
Anticipating a future where important things will be learned from environmental audio recordings, university researchers in Australia have formed the Australian Acoustic Observatory (A2O) and are currently installing 100 sets of recorders in different ecosystems across that country, each set consisting of four recorders. The sites have been chosen to cover a wide range of habitat types, including natural and agricultural areas, and each quartet of recorders is placed so that two are in wet or riverine areas and two are in nearby drier areas. The solar-powered machines continuously record the soundscape within the frequency range of human hearing, and the recordings are physically collected from each device once or twice a year and then uploaded to cloud servers.
The aim is to run recorders at these sites “forever,” said David Watson of Charles Sturt University, one of the A2O’s chief investigator managers. One of the obvious uses of the recorders is to monitor the effects of weather and climate change. The data will be openly available not just to scientists, but to artists, school children, and the general public.
Sieve Analytics’ Aide emphasized that archived soundscape recordings could be used for purposes we haven’t yet thought of. Watson agrees: “The beauty is [that] a kid who hasn’t been born yet will be writing algorithms that’ll be used to analyze historic data that we’re collecting and storing right now.”
