Earth Species Project

How can artificial intelligence help us listen to nature? At the Dubai Future Forum next week, our Director of Impact Jane Lawton will join global thought leaders for a panel discussion on the ways AI is transforming our ability to connect with the natural world. Joining Jane are: ?? Koert van Mensvoort, Founder & CEO, Next Nature ?? Abdelrahman AlMahmoud, UAE Office on AI ??? Moderated by Ariel Muller Ariel Muller, Global Director of Futures, Forum for the Future Together, they’ll unpack the exciting possibilities of technology as a bridge between humanity and the ecosystems we depend on.

We are excited to introduce NatureLM-audio, the first large audio-language model designed specifically for animal sounds. Check out the preprint in arXiv here: https://lnkd.in/g-hm-mxb And our demo: https://lnkd.in/gVazhfqa Built from millions of recordings in bioacoustic archives and enriched with speech and music data, NatureLM-audio begins to addresses some of the greatest challenges in bioacoustics by making it easier to understand the complex vocalizations of animals across the tree of life—from birds to marine mammals. ?? Zero-shot classification and detection: NatureLM-audio can classify animal sounds across thousands of species without additional training, delivering “out-of-the-box” results that enable broader and faster animal communication research and biodiversity assessments. ?? Generalizing to unseen species: It can identify species it's never encountered before based on genus patterns in scientific names—addressing critical data scarcity issues in bioacoustics. ?? Natural Language Prompts: NatureLM-audio works with audio and natural text prompts and generates natural language responses. ?? NatureLM is generalizable beyond standard tasks: - Predicting life stages of birds (e.g., chicks vs. juveniles) - Classifying call types (calls, songs) across species - Captioning bioacoustic audio to provide rich, descriptive details on animal sounds This is just the beginning but we're excited about these early results and what they mean for the study of bioacoustics and animal communication. We'll be releasing the code and an intuitive UI soon, as well as working on incorporating new forms of data (video, motion, etc.). Big congrats to our research team and especially authors Masato Hagiwara Marius Miron David Robinson and Olivier Pietquin!

?? This Sunday at #SMM2024, David Robinson, Senior AI Research Scientist, will be presenting during a workshop on AI and Marine Mammal Science. If you’re attending, come join us! cc: Society for Marine Mammalogy #MarineMammalResearch #AI #SMM2024 #MachineLearning #EarthSpecies

?? Exciting News! ?? We just secured $17M in new grant funding to fuel our mission of decoding animal communication with frontier AI. ???? A massive thank you to Reid Hoffman and our friends at Waverley Street Foundation for their incredible support. This funding will supercharge our work in developing multimodal foundation models for animal language processing. It will also significantly further the impact our work will have in the world. We believe understanding the communication of other species can transform how we address today’s biggest challenges, including climate change and biodiversity loss. Read more in @Forbes: https://lnkd.in/gYPAxCZj Big HT to Phoebe Liu for the story! #AI #AnimalCommunication #EarthSpeciesProject #ClimateChange #FrontierAI #Biodiversity

“We humans have long tried, often mistakenly, to differentiate ourselves from nonhuman animals—by arguing that only we have souls, or use tools, or are capable of self-awareness. Perhaps we should see what the birds have to say.” Rivka Galchen at The New Yorker featured Earth Species Project in a beautiful article examining how researchers and scientists are working to unravel the complexities of bird vocalizations. Our senior AI research scientist Maddie Cusimano, along with co-founder and CEO Katherine Zacarian, discussed our efforts to develop a model capable of distinguishing the sounds produced by different crows and relating those sounds to behaviors… and eventually, meaning.? We, along with many other researchers cited in the piece, are getting closer and closer to uncovering a previously hidden language that surrounds us every day. ?? Read the piece here:?https://lnkd.in/gzPHhFTm

Senior AI Researcher Masato Hagiwara will be at COLM 2024 (https://colmweb.org/) in Philadelphia next week and eager to learn from and connect with fellow language modeling experts. If you're curious about how AI can help decode animal communication and want to learn more about our work in Animal Language Processing, look for us in Philly!

"The Earth Species Project, to me, has been really exciting because it is potentially one of these world changing moonshots in how we understand who we are in our relationship to the world." – Sean White ?? Earth Species Project was mentioned in the latest episode of Possible, featuring our board member Sean White, CEO of Inflection AI, in a thought-provoking conversation with another incredible ESP supporter Reid Hoffman and Aria Finger. ??? Listen here: https://lnkd.in/grGVNc9Y #AI #AnimalCommunication #EarthSpeciesProject

We're excited to chosen as a 2024-25 Tech for Good Global Laureate by The Tech Interactive! Can't wait to celebrate alongside other leaders leveraging AI and robotics for a better world.

Aza Raskin speaks with Hannah Fry on Bloomberg Originals about building multimodal foundation models to decode the communication signals of our fellow species. Our work is accelerating animal communication research and helping ethologists to uncover patterns in massive multimodal data sets that were previously undetectable. https://lnkd.in/gejXzhWs

?? Next week, our CEO Katherine Zacarian will be attending Climate Week in New York. We’re looking forward to engaging in important conversations about the role of AI in the climate and biodiversity crises and how technology can drive a renewed relationship with nature. If you’re attending, let us know—we’d love to connect! #ClimateWeekNYC #LetsCreateTheFuture #AIForGood #EarthSpeciesProject