wildflow

[PROTOCOL] How to 3D-map Coral Reefs with just a few GoPros ???? A simple, low-cost way of creating permanent high-res snapshot of coral reefs for divers using 3D photogrammetry. ?? We were recently mapping coral reefs in East Java and I was running a mini photogrammetry workshop. A lot of you friends were reaching out about how to capture a 3D model of a coral reef so that it could be analysed and guide conservation/restoration decisions. ?? https://lnkd.in/ejq_bHwM -- here is a step-by-step tutorial explaining how to map the reef using relatively cheap equipment! ?? ?? Why? Coral reefs are so precious, beautiful, incredibly complex and threatened ecosystems. We need to understand deeply how coral reefs function, what practices work, what don't, to coordinate precise actions to protect and restore them. Coral reefs are the key to protecting other ecosystems! That's why we doing it. ?? Sadly, In 84% of the restoration cases in Indonesia [1] people never come back to monitor their reefs. In many cases those corals would likely just die (same in other countries). The hard truth is that restoration without N years of monitoring is basically a money laundry. But we can’t blame people for that! One of the problems is the huge monitoring costs, it’s just not accessible to everyone. Once monitoring is cheap, a lot of money would be channeled towards projects that work. The whole process would be more streamlined. ?? There's a solution! There are thousands of scuba diving centers in the world. Almost any dive center or its visitors have an affordable camera like GoPro. It’s fairly easy to swim with a few GoPros around the reef every 6 months or a year. ?? The tech has moved forward very far! Why 3D Photogrammetry? It allows monitoring hectares of coral reefs in centimetre resolution: identifying corals, benthic cover (rock, sand), detecting diseases and their spread. We can track coral demography through time, calculate growth rates, carbonate budget, and more. It's not a silver bullet, but it's an awesome tool! ?? Fusing 3D data with other modalities like acoustics, genomics and environmental data (water temperature, salinity, dissolved oxygen etc.) unlocks a comprehensive understanding of coral ecosystem dynamics, and how we can protect and restore the reefs. ?? There's already a lot of organisations using this. You should give it a try as well! Watching your reef come to life in 3D is incredibly rewarding. You’ll see every coral, every nook, and every cranny. Remember, you’re not just collecting data, you’re creating a digital twin of the reef that can be studied, shared, and preserved for years to come! ?? Huge thanks to John Stratford, Jason Lynch, William Barnes, Ben Williams and other amazing people for sharing the best practices on data collection! ?? We could become the first generation that leaves nature behind better than we found it, if we work together! ????

I just had the most EPIC couple of weeks in East Java 3D-mapping coral reefs and building a spawning lab for local communities! Bloody hell, the problems here are insane - plastic pollution, corruption, blast fishing... It's absolutely overwhelming. ?? BUT I’m hopeful and very excited about the future! The people I worked with are just incredible - the kindest, smartest, hardest working humans I've met. I'm completely in love with Indonesia and had to share it with you!! ?????? ?? Why bother? Coral reefs are so precious, beautiful, incredibly complex and threatened ecosystems, supporting over a billion people and a quarter of marine life. They are dying fast.. But there are ways to protect and restore them! A lot of amazing people are fighting for their life. Coral reefs are the key to protect and restore other ecosystems on Earth. That’s why we are doing it! ?? Epic Quest (1): Coral Spawning Lab - Prof Michael Sweet, Dr Jamie Craggs and our team set up this incredible lab focused on large polyp stony corals. We're basically coral matchmakers, helping them reproduce in controlled conditions! The coolest part? It's self-funding - the local community sells some corals to aquarium hobbyists which pays for the lab, while most go back to restore damaged reefs. Genius, right?! And now Leigh Peters would be scaling this circular economy globally via his PhD. There's way more to it! ?? Epic Quest (2): Mapping the Reefs - would you believe 84% of restoration projects in Indonesia NEVER monitor their reefs after planting because it's so expensive?! We solving this using regular GoPros, photogrammetry and new tech like 3D Gaussian Splatting to create large and insanely detailed 3D models. Game-changer!! We're building this global digital twin of coral reef ecosystems (basically "Google Maps for Coral Reefs") to make quality decisions on how to protect/restore the reefs. 3D is only the first step. ?? We mapped 13 25×25m coral sections and we're open-sourcing ALL of it: raw images, colour-corrected images, 3D reconstruction, Gaussian Splatting models! Anyone can train their AI on our data once we finish processing. Just imagine researchers worldwide having access to this and training AI for species/genuses classification, disease detection, etc! What you see on the video is a tiny sneak peek, without colour correction. You can fly over all the reefs on your laptop on wildflow in the next couple of weeks! \o/ ?? Triona Barker collected eDNA for these sites, so we can make even better decisions. ?? This wouldn't happen without Mujiyanto Mujiyanto, Raymon Rahmanov zedta, Dr. Ofri Johan, M.Sc, Dr. Rita and many other incredible people in the team who worked day and night! ?? BILLIONS of people depend on our ability to protect these ecosystems. There are NO laws of physics saying we can't fix this mess! We can be the first generation that leaves nature BETTER than we found it. Can't wait to tell you all the crazy stories when I'm back! ????

Just wrapped up my 7th conference of the year at ReefFutures! ?? So much joy, inspiration, brilliant and kind people working on protecting and restoring our oceans. Grateful for every selfie, every conversation, and every shared vision! 2024 has been a wild ride, and the best is yet to come! ?? ?? #ReefFutures2024

3D Coral Reef + At Night + Fluorescence -- could you help me with pics? ????? ?? We create photorealistic 3D models of coral reefs that work in the browser using 3D Gaussian Splatting (https://lnkd.in/ebjkWWCc). ?? Coral reefs look wild at night! Would be so much fun to create a 3D model of coral reef using fluorescence! ?? Could anyone help me with that in the next few weeks? We just need multiple pictures from different angles. ?? This truly incredible shot is by Adriana Basques, check out her other photos!

Let's gooo!! ?? Try our 3D "Street View for Coral Reefs" prototype 2.0! ?? We did it! Today we've hit a new milestone! It's been a long journey, but we're one step closer.. We have a 50x times larger 3D Gaussian Splatting model now, covering over 1000 sq meters of corals. So stoked to tell you \o/ ?? Why? Coral Reefs are so precious, beautiful, incredibly complex and threatened ecosystems. They are dying fast.. But there's a way to protect and restore them! A lot of amazing people fighting for their life. We need to understand deeply how Coral Reefs function, what methods work, what don't, to coordinate precise action. They are a key to protect and restore other ecosystems. That's why we're building a digital twin of coral reef ecosystems -- our first product. ?? https://lnkd.in/esYyuupw -- today you can try our second scrappy prototype (20% done) -- works best from computer. ?? What you see: it's a 3D "Street View" for Coral Reefs. Someone swam around a reef with a few GoPros, and you can now see high-res photorealistic 3D model of coral reef in the browser! It's a super cheap way of monitoring. Soon you will fly over square kilometres of reefs and see them in centimetre resolution. You can compare reefs in different countries and see how they evolve over time, easy way of getting rid of biases in data. ?? The prototype is terrible. Still work in progress (20%), just another sneak peek. We're moving fast and testing one hypothesis after another. Right now we still don't have a smooth progressive loading, there are some wacky splats here and there. ?? But it's already much better. This time we used 8382 images from GoPro 10 (instead of 70) and trained over 25x50m model (last time it was about 5x5 m). We respect correct geometry from Metashape (classical photogrammetry tools). Centimetre resolution! I think it's still the best in the world quality you can find. ?? Next steps: 3D time-series data (see how coral used to look like 3 months ago), 3D segmentation models, classifying coral species, adding other modalities (acoustics, eDNA, geospatial)... Allowing anyone to run analytics against all our data... Foundation models for biodiversity... Becoming the first generation that actually leaves behind nature better than we found it! ?? ?? This version of a prototype wouldn't happen without these incredible people: - Ben Williams -- it's all Ben's fault I'm doing this now, his idea and support - Jason Lynch -- Metashape/photogrammetry support - Greg Tkachenko -- powerful compute cluster - Rindah Talitha Vida, Tim Lamont, Mars BuildingCoral team -- collecting the data, doing scientific research and restoring this reef you see - and many many more fantastic people! We've got terabytes of data from multiple orgs, now scaling the compute and processes. Soon you can play with more and more coral reefs! What a time to be alive! \o/

Coral Reef Tetris! ?? ?? ?? ?? Would you play this game? ?????? Pivoting wildflow into a gaming company now haha! ???Coral reefs are incredible ecosystems! While being the size of New Zealand they are supporting 25% of marine life and at least half a billion people! Lot's of amazing people fighting to protect their lives. ???We've got terabytes of 3D coral reef data (images from GoPros) from a bunch of different locations in the world. And building 3D "Street View" for coral reefs, so that anyone could see them and scientists could run analytics against all that data. ???Current challenge is how do you even visualise it? We use new 3D Gaussian Splatting techniques. Last weekend we trained 50x25 meters photorealistic model of coral reefs in centimetre resolution. Respecting all the correct geometry from photogrammetry. Now we need to visualise it in the browser. Splats are great, they are 10-30 times smaller than point clouds or meshes. But it's 36GB of 3D Gaussian Splats still! ???What you see here is a debug mode of me implementing a custom 3D visualisation engine, based on Three.js + WebGL + mkkellogg/GaussianSplats3D. Effectively we need progressive loading of data, so that we load only everything that's close to the camera, not all the data at once. ???This is called LOD (levels of details). E.g. everything within 10 meters is highest level of details, then everything within 30 meters is medium level of details. And if it's further away, it's the lowest resolution. Basically same way as Google Maps/Earth or different games do. Usually using Octree or similar data structures. ???It's so much fun working on it, and playing with it. I'm so sad the final users would never see these circles, because it's a debug mode. Maybe I should make a game out of it? WDYT?

wildflow coral -- our first product, a comprehensive digital twin of coral reef ecosystems based on multimodal foundation models for biodiversity. ?? Why??Billions of years of evolution created Earth's vibrant ecosystems -- they are inconceivably rich and incredibly complex! We have so much to learn from this beautiful complexity. We want to build a future where humans and nature thrive and enrich each other. That’s why we chose to start with coral reefs. They are the most complex, beautiful, endangered, and important ecosystem. They are home to 25% of marine life while occupying only 0.1% of the ocean. Sadly, we’re on track to lose 90% of coral reefs by 2050. The time to act is now. Over half a billion people rely on coral reefs. Already vulnerable communities are the first to suffer the consequences. We have to make a lot of critical decisions about them! They are a key to model entire biosphere (protect nature) down the road. ?? How??We take all the data across all modalities, such as 3D photogrammetry, bioacoustics, underwater videos, remote sensing, eDNA, environmental data like currents, and more, to create the ultimate digital twin of any coral reef ecosystem and make it available to the world. And do it at a planetary scale. Core tech is multimodal foundation models for biodiversity. ?? What? This enables deep modelling of complex ecosystem dynamics, such as population dynamics, predator-prey dynamics, energy transfer, phenological events (like spawning), and computes the ecosystem's health and resilience metrics. Via rigorous analysis of multimodal data, it uncovers precise mechanisms driving ecosystem change, offering humans irrefutable evidence to steer their actions. It shows quantitatively how coastal development, agriculture runoff, invasive species, pollution, increasing water temperatures, and other pressures affect the coral reef ecosystem. It shows what the coral reef ecosystem gives us back through its services, such as coastal area protection and oxygen production. It coordinates conservation and restoration efforts worldwide. We know which practises work and which don’t. It guides human activities to understand, protect and restore coral reef ecosystems. ?? Next steps: ? ? 3D "Street View" for coral reefs prototype [DONE] ? 3D digital twin of coral reefs ??multimodal twin of coral reefs ??model ocean ecosystems ??model biosphere ??become the first generation that actually leaves behind nature better than we found it! ?? ??

?? Coral Reefs are some of the most critical ecosystems in our oceans, yet they're disappearing at an alarming rate. I spoke to wildflow CEO, Sergei Nozdrenkov about their use of Radiance Fields to create a "3D Street View" for Coral Reefs. This represents the beginning of both visualization, but also larger deep (no pun intended) learning about our oceans and the environments that drive them. And yes, Radiance Fields do work underwater. Join me as we continue to examine the real world applications of Radiance Field based technologies and the people building with them. Full article here ?? https://lnkd.in/dU_XJqUm

Sneak peek into the new larger 3D "Street View for Coral Reefs" model: - ? geometry is not good - ? no cleanup of wacky splats - ? used 200 photos from 2 GoPros (almost one swim-through) - ? 5 x 7 sq metres --> 15 x 3 sq metres (link in the comment)