Earthmover

???Today we are thrilled to announce the release of the Icechunk storage engine, a new open-source library and specification for the storage of multidimensional array (a.k.a. tensor) data in cloud object storage. Icechunk works together with?#Zarr, augmenting the Zarr core data model with features that enhance performance, collaboration, and safety in a multi-user cloud-computing context. With the release of Icechunk, powerful capabilities such as isolated transactions and time travel, which were previously only available to Earthmover customers via our Arraylake?platform, are now free and open source. We truly believe that Icechunk will be a game changer for the scientific data community, enabling organizations large and small to work with cloud data in a safe, robust, and high-performance manner. Read our blog post about Icechunk here! https://lnkd.in/gMuuz7JC

We're stoked to be part of the #GeospatialRiskSummit! Join us in NYC on January 29th for Ryan's tech demo and the conference on the 30th. We're excited to meet and learn from geospatial risk management practitioners across the private sector.

Last week Cecil and Earthmover co-hosted the second Data & Dumplings NYC. We had over 140 registrations, and the room was packed with data scientists, data analysts, software engineers, climate scientists, and product builders. I'm grateful for Mansi Shah from Cecil and Ryan + Ilana from Earthmover for helping make this event a success. Also, for those who returned for a second time including Samuel Guttell, Yuri Shlyakhter, and more. In the future we want to find amazing guest speakers, and provide opportunities for others to present their work to this growing community. Jed Sundwall, Karl Burkart, or Somil Aggarwal is there anyone in your network who we should have present?

I'm excited to be speaking at the Cloud-Native Geospatial Forum (CNG) Virtual Conference today! Catch my talk about "Icechunk: Open-Source, Cloud-Native Transactional Tensor Storage Engine" at 2:05pm ET. https://lnkd.in/gr_38PUi

Come learn about recent Xarray GroupBy improvements at tomorrow's (Wed, Nov 13) Pangeo Showcase! Details here: https://lnkd.in/dA_iKhAy

Read our latest newsletter for a sneak peek of our upcoming release, Query Service, which allows users to access data in Arraylake via a range of standards-compliant web services, including EDR, WMS, and DAP/OPenDAP.?

I'm proud and excited to join the Cloud-Native Geospatial Forum (CNG) Founding Editorial Board! Let me explain why I believe that cloud-native is so important for the future of science. To me, "cloud" doesn't mean AWS, or any specific set of technologies or standards. Those will come and go. Instead, "cloud" represents the opposite of the status quo. What is the status quo? A world in which individuals and organizations amass large volumes of data in a way that is only accessible to a tiny number of users. (Chelle Gentemann calls this a "Data Fortress".) This makes it effectively impossible to collaborate outside a narrow circle, which holds back the rate of discovery and innovation. "Cloud" is about creating spaces where people can come together, collaborate around data, and move science forward (a "Data Watering Hole" according to Fernando Pérez). This is why cloud and open science are so linked. I'm more convinced than ever before that global-scale collaboration--around EO data, AI, and climate solutions--will be a key part of how we confront and overcome the challenges our planet is facing. Cloud-native technologies are the tools that make this possible. Facile comparisons of the cost to store a GB of data in AWS vs. a hard drive on your desk often miss this broader point. What's the cost of failing to collaborate effectively? Join us on Wednesday at the CNG Virtual Conference to hear more about this vision from a talented and diverse group of speakers. https://lnkd.in/gVVZ7Ceh

Join Ryan Abernathey and other geospatial leaders on Wednesday for the Cloud-Native Geospatial Forum (CNG) Virtual Conference. ?? Ryan will be giving a demo of Icechunk at 2 PM that's not to be missed! Register here: https://lnkd.in/eii3b9_q

Most developers today wouldn't dream of not using version control for their code... However, the same principles can be applied to data! Earthmover's new open source project--Icechunk--includes version control features built specifically for the #Zarr data model, bringing powerful data version control to the world of massive multidimensional arrays. Features include * All updates occur atomically in isolated snapshots * Tags - immutable pointers to snapshots * Branches - mutable pointers to snapshots With Icechunk, you can safely experiment with changes to your data on a "dev" branch before propagating those changes to "main." You can publish an immutable version of your dataset (tag) while continuing to evolve towards the next version. Or you can simply revert incorrect changes back to an earlier version of your data. These capabilities make life so much easier for data scientists and teams using array data in production. I've been using data version control with Zarr for the past year via our Arraylake platform, and I'm thrilled that these capabilities are now fully open source. I can't imagine going back to the old way of working. Learn more at https://icechunk.io/

We had a blast hosting a weather & climate data happy hour in Seattle last night! Our entire team was in town and got to mix and mingle with folks from?RWE, Amazon, University of Washington, Rhodium Group, WattIQ , EPRI, Google, Pachama, CarbonPlan, NASA - National Aeronautics and Space Administration, Coiled, Development Seed and more!