Why Amazon’s embrace of Iceberg is getting infrastructure software builders so excited

Welcome back to Vertex Angles, the weekly newsletter from Vertex Ventures US . We’re a boutique venture capital firm, investing in exciting companies across software infrastructure, developer tools, data, security, and vertical SaaS. If you prefer email, you can subscribe here to get Vertex Angles in your inbox every week.

This week, Megan Reynolds recaps her first-ever trip to AWS re:Invent last week — and highlights how Amazon is making waves with its embrace of Apache Iceberg.

Last week I went to AWS re:Invent for the first time, the cloud giant’s flagship conference in Las Vegas. It was as overwhelming as everyone told me it would be, and also far more rewarding. Nowhere else in the world can you find this density of talent in infrastructure software. It’s the perfect place to be, and the right crowd with which to surround yourself, when game-changing announcements that shift the cloud landscape are made.

Speaking of the right crowd, Vertex US hosted a breakfast in Vegas, bringing together the founders and execs of our portfolio companies – including LaunchDarkly , Hasura , and Gitpod – with enterprise engineering leaders. Almost all of the buzz this year was around AI, which is hitting a saturation point, judging by what we saw plastered across every large vendor booth. For its part, Amazon made big AI moves including revealing its new foundation model and unveiling plans for Trainium3 chips.

But the most surprising announcement, and the one that seemed to generate the most buzz at re:Invent, was S3 Tables.

As the name might imply, S3 Tables brings native support for the Apache Iceberg table format to S3, Amazon’s cloud object storage service. Iceberg itself is having something of a moment: Databricks recently acquired Tabular, the startup that first created the Iceberg project, for $1 billion dollars. Now, Amazon is offering very similar functionality to Databricks’ Tabular, natively in S3. Ouch?

I’ve spent the days since re:Invent ended speaking to industry leaders and smart friends to learn about why S3 Tables has everybody so excited.

In short, the consensus is that embedding Iceberg into S3 represents a major competitive move from AWS against the momentum of data platforms like Snowflake and Databricks, simplifying streaming systems and data lakes within its own offerings so that customers don’t feel the need to look elsewhere. However, AWS notably decided to launch S3 Tables without any pre-integrated partners, which means that it will take time for these other data platforms to integrate. Their end customers will feel that friction.

The hype comes in when you consider what this means at the macro scale. I’ve spoken before about the ongoing rise of BYOC, or bring-your-own-cloud, where customers are empowered to freely decide which cloud platform makes the most sense for their individual data workloads. Iceberg is a useful building block for enabling BYOC, given that it provides a consistent table format that allows different compute engines to directly read and write data in S3. This new direct access should improve performance and reduce complexity in data architectures. That’s great news for both large companies like Confluent who have been public about their challenges delivering BYOC, and for earlier stage start-ups starting to figure out their own BYOC architecture. Amazon embracing Iceberg in such a meaningful way reflects the demand we’ve been seeing in the market for more open and flexible consumption models.

What Amazon gets out of this is making AWS and S3 an even more appealing environment for AI-native enterprise application development, as it streamlines data access, lineage, and processing. On the other hand, this could be worrisome news for the new breed of infrastructure software startups. If other Amazon services like Aurora, a cloud database, start writing to S3 Tables and customers get access to similar benefits natively could this cause problems for competitive serverless databases in the market like Neon or PlanetScale?

With such a strong announcement, promising clear and significant value to developers, it begins to feel as if Amazon is entering a new era of doubling down on its strengths with core products like S3. If so, that would be a welcome change from its recent era of releasing commercial products that are essentially copy-and–pasted from popular OSS projects.

Megan is the creator of infra.community, and regularly hosts events for infrastructure software?founders and builders in New York City and San Francisco. Sign up for the mailing list here to get notified about upcoming events.

Vertex portfolio job of the week: Growth Lead at Northflank

Northflank is looking for a Growth Lead to help make its story resonate across the developer and engineering community. Northflank empowers developers with a self-service platform to build, deploy, and scale any workload on their preferred cloud infrastructure.

Apply Now

Find more open roles at Northflank here.

For more startup jobs from across the Vertex Ventures US portfolio, check out our jobs portal.