Lakekeeper转发了
I know the Apache Iceberg space may feel a bit crowded now that Godzilla?? and King Kong ?? have teamed up on the same table format. I've sensed a bit of an eye rolling sentiment when more Iceberg catalogs are being announced and I just want to remind you all that this is good and exactly what consumers need. We've grown too accustomed to having choices stripped away from us that we kind of now secretly desire the lack of choice because we anticipate a monopoly or oligopoly to form and so we coast on decision making until we see a "winner". I want to encourage you all not to follow that pattern as consumers in this data space. Imagine if standards like Iceberg could replace the consistency and interoperability that we as humans need and currently seek out from corporate ecosystems. Having your cake and eating it too! Imagine if the Iceberg standard becomes as ubiquitous as SQL or TCP/IP and we mold that format to meet the needs of scalable warehouse (aka lakehouse). When Tabular was purchased, I noticed a bit of a vaccum that was left its wake. We really need a permissively licensed project that provides not just a REST catalog implementation, but one that overlaps with security frameworks like OpenFGA and OpenID Foundation to continue the mission of centralizing governance around the most interoperable table format we have to date. This will continue putting pressure on our Godzillas and King Kongs and keep the will of the consumer alive in this industry. I believe Viktor and Christian have the right vision to bring a lot of what Tabular had to open and keep building a engine and vendor neutral platform that fills the Tabular void. Check out Lakekeeper! I'll be working on improving their docs along with the Iceberg docs so that those new consumers venturing into Iceberg can make informed decisions! #dataengineering
?? Big News: Lakekeeper 0.5 is Here! ?? We're thrilled to announce that Release 0.5 is officially out—and it's our biggest release yet! ?? Packed with long-awaited features, it's a major step forward for our mission to simplify and empower Lakehouse management. Here’s what’s new: ? UI: The Lakekeeper Console is here! Host it with the Lakekeeper binary or separately—it’s baked into all pre-built Docker containers and binaries. ?? Docs: Huge shoutout to @Brian for making our documentation awesome! Whether you’re just getting started or configuring authentication, check it out at docs.lakekeeper.io. ?? Access Controls: Table-level permissions are here! Fine-grained authorization made easy. ?? Authentication: - Native support for Kubernetes Service Accounts. - Better external IdP support, now with docs for EntraID and Keycloak. ?? Hierarchical Namespaces: Organize your Iceberg Warehouse more effectively. ?? Normalized DB Model: Lakekeeper now stores TableMetadata across multiple tables instead of a big binary blob—making updates faster and opening doors for powerful future endpoints. ?? Helm Chart 0.2.0: Deploy secure setups with authentication and authorization in no time! ?? Fixes & Improvements: - Performance boosts and bug fixes. - Default port switched from 8080 → 8181. - Resolved property case limitations. ?? And keep an eye out for the Lakekeeper Kubernetes Operator (https://lnkd.in/e_jNnQG7 thanks to Peter McClonski for driving this forward! This release wouldn’t have been possible without our amazing contributors and community! ?? ? If you’re excited about Lakekeeper 0.5, show your support by giving us a star on GitHub! https://lnkd.in/etDTyF5e ?? Check out the release notes for more details, and let us know what you think! https://lnkd.in/e8qNryFK #Lakehouse #OpenSource #ApacheIceberg #DataInfrastructure #Kubernetes #Lakekeeper
Brian Оlsen, Welcome to the world of Apache licensing and open source. You create a private for profit company that is built upon an Apache project, and as that project's code becomes popular, the bigger companies come in w their flavor that ties that product to their products. (e.g. Iceberg storage and S3 or whatever...) That's a natural occurrence. There's also a bit more irony here. But I doubt many will see it.
Lakekeeper has really good foundations… it’s a project that I can see achieving escape velocity despite the major players coming into the mix because the vision is so clean!
I’ve also been watching the pundits hot takes on AWS S3 tables but have been wondering what your take is on that Brian Оlsen. Maybe for a separate post, but it’s signaling appetite of big players to implement product offerings around iceberg. Whether the implementation is great or not, it still feels positive to me. Curious your take though.
Adding OIDC support to iceberg will be huge for interoperability and security.
Product @ Databricks
3 个月IMO this is easily doable in Polaris, but the CSPs will drag their feet because they are happy to get people covertly locked in.