Last Week in a Byte on Delta Lake | 2022-12-13

Now you can watch or read the latest Delta Lake news a week late (2022-12-13 edition)!

Releases

• Delta Sharing 0.6.0 which now supports using a delta sharing table as a source in structured streaming
• Delta Connectors 0.6.0 which includes Column Mapping support in the Delta/PowerBI connector, and adds support for the?Flink/Delta Connector?on?Apache Flink? 1.15.3.
• Delta Spark 2.2 which supports Apache Spark 3.3 with performance features like LIMIT and aggregate pushdown into Delta scan.

Interesting Interviews

We had two great D3L2 vidcast/podcast sessions:

• D3L2: Discussing Data Integration with Airbyte and Delta Lake with Simon Sp?ti of Airbyte
• D3L2: The Genesis of Delta Rust with QP Hou on how QP started with Rust and created the Delta Rust project.

Use Cases

A great story by R. Tyler Croy, Delta Lake maintainer and Director of Platform Engineering at Scribd on how Delta Lake is the foundation of their open data platform and how Delta Lake allows them to manage, grow, and modify storage in incredibly helpful ways that benefit their users.

A great blog on how Data Sharing across Government Agencies using Delta Sharing where the authors discuss how Delta Sharing is deployed and enhanced at the?United States Citizenship and Immigration Services (USCIS)?to satisfy several inter-agency data-sharing requirements.

Linux Foundation Annual Report

In Linux Foundation’s 2022 Annual Report, there was a call out to the Delta Lake project, check it out on page 131-132 including this great quote from Florian Valeye, Delta Lake maintainer and Staff Data Engineer at Back Market.

“Contributing to open source is key to learning how to solve problems within worldwide, benevolent communities of people."

New Maintainers

We are happy to announce that Will Jones from Voltron Data and Robert Pack from BASF are new Delta Lake maintainers!

Connect with us!

Want to learn more about Delta Lake and chat with other users and contributors? Join us at?delta.io,?Slack,?LinkedIn, and?GitHub.