SDF Labs

SDF, the evolution of data transformation, is officially in Public Beta ??. We raised $9M from RTP Global, Two Sigma, and Founders Co-op From four founders obsessed with SQL to a dedicated team that feels more like family, we are building the next generation developer platform for data. What started at Meta as a solution to data at scale has grown into the best way to build a reliable and optimized data warehouse. SDF is free to use and built on an open source core. Install our CLI (https://lnkd.in/ga_sRzfn) and join our slack community! (https://lnkd.in/gpuCHKgF) Unlike other data transformation tools like DBT, SDF extracts SQL compilers from their clouds, understanding proprietary dialects of SQL (like Snowflake) so deeply that it can ultimately execute them. (See blog post in comments for more) A lot has been wrapped into SDF, and we’ve learned so much from our customers along the way. We’d like to send a huge shoutout to some incredible engineers at Patreon, Deel, ClassDojo, Cybersyn, Obie, Linqto and others at our partner F500 companies. SDF would not be here without you. Thank you.

?? Transform Your SQL Development Experience with SDF Lint & Format Hygiene, consistency, and readability in SQL are essential for scaling data teams and enabling self-serve analytics. That’s why we’re thrilled to introduce SDF Lint and Format: ?? Up to 1000X faster than the current standard SQLFluff ? Built-in support for Jinja macros and deep semantic SQL understanding ??Zero-cost linting in daily development and CI/CD (because it’s so fast!) ?? Designed for developer productivity with sensible defaults and easy configuration Our Rust-powered SQL Linter and Formatter brings industry-leading speed and accuracy to SQL workflows, making it easier than ever to enforce standards and maintain clean, consistent code. Ready to take your SQL workflows to the next level? ?? Check out our blog post in the comment below to learn more and get started.

Want to level up your data platform knowledge over your morning coffee? ? Join SDF this Friday 9 am PT at Dagster Labs's Data Platform Week. You won't want to miss these insider insights into the tools and processes shaping the data landscape on this star-studded panel with Sigma Computing, Dagster, and SDF. Link to register in the comments below. ?

Sharing a sneak peak of something we’ve been working on: A (very) high performance, dialect aware SQL linter that works with jinja, is embedded in SDF, and compatible with DBT projects. It's up to 2000X faster than SQLFluff and of course, written in Rust ?? Highlights: - Native integration in the SDF cli - Compatibility with DBT projects - Extensive auto-fix capabilities for basic linting issues - A similarly high performance SQL formatter - Almost 1:1 mapping with SQLFluff ruleset for easy onboarding Notes about video - It’s not sped up (I promise!) - SQLFluff est 2+hrs to lint the same directory - The ~16k SQL files contain 634k lines of SQL code - No crazy hardware, just my daily workstation (32 cores, 96gb mem) SDF’s linter will initially have Snowflake support only, with other dialects coming soon. Stay tuned for an official release date.

Our webinar on Getting Started with SDF Labs and Snowflake is tomorrow! Register now before it's too late: [https://lnkd.in/gM_YzkBh] Want to sharpen your SQL skills? Don’t miss out on our free, hands-on workshop designed to get you up and running with your first SDF project on Snowflake. ?? Date: Tuesday, October 15th, 2024 ?? Time: 9:00 AM PT | 12:00 PM ET | 5:00 PM GMT In just 60 minutes, we'll walkthrough: - Static analysis & SQL validation with SDF - Materialization strategies on Snowflake - Data testing to improve quality - Column-level lineage for SQL queries - Using data classifiers to govern data proactively #Snowflake #DataTransformation #SDFLabs #DataQuality #SQL

We're excited to announce Seattle's first ever Datafusion meetup, sponsored by SDF Labs and Spice AI. Join Wolfram Schulte and others in the Datafusion community to learn how Datafusion is becoming the LLVM of query engines! Monday, October 14th, 5:30p to 8:30p University of Washington: 1100 NE Campus Pkwy 200, Seattle, WA 98105 Reserve your spot via this link: https://lu.ma/tnwl866b

Want to get started with SDF Labs on Snowflake? Join the upcoming webinar on October 15th! In this hands-on workshop SDF Engineers will walkthrough: - Static analysis & SQL validation with SDF - Materialization strategies on Snowflake - Data testing & expectations to improve data quality - Column level lineage as you author new SQL queries - Data Classifiers, and what they can be used for Use the time to follow along, ask questions, and remember, there are no null values in curiosity! ?? Register here: https://lnkd.in/gM_YzkBh #datawarehouse #cloudcomputing #snowflake #sdflabs #webinar

The wait is over… We’re thrilled to announce column-level Impact Analysis is now GA on SDF Cloud! ?? This integration automatically comments on your GitHub PRs with a natural-language summary of the changes made and their impact on all the tables and columns in your warehouse. ?? ?? Seamlessly integrate into your pre-existing workflow with automatic comments on your GitHub PRs ?? Truly understand the ripple effect of your table & column changes on downstream models ?? Detect overlapping models to avoid unnecessary table creation, optimize your warehouse, and reduce compute costs ?? Help every stakeholder quickly view the key impacts through AI-powered summaries, streamlining communication across teams This will optimize how data engineers manage their pipelines and ensure a smoother, more intelligent development process. Ready to transform your workflow? Dive in and try it out today: https://lnkd.in/gZDygcRv! ?? #DataEngineering #GitHubIntegration #ImpactAnalysis #SDFLabs #Innovation #Automation #AI

Just wrapped up an incredible 2024 onsite at SDF Labs! ?? Building a company post-covid means a lot of conference calls and virtual interactions. It's harder to get colleagues in a room to whiteboard out a complicated problem, but easier than ever before to work with incredible talent across the country (and across the world!). Then, to bring everyone together in person and see the passion, excitement, and joy at collaboration is really special and made this one of the best weeks of the year for me. Our onsite ended up turning into a hackathon, with the goal of pushing the boundaries of what's possible in the world of SQL. There's some polish needed, but we can't wait too share what we've built ???

It is such a pleasure to welcome Milos Gligoric as a Visiting Researcher to our team at SDF Labs! Milos brings a wealth of expertise in program analysis, program transformation, and software testing, having published over 100 papers and secured funding for his research at UT Austin from leading U.S. agencies and companies. A data enthusiast himself, he has also contributed to KatanaGraph for more than two years. "SDF's approach to SQL analysis and data transformation is built on advanced theories that offer a comprehensive understanding of the language and its data dependencies. I'm eager to help translate cutting-edge research into a robust product and contribute to setting a new standard in the transformation space." Glad to have you on our team, Milos!