Tobiko

? In case you need (yet) another reason to come to Tobiko's data engineering conference, we've added an exciting lineup of speakers from Bessemer Venture Partners, New Enterprise Associates (NEA) and Unusual Ventures to our VC panel! ????♂? Register now at https://lu.ma/j15cttbw for your spot at ?????????? ???? ???????????? ???????? ????????????????, ?????????? ???? in San Francisco. See you there!

Tobiko's first data conference is happening in less than 3 weeks! ?? Alon Nafta, CEO of Foundational, is the latest to join our list of guest speakers. Register at https://lu.ma/j15cttbw to see how we're working with Foundational to automate data contracts across the tech stack - and with other partners, like Databricks, Avios, Kariba Labs, SYNQ, to improve velocity and productivity in data engineering!

SQLMesh + ClickHouse: Transform your data at ClickHouse speed ?? We raced here to share the news that SQLMesh (by Tobiko) now supports ClickHouse to bring precise, error-free data transformation to one of the fastest analytical databases today ??? With SQLMesh, data teams can safely develop, test, and deploy changes to their models—with little to no reprocessing or disruption to production. Pair this with ClickHouse’s lightning-fast query performance, and you get real-time analytics at unparalleled scalability and efficiency ? Want to see it in action? Read our integration announcement post (link in comments) to learn more about how SQLMesh and ClickHouse work together. #SQLMesh #ClickHouse #TobikoData #DataTransformation #Analytics #DataEngineering

As the Director of Engineering at ATM.com, Nico Ritschel has many years of experience as a software engineer navigating intricate data complexities. In today's data-driven landscape, businesses often grapple with the complexity of managing multiple data infrastructure tools. In his recent blog, he outlines how SQLMesh and the team at Tobiko have played a crucial role in simplifying and optimizing ATM.com's data stack. Nico discusses how each additional tool introduced new integration points, which led to increased maintenance efforts for his team and potential points of failure. To address this, Nico streamlined the data stack by consolidating functionalities and reducing the number of tools in use. This approach not only simplifies ATM.com's operations but also enhances efficiency and scalability. By focusing on a simplified data stack, organizations can minimize integration challenges and concentrate on deriving actionable insights from their data. Embracing simplicity in your data infrastructure is a strategic move towards more agile and effective data management. Read his latest blog post to learn more!

???Data engineers talk a lot about deployment — but ?????? ???????????? ???????????? about what happens once code reaches production. Tobiko co-founder and Chief Architect Iaroslav Zeigerman shared his thoughts on just that when he joined Lakeway’s Doron Porat and Amazon Web Services (AWS)’ Yonatan Dolan last week at the Tel Aviv Next Generation Data Engineering Meetup. And he came in strong with bold opinions on how deployment almost always falls short when working with the tools and frameworks most teams are using today: ???Today’s deployment processes are too slow and risky—once a conflict reaches production, it’s too late to prevent errors from triggering, or firedrills from being set off. ?? Deployments should not be treated as simply “moving code from dev to prod” but applying code to production in a controlled and predictable way. ? With SQLMesh and Tobiko Cloud, data engineers can achieve zero-downtime deployments, trigger changes on merge, and (soon!) roll back instantly with one click. ? Data transformation should give data engineers the means to deploying with the confidence that someone else’s pipeline changes won’t break theirs in production — or vice versa. So what should an ideal-state deployment look like? ?? Iaroslav delivered a ????????-????-???????? ???????? to show just how it can be done with Tobiko. Give it a watch, and schedule a chat with us to learn more, through the links below!??

A look-back at January — a month of big moves at Tobiko ??? January has been a milestone-packed month for Tobiko. With an acquisition and a first-of-its-kind product launch already behind us — and our first data engineering conference on the horizon — we’ve been firing on all cylinders to better the developer experience with data transformation ?? In case you missed us, here’s a recap of everything we announced last month: ???????????????? ???????????????? ??????????. We started the year with a bang, acquiring Quary (YC W24) to integrate their novel approach to workflow transparency and data access, and to bring Rust-powered speed to #SQLGlot, our open-source parser and transpiler. This has already paid handsome dividends — SQLGlot parsing is now 35% faster. We’re also thrilled to be joined by Quary founder Benjamin King, who has hit the ground running, building on SQLGlot enhancements while also driving improvements to both #SQLMesh and Tobiko Cloud. ????????????????? ?????? ??????????-???????????? ????????????????. For the first time, data engineers can ?????? ?????? ?????????? ???????????????? ?????????? ?????????????? ???????????? ?? ???????????? ??????????????. Our multi-engine support gives teams the ability to assign the best engine to specific models and tasks — with no need to spin up separate projects. Even better? Unlike our competitors, we’ve made this functionality ???????? ?????? ????????-????????????. ???Our perspective on dbt + SDF. With dbt’s recent acquisition of SDF Labs, a spotlight has been cast on the importance of SQL parsing. We broke down why the combination of SQL understanding with ?????????? ?????????????????? is the only path to transformation efficiency, and how SQLMesh’s built-in virtual data environments solve pain-points that dbt and SDF’s stateless approach cannot.? ???Announcing…our first data engineering conference, ?????????? ???? ????????????! On March 20, we’re bringing together the #SQLMesh, #SQLGlot, and Tobiko Cloud communities for our inaugural conference in San Francisco. Featuring: deep dives into our latest product launches, talks led by our customers, partners, and investors, and a day of networking with data experts. Read our announcements - and register for GROUP BY - via links in the comments. Stay tuned for more innovation, impact, and news about the ways Tobiko will be empowering data engineers. And drop your thoughts below on which announcement you’re most excited about! #DataEngineering #DataAnalytics #DataTransformation #Tobiko #OpenSource #SQL

It’s always a good time at Peninsula Data Happy Hour ?? Last week, we teamed up with typedef to host the January edition of our ?????????????? ?????????????????? ???????? ??????????????. For those of you who’ve not yet joined one of our gatherings, we alternate between casual, free-form networking and more structured deep dives - and we kicked off 2025 with the latter, featuring a strong lineup of talks: ?? Tobias (Toby) Mao walked us through how SQLGlot works at the core of SQLMesh to power column-level lineage - giving another tribute to Tobiko’s commitment to the open-source ecosystem that’s supported us from Day 1. ?? Rohit Rastogi got technical & topical about typedef’s vision for query engines and the importance of mixed workloads. He explored how semantic operators make it possible for LLMs to run analytics across progressively larger and more diverse datasets. ???? Tomasz Tunguz, one of our key investors from Theory Ventures, kept the spotlight on query engines (and how to select the right one). He also highlighted the rise of #Iceberg table formats and smaller, faster AI models, and the astronomical surge in AI-written SQL queries ??, ending the night with thought-provoking insights on how these trends could drive a restructuring - and possibly a flattening - of data teams. If you missed us this time, don’t worry - we’ll be taking it up a notch in March at our first ???????? ?????????????????????? ????????????????????, ?????????? ???? (registration linked in comments below) ?? We’ll dive even deeper into ??????????????, ?????????????? and exciting developments like semantic definitions and auto-generated models. And Tomasz Tunguz will be joining Tobias (Toby) Mao, Tyson Mao and Iaroslav Zeigerman to expand on the themes to watch out for in data transformation this year and beyond ?? #DataEngineering #SQLMesh #SQLGlot #ArtificialIntelligence #LLM #DataAnalytics

???????Tobiko is hosting our ??????????-???????? ???????? ?????????????????????? ????????????????????, GROUP BY! Hear about the exciting new features we’re bringing to #SQLMesh, #SQLGlot, and Tobiko Cloud ??and stay ahead on the latest innovations in the data space with updates from investors like Theory Ventures, and partners like Databricks and Avios! What you can expect from our inaugural event: ???A look at what we’ve built for our transpilation, transformation, and cloud products - and a preview into what else we’re cooking up ???Talks about trends that will shape the industry this year, tools that can supercharge your workflows, and what our customers and partners have learned - and gained - from using Tobiko ???A chance to meet & mingle with other SQLGlot and SQLMesh users, and to get any questions you have about our products answered by our team and founders Tobias (Toby) Mao Iaroslav Zeigerman Tyson Mao ???And of course, food and drinks aplenty! GROUP BY will be hosted on ????????????????, ?????????? ???? at the SVB ???????????????????? ???????????? on Market Street in San Francisco ??? Save the date! Learn more at groupby.tobikodata.com Or register directly at https://lu.ma/j15cttbw

?? Last week, dbt Labs announced their acquisition of SDF Labs — a move aimed at tackling long-standing developer pain-points like the lack of syntax checks and slow compile times. I want to extend my congratulations to both teams, and applaud these improvements. But I’m disappointed that there are no open source plans, and skeptical that SDF’s integration will meaningfully address core challenges tied to dbt’s stateless architecture, such as inefficient recomputation and complexities of managing incremental models at scale. In my latest blog (linked in the comments), I share my thoughts on: ?? The impact and limitations of dbt + SDF, and why SQLMesh takes a different approach to these problems with state management and Virtual Data Environments ?? How SQLGlot, our open-source multi-dialect SQL parser and one of the most widely adopted tools of its kind, lies at the heart of SQLMesh, making a feature that’s core to developer experience available to all Tell us your thoughts, and if you like what you read about SQLMesh and SQLGlot, join our community Slack (linked below) and join us in building! ???

?? It’s a big week here at Tobiko, and we’re keeping the momentum going with more big news: we’re launching support for multi-engine projects! On the heels of our announcement yesterday that we’ve acquired Quary, I’m excited to share a new and game-changing feature for SQLMesh and Tobiko Cloud. For the first time, data engineers can mix and match query engines within a single project, leveraging the most suitable engine for each task. As the industry embraces data lakes and open standards, flexibility and interoperability are becoming non-negotiable. That’s why we built our multi-engine support to help users: ?? Optimize workloads with the right engine for each use case—be it SPARK for heavy-duty analytics, Trino for mid-sized loads, or DuckDB for cost-efficient local processing. ?? Consolidate all engine configurations in one project, eliminating the distractions of spinning up new environments. ?? Enjoy free & open access. Unlike competitors like dbt?, whose cross-platform mesh is restricted to their cloud offering (and also requires spinning up a new project for each engine), our multi-engine support is available for all open-source and paid users. And it doesn’t stop there. Tobiko’s multi-engine support integrates with our powerful open-source parser and transpiler, SQLGlot. Now, you can effortlessly query datasets across dialects for true cross-engine compatibility ?? ???♂? No paywalls, no extra work, just capabilities designed to help data engineers work smarter, faster, and with fewer distractions ?? Ready to dive in? Join our community Slack, talk to our team or check out our blog to learn more and see a guide to configuring your multi-engine projects - all via our links in the comments below! #DataEngineering #SQLMesh #SQLGlot #OpenSource #ModernDataStack #DuckDB #Trino #Spark #PostgreSQL