Your December Dose of Data & AI

Thanks for following the Data Council newsletter. Each month, you'll get important updates about the Data Council conference, an in-depth feature on OSS projects, curated technical reads and select job listings from the data & AI community. Happy Reading!

Team Data Council ????

Ready to be part of something extraordinary? Data Council 2025 is coming back to The Bay, and trust us – you won't want to miss this! Next year will be our biggest Data Council yet with real-life insights, breakthrough discussions and new connections that could shape your next big move.

Next year’s conference spans the full spectrum of modern data and AI, with dedicated tracks covering:

• Data Engineering & Infrastructure
• AI Engineering
• Data Science & Algos
• Foundation Models
• Analytics & BI
• MLOps & Platforms
• GenAI Applications
• Databases
• AI & Data Culture

Early-bird tickets are live, but they won't stay that way for long. Last year's conference sold out in record time, so don’t just mark your calendar – secure your spot. The future of data is calling.?

Featured OSS Project: LlamaIndex

LlamaIndex is a data framework that connects custom data with large language models for building RAG applications with the goal of helping companies use AI to intelligently search and understand their own data and documents. The framework provides comprehensive tooling across data ingestion, indexing, querying, and evaluation workflows, supporting both simple question-answering systems and complex cognitive architectures, with capabilities for diverse data handling, flexible retrieval, and structured outputs.?

Notable Features:

• Data Ingestion: Multi-format connectors, automated chunking, metadata extraction, 50+ data loaders for all source types
• Retrieval: Hybrid search combining dense/sparse vectors, reranking, and customizable relevance metrics
• Output Generation: Structured responses with JSON/SQL parsing, templating, and programmable synthesis Query Engine: Auto-routing, recursive summarization, sub-question decomposition with caching
• Integrations: Native support for Pinecone, Weaviate, ChromaDB, OpenAI, Cohere, and custom stores

Monthly Reads - The Latest in Data

• GraphRAG: Microsoft’s Open-Source Solution for Enhanced Document Understanding Towards AI
• The Semantic Layer Movement: The Rise & Current State Modern Data 101
• How to Improve LLM Safety and Reliability Arize AI
• Beyond LLMs: Compound Systems, Agents, and Whole AI Products The Technomist
• How to Extract Analytics from Bluesky, the New Open Social Network MotherDuck
• PyIceberg: Current State and Roadmap Ju Data Engineering Newsletter
• Promptim: An Experimental Library for Prompt Optimization LangChain
• Beyond the Cloud: Distributed AI and On-Device Intelligence Alconomics
• Context-Aided Forecasting: Enhancing Forecasting with Textual Data Towards Data Science
• Shift Left: Headless Data Architecture, Part 2 Confluent

Zero Prime Podcast

E26: Balancing Open Source and Business with Spencer Kimball (Cockroach Labs) In this new episode of the 0P podcast, we welcome back Spencer Kimball, co-founder and CEO of Cockroach Labs, to discuss the evolving landscape of software licensing in the cloud era. Spencer shares insights on how companies are innovating with new licensing models to bridge the gap between open source ideals and sustainable business growth.

Drawing from his experience leading Cockroach Labs, a distributed database company, he offers a unique perspective on finding equilibrium between community collaboration and commercial success.

??? Listen to the episode here?

Jobs

Motherduck: Software Engineer - DevOps, Seattle | Software Engineer - Platform, Seattle

Clusterfudge: Senior Backend Engineer, London

Hex: Compute Technical Lead, SF, NYC | Remote | Product Engineering Lead, NYC

Dagster: Engineering Manager - Platform, Remote

Soda: Senior Data Engineer, Europe | Senior Python Engineer, Europe

Vectara: ML Engineer, Palo Alto

Reads written by: Mosbeh Barhoumi, Animesh Kumar, Eric Xiao, Adel Zaalouk, Simon Sp?ti, Mehdi Ouazza, Julien Hurault, Kevin Liu, LangChain, Siddhant Sahu, Nikos Kafritsas, Adam Bellemare

Jobs at: MotherDuck Clusterfudge Hex Dagster Labs Soda Vectara