?? DATA Pill #123 - Stateless vs. Stateful Stream Processing, BigQuery Engine for Apache Flink

Hi,

Ready to level up your data skills? This week's DATA Pill is packed with expert insights on everything from AI-assisted product onboarding to mastering Databricks tools. Whether you're looking to streamline your workflows or explore the latest in AI, I've got you covered!

ARTICLES

Databricks SDKs vs. CLI vs. REST APIs vs. Terraform provider vs. DABs | 6 min | Data Engineering | Alex Ott | Personal Blog

This comprehensive comparison explains when to use Databricks REST APIs, SDKs, CLI, DABs, and Terraform based on your flexibility, simplicity, or complex environment management needs.

Stream Processing Demystified: Stateless vs. Stateful | 4 min | Stream Processing | David Fabritius | Decodable Blog

Explore why stateful processing is essential for complex real-time analytics, handling event correlation, and maintaining context across streams, while stateless processing shines in more straightforward use cases.

Step-by-Step Guide to Creating Your Own Large Language Model | 6 min | LLM | Sciforce Blog

Learn how to build and fine-tune private LLMs, covering data curation, training, customization, and data security advantages.

In MORE LINKS you will read:

• Content Creation Copilot - AI-assisted product onboarding by Zalando

{ MORE LINKS }

TUTORIALS

From keywords to relationships: Reveal deeper insights with full-text search and Spanner Graph | 5 min | Data Engineering | Bei Li, Jeff Sosa | Google Cloud Blog

Learn how integrating full-text search with Spanner Graph streamlines data retrieval and relationship modeling for improved workflow efficiency.

NEWS

BigQuery Engine for Apache Flink overview | 3 min | Data Processing | Google Cloud Blog

BigQuery Engine for Apache Flink simplifies infrastructure management for running Apache Flink, offering autoscaling and easy integration with other Google Cloud services.

PODCAST

Unlocking the Power of LLMs with Data Prep Ki | 38 min | LLM | Ben Lorica, Petros Zerfos, Hima Patel | The Data Exchange Podcast

A deep dive into Data Prep Kit’s scalability, cloud-native architecture, and integration with popular tools like Ray for large-scale LLMs.

In MORE LINKS you will listen to:

• Looking under the hood at the tech stack that powers multimodal AI

{ MORE LINKS }

DATA TUBE

AI prompt engineering: A deep dive | 1h 17 min | AI | Amanda Askell, Alex Albert, David Hershey, Zack Witten | Anthropic

Anthropic's prompt engineering experts discuss the evolution of prompt engineering, offering practical tips and insights into how prompting might change as AI capabilities advance. Key topics include refining prompts, model reasoning, and the differences between enterprise, research, and general chat prompts.

CONFS, EVENTS AND MEETUPS

MOPS - Meetup #5 | Warsaw | 25th September

Join MOPS #5 for an evening of insightful discussions on cutting-edge AI topics, including the power of Small Language Models for on-device intelligence, deploying generative AI at scale with NVIDIA NIM, and practical strategies for self-hosting LLMs.?

_______________________

Have any interesting content to share in the DATA Pill newsletter?

? Join us on GitHub

? Dig previous editions of DataPill

Adam from the GetInData | Part of Xebia