How Uber Saved 140,000 Hours Monthly Using Generative AI Agents

Video

The Problem at Hand

Uber's data platform processes approximately 1.2 million interactive queries monthly, with 36% of these coming from the operations organization. This group—comprising engineers, data scientists, and operations professionals—analyzes data from hundreds of thousands of tables across various domains to derive actionable insights.

However, the process of composing and executing queries was a bottleneck:

• 10 minutes per query: Each actor spent an average of 10 minutes composing a query.
• Inefficiency Loop: Users would sift through datasets, run queries, and validate results in a repetitive cycle.
• Wasted Time: The cumulative effect of this inefficiency led to significant lost productivity.

This challenge is not unique to Uber. It resonates across industries, from e-commerce to customer support, where operations teams grapple with similar inefficiencies.

Enter QueryGPT: The Hackathon Solution

In 2023, a team at Uber's hackathon introduced QueryGPT, a prototype designed to streamline the query-generation process. Here's how it worked:

1. Metadata-Driven Query Generation:
2. Few-Shot Prompting:
3. Initial Results:

While this was a promising start, the prototype faced scalability and technical challenges, necessitating further iterations.

Challenges and Iterative Solutions

Key Challenges

1. Prompt-to-Schema Mismatch:
2. Token Limitations:

The Final Architecture

The refined system, powered by Azure OpenAI and GPT-4, demonstrated remarkable efficiency:

• Context Optimization: Leveraged a context window of 128k tokens to handle large schemas.
• Human Validation: Ensured precision through user acknowledgment of suggested tables.
• Scalable Design: Addressed the challenge of querying across hundreds of thousands of datasets.

Uber's engineering team implemented a robust architecture combining SQL, RAG, agents, and custom configurations. Here's a breakdown:

1. Domain-Specific Curation:
2. Intent Agent:
3. Table Agent:
4. Enhanced RAG Pipeline:

Real-World Impact

By the 20th iteration, Uber's Query GPT achieved a staggering 140,000 hours saved monthly across its operations organization. This success underscores the value of combining AI, domain-specific curation, and user-centric design.

Key Takeaways

Uber's solution offers valuable insights for tackling similar challenges in other industries:

1. Break Down Data Silos
2. Implement Intent Detection To Identify The Silos
3. Leverage Human-in-the-Loop Systems
4. Iterate for Scalability

The Future of AI in Operations

Uber's journey with QueryGPT exemplifies the transformative potential of generative AI in operational analytics. By reducing manual effort and empowering teams with intelligent tools, businesses can unlock unprecedented productivity gains.

Whether you're in e-commerce, customer support, or any data-intensive field, the principles behind Uber's success can guide your own AI-driven innovations.

Want to delve deeper into the technical details? Check out Uber's engineering blog here for the full story.