LLM? What to do with it.

This is not a coding tutorial but highlight of architecture and techniques that I have implemented with my work.

Large Language Models (LLMs) are sophisticated AI systems trained on vast amounts of public data to understand human natural language. Examples range from open-source models like Meta's Llama3 to closed-source models like OpenAI's GPT series.

While there’s a lot of buzz around AI, Generative AI (GenAI), and LLMs, the real question is: what are these language models, and what can they do beyond writing emails or telling jokes? That’s what I aim to discuss.

What are LLMs and Their Capabilities?

LLMs excel at comprehending human language and the semantics behind sentences or inputs. Leveraging this capability, we can achieve feats that traditional software development couldn't. We can think of this intelligence as a child who follows instructions. If we train this child to become a doctor, engineer, or lawyer, it’s merely a matter of the child learning the necessary knowledge. Although we are years away from achieving Artificial General Intelligence (AGI), we can use current technology to let the intelligence acquire specific knowledge without needing to retrain or fine-tune the model.

Practical Example:

1. SQL Expert Tool

Let's delve into a practical example to illustrate this:

User Input:

"What is the sales report for this year by product?"

Expected Output:

SELECT SUM(sales) AS total_sales, product, MONTH(sales_date) AS month
FROM product_sales
WHERE YEAR(sales_date) = 2024
GROUP BY product, MONTH(sales_date);        

In this example, we are building a tool using an LLM that specialise in generating SQL queries based on user input.

Steps Involved:

1. SQL Expert: This is a Retrieval-Augmented Generation (RAG) model that uses a system prompt containing context and knowledge for generating SQL queries. It has an understanding of the product sales report schema, example data, and enough background to generate the desired SQL output using any LLM.
2. SQL Executor: This component is a simple SQL execution engine connected to a database. Its role is to verify if the generated SQL query produces the expected results.
3. Debugger: If the SQL query does not produce the correct results, the debugger extracts the error message, updates the system prompt with the previously generated SQL and the identified issues, and sends it back to the SQL Expert for validation and regeneration. This process repeats until the correct result is obtained or the iteration limit is reached.

Process Overview:

1. User Input: The user asks a question, such as, "What is the sales report for this year by product?"
2. System Understanding: The LLM, equipped with LangGraph and AlphaCodium techniques, interprets the user’s request, understanding the underlying intent and required data.
3. SQL Generation: The SQL Expert, informed by the system prompt, generates an appropriate SQL query based on the product sales schema.
4. Execution and Verification: The SQL Executor runs the generated query. If the query is valid and returns the correct results, the process ends here.
5. Debugging and Iteration: If the query fails, the Debugger identifies the error, updates the prompt, and the SQL Expert generates a revised query. This loop continues until a valid result is achieved or a maximum number of iterations is reached.

2. Python Code Generation Tool

User Input:

"Write a Python function to calculate the factorial of a number."

Expected Output:

def factorial(n):
    if n == 0:
        return 1
    else:
        return n * factorial(n-1)
        

Steps Involved:

1. Python Code Expert: This component is a Retrieval-Augmented Generation (RAG) model tailored to understand and generate Python code. It uses a system prompt with context and examples related to Python programming, enabling it to produce accurate code snippets.
2. Python Executor: A runtime environment that executes the generated Python code to check if it works correctly and meets the user’s requirements.
3. Debugger: If the generated code fails or produces errors, the debugger captures the error messages and feedback, updating the system prompt with this information. The revised prompt is then used to regenerate the code until it works correctly or the iteration limit is reached.

Process Overview:

1. User Input: The user asks for a Python function to calculate the factorial of a number.
2. System Understanding: The LLM, equipped with LangGraph and AlphaCodium techniques, interprets the user’s request, understanding the task and the necessary Python constructs.
3. Code Generation: The Python Code Expert generates a Python function that calculates the factorial of a number based on the user’s input.
4. Execution and Verification: The Code Executor runs the generated Python function with test cases to verify its correctness. If the function works as expected, the process ends here.
5. Debugging and Iteration: If the function fails or produces incorrect results, the Debugger identifies the issues, updates the prompt with the error details, and regenerates the function. This loop continues until a valid function is achieved or a maximum number of iterations is reached.

Implementation:

You can come up with endless possibilities of this LLMs tools and combine them to solve a single problem.

Let say we have an objective to create a visualisation of the sales data mention above into a bar chart.

User Input:

"What is the sales report for this year by product? Draw a bar chart"

Expected Output:

A Python code that includes SQL to fetch the data and generate the visualisation.

Steps Involved:

1. SQL Expert: Generates the SQL query to fetch the required sales data.
2. Python Code Expert: Uses the fetched data to generate a bar chart visualisation.

Implementation Workflow:

1. SQL Expert

User Input: "What is the sales report for this year by product?"

Generated SQL Query:

SELECT SUM(sales) AS total_sales, product, MONTH(sales_date) AS month
FROM product_sales
WHERE YEAR(sales_date) = 2024
GROUP BY product, MONTH(sales_date);        

SQL Executor

The SQL Executor runs the above query and fetches the data.

3. Python Code Expert

Objective: Generate a Python script to create a bar chart using the fetched data.

Generated Python Code:

import pandas as pd
import matplotlib.pyplot as plt
import sqlite3

# Connect to the database (assume the database connection is established)
conn = sqlite3.connect('sales.db')

# Execute the SQL query
query = """
SELECT SUM(sales) AS total_sales, product, MONTH(sales_date) AS month
FROM product_sales
WHERE YEAR(sales_date) = 2024
GROUP BY product, MONTH(sales_date);
"""
data = pd.read_sql_query(query, conn)

# Close the database connection
conn.close()

# Create a pivot table for visualization
pivot_table = data.pivot(index='month', columns='product', values='total_sales')

# Plot the bar chart
pivot_table.plot(kind='bar', stacked=True)
plt.title('Sales Report for 2024 by Product')
plt.xlabel('Month')
plt.ylabel('Total Sales')
plt.legend(title='Product')
plt.show()
        

Conclusion

By combining the capabilities of SQL and Python experts, we can create an end-to-end solution that not only fetches the necessary data but also visualises it effectively. This approach demonstrates the power of LLMs in integrating multiple tasks to achieve a comprehensive solution.

Reference:

Paper on AlphaCodium