Transforming data analysis with AI & Autogen : a new era for data analysis

Introduction

Getting large language models (LLMs) like ChatGPT to automate and replicate any of what commercial data analytics teams do is tough, not least because experienced human analysts know about the business needs in detail, develop a deep understanding of the data they are working with, its quirks and limitations, and can come up with novel solutions to tricky problems.

Most importantly, human analysts frequently work with colleagues in the same team to critique and review important work before sharing, helping to reduce errors and catch bad assumptions.

Steps towards automation through Autogen

Whilst we still have some way to go before AI can imitate this behaviour in full, tools like Autogen are getting us closer by providing the ability to use ChatGPT and other models to build teams that can learn from specific abilities and knowledge, coordinate their work so they can build off each others outputs in sequence, and reflect and criticise on work completed - all within one workflow.

Fully worked example

To show this is possible, I have created a full example of how a team of three virtual analysts (agents) were asked to:

1. extract and understand the structure of some raw unaggregated synthetic marketing pipeline data
2. from this understanding, aggregate and count marketing qualified leads, opportunities and closed won deals for a sub-segment of the data, and cohort the data by MQL created date
3. use that data to create a subsequent simple linear forecast for each metric
4. comment on the results and suggest improvements and enhancements to the forecast
5. create a visualisation of the forecast

Once the sequence was started, all of this was achieved in order, using multiple agents, with very limited intervention from a human, other than asking ancillary questions and approving outputs.

You can find a full explanation here, including the entire conversation and output.

https://gpt-training.ai/blog/blog-posts/transforming-data-analysis-with-autogen-a-new-era-for-analysts

And you can find the raw code and synthetic data used on github here. Feel free to download and use for your own investigations.

https://github.com/jjones7/autogen-analytics-workflows

Conclusion

Whilst this is a relatively straightforward workflow, I believe it underscores the immense potential for conducting data analysis through interactive collaboration with teams of automated agents.

Consider, for instance, the ease with which I could have directed the 'Senior Analyst' to revise the forecast based on its evaluation of the initial linear regression model.

Similarly, requesting multiple visualizations from the 'Visualization Analyst' for comparative forecast analysis could have been effortlessly executed without disrupting the workflow, thanks to the predefined rules I was able to code into the conversation flow.

The flows within Autogen make this possible by ensuring that all agents can seamlessly transition to subsequent tasks once one is complete and approved by the user. The work from the previous task is then handed across to the next agent to work from.

Moreover, the AI-generated code and knowledge the completion of the workflow creates can be learned from and used again by other workflows. It's also possible to provide our virtual analysts with access to business specific documentation documentation (perhaps KPI definitions) that would enhance their understanding of specific data sources, fostering a learning environment where they can build on their previous experiences.

I will experiment further to enhance these workflows and share the outcomes. Should you be interested in learning more about how such agent led analytics workflows can benefit your business, I am more than willing to offer any assistance. Feel free to reach out to me at [email protected].