ChatGPT takes on the "Volume" Problem

2024 will bring an even more rapidly evolving professional landscape, and our journey takes us to a pivotal juncture where innovation meets practicality.? Reflecting on my previous post introducing four core challenges in our professional spheres, especially in corporate departments like finance, it's clear that these challenges - volume, complexity, team development, and knowledge transfer - are opportunities ripe for transformative change. In today's conversation, we will focus on the challenge of “volume”: the daunting and interminable mountain of tasks that professionals face daily. From the hundreds of tax filings our team faces to overwhelming inboxes, the sheer quantity of work can often overshadow the quality of our output and the growth of our teams. But what if we could turn this tide with the power of generative AI (genAI)???

In many industries and for many types of analytic tasks, the discussion is no longer about whether to adopt AI but rather about how to use AI.? Thus, we’ll dive into some real-world applications and firsthand experiences with custom GPT models, highlighting how genAI, including tools like ChatGPT, can significantly enhance our workflows, improve output quality, and reduce the cognitive strain of high-volume tasks.??

Empirical Evidence: How GenAI is Changing the Game

For those who prefer empirical evidence over anecdotal experiences, recent studies provide compelling data on the effectiveness of genAI in professional contexts. Let's examine two significant studies that showcase how AI tools like GPT-4 are revolutionizing work efficiency and output quality.

Boston Consulting Group Study

In a large-scale controlled trial published in HBR , the impact of GPT-4 on consultants' performance was measured.

Key Findings:

• An average increase of 12.2% in task completion.
• A 25.1% improvement in task completion speed.
• A remarkable 40% enhancement in the quality of results.

This study demonstrates that the publicly available version of GPT-4 can significantly boost performance in complex work tasks.

Lawyering in the Age of Artificial Intelligence

This research explored GPT-4's impact on legal analysis.

Key Insights:

• GPT-4 led to modest, albeit inconsistent, improvements in the quality of legal analysis.
• The tool significantly increased the speed of task completion across skill levels.
• The most notable quality improvements were observed among the least skilled participants.
• Participants reported higher satisfaction when using AI for legal tasks and could accurately predict where GPT-4 was most effective.

Photo credit: Ethan Mollick

These studies are testaments to the transformative potential of AI in professional settings. They invite us to rethink how we approach our workloads.

A lofty goal is either offload tasks to the AI or use the AI in some way to lighten the load, not simply speeding up the tasks so we can process more by ourselves! But this is brand new technology and many struggle to get out of old paradigms from decades of refining their skills on the incumbent tech.

Some Misconceptions About GenAI

Misconception: Proper Use of GenAI

• Myth: GenAI tools like ChatGPT use the same skills you’ve developed to use search engines.? Its failure to provide verbatim outputs like extracts of legislation or precise case references completely undermines its usefulness in professional departments.
• Reality: Large Language Models (“LLMs”) like ChatGPT, Claude and other open source models, excel at generating insights, suggestions, and synthesizing information rather than executing direct automation like robotic process automation (RPA) or providing exact citations from legislation or case law. Expecting it to function as a precise search tool is a common mistake.

Misconception: AI Replaces Human Judgment

• Myth: GenAI tools can automate and replace the need for human expertise and judgment.
• Reality: GenAI is designed to complement and augment human capabilities, not replace them. It assists professionals by providing analyses, insights and suggestions, but critical thinking, review, and decision-making remain firmly in the hands of human experts.? You must review the output as you would any report received from external advisors or internal staff.

Misconception: AI Hallucinates; therefore, is unreliable

• Myth: Because all genAI output is “hallucinated”, it will fabricate incorrect responses that slip past your review and therefore is too risky and has limited value.
• Reality: Concerns about AI "hallucinating" or fabricating information can be navigated similarly to how we evaluate information from human experts. Just as with diverse expert opinions, the value of genAI's output increases with the specificity of the context provided and the clarity of the question asked.

A Small Sample of Use Cases for GenAI in Professional Contexts

If genAI isn't about reciting Shakespeare or tax laws with pinpoint accuracy, then how can it transform our workflow?? Let's examine some diverse ways that tools like GPT-4 can offer solutions to alleviate the burden of high-volume tasks.??

1. Documentation Generation: Highly relevant for creating compliance documents, reports, and memos, significantly reducing manual effort.
2. Legal Document Analysis: Essential for summarizing, extracting key points, and analyzing legislation, contracts and legal filings, aiding in quicker decision-making.
3. VBA Code Writing for Workbook Automation: Use GPT-4 (or 3.5) to write code to automate repetitive Excel tasks, such as data manipulation and analysis.
4. Contract Review and Drafting Assistance: Draft, review, and suggest edits to legal contracts based on predefined templates and guidelines, enhancing efficiency and accuracy.
5. Process scanned mail: translate, summarize, highlight important information, and identify action items.

Getting down to Brass Tacks: 3 Practical Examples of GenAI Applications Used in our Department to Tackle Volume

Automating a monthly data manipulation process using English

Natural Language Processing (NLP) makes the advanced capability of "coding" or automation accessible to those like me who have no background in programming. We instruct GPT-4 using everyday English language, making it possible to automate complex data manipulation tasks involved in some of our monthly sales tax filings.

What we used:

• English language prompt laying out which data to be filtered and manipulated, and in what order to execute.
• chatGPT-4 code interpreter (to invoke python libraries) to execute the prompt.
• 2 excel inputs of source data (you can add more)

What chatGPT did:

• Filtered multiple rows, columns and other parameters
• Cross-referenced the data in 2 excel documents.
• Created and exported an excel-based exception report
• Reduced 100+ click exercise down to ~20.
• Saved 1.5 hours (per month)

What chatGPT did NOT do:

• Cannot cause any action to be performed on any of our systems (unlike what Robotic Process Automation might achieve).
• Make errors (python does not “hallucinate”)

After conducting concurrent testing over four months, we're fully satisfied with the precision and accuracy of using GPT-4 for automating this task. This period of experimentation and iterations, or "NLP debugging," allowed our analyst to refine the prompts, ensuring the process became increasingly efficient and error-free. This careful, iterative approach was key to integrating GPT-4 effectively into our workflow.

VBA Code Writing for Workbook Automation (Macros)

I created a custom GPT, dubbed "Excel Maestro ," to specifically help with Excel based automation tasks from formula writing, VBA code generation or Power Query guidance.? My goal was to automate the roll-forward processes in a complex Excel workbook requiring 20-30 hours to complete and following a clear order of operations. Automating this effort would have previously required an advanced understanding of Excel, VBA code and related automation best practices.

What I used:

• chatGPT-4 code interpreter
• English (NLP) prompting
• Custom GPT (Excel Maestro )
• Screenshots to provide context to GPT-4
• Excel dummy workbook for testing and debugging

What chatGPT did:

• Wrote the automation outline to follow and practical step by step instruction
• Wrote VBA code
• Provided best practices (ex: efficiency enhancers, error handling etc…)
• Guided where to add small code blocks into larger code sections
• Debugged code

What chatGPT did NOT do:

• Get it right the 1st time
• Test the code inside the workbook (I had to do that).

This process taught me a lot about approaching VBA / macro style automation.? My learning led to improved instruction to GPT-4 thus improving the efficiency and quality of the outputs as I automated more complex tabs.? It took weeks of work, but will save a lot of time, mental energy and prevent errors in the future.

Processing Scanned Mail

We operate in a multinational environment and receive mail from Revenue or Tax Authorities mostly in English, but many times in foreign languages.? Processing such mail using Google Translate can be time consuming and sometimes impossible if we’ve only received a screenshot of the mail instead of a scan.? Imagine trying to retype a screenshot written in Ελληνικ? γρ?μματα και αριθμο?!

I created a custom GPT, dubbed “Mail Master ”, to both speed up and lighten the mental effort of mail processing.

What I used:

• chatGPT-4 in-built Vision capabilities (to read screenshots)
• SOPs (standard operating procedures) to dictate what to extract from the mail when processing.

What chatGPT (Mail Master) does:

• Efficiently detects language,?
• Translates content to English,?
• Extracts crucial information (due dates, amounts, banking details, contact information etc.)
• Assesses priority and sensitivity,?
• Suggests the most appropriate department to route the mail.
• Drafts summary emails with clear action recommendations.?
• Uses a highly professional tone

What chatGPT (Mail Master) does NOT do:

• Actually route the scanned mail to the appropriate department

ChatGPT is very poor at analysing PDFs using python and I’ve not been impressed with the plugins and other “wrappers” that claim they’ve solved it.? GPT-4 Vision is quite capable though.

The Custom GPTs

With a paid GPT-4 subscription, you can create custom GPTs that are more precisely oriented towards certain tasks, domains, areas of expertise, and desired modes of interaction and voice.? The links provided below are just 2 examples of custom GPTs and will take you to a new chat so you can try it out!

Excel Maestro is a great Excel companion and can function on GPT 3.5, but works best on GPT-4.

Mail Master must be used on GPT-4 and works best with screenshots of your mail.

Even though creating a custom GPT isn’t that hard, it’s often more convenient to use one someone else has created.? Future articles will expand on the practical applications of chatGPT within a corporate work environment and I will showcase various custom GPTs for achieving better quality and efficiency.

These tools exemplify how targeted applications of generative AI can address the perennial challenges of volume, complexity, and efficiency in professional settings. The journey we've embarked on is just the beginning. As we continue to innovate and refine our approaches, the potential for AI to revolutionize our workflows and enhance our productivity is boundless. Stay tuned for further insights as we delve deeper into the practical applications of ChatGPT and beyond, charting a course towards a smarter, more efficient future in the corporate world.