Anticipating the dawn of LLM agents in legal work

[The following originally appeared in the Ontario Bar Association's "JUST" magazine and is a lightly edited excerpt from my keynote address at OBA TECHxpo: AI Edition, delivered on August 21, 2024 at the OBA Conference Centre.]

Agentic AI frameworks (basically, a “team” of AI and other software capacities with tool and data access guided by a combination of analysis and reasoning engines) are increasingly being used in software development and customer service environments. In fact, there are a lot of agentic frameworks, as they're called, both visible and invisible.

Proliferating elsewhere, perhaps, but is there a place for them in law? And do we even need them when we have companies that are building AI tools that serve our particular needs?

To answer that, let’s start with a familiar scenario probably playing out today in the downtown Toronto office tower of your choice.

Imagine a senior partner with a business client contemplating the sale of one of their divisions, and they want to start thinking about the tax implications.

Senior partner calls a junior associate to her office and gives a preliminary instruction. The associate nods, trying to conceal his anxiety.

The associate heads back to their desk, flips open the online research tools, and also starts skimming books and other reference resources. He very quickly realizes that he’s not quite sure what where he’s going.?He had more questions he didn’t ask, so he goes back and asks the partner the questions.

This first check-in goes smoothly, but the interaction is something that cycles through a few times, either because the associate comes up with more questions he hadn't thought of before or maybe he stumbles on a resource that changes his thinking and the direction. He ends up going back and back and back again to the partner for a little bit more information each time. And sometimes, in one of those visits, the partner probably says, “Oh, by the way, I forgot to tell you about this.”

Ultimately, we get to the point where the associate has written the memo to the partner. The partner looks at it and says, “this seems pretty good. You went off in a lot of detail in this direction that I wasn't quite expecting. Given that this is our particular focus, why did you do that?”

And we're about to enter another cycle of rewrites before we get anything to the client.

There's got to be a better way. Right?

Enter, agents.

In an early August interview, Eric Schmidt, former chair of Google, decades-long influencer in Silicon Valley, and significant investor in a lot of AI activities, said as follows:

In the next year, you're going to see very large context windows, agents and text-to-action. When they are delivered at scale, it's going to have an impact on the world at a scale that no one understands yet. Much bigger than the horrific impact we've had on social media.

What does that mean? Well, in the realm of AI, he’s suggesting the experience of the past two years was just an appetizer.

“Large context windows” means the ability of language models to hold enormous amounts of specific and highly relevant detail in short-term and long-term memory while processing tasks. And engaging an agent is not asking one Mega Model to do something. It’s triggering a cascading, looping and potentially self-correcting set of actions where one or more language models have, for lack of a better word, “agency” to make decisions to confirm and carry out your intent. You're essentially inviting one model to use a tool, and then maybe another model to check it, and so on, and so on, and so on. Finally, text-to-action is the idea of initiating computer code and programming directions through natural language prompts, “written” by you, or possibly an agent in your framework.?

He goes on to describe how through this combination of capacities, it will be simple to spin up complex software in minutes through prompts, and he’s basing this prediction on what he's seeing now, with an anticipated wave starting in the next year.?

So, let's return to the legal industry and imagine these events occur on the timeline that Eric Schmidt is talking about.

Turning back to our senior partner, a new story plays out.

It no longer begins as an initial engagement with the senior partner and the associate. It's an initial engagement with the lawyer and a tool. Let's call it LegalMind.

It begins conversationally as the partner leans over to a computer and says, “Hey, I got a call from a client. This is what they're doing. Here are some key considerations that I think are relevant.”

LegalMind, the lead agent in the framework, having a large enough context window to know what kind of questions to ask and having no fear akin to a junior associate to ask them, responds with a few questions and engages the partner in an iterative back and forth. Two minutes in and the computer says, “Okay, I think I got it. I'll come back if I have another question.”

I’m describing something that is possible right now. We're not seeing it in a lot of tools, but this idea of a system beginning to generate its activities is achievable – as is the orchestration of functions such that when it reaches a point of uncertainty, it can come back and ask and reinform itself.

Computer: <ping!> I thought of this, is this relevant?

Senior lawyer: Yeah, actually, that's a good point. That's relevant. Please put that in.

Computer: No problem. I'll be back with the memo. <ping!> The memo is ready.

The senior lawyer, having spent maybe 15 minutes instructing, reinstructing, and iterating with the computer on the issues that matter, followed by her initial review and conclusion that it seems correct, invites the associate in to say, “Make sure this is right.”

Now “this” isn't “Do the research over.” No, this is very much, “this seems right, but let's be sure, before we actually move forward.”

The scene changes to the client's office. Maybe the next day, maybe that same day.

The client checks their email to see that the lawyer has now written up a summary of the memo and given useful advice supported by meaningful research that has been twice validated and client says: ?“This is great. This is what I needed. Oh, and look at that small bill. Am I glad I hired this law firm!”

Full recordings from TECHxpo: AI Edition are available under the Video tab here.

If you’ve questions for OBA's Innovator in Residence, please drop by Colin's?Office Hours, taking place Tuesdays at noon – register for the next session or find recordings of previous sessions here.