Checking out ChatGPT for the First Time – Revelations for Law and Everything Else

By Paul Pryzant and James Dorough-Lewis

We’re both M&A attorneys.?We spend most days (and nights!) helping clients buy and sell businesses.?We also love technology, and finding ways to use it to make our work faster and more efficient.?So with all of the hype about ChatGPT, we thought it was time to check it out to see how it will change what we do and how we do it in the future.

Our first use case was non-legal.?One of us (Paul) needs a new roof, and was checking out different types of roof shingles.?30-45 minutes of research on Google provided some good information on the differences between shingle A and shingle B (longevity, durability, etc.).?But it took time to separate the hype and fluff (from the manufacturers) and get down to some core information to help make a decision.?After the research, we felt pretty knowledgeable on the subject.?What was hard to find was the cost difference.

On a lark, we asked ChatGPT “what are the differences between shingle A and shingle B?”?In literally seconds, it spit out a list of four advantages of shingle A over shingle B.?Based on the earlier research, the answer was spot on!? But then we went to the next step, and asked “how much more does shingle A cost over shingle B?”?The immediate answer:?20-30% more.?The next day, we learned from a roofer that the cost would be 17% more.?In our book, that is pretty darn close!

But what about using it for law??We first used it to ask about non-competition law in M&A transactions.?One of us had recently written an article on a recent opinion from the Delaware Chancery Court, and we deal with this question on a regular basis in our practice. So we know something about the subject matter.?Here the results from ChatGPT were good, but not quite ready for prime time.?The general law was correct, but the specifics on the recent opinion were not.?Here we found the first limitation – ChatGPT’s knowledge base is limited to data from September 2021.?So more recent court opinions don’t make it into the answers, and it does not alert you to that limitation.?ChatGPT can also get the facts incorrect.?It thought the case was before the Delaware Supreme Court rather than the Delaware Chancery Court. So ChatGPT is certainly not “client-ready”.

Then we got a question from a colleague generally asking to find court opinions on the measure of damages for a breach of contract in a specific situation.?Again, knowing the limitation on ChatGPT’s data set to September 2021, we had to recognize that recent cases would not be included.? But the results were nothing short of amazing – especially with the speed.?Mere seconds which even the most knowledgeable expert could not hope to match.?We learned how to drill down from the first very general answer to get the more specific information we wanted.?We also learned that re-framing the question produced better answers.?We had first asked for court opinions on the subject matter, and got more generic opinions that were not that helpful.?But then we asked a very specific question – find us cases in which a particular way to measure damages did not prevail.?This time, the results were pretty good. Drilling down with follow-up questions gave even better answers – a list of opinions with short, concise descriptions of the rulings that would serve as a great springboard for further research using traditional (and more authoritative and current) means of legal research such as Westlaw or LexisNexis.?

Like any tool, whether a hammer or saw, or a research tool, we learned that you have to know how best to use ChatGPT.?You have to understand the limitations (i.e., the data set being used) and how to ask the questions to get the best results.

So what does the future hold for ChatGPT’s use in the legal profession??It’s hard to imagine.?So many possibilities for uses…and abuses, just like any other new technology.?As Yogi Berra famously said, “It’s hard to make predictions, especially about the future”.?When we held an iPhone in our hands for the first time in 2008, it was hard to foresee the profound impact it would have on our society.?In January 2023, over 100 million users had used ChatGPT in a mere two months, making it the fastest-growing consumer application ever in such a short period of time.?Can there be any doubt that ChatGPT (and other applications of generative AI) will change the way that lawyers do their work in the future? D. Casey Flaherty , the Chief Strategy Officer at LexFusion, has a succinct prediction that “the composition of work for lawyers will change, slowly at first and then quite suddenly (the road to product is long but clear)”.

Three luminaries, Henry Kissinger, Eric Schmidt and Daniel Huttenlocher, published a long and very thoughtful opinion piece in The Wall Street Journal on February 25, 2023 entitled ChatGPT Heralds an Intellectual Revolution.?For anyone looking to get beyond the hype of ChatGPT, this article is highly recommended.?They point out the huge potential for generative AI, as well as the many unanswered questions that it will pose.?To paraphrase just a few of their comments:

• The ChatGPT developers don’t know exactly how it works, and the outputs remain difficult to explain.
• The lack of citations in the answers makes it difficult to discern truth from misinformation.
• The speed and highly articulate answers could encourage unquestioning acceptance of the output from ChatGPT.
• The cost is staggering at present – over $1.0 billion per model.?While subscription models are likely, so that many people and businesses will have access to them (similar to cloud storage), the design and control of these generative AI models will be highly concentrated.
• Education will need to adapt, and quickly.

A few days ago, we asked a law student we know how many students at her law school were using ChatGPT.?The immediate answer was “everyone”.?While the legal industry has been slow to change in the past, this time may be very different.?ChatGPT, and its future iterations, will be a game changer, but at present, we don’t know how the game will change.