Every one of my clients now leverages GenAI in their stack and it's been a richly educational experience to help design, implement and market these capabilities. So here are some new additions to my list of learnings about legal tech, this time with GenAI in mind. (I hope you'll share your learnings, too.)
- Even though benchmarks are emerging to measure the legal performance on legal tasks, the best measure of performance for your generative output is what the lawyer actually does with it. Build that feedback loop into your product.
- Whatever your AI product might do, generic services like ChatGPT, Copilot and Gemini are your first line of competition with users. You must be demonstrably better.
- If your AI product generates legal or factual material, you can count on legal users wanting to verify the source of the output. Attribution needs to be inherent in your design.
- Testing AI software against legal standards is maddeningly hard: output is non-deterministic, prompts are sensitive, and public model outputs differ, among other challenges. This puts a premium on capturing human feedback.
- Despite the data duopoly of Westlaw and LexisNexis, there's a surprising amount of open legal source data available for training and testing output (Including the awesome Caselaw Access Project.)
- Under ethical rules, law firms generally don’t need client consent to use AI tools with client data. But to avoid misappropriation, consent is required if client data is used to train a model to be used outside of the client's work.
- You can and should anonymize or pseudonymize client data before using it to train your model, and you can do this using packaged alternatives. This creates a powerful protective buffer for data and a stronger assurance for clients.
- Even private AI environments on hosts like Azure may be subject to human review of stored data. When using client data in these environments, you may need to secure an exception or disclose the possibility to your customers.
- There's a reckoning coming on the provenance of training data. Keep careful track of yours and play fair.
- Responsible AI practices will soon be certified just like security. This will move the market forward on AI just as SOC-2 and the like smoothed the way for SaaS, but it's another compliance task for you.
- AI promises to drastically reduce the cost of litigation. As a result, there will be a lot more litigation, which will strain and change the judicial framework. When courts use AI, lawyers will have no choice but to follow.
- Legal AI is gaining ground quickest in practice areas where it doesn’t threaten the billable hour, like personal injury.
- The success of companies like EvenUp shows the power of GenAI married with a human in the loop reviewing all output. Getting to market with human review gives you the data you need to perfect your output.
- The rapid advancement of no-code AI tools means law firms, practice groups and even individual attorneys will soon be able to build customized solutions quite easily. To last, your product needs to do something they can’t do on their own with generic tools.
- Every legal software company needs a Microsoft Copilot strategy.
I hope this list gets you inspired with questions and ideas. Get in touch to talk about your project.?
Scientific Information Services │ Helping clients make critical business decisions by researching and organizing scientific and regulatory information │Historical Research Forensics │ Regulatory Information Services
6 个月Thanks for the post--one important point not mentioned and another 'human in the loop' element needed is to verify all the outputs that are AI generated. I was just explaining to a PhD scientist that some of the AI generated references that they had submitted for article were fake; he was really surprised because he was using one of the cutting edge AI products. AI won't reach its promised potential until the problem of source hallucination is solved. The person that does it will be just as rich and famous as Jeff Besos or Elon Musk. Until then, all AI generated content needs to have the human in the loop to verify its authenticity.
Code & Counsel, PLLC | HeyCounsel | Legal AI Engineer | Fractional General Counsel | Fractional Chief Artificial Intelligence Officer | LegalTech Enthusiast
6 个月This point. If my clients didn’t hear anything else from this fantastic article, please hear this: “The success of companies like EvenUp shows the power of GenAI married with a human in the loop reviewing all output. Getting to market with human review gives you the data you need to perfect your output.”
Advocate.. High Court ,at Calcutta
6 个月Interesting!
Experienced Managing Director | Strategy, Marketing, and Turnaround Specialist | P&L Leadership Across Digital Publishing, SaaS, and Consulting | Board Member & Advisor | Blockchain and AI Advocate
6 个月Hi Jim, thank you for the excellent post. Your domain experience in Law and technology is a huge plus for me and the readers of your blog post.
Lawyer | Legaltech Journalist | Principal Legal Insight Strategist, MyCase, LawPay, CASEpeer, & Docketwise, AffiniPay companies | Author | I bridge the gap between lawyers and emerging tech like genAI & law firm software
6 个月This is a fantastic post full of lots of great insight. I find your posts on legaltech to be invaluable. Thanks for sharing!