Feb 26, 2025 | AI

“Welcome to the era of the AI superlative. While the first two years of generative artificial intelligence (GenAI) development were an all-out sprint to create new models, establish proof-of-concept solutions, and define optimal use cases, the next phase to deliver increased efficiency and better work product to clients in the AI lifecycle will be dominated by marketing as well.”

Raghu Ramanathan, president of Legal Professionals at Thomson Reuters, opened with these statements and shared his view on industry benchmarks in an article on Above the Law titled Beauty is in the AI of the Beholder.

He noted as more companies develop AI solutions and start-ups seek capital investment, customers will look for benchmarks to evaluate these tools. Thomson Reuters does see value in benchmarking, however, Ramanathan added benchmarks must measure products the way they’re designed to be used and should focus on results customers care about.

“The challenge is that one-dimensional metrics do not offer a reliable representation of the real value of GenAI in the legal research process,” stated Ramanathan. “No LLM-based legal research products in the market today provide answers with 100% accuracy, so users must engage in a two-step process of 1) getting the answer and 2) checking the answer for accuracy.”

Chief Legal Operations Officer Meredith Williams-Range from Gibson, Dunn & Crutcher LLP discussed how they are using and seeing results from AI-enabled resources. “There is a widespread misperception around how law firms are using AI and how we conduct legal research. We are not bringing in AI and saying: ‘Go do all the research and write a brief,’ and then replacing all of our junior associates with automated results. We’re using AI-enabled tools that are integrated directly into the research and drafting tools we were using already, and, as a result, we’re getting deeper, more nuanced, and more comprehensive insights faster. We have highly trained professionals doing sophisticated information analysis and reporting, augmented by technology.”

Read the full article on Above the Law, and as Ramanathan concludes, “the value of legal AI – of any technological innovation for that matter – is in how it gets used in the real world and how well all the different components come together to help lawyers do their jobs more effectively.”