Recap: Use Cases, ROI, and What’s Next for 2025?
With the dramatic innovations from generative AI, companies want to move beyond the experimentation phase to tangible business results. But moving from pilot to production requires business leaders to make important decisions about which tools to use, how to mitigate risks and when to scale.
During a recent live video summit, Jon Victor from The Information explored the future of AI with three leaders in the field:
- Dr. Walter Sun, SVP and Global Head of AI, SAP
- Percy Liang, Associate Professor of Computer Science, Stanford University
- Saurabh Baji, SVP Engineering, Cohere
Wide-Ranging Use Cases
Although most industries have not seen their business models upended by a “silver bullet” AI application, Sun noted that companies are getting value from use cases including transportation logistics, marketing and sales. SAP supports nearly 30 large language models, evaluates each for specific capabilities, and then connects its customers with the most cost-effective tools for the tasks they need done.
One SAP customer, a large European automaker, is using generative AI to track incoming deliveries more efficiently and accurately. They are “taking the docs off the docks” by enabling truck drivers to upload photos of printed delivery forms. This makes the entire process of unloading and loading trucks more accurate and efficient. This is a game changer for truck drivers, who want to get back on the road quickly to earn more money, and the business owner, because inventory does not have to be manually updated. “Humans create errors when they type in what they receive, or they lose the paperwork," Sun said. “Using AI, the customer can take a photograph, and it gets digitized automatically and processed into a structured format.”
In a Cohere use case, their platform helps employees prepare for performance reviews by summarizing their work and impact after scanning through project documents. “Of course, the employees review to make sure their contributions are represented the right way,” Baji said. “But it certainly cuts down on the amount of time and leads to increased productivity.”
Overcoming Hurdles
Still, Baji noted, some business leaders have felt burned by investments in AI proof-of-concept projects that didn’t pay off. He advised companies to start small with proven applications, and then scale up the applications that show the highest ROI.
The biggest blockers to enterprise AI adoption, Baji said, are concerns around data privacy and security. “We’ve noticed that being able to deploy inside the customer’s network—having our model actually go to their data, rather than asking them to bring their data to our model—goes the furthest” in solving this problem.
Liang, who was instrumental in the creation of the Holistic Evaluation of Language Models (HELM) at Stanford, noted the importance of third-party standards and benchmarks in helping business leaders better understand the ever-expanding ecosystem of AI platforms.
“The pace of innovation is incredible,” Liang said. “I see our role as providing this third-party evaluation, where we have an objective view of the lay of the land.”
Looking Ahead
Much of the excitement surrounding AI has centered on AI agents that can independently perform the duties of existing roles. Liang said he sees two categories of AI agents: those that can present a large number of (sometimes wrong) answers to humans for verification, and those that need to work with nearly perfect accuracy.
“If you want an agent that finds vulnerabilities in your code, you can, and half the time it might be nonsensical, but it doesn’t matter,” Liang said. “But if you need agents to be very reliable, then I think the general understanding is that they’re not quite here yet—at least for some of the ambitious tasks that we hope that they can do eventually.”
Sun proposed a multi-agent scenario, with several smaller LLMs trained in specific content areas cooperating in tandem. A multi-agent approach, Sun said, will help organizations fine-tune their models and lessen worries that agents will introduce errors into business processes.
“Having expert agents reduces hallucination rates,” Sun said. “That's a really big concern. If something is only 90% right, it’s not usable, because you can’t trust it. But if you can get multiple expert agents, they can do much better.”