Recap: What’s Working and What’s Not With AI Agents
Although many businesses are still experimenting with LLMs, some enterprise leaders are already eyeing the next evolution of AI: autonomous agents that act on their own rather than merely generating outputs on command.
But which early use cases are the most promising? What benchmarks can organizations use to measure the success (or failure) of their new digital workers? And how can leaders overcome the skepticism of employees who are worried AI will replace their jobs?
During a live video summit, “AI Agents: What’s Working and What’s Not,” Jon Victor, a reporter at The Information, explored these and other questions with a trio of AI leaders:
- Shishir Mehrotra, CEO, Grammarly
- Naveen Rao, vice president of AI, Databricks
- Kaylin Voss, EVP of Agentforce and Data Cloud, Salesforce
Putting AI Agents to Work
Mehrotra called Grammarly, a tool that provides personalized writing suggestions for tone, style and clarity, “one of the original” AI agents. “We built this superhighway to get agents right to the edge where they can work alongside you,” he said. “Right now, we only really have one car running on the superhighway. That’s the Grammarly agent, and we’re working to add more.”
Rao noted that most tools marketed as AI agents aren’t truly autonomous—at least not yet. “It’s more workflow automation, more of an assistant tool,” he said. “That actually can add up [to] quite a lot, and we’re starting to see this in our customers’ financial statements. Maybe eventually we’ll get to a point where agents truly find insights on their own, but the technology is still getting there.”
Salesforce is deploying its Agentforce AI platform both internally and to its customers to help automate rote work. Voss noted that the company’s 25,000 sellers currently spend 70% of their time on activities other than selling, presenting a prime opportunity for automation. Also, she said, 84% of visits to the Salesforce customer support website are now handled by agentic AI, with only 2% of requests requiring escalation to humans. “It is humans and agents together that are delivering customer success,” she said. “That’s what we’re seeing most frequently in the market right now.”
Measuring Success, Averting Pitfalls
Much like human employees, Voss said, AI agents need training, clear goals and ongoing oversight to succeed. “You wouldn’t just put an employee into your company and say, ‘You got it,’” she noted. “There’s the measurement of KPIs, there’s fine-tuning, there’s coaching. We very much are working alongside our customers to do that.”
Data, of course, is key to powering agentic AI, and Voss pointed out that small issues can cause big problems. For instance, if different departments have different names for the same data (such as “leads” versus “opportunities”), an AI agent will likely struggle to identify and interact with records consistently throughout the enterprise.
Mehrotra said Grammarly saves companies about 19 days of working time per employee each year—an easily understood, clearly valuable metric. “You have to start with: What problems are you trying to solve?” he said. “The other challenge is, you need it to work in the way that the users are actually working.”
Encouraging Adoption
Rather than framing AI agents as a replacement for employees, Voss said, companies should talk to workers about how these tools can take the toil out of their own jobs. “When I talk to sales teams about how they can have an agent create a quote for them, are they sad about that?” she said. “No, that’s great. When you look at the majority of companies, 41% of their time is spent on the drudgery, the not-fun part of work. When agents can take on those highly automatable or repetitive tasks, I think it’s more of a business case around increasing win rates and velocity of the business.”
Costs for AI tools have dropped dramatically over the past few years, but Rao noted that accuracy has not improved at the same rate. Once business leaders can be confident that AI agents won’t make critical mistakes, he said, agentic AI will see a “massive explosion” in adoption.
“Once we unlock increasing reliability, and then really understand the user experience side,” Rao said, “it’s going to start to fill in all the nooks and crannies of things we can’t even imagine.”