Apple

The Seven-Year Itch: How Apple’s Marriage to Siri Turned Sour

Apple’s 2011 “Let’s talk iPhone” event at its Cupertino headquarters was a pivotal moment for the company. It was the first big showcase for new CEO Tim Cook as he tried to fill the shoes of the legendary but ailing Steve Jobs. Yet the event threatened to disappoint because the company’s new iPhone, the 4s, was simply an incremental update and not a flashy new iPhone 5.

Siri saved the day. Apple unveiled the digital assistant and showed off its ability to respond to voice commands that created reminders, scheduled calendar events or brought up information on restaurants or weather. Critics marveled at Siri’s potential and speculated that Apple might have developed another revolutionary product. Customers seemed to agree, and three days after the iPhone 4s launched, Apple had sold four million devices, at the time making it the fastest-selling iPhone ever.

Fast forward seven years, and Siri is a problem. It’s arguably the main reason that Apple’s latest product launch—a $349 smart speaker called HomePod—has underperformed, based on early estimates from analysts. Although the speaker won plaudits for its sleek look and audio quality, review after review trashed the Siri functionality with words like “dopey, ”“annoying” and “embarrassingly inadequate.”

The Takeaway
Siri was once considered Apple’s next major innovation. But seven years after it launched, it’s the worst feature dragging down its latest product. This inside look shows how Apple turned Siri’s promise into a major problem for the company.

To determine how Apple squandered its own head start over rivals Amazon and Google in the digital assistant realm, The Information interviewed a dozen former employees who worked on various teams responsible for creating Siri or integrating it into Apple’s ecosystem. Most of them agreed to speak only on the condition that they not be named, citing non-disclosure agreements they had signed or concerns about retaliation from Apple executives.

The experiences and opinions shared by the employees amount to a rare look inside Apple’s struggle to remake itself as a company that succeeds at creating innovative and effective online services, and not just hardware design. Apple has had previous blunders in its efforts to build up its online services business—most notably in its subpar iCloud service. The company’s enduring challenges with online services could hurt the company down the road as iPhone sales continue to slow.

Many of the former employees acknowledged for the first time that Apple rushed Siri into the iPhone 4s before the technology was fully baked, setting up an internal debate that has raged since Siri’s inception over whether to continue patching up a flawed build or to rip it up and start from scratch. And that debate was just one of many, as Siri’s various teams morphed into an unwieldy apparatus that engaged in petty turf battles and heated arguments over what an ideal version of Siri should be—a quick and accurate information fetcher or a conversant and intuitive assistant capable of complex tasks.

Presiding over it all has been a revolving door of team leaders and middle managers who lack the kind of vision or clout possessed by Mr. Jobs, who passed away from pancreatic cancer the day after Apple introduced Siri. The absence of such leadership and the constant turnover has held Siri back in key ways, these former employees said, most notably in the failure to open up Apple’s notoriously closed culture to allow outside developers a greater opportunity to create a broader array of useful Siri apps.

For most of Siri’s history, the unit has been run by Bill Stasior, a quiet and unassuming engineer who joined Apple in late 2012 after distinguishing himself as a search expert at Amazon. Mr. Stasior oversees Siri’s core technology—a speech recognition system, a natural language processing system and Apple’s various search capabilities. Those divisions have competed with each other for attention and resources and have clashed often, former Siri employees said.

An Apple spokesperson declined to make Mr. Stasior available for an interview. In a written statement, a spokesperson called Siri “the world’s most popular voice assistant” and touted its ability to work across the company’s devices.

“We have made significant advances in Siri performance, scalability and reliability and have applied the latest machine learning techniques to create a more natural voice and more proactive features,” Apple wrote in its statement. “We continue to invest deeply in machine learning and artificial intelligence to continually improve the quality of answers Siri provides and the breadth of questions Siri can respond to.”

To be sure, no company boasts the kind of flawless digital assistant seen in sci-fi movies, and Apple’s product is ahead of Amazon’s and Google’s in certain areas. Siri supports 21 languages, while Amazon’s Alexa supports only three and Google Assistant supports eight (Google said last month that Assistant will soon support more than 30 languages). Siri is used on more than half a billion devices each month and processes two billion requests a week.

Siri is more responsive than it was when it first launched, and because it resides in the iPhone—one of the best selling products of all time—Apple can credibly claim that more people have access to Siri than any other digital assistant.

But Siri’s capabilities remain limited compared to the competition. Apple in 2016 introduced SiriKit, a tool that allows outside developers the opportunity to create new capabilities for Siri. However, it hasn’t yet led to the explosion of developer interest that its founders envisioned.

Meanwhile, Amazon’s Alexa has 25,000 “skills” built by third-party developers. A Google spokeswoman said Google Assistant has more than a million "actions," which include commands that Google has programmed into the digital assistant, such as navigation via Google Maps, as well as features built by third parties. (Google declined to break out the number of actions built by third parties.) These competing platforms are also catching up fast: Google claimed in January that its Assistant is on 400 million devices, while Amazon said it sold “tens of millions” of Alexa-enabled devices over the holiday season (Amazon doesn’t break out sales of its Echo speakers).

“When Alexa shipped, it was rock solid from day one,” said a former Siri team member. “For Siri to get there, they almost need to set everything aside and start over.”

‘A Ding in the Universe’

Siri Inc. was formed in 2007, born out of technology built at SRI International, a Menlo Park-based nonprofit research group. The original vision was for Siri to be an intelligent digital assistant capable of responding to a user’s request and also interacting to get the desired response. For example, an ideal version of Siri wouldn’t just produce a restaurant’s phone number, it would go back and forth with the user to suss out details like a date, time and the number of people in the party—and then go ahead and make the reservation all on its own.

Siri’s founders believed that the only way to achieve this result was to create an open platform that allowed outside developers to integrate into Siri. By doing that, the assistant’s capabilities could scale far beyond what the team could build in-house.

“It was about making the App Store for AI,” one of Siri’s original executives said. “It was supposed to be a way to orchestrate the internet through conversation.”

Norman Winarsky, who helped found Siri Inc. and sat on the company’s board, said Mr. Jobs was a true believer in Siri’s potential. Mr. Jobs called Siri CEO Dag Kittlaus one day in 2010 and invited the company’s three co-founders over to his house for a meeting that lasted for hours. Mr. Jobs ultimately convinced the group to ignore their other suitors and cast its lot with Apple, where Mr. Jobs said they would be able to realize their goals for Siri on a much bigger scale.

Mr. Jobs “motivated the team enormously,” Mr. Winarsky said of the Siri co-founders. “He made them believe they could make a ding in the universe. He made us all believe that.”

Apple acquired Siri in April 2010 for more than $200 million. Siri had just 24 employees at the time, but Apple quickly hired more people to form the core Siri team. By the time Siri rolled out with the iPhone 4s 18 months later, the Siri team had grown to almost 100 people.

Mr. Jobs remained closely involved with Siri after the acquisition, and former members of the team said it wasn’t unusual to see him show up for weekly meetings to give his input on features it would launch with. Scott Forstall, who at the time was one of Apple’s senior vice presidents, was another top executive who backed the Siri team. Mr. Forstall, whose broad portfolio included Siri’s development, was known for his confrontational manner, but the team appreciated that he had a strong vision for what the product could do and that he was extraordinarily hands-on. He played around with the technology every day and would often come into meetings and say, “Here are 10 things I found last night,” a former Siri employee said.

To ensure that Siri launched on time, the team made some compromises to the original vision. Opening up the product to outside developers—the “App Store for AI” plan—had to be set aside. Mr. Jobs wanted Siri to focus on a few things that only Apple would have complete control over, but the early Siri team members said he promised them that they’d be able to work on building out a third-party ecosystem later.

The day after Apple introduced Siri on October 4, 2011, Mr. Jobs passed away following a prolonged bout with cancer. His death essentially marked the end of what the original Siri team had planned to accomplish, a member of the original Siri team said.

“When Steve died the day after Siri launched, they lost the vision,” the ex-employee said of Apple. “They didn’t have a big picture.”

The Messy Beginning

The Siri team still had Mr. Forstall, but his attention was divided by other major projects, including the upcoming launch of Apple Maps. Mr. Forstall installed Richard Williamson, one of his deputies on the Apple Maps project, to head up the Siri group and get things back on track.

Several former employees said Mr. Williamson made a number of decisions that the rest of the team disagreed with, including a plan to improve Siri’s capabilities only once a year. That was the approach Apple typically employed with iOS, and Williamson’s background was in making software run on phones that received updates from backend servers. Team members said they argued in vain that that model was wrong for Siri, which they believed needed to be an online service that continuously improved, not updated annually. While the server software received many updates relating to stability and performance, there were no architectural changes to Siri in the first year, say former employees.

Mr. Williamson, in an emailed response to an interview request, wrote that it’s “completely untrue” that he decided Siri shouldn’t be improved continuously. He said decisions concerning “technical leadership of the software and server infrastructure” were made by employees below his level, while he was responsible for getting the team on track.

“After launch, Siri was a disaster,” Mr. Williamson wrote. “It was slow, when it worked at all. The software was riddled with serious bugs. Those problems lie entirely with the original Siri team, certainly not me.”

After publication, Mr. Kittlaus tweeted in response: “This statement, wholly false, was made by the architect and head of the biggest launch disaster in Apple history, Apple Maps. In reality Siri worked great at launch but, like any new platform under unexpectedly massive load, required scaling adjustments and 24 hour workdays.”

When Siri first launched, the surge of users exceeded expectations, and the software started breaking down. Engineers scrambled to keep the backend up and running, but the infrastructure had simply not been built to handle that amount of incoming traffic. One employee who was brought on to improve the shaky backend infrastructure remembers a Siri task that originally needed 500 servers coming down to only five servers after improving the code.

“It was like a war,” another former employee said.

Luc Julia, who worked previously with Siri co-founder Adam Cheyer at SRI in the 1990s, was brought in to manage the Siri team and provide “seasoned leadership.” Several former Siri employees said they enjoyed working for Mr. Julia, but Mr. Williamson wrote that he clashed with a number of people at Apple, including Mr. Forstall. When Mr. Forstall pushed for Mr. Julia to go, Mr. Williamson wrote that he complied. Mr. Julia left after running the Siri team for less than a year.

Mr. Williamson wrote that he tried to get the team to implement SiriKit and allow for outside developers to improve Siri’s functionality, but the team resisted because Siri’s “original software was so brittle and inflexible.”

The disagreements between Mr. Williamson and the Siri team led to the June 2012 departure of cofounder Mr. Cheyer, among the most zealous proponents of getting Siri to reach its potential. Mr. Cheyer’s exit—which came about a year after the departure of Mr. Kittlaus, another Siri co-founder—soon resulted in several more members of the original Siri team leaving the project.

Mr. Forstall and Mr. Williamson would also be out before the end of the year. Apple fired both of them following the disastrous, error-riddled launch of Apple Maps in iOS 6. The former Siri employees said they lamented losing Mr. Forstall, a top executive who believed in what they were doing.

A Shift to Search

Shortly after Apple acquired Siri, Mr. Jobs offered to the public some rare insight into what he envisioned.

"[Siri is] not in the search area,” Mr. Jobs said in June 2010 at the Wall Street Journal’s tech conference, All Things Digital. “They’re in the AI area. … We have no plans to go into the search business. That's not something we know about. It's not something we care deeply about. Other people do it well.”

Still, a quality search apparatus is a critical component to creating a useful digital assistant. When a user asks a question, the AI needs to tap into a source of knowledge and quickly identify the right response.

But the point Mr. Jobs was making was one expressed by many members of the original Siri team: Siri was going to be bigger than search. One team member said their vision of an ideal Siri was similar to the 2013 Spike Jonze movie “Her,” in which Joaquin Phoenix plays a lonely man who falls in love with “Samantha,” a conversant operating system.

‘It was like a war.’

The team members who shared that vision said they were immediately skeptical upon learning that a search expert like Mr. Stasior had been tapped to lead Siri. Mr. Stasior had a Ph.D. in computer science from MIT and helped found A9.com, Amazon’s search unit, where he served as its president and CEO from 2006 until 2012.

Some people working on the Siri team under Mr. Stasior felt like he didn’t have the right background in speech or natural language processing—two other critical Siri divisions—to successfully lead the team.

“I get the sense that as much as he was very interested in Siri when he came to Apple, the long-term goal was always to build a search engine,” said one former long-time employee under Mr. Stasior. “Siri was just a jumping off point.”

When Mr. Stasior joined Apple, its search engineering capabilities were scattered across the company with lots of duplication. Mr. Stasior’s job included pulling together all the disparate resources among Siri and Apple’s existing search capabilities to improve Siri’s core performance, said people there at the time. He also oversaw an effort to bolster Apple’s presence in search technology, including web search, media search and sorting through Wikipedia data.

“His job was making search a core asset and making it world class at Apple,” said a person who worked for Mr. Stasior on the search team.

Mr. Stasior led a number of acquisitions to bolster Apple’s in-house search talent and technology. In October 2013, the team bought Cue for over $40 million, according to a TechCrunch report. The startup had built a personal assistant app that searched through a user's emails to spit out a personal agenda.

Mr. Stasior made another big acquisition in 2013 by purchasing Topsy for more than $200 million. The Topsy technology was acquired to be used in Spotlight, the Apple search feature built into the operating system of macOS and iOS that first launched in 2004.

“Bill had free rein if he found something good to jump on it,” said an Apple employee who worked on the due diligence for acquisitions. “The goal here was trying to acquire and consolidate as much good search talent under Bill as they could.”

The Topsy team ultimately grew into a massive organization under Mr. Stasior that now nearly rivals the number of employees on the Siri team, said one former employee. Topsy CEO Vipul Ved Prakash continues to lead that search group and reports directly to Mr. Stasior.

Uniting the existing Siri team with the expanding search unit under Mr. Stasior proved troublesome. Members of the Topsy team expressed a reluctance to work with a Siri team they viewed as slow and bogged down by the initial infrastructure that had been patched up but never completely replaced since it launched.

“There was a feeling that, ‘Why don’t we just start over and build what we need to build, and then worry about reconciling those two later?’” said a former member of the search team. “They’re still reconciling it.”

Core Siri and Spotlight are powered by a combination of both Topsy's technology and Siri Data Services, which is based on older search technology ported over from iTunes search but modified for Siri and launched in 2013, said the former employee. Siri Data Services deals with things like Wikipedia, stocks and movie showtimes, while Topsy sorts through Twitter, news and web results. The Siri Data Services team was eventually lumped into the Topsy team under Mr. Prakash with the plan to integrate all of the tech into a single stack. But they're based on two different programming languages and are tricky to reconcile.

The difficulty integrating the search teams led to some embarrassing outcomes. Users could get completely different responses to the same question based on whether they were using Siri or Spotlight—which were powered by two different search technologies built by two different teams.

Turf Wars and Basketball

Siri’s original technology dealt with natural language processing, which is about understanding the user’s intention. For speech processing, which is software that can detect spoken words, Apple initially outsourced the technology to Boston-based Nuance. Under Mr. Stasior, Apple went to work on building its own speech recognition software.

In 2013, the company hired Alex Acero, an experienced speech recognition expert from Microsoft, to lead a new speech recognition team. Apple also acquired speech startup Novauris Technologies in 2013 for the startup’s speech expertise. The plan was to combine the new speech recognition technology with the core Siri technology under Mr. Acero to improve Siri’s conversational capabilities. The Siri team felt that combining the units made sense and could lead to major improvements for Siri.

But in late 2014, the teams split apart after working together for only six months. Mr. Acero continued to lead the speech recognition team, while the natural language team was overseen by Anoop Sinha, a management consultant who had worked at the global consulting firm McKinsey & Company.

The former employees who spoke with The Information said they never received an explanation for the split, but the assumption was that it happened so Mr. Sinha had his own turf to manage.

Several members on the Siri team took an immediate disliking to Mr. Sinha, who had no background in the natural language processing world. One former employee said Mr. Sinha’s decisions seemed to be driven by office politics instead of science.

Mr. Sinha and Mr. Acero openly feuded, and each would push Mr. Stasior during meetings to clarify which one of them would own which project, according to one former employee who recounted one of the meetings. The tension between the various factions triggered another spate of employee departures, including experienced speech scientists Gokhan Tur, Chuck Wooters, Tom Kollar and Larry Gillick.

Mr. Sinha departed in 2016 and is now an engineering manager at Facebook, according to his LinkedIn profile. He didn’t respond to a request for comment.

A number of Siri’s onetime employees who left over the years joined back up with Messrs. Kittlaus and Cheyer, who formed a new company called Viv Labs to continue the pursuit of a better digital assistant. Mr. Stasior wasn’t happy about the rival effort and fumed to colleagues, “Can you believe those guys?” recounted a former employee.

After learning that Messrs. Kittlaus and Cheyer continued to visit the Apple campus to play basketball with their former Siri colleagues, Mr. Stasior worried that they were there to poach members of his team. Soon after, the two were banned from Apple’s campus, according to Siri employees who were there at the time.

Rivals Emerge

Mr. Stasior has continued to make ambitious acquisitions aimed at improving Siri. In October 2015, he led the acquisition of VocalIQ, a United Kingdom-based AI startup working on a dynamic learning system instead of one that uses programmed responses.

The VocalIQ team viewed Siri as a “manually-crafted system” and felt their technology could help improve it, said a former VocalIQ employee. VocalIQ’s technology is designed to continually finetune its accuracy by ingesting and analyzing data from voice interactions, he said. Apple has successfully integrated the VocalIQ technology into Siri's calendar capabilities, sources familiar with the project said.

Apple has also introduced more automated machine learning techniques into its natural language processing system. Former Siri employee Rushin Shah led an effort to integrate more automated machine learning into domain choosing —such as when Siri needs to figure out if the request is for messaging or ride sharing—and intent understanding for particular domains. Mr. Shah left Apple in 2017 for a senior manager role on Facebook’s natural language understanding team, according to his LinkedIn profile.

Meanwhile, Apple has ceded its three-year head start in the digital assistant realm to Amazon and Google. A  Wall Street Journal story published in June 2017 predicted that HomePod would “enter the home-speaker market a distant third, at best.”

In a sign of how unprepared Apple was to deal with a rivalry, two Siri team members told The Information that their team didn’t even learn about Apple’s HomePod project until 2015—after Amazon unveiled the Echo in late 2014. One of Apple’s original plans was to launch its speaker without Siri included, according to a source.

The sudden shift to add Siri into the speaker project resulted in weekly meetings between teams from Siri’s speech recognition unit and Beats, the audio product company Apple had acquired the year before for $3 billion. Roughly 25 people attended those early meetings, including a few Beats employees via video conference, said Chuck Wooters, a former Apple speech recognition employee who sat in on the meetings.

The Beats team worked on algorithms for noise reduction and beamforming on microphones to better pick up a user’s voice commands. Apple stashed microphones in apartments around the Bay Area to collect data and replicate a home environment. The team called the data it was collecting “room impulse responses,” Mr. Wooters said.

But the most notable failure in Siri’s evolution is that it still lacks the third-party developer ecosystem considered the key element of the original Siri vision. Apple finally launched SiriKit in 2016 after years of setting aside the project and shifting resources away to other areas, according to former employees who worked on various aspects of the developer kit. Apple had been working on a developer kit off and on since 2012, said multiple sources.

Apple ultimately tasked Vineet Khosla, one of the original Siri employees, with leading the SiriKit initiative. By enabling developers to connect their apps with Siri, Apple looked poised to boost the digital assistant’s functionality instead of relying on a limited set of native apps for making phone calls or sending text messages.

But SiriKit has yet to fulfill its promise. So far it includes just 10 activities—Apple calls them “intent domains”—such as payments, booking rides, setting up to-do lists and looking at photos. Several senior engineers who worked on SiriKit have left Apple or moved off the project. Mr. Khosla no longer leads SiriKit, according to several sources with knowledge of the situation, although he remains employed by Apple. Robby Walker, the current head of Siri, has taken over responsibility for SiriKit.

Among all these challenges, former Siri members noted that while Apple has tried to remake itself as a services company, its core is still product design. The company’s commitment to creating flawless online services is less of a priority than designing transformational hardware, and that can cause problems for a project as complex as the one Mr. Stasior manages.

“The structure of Apple works against those efforts,” a former Siri employee said. “I think what Bill’s had to struggle with is working against that mentality.”


Image title



Kevin McLaughlin has been a reporter at The Information since 2016, covering cloud computing, enterprise software and artificial intelligence. He is based in San Francisco and you can find him on Twitter @ KevKubernetes.