Apple Eyes 2022 Release for AR Headset, 2023 for Glasses

The talk included videos and explanations of some of the devices’ features along with updates from other teams involved in the projects, the people said. Executives also discussed the hardware’s “thermal architecture,” or its methods of dissipating heat, which suggests the development of the devices is progressing.

The new details, along with Apple’s unusual decision to present them to such a large group of employees, is a further sign that augmented reality could become one of the tech industry’s next major battlegrounds. Facebook is investing heavily in the development of AR, as is Microsoft with its HoloLens headset. Smaller companies such as Snap and Magic Leap have also demonstrated early work in AR headwear.

Hardware Features

While VR headsets such as Facebook’s Oculus immerse people completely in virtual environments, AR devices promise to display digital imagery and information on lenses in front of people’s eyes without completely obscuring the view of their physical surroundings. If companies can make the technology reliable and lightweight enough—which could take years—they could eventually replace mobile phones, some technologists believe.

Apple’s headset, code-named N301, will offer a hybrid of AR and VR capabilities, according to people familiar with the device. It resembles the Oculus Quest, a Facebook virtual reality headset released earlier this year, but with a sleeker design, these people said. Cameras will be mounted on the outside of the device, allowing people to see and interact with their physical surroundings, they said. Apple wants to make heavy use of fabrics and lightweight materials to ensure the device is comfortable to wear for extended periods of time, executives said in the presentation in October.

The headset will have a high-resolution display that will allow users to read small type and see other people standing in front of and behind virtual objects. The technology will be able to map the surfaces, edges and dimensions of rooms with greater accuracy than existing devices on the market, executives said at the meeting. To illustrate these capabilities, attendees at the October meeting were shown a recording of a demonstration in which a virtual coffee machine was placed on a real kitchen table surrounded by people in a room. The virtual coffee machine obscured people standing behind it in the room.

Apple is planning to reach out to third-party software developers as early as 2021 to encourage them to build apps for the new hardware, the company told employees at the October meeting.

In contrast, Apple’s AR glasses, code-named N421, present bigger technical challenges than the headset and are further from release. They are meant to be worn all day, and current prototypes look like high-priced sunglasses with thick frames that house the battery and chips, according to a person who has seen them.

Additionally, Apple has explored the use of lenses for the glasses that darken when people are using AR on them, a way of letting others know the wearer of the glasses is distracted, said another person involved with the project. Apple applied for a patent for such a feature earlier this year. Apple senior managers have told employees they believe later versions of the AR glasses could replace the need for iPhones in roughly a decade, the person said.

People familiar with the October meeting said it was unusual for Apple, one of the most secretive companies in Silicon Valley, to brief so many employees at once about product roadmaps. One person at the meeting said attendees were required to wrap their iPhones in tamper-proof stickers with unique QR codes that blocked the front and back cameras, preventing them from photographing the event.

An Apple spokesperson said the company “doesn’t comment on rumors and speculation.” It is possible that the final AR-VR products Apple releases won’t have all of the features discussed in the presentation and the planned timelines for shipping the devices could change. Apple is known to change or remove features, and even kill products altogether, late in their development.

The Next Platform

Apple has shown its growing interest in AR and VR hardware through a string of acquisitions. In 2017, Apple bought a Canadian company called Vrvana that was behind a headset capable of both AR and VR experiences similar to what Apple has in the works. That same year, Apple acquired a firm called SensoMotoric Instruments that was developing eye-tracking tech for smart glasses. And last year Apple purchased Akonia Holographics, which was making holographic lenses for AR glasses.

The number of Apple employees working on AR projects across multiple divisions was estimated to be roughly 1,200 by the end of 2018, with hundreds working directly on the headset and dozens working on the glasses, according to a person familiar with the matter.

As sales of iPhones slow, Apple has a strong incentive to become a major player in whatever the next big computing platform turns out to be. But it isn’t at all clear whether VR or AR will be that platform. When Facebook acquired Oculus VR for $2 billion in 2014, CEO Mark Zuckerberg predicted that VR could be as broadly appealing as mobile, but so far VR remains a niche product category with no obvious mainstream appeal.

Apple’s Technology Development Group, the AR team led by Rockwell, a former Dolby executive, consists of recent hires with backgrounds in video games, audio direction, computer vision and special effects, along with several longtime Apple employees experienced in supply chain and program management, according to a review of Apple’s internal organizational chart that was previously reported by The Information. Earlier this year, The Information also reported that Apple transferred Kim Vorrath, a high-profile software executive, to Rockwell’s AR team, a move seen internally as intending to bring more order to the nascent group.

Apple has been exploring and building prototypes of AR and VR devices as far back as 2015 under the umbrella code name of T288, according to people involved in the efforts. Initial prototypes used heavily modified HTC Vives outfitted with external cameras that were connected to large desktop computers or backpacks. Early demos allowed employees wearing the headsets to participate in virtual-reality meetings and paint 3D images while still letting them view their physical surroundings, one of the people said.

Apple has already begun warming up outside developers to the possibilities of AR with software tools for the iPhone and iPad called ARKit, released in 2017, and RealityKit, a more recent toolkit that allows developers to build 3D objects for AR experiences in mobile apps. Bloomberg earlier reported details about Apple’s efforts to build AR devices, saying in one story that Apple was targeting 2020 as the release date for a headset.

The later target release date for Apple’s full-fledged AR glasses likely reflects technology hurdles that everyone in the industry is seeking to overcome. Many of the software features in Apple’s upcoming devices, such as advanced human detection, are available today but are computationally intensive, demanding mobile-processing and battery power that is beyond what is currently capable in the form factor of glasses.

Making these devices lighter so they can be worn for longer periods of time is also a major challenge. Apple has explored using lightweight aerospace or composite materials for the AR devices, but these materials continue to be expensive for mass production, according to a person familiar with the matter.

Apple’s work on AR hardware comes as other tech giants are investing heavily in the category. Facebook, for example, is eyeing a similar ship year of 2023 for its own augmented-reality glasses, code-named Orion, according to people familiar with the effort. In the meantime, Facebook is also developing a simpler, shorter-term pair of camera-equipped glasses in partnership with Ray-Ban, The Information previously reported.

Meanwhile, Snap is currently developing a fourth-generation version of its Spectacles camera glasses, code-named Hermosa, with smart lenses capable of showing AR effects. (A Snap spokesperson declined to comment on future hardware.) Snap CEO Evan Spiegel has said previously that AR glasses won’t become mainstream for roughly a decade.

In 2017, Apple CEO Tim Cook told The Independent that the technology “doesn’t exist” to build AR glasses “in a quality way.”

“I regard it as a big idea like the smartphone,” Cook said in another interview with The Independent that same year. “The smartphone is for everyone. We don’t have to think the iPhone is about a certain demographic, or country or vertical market; it’s for everyone. I think AR is that big.”