Y Combinator’s Garry Tan Goes to the MatRead more

OpenAI's Greg Brockman (left) and Google's Demis Hassabis (right). Photos by Getty.

OpenAI Hustles to Beat Google to Launch ‘Multimodal’ LLM

Photo: OpenAI's Greg Brockman (left) and Google's Demis Hassabis (right). Photos by Getty.

As fall approaches, Google and OpenAI are locked in a good ol’ fashioned software race, aiming to launch the next generation of large-language models: multimodal. These models can work with images and text alike, producing code for a website just by seeing a sketch of what a user wants the site to look like, for instance, or spitting out a text analysis of visual charts so you don’t have to ask your engineer friend what these ones mean.

Google’s getting close. It has shared its upcoming Gemini multimodal LLM with a small group of outside companies (as I scooped last week), but OpenAI wants to beat Google to the punch. The Microsoft-backed startup is racing to integrate GPT-4, its most advanced LLM, with multimodal features akin to what Gemini will offer, according to a person with knowledge of the situation. OpenAI previewed those features when it launched GPT-4 in March but didn’t make them available except to one company, Be My Eyes, that created technology for people who were blind or had low vision. Six months later, the company is preparing to roll out the features, known as GPT-Vision, more broadly.

Access on the go
View stories on our mobile app and tune into our weekly podcast.
Join live video Q&A’s
Deep-dive into topics like startups and autonomous vehicles with our top reporters and other executives.
Enjoy a clutter-free experience
Read without any banner ads.
Former Apple design chief Jony Ive and OpenAI CEO Sam Altman. Photos by Getty.
Exclusive
Designer Jony Ive and OpenAI’s Sam Altman Discuss AI Hardware Project
Jony Ive, the renowned designer of the iPhone, and OpenAI CEO Sam Altman have been discussing building a new AI hardware device, according to two people familiar with the conversations.
From left to right: Blair Effron, Robert Pruzan and David Handler. Photos by Getty; Tidal Partners.
Exclusive Finance
Disputes, Employee Misconduct Rattle Centerview’s Silicon Valley Dreams
The San Francisco Bay Area–based bankers at Centerview Partners, the investment bank that advised Silicon Valley Bank’s owner and Credit Suisse through recent turmoil, got two doses of bad news last week.
Art by Clark Miller
Exclusive startups entertainment
MasterClass Takes a Crash Course in Frugality
MasterClass had a problem with the shoot featuring its latest star instructor, Walt Disney Co. CEO Bob Iger.
OpenAI CEO Sam Altman and Microsoft CEO Satya Nadella. Photos via Getty
Exclusive microsoft ai
How Microsoft is Trying to Lessen Its Addiction to OpenAI as AI Costs Soar
Microsoft’s push to put artificial intelligence into its software has hinged almost entirely on OpenAI , the startup Microsoft funded in exchange for the right to use its cutting-edge technology.
From left: Paul Graham, Garry Tan and Michael Seibel. Photos by Getty. Art by Mike Sullivan.
Exclusive startups ai
Y Combinator’s Garry Tan Goes to the Mat
Garry Tan was in his happy place. Surrounded by food trucks and techies basking in San Francisco’s September sun, the CEO of Y Combinator snapped selfies with entrepreneurs as he meandered through a crowd of 2,700 attendees at the startup accelerator’s annual alumni event.
Dave Rogenmoser, cofounder of Jasper. Photo via Getty.
Exclusive startups ai
Jasper, an Early Generative AI Winner, Cuts Internal Valuation as Growth Slows
Jasper AI, an early darling of the generative artificial intelligence boom, has cut the internal value of its common shares 20%, according to former employees who were notified by the company.