Before we get to today’s column, check out this story about last week’s Neural Information Processing Systems conference, where a number of researchers cast doubt on some of today’s most popular techniques for improving AI. Some researchers, including at OpenAI, have ideas for new techniques that could produce better results.

Other researchers such as David Luan, who leads an Amazon AI research unit, have issue with the order and focus of steps in the model training process.

For instance, if a model will be used to help customers with refunds as a part of a customer service bot, it will typically be taught to handle such tasks in the latter stages of its development rather than in the initial “pretraining” phase. Luan argues that the model should be taught such tasks earlier on.

“It‘s really weird right now that in pretraining, we force the model to spend like 50% to 80% of its compute budget learning an entirely unrelated set of tasks [to what it’s eventually going to do], hoping that it transfers to the problem at hand,” he said.

On to today’s column…

With OpenAI and a handful of high-profile AI names commanding so much attention, it can be easy to forget that the longer tail of heavily-funded startups are charting their own paths—and burning a lot less cash because they aren’t developing state-of-the-art models.

For instance, Canadian startup Cohere, which sells AI to enterprises while also developing its own models, enjoys gross profit margins that are “much closer to traditional software sales” margins than to the margins of bigger AI developers, CEO Aidan Gomez said, without specifying the number.