A wave of new chips designed to run artificial intelligence software will soon be hitting the market. Device makers are waiting to use them in products like cameras that can identify images faster, and cloud computing providers want them to give their services a technical edge. First though, the companies have to figure out which chips perform the best.
That’s why some of the biggest companies in tech are banding together to create a new standardized set of tests, called a benchmark, to analyze how well the chips perform at various tasks. While esoteric, the process of devising benchmarks can be surprisingly contentious, involving fierce technical battles and corporate politics. Participants are often the same companies that have heavy stakes in the results of the tests—namely, chip makers and cloud computing providers who use the scores to publicly boast about the advantages of their products and services. It is a bit like inviting students to craft the questions for an exam they’re about to take.