The price of the most important “raw materials” underpinning the current AI boom is falling fast, which will help the technology move more quickly into the mainstream. However, it also threatens the finances of start-ups hoping to cash in on the boom, and could lead to the concentration of industry dominance in the hands of a small group of companies.
The raw material here refers to the processing power of large language models (LLMS). These models underpin services such as ChatGPT and Microsoft’s new Bing search.
The high computing costs involved in running these models can be a serious drag on their widespread use. Richard Socher, chief executive of the search engine You.com and a prominent computer scientist, said that just a few weeks ago it was 50 percent more expensive for You.com to use artificial intelligence to deliver its search than traditional Internet search. But by the end of last month, that cost difference had dropped to only about 5 percent, thanks to intense competition among big language modeling companies OpenAI, Anthropic, and Cohere.
A few days later, OpenAI released a new service that lets developers use ChatGPT directly and cut the price of using the technology by 90 percent.
That’s great for customers, but it could be devastating for OpenAI’s competitors. A number of companies, including Anthropic and Inflection, have completed or are raising funds to support the development of their respective large language models.
Few technologies have evolved so quickly from laboratory research to large-scale commercial use, prompting researchers to “industrialize” the process of developing laboratory environments. Most of the performance gains and cost reductions come from optimizing the underlying computing platform that runs the large language model and improving the way the model is trained and run.
From one perspective, the dramatic drop in hardware costs benefits all market participants. These include high-performance chips designed to meet the requirements of the latest AI models, such as the Nvidia H100 GPU. Microsoft runs OpenAI’s model on its Azure cloud computing platform and offers the same cost-effective hardware support to other large language model companies.
However, large language models are as much art as science. Since December, ChatGPT has made a “series of system-wide optimizations” to the way it handles queries that have reduced costs by 90 per cent, leading to significant price cuts for users, OpenAI said.
It costs tens of millions of dollars to train large language models, and the technology for handling such tasks is changing fast. At least in the short term, a small group of people with model development and training experience will gain a greater advantage.
By the time the best technology is widely understood and adopted, early participants may have gained a first-mover advantage. Scott Guthrie, Microsoft’s head of cloud computing and artificial intelligence, pointed to new services like GitHub Copilot. The service, which launched last summer, offers code suggestions to software developers. After widespread use, such services will be quickly optimized. He told a Morgan Stanley investor conference this week that “signals” from users of such services will soon become an important point of differentiation.
The main hope of OpenAI’s competitors is to provide additional services that make it easier for developers and large enterprise customers to use large language models, as well as to explore models that meet specific business needs in niche markets.
This week, for example, Israeli startup AI21 Labs released its latest large language model, along with a set of apis (application program interfaces) to provide more advanced services such as text summarization or rewriting.
Ori Goshen, AI21’s co-chief executive, said most companies would not use a generic model like Chatgpt-but rather models trained for industries such as finance or healthcare, or based on a company’s own data.
He believes that the large language model is still in its infancy and much work needs to be done, such as reducing the model’s tendency to lie and preventing the model from “hallucinating” and providing plausible answers that have nothing to do with fact. Ai companies also need to keep pushing the frontiers if they want to succeed.
But the fact of the matter is that the underlying cost of these generative ais is falling dramatically. OpenAI’s price cut is a signal that the technology will move very quickly into large-scale commercial use. But it also comes with a warning that there may not be too many players in the industry that can afford to play in the future.