OpenAI released its beta version of GPT-3 (Generative Pre-trained Transformer 3) on June 11th, its brand new autoregressive language model that uses deep learning to produce human-like text. Since the release, the GPT-3 was the main discussion topic of many developers, entrepreneurs, journalists, and researchers. Some have even called it the next big thing in tech since blockchain.
What is GPT-3 exactly, and why should we care?
GPT-3 has 175 billion parameters. To put it in context, its predecessor GPT-2 had 1.5 billion parameters, which represents a 117x increase. It is also one order of magnitude larger than Microsoft’s Turing NLP. The model had to learn to fill blanks in sentences using only the surrounding words as context. It was trained using a dataset containing one trillion words. The model went through the LAMBADA, HellaSwag, and StoryCloze tests to measure its accuracy and was impressively accurate since it wasn’t tuned for the specific test.
The model is exceptionally effective in performing specific tasks without any special tuning. The GPT-3 is a powerful general-purpose “text in, text out” interface that needs only a small amount of guidance on the task. Businesses are now interested in seeing how they can use this model.
“GPT-3 is a powerful
“text in, text out”
interface that needs
only a small amount
of guidance on tasks.
Businesses are now
interested in seeing
how they can use
What does this mean for businesses?
OpenAI just went from a not-for-profit to a for-profit company to cover the costs of research and development. They are also facing pressure from the tech companies and VC firms, which have been financing their research. Microsoft invested one billion dollars in the lab following its switch to for-profit. So, unsurprisingly, the AI will be available through a commercial API.
OpenAI has chosen a subscription-based business model with higher prices for bigger projects. This plan is particularly well-fitted for SaaS and B2B. OpenAI is betting on the businesses’ need for this service to break-even on its research cost.
Weeks after its beta release, developers have shared on Twitter applications of the AI. Generating code, creating spreadsheets, semantic searches, the AI seems to execute these tasks from simple queries nicely.
The GPT-3 could be a changemaker in several industries such as Accounting, Audit, Compliance, Legal services, and other industries where you can automate low-level tasks. According to the Financial Times, some developers have already tried implementing the model to output some interesting language tricks such as ” turning a piece of dense legal writing into more general language,[…] [gleaning] financial information from a piece of text and enter it into a spreadsheet, turning it into a form of an automated accountant. The model could also significantly improve workplace productivity by making professionals focus on higher-level tasks and evaluate the assumptions of their automated models.
The cost of training GPT-3
While the size of language models keeps increasing tenfold every year, the cost of training these networks also increases exponentially. In fact, according to one estimate, training the AI would take about 355 years and $4.6M using a Tesla V100 cloud instance. This has created concerns on AI’s accessibility in the future. As Stanford’s AI Lecturer Younes Mourri mentions: “I am worried that AI might become a field where only the rich will be able to compete, creating further gap inequalities.”. Coding used to be one of the least capital intensive industries where one could start without substantial upfront costs. Now, many startups in Silicon Valley are forced to change their business models due to the higher costs of running their business. This could potentially result in innovation being centralized by big tech companies.
Growing ethical concerns about AI research
Like any significant tech innovation, the GPT-3 is a double-edged sword. The GPT-3 can also be used for evil purposes. If a handful of highly valued companies control the AI research output, they would be able to get away with sharing what benefits them. In fact, they could also use state-of-the-art algorithms to improve their ad recommendations and control their users’ psychology.
Additionally, the AI sounds a lot like a human. It didn’t pass the Turing test, which gauges whether people can detect if they are talking to machines or humans, due to two responses: “A pencil is heavier than a toaster” and “there are three bonks in a quoit.” The model is still impressively human-sounding, which raises a few concerns. It was relatively easy to check if a Twitter or Facebook account was a fake previously, but it will be increasingly harder with AI’s advancements. This could lead to a greater spread of fake news on social media platforms and could potentially threaten elections and polling.
“It was relatively easy to
check if a Twitter or
Facebook account was
a fake previously, but
it will be increasingly
harder with AI’s
Will GPT-3 lead a paradigm shift in business?
To conclude, GPT-3 is a major advancement in the field of AI. We’ve never been closer to a human-sounding interface, and the fact that it’s a general algorithm is its real strength. However, the GPT-3 still needs to pass the business model test. The model needs to prove itself as a value-adding service for businesses. Moreover, the OpenAI CEO, Sam Altman, said that the “GPT-3 hype is way too much”. Like many other AI models, people often jump to conclusions about whether it would revolutionize the industry. If it passes the business model test, then the language model could have a tremendous impact and could slowly be implemented in finance, customer services, writing, programming, and many more areas.
Written by Wassim Boutabratine