OpenAI released its beta version of GPT-3 (Generative Pre-trained Transformer 3) on June 11th, its brand new autoregressive language model that uses deep learning to produce human-like text. Since the release, the GPT-3 was the main discussion topic of many developers, entrepreneurs, journalists, and researchers. Some have even called it the next big thing in tech since blockchain.

What is GPT-3 exactly, and why should we care?

GPT-3 has 175 billion parameters. To put it in context, its predecessor GPT-2 had 1.5 billion parameters, which represents a 117x increase. It is also one order of magnitude larger than Microsoft’s Turing NLP. The model had to learn to fill blanks in sentences using only the surrounding words as context. It was trained using a dataset containing one trillion words. The model went through the LAMBADA, HellaSwag, and StoryCloze tests to measure its accuracy and was impressively accurate since it wasn’t tuned for the specific test.

The model is exceptionally effective in performing specific tasks without any special tuning. The GPT-3 is a powerful general-purpose “text in, text out” interface that needs only a small amount of guidance on the task. Businesses are now interested in seeing how they can use this model.

“GPT-3 is a powerful

general-purpose

“text in, text out”

interface that needs

only a small amount

of guidance on tasks. 

 

Businesses are now

interested in seeing

how they can use

this model.‘ 

 

 

What does this mean for businesses?

OpenAI just went from a not-for-profit to a for-profit company to cover the costs of research and development. They are also facing pressure from the tech companies and VC firms, which have been financing their research. Microsoft invested one billion dollars in the lab following its switch to for-profit. So, unsurprisingly, the AI will be available through a commercial API.

OpenAI has chosen a subscription-based business model with higher prices for bigger projects. This plan is particularly well-fitted for SaaS and B2B. OpenAI is betting on the businesses’ need for this service to break-even on its research cost.

Weeks after its beta release, developers have shared on Twitter applications of the AI. Generating code, creating spreadsheets, semantic searches, the AI seems to execute these tasks from simple queries nicely.

The GPT-3 could be a changemaker in several industries such as Accounting, Audit, Compliance, Legal services, and other industries where you can automate low-level tasks. According to the Financial Times, some developers have already tried implementing the model to output some interesting language tricks such as ” turning a piece of dense legal writing into more general language,[…] [gleaning] financial information from a piece of text and enter it into a spreadsheet, turning it into a form of an automated accountant. The model could also significantly improve workplace productivity by making professionals focus on higher-level tasks and evaluate the assumptions of their automated models.

 

The cost of training GPT-3

While the size of language models keeps increasing tenfold every year, the cost of training these networks also increases exponentially. In fact, according to one estimate, training the AI would take about 355 years and $4.6M using a Tesla V100 cloud instance. This has created concerns on AI’s accessibility in the future. As Stanford’s AI Lecturer Younes Mourri mentions: “I am worried that AI might become a field where only the rich will be able to compete, creating further gap inequalities.”. Coding used to be one of the least capital intensive industries where one could start without substantial upfront costs. Now, many startups in Silicon Valley are forced to change their business models due to the higher costs of running their business. This could potentially result in innovation being centralized by big tech companies.

Growing ethical concerns about AI research 

Like any significant tech innovation, the GPT-3 is a double-edged sword. The GPT-3 can also be used for evil purposes. If a handful of highly valued companies control the AI research output, they would be able to get away with sharing what benefits them. In fact, they could also use state-of-the-art algorithms to improve their ad recommendations and control their users’ psychology.

Additionally, the AI sounds a lot like a human. It didn’t pass the Turing test, which gauges whether people can detect if they are talking to machines or humans, due to two responses: “A pencil is heavier than a toaster” and “there are three bonks in a quoit.” The model is still impressively human-sounding, which raises a few concerns. It was relatively easy to check if a Twitter or Facebook account was a fake previously, but it will be increasingly harder with AI’s advancements. This could lead to a greater spread of fake news on social media platforms and could potentially threaten elections and polling.

 

 

It was relatively easy to 

check if a Twitter or

Facebook account was 

a fake previously, but

it will be increasingly

harder with AI’s

advancements.”

 

Will GPT-3 lead a paradigm shift in business?

To conclude, GPT-3 is a major advancement in the field of AI. We’ve never been closer to a human-sounding interface, and the fact that it’s a general algorithm is its real strength. However, the GPT-3 still needs to pass the business model test. The model needs to prove itself as a value-adding service for businesses. Moreover, the OpenAI CEO, Sam Altman, said that the “GPT-3 hype is way too much”. Like many other AI models, people often jump to conclusions about whether it would revolutionize the industry. If it passes the business model test, then the language model could have a tremendous impact and could slowly be implemented in finance, customer services, writing, programming, and many more areas.

 

Written by Wassim Boutabratine

LEARN MORE ABOUT AI

Privacy of the Commons: or why you should pay more attention to GDPR and your data privacy rights
Concerns about data privacy are changing   On the Internet, stranger danger is no longer a primary worry as consumers become more savvy, but data privacy is rapidly becoming an urgent issue...
Emotional AI: The New Step in Human Relationships
Emotional AI: The New Step in Human RelationshipsThe modern world is tough, fast and unforgiving. The increased expectations for individuals in both their work and leisure means that the way in...
What is AI for Good?
AI is often discussed in the context of incorporating ethics into automation or data analysis. This is different from AI for Good. At the AIBE Summit 2020, Manoj Saxena spoke about AI Global, which...
AI: Formula 1 and the Jeopardy Dilemma
Formula 1 (F1) is as much a battle between the world's best engineers as it is the world's best drivers. With pit stops being taken in under 2 seconds and drivers hitting speeds as high as 370kmh...
Checklists and frameworks for ethical AI design in business
While the top ethical issues in AI have been identified and are fairly well-known, we often do not know what this means on a technical level. Our ability to think through the consequences needs to...