“Here’s the sort of startup that might make a go of it on Mars: One that makes money by taking advantage of local resources. For example, a startup that uses the Martian atmosphere to make fertilizer or methane fuel. A startup that sells water from Martian rivers. A startup that uses minerals from the Martian soil. A startup that uses solar power from the Martian sun. A startup that uses a new kind of solar panel that’s more efficient at converting sunlight into electricity.

I would even invest in a startup that did nothing but extract the helium-3 from the Martian atmosphere and send it back to Earth.If you can find a way to make money by using local resources, there’s probably a business there.”

As some of you might have assumed from the title of this article, the above words aren’t from an essay authored by any of the prominent futurists or entrepreneur. In fact, it isn’t even authored by a human. They are the work of the latest iteration in the long line of language models called GPT-3 created by OpenAI based on a short prompt “how to make Mars a startup hub “given by Paul Graham and published on paulgraham.com.

This isn’t the first time language models had blown our minds, the predecessor of GPT-3, the GPT-2, itself was quite powerful that we even had it write long-form articles for this portal, at the risk of triggering the next world war. However, unlike all the other breakthrough language models since BERT, GPT-3 is armed with something a unique and powerful characteristic, which is its ability to be “very good at few-shot learning”, or the ability of the program to learn from a very small set of data. 

Nothing explains this superpower better than the short story titled “The importance of being on Twitter,” shared by Mario Klingemann, authored by the model in the style of legendary writer Jerome K. Jerome. 

The short story opens with the following line, surprisingly similar to the writing style of Jerome K Jerome: “It is a curious fact that the last remaining form of social life in which the people of London are still interested is Twitter. I was struck with this curious fact when I went on one of my periodical holidays to the sea-side and found the whole place twittering like a starling-cage.” 

What’s more shocking here is that Klingemann only gave the title, the author’s name and the word “it,” as a prompt to produce the above prose, demonstrating the power of GPT-3’s “few shot learning” capability, which marks a milestone in the swift progress NLP has been making in the last decade or so. 

Following the huge success and fanfare of GPT-2, GPT-3, or the Generative Pre-Trained Transformer-3, was first described in a research paper published in May 2020 titled ‘Language Models are Few-Shot Learners.’ 

For many years, NLP, despite being a prominent subset of Artificial Intelligence, was often considered as the second class citizen of the AI world as Computer Vision stole all the attention following the ImageNet breakthrough. 

Things started to take a turn in NLP after the introduction of Recurring Neural Networks in 2013, and the Attention Architecture in the following year. But NLP’s development remained unremarkable as these new models were only able to handle phrases or short sentences. The much-awaited breakthrough, the ImageNet equivalent event in NLP, happened in 2018, thanks to the brilliant minds at Google who created a model called Bidirectional Encoder Representations from Transformers or BERT. 

BERT introduced the Transformer Architecture, which used a parallelization technique, as multiple neural network encoders and decoders worked together to understand the key elements of a sentence such as part of speech tags, constituents, dependencies, semantic roles, coreferences and relations, which were not possible in the early RNN powered models. Transformers provided the NLP models with the much-desired capability to process whole sequences of sentences at once. 

It is based on this same Transformer Architecture OpenAI built their GPT models. In fact, the GPT-3 itself isn’t much different from GPT-1 or GPT-2 in terms of its core technology. But where it differs is in the humongous datasets it was trained on. 

While the GPT-1 and GPT-2 were trained on 110 million and 1.5 billion parameters respectively, GPT-3 was trained on “an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting,” as pointed out in the research paper. As a result, it was found that “GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans,” according to the researchers. 

To date, GPT-3 is the most powerful language model ever created, but comparing it to the beginning of the creation of truly human-like Articifical General Intelligence by various news outsources are misplaced. GPT-3 can “generate amazing human-like text on demand but won’t bring us closer to true intelligence,” writes MIT Technology Review’s Senior Editor Will Douglas Heaven in his article. 

A much simpler way of explaining GPT-3’s abilities comes from Arran Sabeti, who is one of the developers who got early access to GPT3. “I’ve gotten it to write songs, stories, press releases, guitar tabs, interviews, essays, technical manuals. It’s hilarious and frightening. I feel like I’ve seen the future and that full AGI might not be too far away,” he noted on his blog. 

According to Wired Magazine’s Sophie Epstein, “the text generator is arguably the world’s most impressive AI”, as many people have discovered that GPT-3 can generate any kind of text, including guitar tabs or computer code.

A better way to understand the model’s ability and limitation is to see it as a little kid blessed with an exceptional memory and brilliant writing skills, but lack the reasoning or analytical skills of a grown up. 

“GPT-3’s human-like output and striking versatility are the results of excellent engineering, not genuine smarts. For one thing, the AI still makes ridiculous howlers that reveal a total lack of common sense. But even its successes have a lack of depth to them, reading more like cut-and-paste jobs than original compositions,” wrote Will Douglas Heaven in his article. And that’s exactly where lies its biggest problems.

Unlike its predecessors, Open AI has set out plans to turn GPT3 into a commercial product later this year, raising more questions on its reliability and new controversies. In fact, it was the organization’s decision to make GPT-2 public and pivot towards making commercial products from its previous non-profit nature was one of the core reasons that forced co-founder Elon Musk to leave OpenAI a year ago. 

The model had faced much criticism over the last few weeks, especially when Facebook’s head of AI Jerome Pesenti pointed out the bias coming out of a program created with GPT-3. The program mentioned here is a tweet generator in which anyone could type in a word, and the GPT-3 API will come up with a relevant tweet.

“GPT-3 is surprising and creative, but it’s also unsafe due to harmful biases,” he noted in his tweet. “We need more progress on Responsible AI before putting NLG models in production.”

In his test, Pesenti found that the words Jews, blacks, women and holocaust, had come up with some shocking results, exposing the bias contained in its code. “AI systems copying the human prejudices – including, but not limited to, racism and sexism – based on the data they learn from has been well documented. The real question is, what can OpenAI do about it before the system is made commercially available in the future?,” writes Epstein.

We ourselves ran a few tests on the above-mentioned program, and the results we got were nothing short of shocking, especially to the words Muslims and BlackLivesMatters. Some of them are so extreme and insensitive, often coming across as hate speech, we chose not to list them here. 

On the other hand, after being notified about the biases, Sam Altman, the CEO of OpenAI, has made it clear that the apps will undergo safety review before they go live from the current Beta stage. The company has also noted in a Twitter thread that the model “can display both overt and diffuse harmful outputs, such as racist, sexist, or otherwise pernicious language”.

However, the significant concerns GPT-3 raises are in its possible commercial use cases. Mass production of carefully crafted believable misinformation would be one severe outcome of models such as GPT-3, which we have previously explored. On the other hand, the commercial use case of the software could span across anything from creating legal documents to creating academic contents, and in those scenarios, the credibility of the text generated by the model will raise more flags unless the makers can iron out the biases plaguing its code. 

Going into the future, it is becoming more and more important that we should explore the possibility of labelling the content created by programs such as GPT-3 as “machine generated or AI generated”, to provide better transparency as well as to give the human readers the opportunity to overlook some of the biases it may still demonstrate. 

From a technology enthusiast point of view, GPT-3 is indeed a landmark moment in AI, alongside DeepBlue and AlphaGo. On the other hand, the AI Ethicist in me is certainly worried about the biases it has been exhibiting- a sign of AI programs adopting some of the most undesirable traits from its human creators, as well as the possible premature deployment of these tools across the industries in a bid to gain technological superiority by ignoring the moral and ethical norms. Only time will tell the answer, as GPT-3 cannot complete prompts with predictions of future events as of now.