In the last few years, India has consolidated its position as a front-runner in Artificial Intelligence (AI) thanks to a robust IT presence and enviable consumer base. According to IDC1, the AI market in India is expected to reach 7.8 billion USD by 2025. Despite low adoption rates, government agencies have also shown interest, particularly in Natural Language Processing (NLP) models. Understanding the potential of NLP, the government has introduced national-level initiatives, funding, partnerships, and research centres aimed at fueling innovation. 

Why it pays to invest in NLP 

NLP plays a pivotal role in bridging the gap between languages and computer science since human languages are complex to understand and implement for computers. Over time, several applications have been there, like machine translations, text summarization, and email filter applications. In addition, text-based chatbots or virtual assistants have evolved into sophisticated versions capable of handling time-consuming activities. 

Early training models to foundational models 

An early instance of NLP was the first chatbot, Eliza, which used pattern matching and substitution methodology to mimic human conversation, but in a limited fashion. However, being a rule-based model, a massive manual effort went into encoding rules, making systems rigid, brittle, and anything but natural. 

NLP later evolved into supervised machine learning that requires large amounts of well-labelled training data sets for every task. However, data quality also makes systematic errors, such as model bias, a potential challenge. In addition, new applications are slower to develop since creating labelled data is labor-intensive and harder to access.  

Big corporates are investing more in computational resources to enable AI models to consume data efficiently, whilst foundational models are causing a significant shift. Foundational models, a term popularized by the Stanford Institute for Human-Centered Artificial Intelligence, use transfer learning to accomplish tasks and overcome the restrictions of earlier models.  

Demystifying foundational models 

Foundation models use extensive training data and self-supervised learning, such as filling in a missing word or predicting the next word in a running text to learn representations of all words. The knowledge does not require any labelled data; therefore, a considerable amount of data can be used for this purpose.  

The model can then be fine-tuned using very little labelled data for a specific use case. For example, a sentiment analysis model can be trained using very little data containing some reviews and their sentiment polarity. Certain banks employ sentiment analysis to assess borrower sentiments and improve debt collection rates, while human resource departments use the same model to improve employee retention rates. 

The key to the future 

Although the dream of creating applications with human capabilities is far-fetched, given the cost of AI resources, foundational models can change how AI is adopted across industries, domains, and languages. When scaling AI, these models make sense from a future cost perspective and functionality. Imagine performing tasks simply using prompts while using the same foundational model! 

Here is how these models benefit enterprises: 

• Self-supervised learning 

Foundational models can learn at scale, saving the time required to label data sets and reducing the cost of developing applications. In addition, once trained, very little supervised data is necessary to create specific applications, substantially reducing training time, effort, and resources. 

•  Access to pre-trained models 

Enterprises can use pre-trained foundational models to build specific NLP applications as communities democratize. For example, meta AI’s OPT, Big Science’s Bloom, and OpenAI’s GPT3 and ChatGPT are known to open foundational models on which many NLP applications are built. In some cases, open models are enabled with APIs for easy integration. 

•  Consumption of lesser resources 

A common deterrent with training and deploying AI models is the cost of specialized hardware. Typically, clusters of graphics processing units (GPUs) or tensor processing units (TPUs) are needed to train and fine-tune models quickly. But with foundational models, once pre-trained, they can be used for multiple purposes, and therefore enterprises can expect a reduction of overall computing cost. 

•  Training in natural languages 

The excitement with NLP for India is that it ties in with the government’s vision to provide e-services in Indian languages and accelerate its plans to build next-generation conversational applications under the National Language Translation Mission. For this, we need to create foundation models in all official languages in the country. However, a roadblock in creating these models in Indian languages is gathering sufficient digital data in India’s 22 Official languages. But deepening 4G, broadband connectivity, and digital and social networking platforms will eventually make digital data more accessible. 

 With a spate of niche offerings, NLP language models in automation, analytics, speech synthesis, and conversational AI is rising in usage and popularity. New use cases will emerge as further developments occur, bolstering India’s presence globally.