Large language models (LLMs) are gaining popularity in academia and industry due to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level but also at the society level, to better understand their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. 

At the Global IndiaAI Summit 2024, organised at Bharat Mandapam, New Delhi, a session was held on the topic "IndiaAI: Large Language Models". Amitabh Nag, CEO of Bhashini, moderated the session. Srinivas Narayanan, Vice President of Open AI, delivered the keynote address during the session. Other panelists of the session include, Prof. Ganesh Ramakrishnan, IIT Bombay; Ms. Shalini Kapoor, Chief Technologist APJ, AWS; Dr. Mohit Sewak, AI Researcher and Developer Relations, South Asia, NVIDIA; Dr. Pratyush Kumar, Co-founder, Sarvam AI and Dr. Kalika Bali Principal Researcher, Microsoft. 

The session began with the moderator introducing the IndiaAI Mission and INDIAai Innovation Centre. "The AI revolution is the major innovation after the Industrial Revolution," said Nag during the introduction.

While delivering the keynote address, Srinivas Narayan spoke about where we are with LLMs and the possibilities of LLMs. "LLMs are the most natural ways to interact with technology, and this is happening for the first time", he remarked. He also spoke about the challenges that diversity in language poses in India and the importance of safety while developing LLMs. As humans, providing expertise with scale is one of the significant problems we face. AI can help humans in tackling this issue and be more productive.

Developing LLMs in India

AI is a new technology. The process of development is ongoing and has a long way to go. India, as a country, should be more optimistic. "LLMs up the game a bit", said Dr. Pratyush Kumar, speaking about the development of LLMs in India. He opines that India needs to open-source the development process while considering culture and language. Though numerous entities remark production cost as a significant challenge in the process, according to Pratyush Kumar, the development cost is rapidly decreasing since computing is becoming increasingly efficient. This is a favourable trend for creating LLMs in the country.

Data availability and management are significant challenges that Indian engineers face. Shalini Kapoor remarks that using AI models will pump up data creation. She emphasises the need for domain-specific LLMs that can solve domain-specific problems. For instance, an AI model exclusively made for healthcare might scale better than a generic model.

Stakeholder collaboration is significant for the development of AI models. The success of BharatGPT is highly supported by its "public-private partnerships". "Collaboration between industry and academia is a necessity for developing massive AI models", said Prof. Ganesh Ramakrishnan, speaking about the importance of stakeholder collaboration.

Respecting Indian culture

According to Dr. Kalika Bali, LLMs today have an outsider view of Indian culture. For developing an India-centric model, an insider point of view is essential. Furthermore, no definition of bias is agreed upon. "We can never have a bias-free system; it gets complicated when we have many languages. When talking about data collection, we have to be sensitive about that," she said.

India is a nation with large linguistic diversity. "Any model required to deal with these any languages should have a tokeniser vocabulary of 2 lakhs 54 thousand", said Dr. Mohit Sewak. He suggest the nation to have more number of multi-modal models to satisfy the country's conversational needs. "If we want real Indian LLM , we need tokenisers with tens and trillions of data", he added.

Furthermore, the domination of Western companies over Indian startups also centres around India's language challenges. Training Indian models with Indian languages requires higher financial investments than training foreign models. 

Dr Kalika Bali remarked that to bring Indigenous expertise into LLMs, we require specialised datasets. "Big models might not be the right solution; rather, creating a federation of models and having an orchestration around it might be the way forward," she added.

Tackling the challenges

To tackle the challenges of dialect variation, Prof. Ganesh suggests documentation of dialects, federated learning, data sharing among stakeholders, and respecting inclusivity. Because of the vast population of the country, India has the capacity to be the use case capital of the world. Business value derived from the model will define a lot of usage of AI itself.