Large language models, oh how they shine, 

Generating text that's truly divine.

Trained on vast amounts of data,

They can produce text that's truly great.

From stories to poems, they've got it all,

Creating works that truly stand tall.

Their creativity knows no bounds,

Generating text that truly astounds.

But don't be fooled, for they are not truly creative,

Simply drawing on patterns and things that are native.

Yet still they inspire, and help us create,

Large language models, oh how great.

Above lines are from a poem written by ChatGPT on large language models, a conversational AI chatbot developed by OpenAI that gained a significant amount of traction and users — 1 million in five days after being made available to the public.

Can you really believe it - machines writing poems and code just like us, talking and joking just like you and me. Mind blowing! And all this is made possible because of Large Language Models (LLMs). LLMs are designed to process and understand natural language. These models are typically trained on humongous amount of text data, allowing them to accurately analyze and generate human-like text. 

LLM models, such as PaLM, ChatGPT, LaMDA, GPT3 have been shown to achieve state-of-the-art performance on a variety of natural language processing tasks. They are typically trained using unsupervised learning, which means that they are not explicitly provided with the correct output for a given input, but instead must learn to generate reasonable outputs based on the input data.

LARGE Language Models are truly LARGE!

Analysts say that the NLP market is rapidly growing from $11B(2020) to $35B+ (2026). But it’s not just the market size that’s huge. The model size and the number of parameters involved is also large. The below figure (image source: link) shows how the size of LLM models is increasing exponentially over the last few years.

Why are LLMs becoming important?

LLMs are gaining importance for several reasons. 

• Remarkable outcome: LLMs have demonstrated impressive capabilities in generating human-like text. They are able to generate text that is difficult to distinguish from text written by humans. This ability has a wide range of potential applications, including natural language processing, language translation, text generation, and many others.
• Availability of cheap compute resources: Compute resources have become cheaper due to the adoption of Cloud architectures and advancement in technology.
• Change in consumer behaviour: The GenZ and millennials prefer communication using natural language, rather than the “Press 1 to continue” sort of communication.
• Insights from unstructured data: Estimates say that unstructured data makes up more than 80% of the enterprise data, and is growing at the rate of 55% per year. In order for companies to truly get insights from unstructured data, it is important to develop the required technology.

Consuming LLMs

How do companies consume the power of LLMs? Below are a few approaches:

1.Existing LLM Models: Companies do not have to train large custom models from scratch for each task or use case. Instead, they can easily leverage the state of the art commercially available LLMs in the form of API, like the ones provided by OpenAI.

2.Building LLMs: There are industries like financial, banking and healthcare which are highly regulated and secure. They might have restrictions in sending data online. In this scenario, they can think of building LLMs using one of the below approaches:

2.1 Building LLMs from Base Models: Customers can leverage pre-trained transformer models that are already trained on a large corpus of data in a self-supervised fashion.This raw model can be further fine-tuned on a downstream task. There are several base models provided by HuggingFace.

2.2. Building LLMs from Scratch: In case companies are looking for models in specific languages for which no base models are available, then they can think about building the LLM from scratch. But there are a couple of points that they need to keep in mind, these are explained in the next section.

Evaluation Strategy for LLMs

Now that we’ve our LLM model available, how do we evaluate it? There are distinguished methods available to evaluate large language models which are as follows:

• GLUE: General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding tasks, including single-sentence tasks CoLA and SST-2, similarity and paraphrasing tasks MRPC, STS-B and QQP, and natural language inference tasks MNLI, QNLI, RTE and WNLI. These are used to benchmark and compare various LLM models.
• Evaluation based on downstream tasks: Other metrics or assessment techniques may be more relevant depending on the task or domain for which the language model is being utilized. For example, a LLM model used for text classification, might be assessed using typical classification metrics such as precision, recall, and F1 score. But there are other specific metrics that can be explored, depending on the downstream task:
• Perplexity: Perplexity is a statistical measure of how confidently a language model predicts a text sample. A low perplexity indicates that the model can predict the test set well, while a high perplexity indicates that the model is not able to predict the test set well.
• BLEU score: One of the ways to evaluate a LLM for translation is the BLEU (bilingual evaluation understudy) score which is a performance metric that is used to compare the quality of machine translation to a reference translation. Its values lie between 0 and 1. Values closer to 1 indicate that the candidate text is very similar to the reference text.
• Human assessment: While the above evaluation approaches are statistical and automatic, it is strongly recommended to have human evaluators to assess the model in terms of creativity, humor and toxicity. A framework can be developed to score the output of the models depending on the downstream task.

It's worth noting that no single evaluation method is perfect, and it's often useful to use a combination of methods to get a more complete picture of a model's performance.

Risks of using LLMs

The use of large language models carries several risks, both ethical and technical.

• Bias, fairness and toxicity: One of the main ethical risks of LLM is the potential for bias in the data used to train the models. If the data is not representative of the population, the model may make biased or unfair decisions. For example, a model trained on mostly male-authored text may be more likely to associate certain words or phrases with men rather than women.
• Misuse: Another social risk of LLMs is the potential for misuse. Large language models have the ability to generate highly convincing text, which can be used to create fake news or impersonate individuals online. This can lead to the spread of misinformation and harm the reputation of individuals or organizations.
• Adversarial attacks: From a technical standpoint, large language models are also vulnerable to adversarial attacks, where an attacker manipulates the input to the model to produce a desired output. This can lead to the model making incorrect or malicious decisions.
• Private data leaks: If the training data is not masked or de-identified, then there are chances that the LLMs could infer and leak this private or sensitive information. 
• Sustainable LLMs: With sustainability being an important concept, the training and inference of LLMs can incur high carbon footprint and environmental costs.
• Accountability for mis-information: Like any other ML model, accountability is a key issue. If the LLM model gives incorrect suggestions or predictions for a given downstream task, then who will be accountable and responsible.

Overall, the use of large language models carries significant risks, and careful consideration must be given to mitigate these risks and ensure their responsible and ethical use.

Do "LARGE" Language Models mean "LARGE" Pockets?

Even if you are training a model with billion parameters or leveraging commercially available LLMs, it would be heavy on the pockets. So before starting any LLM project, it's super important to make sure that there is a clear business value proposition and ROI to convince the business stakeholders. It’s not advisable to implement the state-of-the-art models without any clear business direction.

Conclusion

Large language models have the potential to revolutionize the field of AI, enabling machines to better understand and interact with humans in a natural language setting. However, their use must be carefully considered and regulated to ensure their benefits are maximized and their potential dangers are minimized. 

Looking forward to seeing what the future holds in this interesting space!