The world is in an intense race to offer the best in AI. ChatGPT, which elevated the graph, recently acted as a catalyst in developing large language models. 

 Large Language Models (LLMs) have shown great promise as highly capable AI assistants that excel in complex reasoning tasks requiring expert knowledge across various fields, including in specialized domains such as programming and creative writing. They enable interaction with humans through intuitive chat interfaces, which has led to rapid and widespread adoption among the general public. 

 Recently, Meta announced partnering with Microsoft to introduce Llama 2, its next-generation open-source language model.  

 The Meta CEO stated that the model will soon be free for researchers and other commercial use. The latest powerful model has been tipped against Open AI's GPT-4, which poses tools like ChatGPT and Microsoft Bing.  

 The announcement was made as part of Microsoft's Inspire event, noting its support for Azure and Windows and a "growing" partnership between the two companies. At the same time, Microsoft revealed more details about the AI tools built into its 360 platforms and how much it will cost. Qualcomm also announces it is trying to introduce Llama to laptops, phones and headsets starting from 2024 onward for AI-powered apps that work without relying on cloud services.

Meet Llama 2 

According to Meta, the decision to open Llama 2 is a way to give businesses, startups, and researchers access to more AI tools, allowing for experimentation as a community. Llama 2 was pre-trained on publicly available online data sources. The fine-tuned model, Llama-2-chat, leverages publicly available instruction datasets and over 1 million human annotations.  

To create the new family of Llama 2 models, researchers began with the pretraining approach using an optimized auto-regressive transformer but made several changes to improve performance. Specifically, they performed more robust data cleaning, updated our data mixes, trained on 40% more total tokens, doubled the context length, and used grouped-query attention (GQA) to improve inference scalability for our larger models. 

 The training corpus included a new mix of data from publicly available sources, which does not include data from Meta's products or services. The researchers tried to remove data from certain sites known to contain a high volume of personal information about private individuals. They trained on 2 trillion tokens of data as this provides a good performance–cost trade-off, up-sampling the most factual sources to increase knowledge and dampen hallucinations. 

 While developing the model, Meta researchers dwelled on practicing responsible AI. In the paper, they explained safety investigations into pretraining data and pre-trained models. They spoke about safety in pertaining, fine-tuning and red teaming. 

Limitations and ethical concerns 

At the project's outset, the researchers preferred supervised annotation, attracted by its denser signal. Meanwhile, known for its instability, reinforcement learning seemed a somewhat shadowy field for those in the NLP research community. However, reinforcement learning proved highly effective, particularly given its cost and time effectiveness. Their findings underscore that the crucial determinant of RLHF's success lies in the synergy between humans and LLMs throughout the annotation process.  

Llama 2-Chat is subject to the same well-recognized limitations of other LLMs, including a cessation of knowledge updates post-pretraining, potential for non-factual generation such as unqualified advice, and a propensity towards hallucinations. The model's performance in languages other than English remains fragile and should be used cautiously. 

 Not everyone using AI models has good intentions, and conversational AI agents could potentially be used for nefarious purposes, such as generating misinformation or retrieving information about bioterrorism or cybercrime. The researchers have, however, tried tuning the models to avoid these topics and diminish any capabilities they might have offered for those use cases.  

According to the analysis, these models have demonstrated their competitiveness with existing open-source chat models and a competency equivalent to some proprietary models on evaluation sets we examined. However, they still lag behind other models like GPT-4. 

Meta AI

Meta research paper