Meta AI has developed PEER, a collaborative language model trained to simulate the writing process. The announcement was via a tweet.

Textual content frequently results from a collaborative writing process where we start with a rough draft, solicit feedback, and make numerous revisions. Unfortunately, the language models of today are to produce only the result, being indifferent to this process. They consequently lack several skills necessary for group writing, including: 

• They are unable to update previously written texts, are challenging to control, and 
• they are unable to plan or justify their actions verbally. 

Writing aid language model

The Meta AI presents PEER, a collaborative language model that mimics the entire writing process, to address these shortcomings: PEER can create drafts, add suggestions, suggest edits, and justify its choices.

To use self-training techniques to increase the quality, quantity, and diversity of training data, the researchers train multiple instances of PEER that can fill in different gaps in the writing process. Enabling PEER to be used in domains without edit histories and enhancing its capacity to adhere to instructions, produce insightful comments, and justify its actions maximizes PEER's potential. The researchers demonstrate that PEER performs well in various domains and editing tasks.

Image source: Twitter

Plan, edit, and explain are the three steps that make up the PEER process, as shown in the illustration above. 

• The action uses an edit after the model or user specifies the plan with the necessary action. 
• The model is then processed by outlining the edits in plain language and citing pertinent references. 
• The process repeats until there are no more updates for the generated text.

Conclusion

Researchers at Meta AI created PEER to enhance the model's capacity to write texts in various domains. With the aid of self-training techniques, the team has trained multiple instances of the model to incorporate different aspects of the writing process to improve the quality and diversity of training data.

Furthermore, with PEER, one can make edits in a variety of fields, which improves its ability to follow directions and cite and quote from pertinent documents. As a result, it is more trustworthy and verifiable than regular large language models.

Read the full paper here to learn more about PEER.