These are the most intriguing AI research papers published this year. It combines artificial intelligence (AI) and data science breakthroughs. It is chronologically organised and includes a link to a longer article.

Meta-X NLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation

Recent developments in multilingual and cross-lingual transfer research, in which supervision is transmitted from high-resource languages (HRLs) to low-resource languages (LRLs), have been observed by the NLP community. Unfortunately, the cross-language transfer is rarely consistent between languages, especially in the zero-shot setting. An intriguing research subject is finding shared structures across numerous tasks with little annotated data. While most languages are low-resource and share some grammatical structures with other languages, multilingual applications may benefit from such a learning environment.

In this paper, the authors offer a unique meta-learning framework (Meta-XNLG) for learning shared structures from typologically different languages using meta-learning and language clustering. It's a step in the direction of uniform cross-lingual transmission for unseen languages. First, the researchers cluster the languages based on their linguistic representations, and then they determine the language at the centre of each cluster. Then, a meta-learning algorithm is trained with all centroid languages and evaluated using the zero-shot setting on the remaining languages. Finally, they illustrate the efficacy of this approach for two NLG tasks (Abstract Text Summarization and Question Generation), five popular datasets, and thirty typographically distinct languages. Steady improvements over solid baselines illustrate the framework's effectiveness. Moreover, due to the model's meticulous design, this end-to-end NLG configuration is less susceptible to unintended translation, which is a significant worry in zero-shot cross-lingual NLG jobs.

OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages

Recent advances in AI technology for Natural Languages have been remarkable. Nevertheless, comparable progress has yet to be realised in Sign Languages, namely in identifying signs as individual words or whole phrases. The researchers present OpenHands, a library for word-level recognition in sign languages that applies four core concepts from the NLP field for low-resource languages.

The researchers released standardised pose files for six distinct sign languages: American, Argentinean, Chinese, Greek, and Indian. Second, they train and provide baselines and deployment-ready checkpoints for four pose-based isolated sign language recognition models across all six languages. Third, the researchers offer unsupervised, self-supervised pretraining to overcome the lack of labelled data. Fourth, researchers compile and distribute the largest pose-based dataset for Indian Sign Language (Indian-SL). Then, they compare different pretraining strategies and demonstrate for the first time that pretraining is effective for sign language recognition by showing: 

(a) improved fine-tuning performance, particularly in low-resource settings, and 

(b) high cross-lingual transfer from Indian-SL to a small number of other sign languages.

Furthermore, the researchers open-source all models and datasets, which are available here.

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages

Multilingual language models that have already been trained, like mBERT and XLM-R, have shown a lot of promise for zero-shot cross-lingual transfer to low web-resource languages (LRL). But because model capacity is limited, the big difference in the sizes of monolingual corpora between high web-resource languages (HRLs) and low web-resource languages (LRLs) does not give enough room for co-embedding the LRL with the HRL, which hurts the performance of LRLs in tasks that come after.

In this paper, the researchers argue that the similarity between languages in the same language family in terms of lexical overlap could be used to get around some of the problems with LRLs' corpora. They suggest Overlap BPE (OBPE), a simple but effective change to the BPE algorithm for making new words that increases the overlap between related languages. Researchers have done a lot of tests on different NLP tasks and datasets and found that OBPE creates a vocabulary that makes it easier for LRLs to be represented by using tokens also used in HRLs. It makes the zero-shot transfer from related HRLs to LRLs better without lowering the accuracy or representation of HRLs. In contrast to earlier studies that didn't think token overlap was significant, the researchers show that it is essential in low-resource language settings. If the overlap is cut down to zero, the zero-shot transfer accuracy can drop by as much as four times.

Image source: Unsplash