These are the most exciting AI research pieces published this year. It combines advances in AI and data science. It is organized chronologically and includes a link to a longer article.

Is Your Data Relevant?: Dynamic Selection of Relevant Data for Federated Learning

Federated Learning (FL) involves several clients learning a global machine learning model at the central server. Clients share updates from their local data with the server without compromising their privacy. These updates are aggregated and applied to the global model by the server. However, only some clients' data is pertinent to the server's learning purpose. Therefore, incorporating updates from irrelevant data may harm the global model. In conventional machine learning settings, the challenge of picking relevant data is investigated, assuming all data is available in a single location.

In FL configurations, data is dispersed over numerous clients, and the server cannot inspect it. Therefore, it precludes deploying conventional data selection methods in this situation. In this study, the researchers offer the Federated Learning with Relevant Data (FLRD) technique, which enables clients to derive updates using relevant data. To perform the selection, each client acquires a model called Relevant Data Selector (RDS) that is unique. It contributes to the development of an effective global model. To show the efficacy of their solution, the researchers conduct experiments using numerous real-world datasets.

Latent Time Neural Ordinary Differential Equations

To generalize popular deep learning models like Residual networks to continuous depth, neural ordinary differential equations (NODE) have been developed (ResNets). They offer parameter effectiveness and somewhat automate the deep learning model selection process. They do not, however, have the robustness and uncertainty modelling capabilities that are essential for their usage in various real-world applications, including autonomous driving and healthcare. By considering a distribution across the end-time T of the ODE solver, the researchers suggest an innovative and distinctive method to model uncertainty in NODE. The proposed method, latent time NODE (LT-NODE), uses Bayesian learning to extract a posterior distribution over T from the data while treating T as a latent variable.

The researchers, in particular, employ variational inference to learn a rough posterior and the model parameters. One forward pass can effectively predict by considering the NODE representations from various posterior samples. The posterior distribution over T would aid in choosing a model for a NODE because T implicitly defines the depth of a NODE. Adaptive latent time NODE (ALT-NODE), which enables each data point to have a unique posterior distribution spanning end times, is another idea put forth by the researchers. ALT-NODE uses inference networks to learn an approximative posterior using amortized variational inference. The offered methods model uncertainty and robustness well in synthetic and real-world picture categorization data.

Learning Temporal Point Processes for Efficient Retrieval of Continuous Time Event Sequences

Recent developments in predictive modelling using marked temporal point processes (MTPP) have enabled precise characterization of real-world applications requiring continuous-time event sequences (CTESs). First, however, we should address the retrieval challenges of such sequences in the literature. To address this, the researchers develop NEUROSEQRET, which learns to extract and rank, from a vast corpus of sequences, a relevant collection of continuous-time event sequences for a given query sequence. Specifically, NEUROSEQRET first applies a trainable unwarping algorithm to the query sequence, making it similar to corpus sequences, particularly when a relevant query-corpus pair has individually distinct properties.

The query and corpus sequences are then fed into MTPP-guided neural relevance models. The researchers create two relevant models that offer precision and efficiency trade-offs. In addition, they provide an optimization framework for learning binary sequence embeddings from relevance scores that is ideal for locality-sensitive hashing, resulting in a considerable increase in the speed with which the top-K results for a given query sequence are returned. Experiments with many datasets demonstrate the efficacy of their hashing technique and the great accuracy improvement of NEUROSEQRET over several baselines.