These are the most intriguing AI research publications that were published this year. It assembles the most recent achievements in artificial intelligence and data science. It is arranged in chronological order with a link to an article that goes into greater detail.

Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows

Accelerated gradient methods are the backbone of large-scale optimization problems driven by data and come up naturally in machine learning and other fields that deal with data analysis.

Here, the researchers introduce a gradient-based optimization framework for achieving acceleration. It is based on the recently introduced idea of fixed-time stability of dynamical systems. The method seems to be a generalization of simple gradient-based methods that are scaled to converge to the optimizer in a fixed amount of time, no matter how the problem is set up at the beginning. The researchers do this by first using a continuous-time framework to design fixed-time stable dynamical systems. They then provide a consistent discretization strategy so that the equivalent discrete-time algorithm can track the optimizer in a fixed number of iterations. They also give a theoretical analysis of how the proposed gradient flows converge. In addition, how well they can handle disturbances that are added together. 

The researchers also show that the regret bound on the convergence rate is always the same because of fixed-time convergence. The meanings of the hyperparameters are easy to understand, and we can change them to meet the needs of the desired convergence rates. They compare the accelerated convergence of the proposed schemes to the best optimization algorithms on several numerical examples. Furthermore, their work gives us ideas about how to make new optimization algorithms by breaking up flows in continuous time.

Conditional Generative Model based Predicate-Aware Query Approximation

Approximate Query Processing (AQP) aims to give fast but "good enough" answers to expensive aggregate queries, making it easier for users to explore large datasets interactively. Compared to traditional query processing on database clusters, recently proposed Machine-Learning-based AQP techniques can provide very low latency because query execution only involves model inference.

But the approximation error for these methods gets much worse as the number of filtering predicates (WHERE clauses) increases. To find insights, analysts often use queries with a lot of predicates. So, keeping approximation error low is important if analysts don't want to come to wrong conclusions.

In this paper, the researchers propose ELECTRA, a predicate-aware AQP system that can answer analytics-style queries with many predicates and much more minor approximation errors. ELECTRA uses a conditional generative model that learns the conditional distribution of the data and, at runtime, creates a small (1000 rows) but representative sample on which the query is run to get an approximation of the result. Their tests with three real-world datasets and four different baselines show that ELECTRA has lower AQP error for many predicates than baselines.

Deep Clustering of Text Representations for Supervision-free Probing of Syntax

The researchers are looking into how deep clustering text representations can help unsupervised models understand and learn syntax. Since these are high-dimensional representations, methods like KMeans don't work very well. So, their approach changes the representations into a lower-dimensional space that is good for clustering and then groups them.

In this work, the researchers look at two ideas about syntax: Part of Speech Induction (POSI) and Constituency Labeling (CoLab). They find it interesting that Multilingual BERT (mBERT) knows a surprising amount of English grammar, maybe even as much as English BERT (EBERT). Their model can be used as an unsupervised probe to look into something that might be less biased. They find that, compared to supervised probes, unsupervised probes benefit from higher layers. In addition, the researchers also say that their unsupervised probe uses EBERT and mBERT representations in POSI in different ways. Finally, they prove their probe works by showing that it can be used as an unsupervised syntax induction method. Their probe works well for both syntactic forms because it changes how the inputs are represented.

Furthermore, the researchers say that their probe did well on 45-tag English POSI, was state-of-the-art on 12-tag POSI across ten languages, and did well on CoLab. They also do zero-shot syntax induction on languages with few resources and say the results are promising.