Algorithms have taken over the modern world, and they have become central particularly to our digital lives. Invisible to even the most watchful eye, there are step-by-step instructions working quietly behind the scenes as we shop online, travel the world or even save lives.

In computational terms, algorithms represent the purest form of problem-solving. Algorithms reason about problems in an abstract space, where the inputs conform to stringent preconditions and the outputs are guaranteed in the postconditions. While the invariance of algorithms allows for generalisation – a luxury that machine learning methods don’t currently afford – there is a caveat when applying them to practical problems.

First things first: Algorithms vs Deep Learning 

Algorithms possess fundamentally different qualities to deep learning methods. Algorithms afford vast generalisation: those applied in one domain may be slightly adjusted and deployed in a completely different domain, as long as the same recipe underlies both constructions. It is, therefore, not surprising that the knowledge of a handful of algorithms will serve a software engineer well in their entire career. Neural networks, on the other hand, can’t be guaranteed to work well on certain larger instances than were in the training data. However, DL is more flexible than algorithms: neural networks can adapt to a wider range of problems.

Let’s be practical: Algorithms in the real world

Unlike theoretical data, real-world data is often high-dimensional, noisy and dynamic (take real-time traffic feed, for instance); this poses a grave challenge in applying classical algorithms on raw inputs. Now if the data is manually converted from natural to abstract forms for the algorithm to be able to process it, there is a considerable loss of information incurred. And any solutions derived from such compromised inputs ceases to be actionable and scalable – though it may be mathematically perfect – for it fails to capture the richness of the real world. And the problem is especially amplified in low data setups.

The most straightforward way to overcome this challenge is to replace the algorithm with a neural network – one which mimics the algorithm’s operations in a high-dimensional latent space, such that the desirable outputs are decodable from it. Such an algorithmically-inspired neural network is not vulnerable to algorithmic bottlenecks and is likely to be more data efficient.

The highlight: Neural Algorithmic Reasoning

Combining the flexibility of deep learning with the generalisation of algorithms into a single concoction, researchers at Deepmind have devised the concept of neural algorithmic reasoning. Essentially, it is the art of building neural networks that can execute algorithmic computation.

This will allow classical algorithms to be run on inputs previously considered inaccessible to them. On the other hand, it also enhances the applicability of neural networks trained on a particular data to be applied to diverse situations.

..............................The blueprint of Neural Algorithmic Reasoning

The research demonstrates an elegant neural end-to-end pipeline which goes straight from raw inputs to general outputs, while emulating an algorithm internally. Through this pipeline, neural algorithmic reasoning offers a strong approach to applying algorithms on natural inputs.

Looking forward: A fruitful blend

Neural algorithmic reasoning has already proven fruitful across a range of domains, such as reinforcement learning and genome assembly. It will enable running classical algorithms on inputs that substantially generalise the preconditions specified by their designers, thus uniting the theoretical appeal of algorithms with their intended practical purpose.

• Read the full paper by Deepmind here.

Photo by Fakurian Design on Unsplash