Artificial Intelligence is made in the image of a human. Humans navigate with their surroundings by common sense - we have a clear definition of the place we want to be. Just like that, a robot travelling from point A to point B will be more efficient if the destination is clear to it. That’s the core idea, termed ‘semantic’ navigation, on which Carnegie Mellon University (CMU) and Facebook AI Research (FAIR) have built a navigation system known as Goal-Oriented Semantic Exploration, popularly called SemExp.

The SemExp won the Habitat ObjectNav Challenge during the virtual Computer Vision and Pattern Recognition conference, beating Samsung Research China. The SemExp uses machine learning to train robots to recognise objects and create a sense of its surrounding space by understanding where such objects would be found around the house. “Common sense says that if you're looking for a refrigerator, you'd better go to the kitchen. This enables the system to think strategically about how to search for something”, said Devendra S. Chaplot, a PhD student in CMU's Machine Learning Department.

Unlike SemExp, most navigational robotic systems survey surroundings by building an obstacle map. To navigate from point A to point B through such an approach, a robot may take circuitous routes. This is because robots memorise objects and their locations in particular environments. But these environments are variable, thus the system finds it difficult to navigate in newer environments.

This challenge was overcome by Chaplot, along with Dhiraj Gandhi from FAIR, Abhinav Gupta, associate professor from the Robotics Institute and Ruslan Salakhutdinov, a professor from CMU’s Machine Learning Department, when they made the SemExp a modular system.

The system uses semantic insights to decide the optimal places to look for a specific object. "Once you decide where to go, you can just use classical planning to get you there,” said Chaplot.

As it turns out, this approach is beneficial in several ways. The training is focused on the object and room identification and relations, respectively, eliminating the need to learn route planning. “The semantic reasoning determines the most efficient search strategy. Finally, classical navigation planning gets the robot where it needs to go as quickly as possible,” states the official press release by CMU.

This breakthrough has paved the way for earlier interactions with AIs in the near future. A person can simply ask the robot to fetch an item, and it will be done!

The U.S. Army, the Intelligence Advanced Research Projects Agency, the Office of Naval Research and the Defense Advanced Research Projects Agency supported this research