Recently, researchers at MIT have devised a technique that enables a robot to learn a new pick-and-place activity from a small number of human examples. For example, this technique could allow a human to teach a robot in around 15 minutes to grab previously unseen objects presented in random positions.

What is the objective?

With an influx of e-commerce orders, a warehouse robot chooses mugs from a shelf and places them into shipping boxes. Everything is OK until the warehouse changes, requiring the robot to grip higher, narrower mugs stacked upside down. Reprogramming the robot requires manually annotating hundreds of photos that demonstrate how to grasp the new cups and retraining the system.

However, a new technique developed by MIT researchers could reprogram the robot with only a few human examples. This machine-learning technique enables a robot to pick up and position previously unseen objects in random poses. The robot would be ready to conduct a new pick-and-place assignment within 10 to 15 minutes.

How is it done?

The technique uses a neural network that MIT researchers created specifically for reconstructing the shapes of three-dimensional objects. With just a few demonstrations, the system can understand new objects comparable to those in the demos by using what the neural network has learnt about 3D geometry.

The researchers demonstrate that their system can effectively manage previously unseen mugs, bowls, and bottles placed in random postures using only ten examples to educate the robot.

Grasping geometry problem

While researchers may train a robot to pick up a specific item, if that item is lying on its side (perhaps due to a fall), the robot perceives this as a completely different circumstance. This process is one of the reasons machine-learning systems have such a hard time generalizing to novel object orientations.

To address this issue, the researchers developed a new form of neural network model called a Neural Descriptor Field (NDF) that automatically learns the three-dimensional geometry of a class of things. The model generates a three-dimensional point cloud, a collection of data points or coordinates in three dimensions. The data points are acquired using a depth camera, which measures the distance between an item and a perspective. While researchers trained the network on a vast dataset of synthetic 3D shapes in simulation, we may apply it directly to real-world objects.

How did they overcome it?

The MIT team created the NDF with a property called equivariance in mind. So, for example, if the model is shown an upright mug and subsequently shown the identical mug on its side, it recognizes that the second mug is the same object, simply turned.

As the NDF acquires the ability to rebuild the forms of similar objects, it also gains the ability to correlate related components of those items. For example, it learns that mug handles are comparable, even though some mugs are taller or broader or have smaller or longer handles.

The researchers can teach a robot a new ability using only a few physical examples by using this trained NDF model. The researchers position the robot's hand on the desired part of an object. Such as the rim of a bowl or the mug's handle, and record the location of the fingertips.

Conclusion

The MIT researchers validated their approach using mugs, bowls, and bottles as items in simulations and on an accurate robotic arm. Their technique achieved an 85 per cent success rate on pick-and-place tasks with novel objects in novel orientations, while the best baseline achieved only a 45 per cent success rate. Success entails grasping an unknown object and arranging it in the desired spot, like mugs on a rack.

While the researchers were pleased with the approach's effectiveness, their method is limited to the object category. For example, a robot trained to pick up mugs will be unable to pick up boxes or headphones, as their geometric features are too dissimilar to those of the cups on which researchers trained the network.

Additionally, the researchers intend to modify the system for nonrigid objects and, in the long run, enable pick-and-place operations when the target region changes.