Historical trends have shown a clear correlation between AI and chip design: progress in the former follows any progress in latter. With the growth of AI, moreover, hardware is fashionable again after years of software being the centre of attraction. In addition, the shrinkage of chip components to the scale of individual atoms has made it increasingly difficult to keep up the pace of Moore’s prediction for the semiconductor industry. This, combined with the slowing of the Dennard scaling, means that AI, now more than ever, needs to enable computers to continue to make improvements in training and inference.

Google’s new reinforcement learning-based approach has significantly sped up the chip design process, reducing graph placement time from several weeks to under six hours. This is expected to lead to a new generation of improved architectures, and thereby, accelerate advancements in AI. As a general rule, reinforcement learning algorithms use positive and negative feedback to learn complicated tasks. Therefore, Google’s approach has the ability to learn from past experience and improve over time.

Chip placement as an RL problem 

One of the most complex and time-consuming stages of the chip design process is chip floorplanning, which refers to the engineering task of designing the physical layout of a computer chip. In simpler words, it is the placement of components on a computer chip. As a part of its latest research, Google has presented a deep reinforcement learning approach to chip floorplanning. The latest findings have been published by Google in the journal Nature. This follows last year’s preprint paper titled ‘Chip Placement with Deep Reinforcement Learning’ on arXiv. 

“We pose chip floorplanning as a reinforcement learning problem and develop an edge-based graph convolutional neural network architecture capable of learning rich and transferable representations of the chip. As a result, our method utilises past experience to become better and faster at solving new instances of the problem, allowing chip design to be performed by artificial agents with more experience than any human designer,” Google researchers write in the paper.  

The semiconductor industry had been unable to automate the tedious process of manual chip floorplanning using commercial EDA tools, despite putting in five decades of research effort. But Google has managed to challenge the norm. “In under six hours, our method automatically generates chip floorplans that are superior or comparable to those produced by humans in all key metrics, including power consumption, performance and chip area,” states the paper. It is worth noting that PPA stands for the three variables – power, performance, area – that have historically been used in deciding how to optimize semiconductor designs. 

Future scope and impact

This new method has been used in production to design the next generation of AI-designed AI accelerators by Google. By saving thousands of hours of effort that is typically put in chip design by human design engineers, this novel method will allow hardware to better adapt to rapidly evolving AI research. Over time, the artificial agents will become more experienced at generating optimised placements for new chip blocks than human designers will ever be. 

The implications of this RL method go beyond the immediate impact on chip floorplanning; the ability to generalise across chips also presents opportunities for co-optimisation with earlier stages of the chip design process. “Large scale architectural explorations were previously impossible, because it took months of human effort to accurately evaluate a given architectural candidate. However, modifying the architectural design can have an outsized impact on performance, and would facilitate full automation of the chip design process,” adds the paper. 

With this advance, we are moving closer to a future in which AI and hardware share a symbiotic relationship. “Automating and accelerating the chip design process can also enable co-design of AI and hardware, yielding high-performance chips customised to important workloads, such as autonomous vehicles, medical devices and data centres,” as per the paper. 

Image from Google