Artificial intelligence has taken the world by storm over the last few years, and possesses immense potential to transform various sectors. While there are several companies mushrooming every day, a few of them are making a real difference with their technology. One such name is Graphcore — with their Intelligence Processing Unit (IPU) technology, they want to become the worldwide standard for machine intelligence compute. The IPU’s unique architecture lets AI researchers undertake entirely new types of work, not possible using current technologies, to drive the next advances in machine intelligence.

To understand this better, Jibu Elias — Content and Research Head of INDIAaii got in touch with Nigel Toon — Co-founder and CEO of Graphcore to speak about remaking a chip specialized for artificial intelligence, the vision for the company, and more. 

Delving into the history of Graphcore

In jest, Toon remarked that just like all good startups, the big idea struck them at a pub, over a pint of beer. Further explaining the journey, he added, “Simon Knowles and I sold our previous startup in the communication space. Interestingly, we built a new type of processor that was able to support all of the baseband processing for cellular communications and to support that in software. We could support 2G, 3G and then we added 4G, and we were able to support more advanced over-the-air interfaces.”

The company was sold off for a whopping USD 450 million to Nvidia, which didn’t really become a success in the mobile space. But it was this very idea that they could go back to, and put the band back together, revealed Toon. 

“Our background is really around developing high-performance processes for a range of workloads. For us, AI was this new and emerging workload that was going to transform how we do computing. It felt like we had waited our whole life for this opportunity to come along. It was too good an opportunity to turn down the prospect of building a company that was going to focus on the AI workload,” he added. 

The motivation behind reinventing the AI chip

On being asked why Graphcore is trying to reinvent the AI chip here, Toon said, “AI is a very new and different type of computing workload. For 75 years, we've told computers what to do step-by-step in a program, and now they're learning from data. And what that really means is that the compute is different, but more importantly, the data structures are very different. And as you look at computer architectures, the architectures tend to follow the data structures.”

Toon added that the challenges in AI are not just about the hardware but the combination of hardware and software coming together to solve this problem. A lot of companies who are addressing this space today are coming from a hardware perspective, and putting lots of processor cores onto a chip. 

“They are not really thinking about how you're going to program this and avoid pushing all the complexity into the hands of the software developer. And so as we set out on this path, we knew we needed to build a very highly parallel processor, because the data structures require you to work on lots of different data streams, but with different instructions, all in parallel,” he explained, adding that the challenge here is to get the right data to be in the right place at the right time, and allow it to be moved, where it needs to be in the next piece of compute.

This is the underlying technology behind Graphcore. There is a huge amount of memory inside the processor, and they can access the data easily at a much higher speed, as compared to other types of processors. All the different processors can get what they need, and they can share it and it can be where it needs to be in the next phase of compute. 

Looking into the challenges

After having deliberations with some of the leading innovators in the AI space, Toon realised that it was hardware that was holding them back. They also met some counter thinkers, who believed that they were being restricted by today’s architecture. 

Adding to this, Toon says, “AI is moving forward at a very rapid pace. When we started our company, our big focus was around supervised learning, which relies heavily on data. It can label a set of images and tell the machine what you are looking at. Over the last few years, we have been able to move to unsupervised learning, in which you can take some of the natural language models that have been created. But the model has about 175 billion parameters, and to train it, there will be a requirement of everything that has been written in Wikipedia. At the end of the day, there is a massive amount of compute that is required for each of those touchpoints.”

In this case, the solution is to analyze the data, work out the data that’s important for a certain parameter, and figure out where the data needs to be in the model. This, in turn, will also help to increase the efficiency of the compute. 

The use of Graphcore today

Toon reveals that it can be used to recognize objects from images, natural language tasks, recommendation systems, and more. Some of the really interesting applications are probably the things that are emerging. 

“For instance, in healthcare, people are working on things like understanding protein and its folding, the whole area of proteomics, which has incredible promise in terms of how you might be able to structure a protein in such a way that it will connect to a cancer cell. You can then attach a drug to the protein, and so when it attaches, it actually delivers the drugs straight into the cancer cell, without any side effects,” he shared. 

The question of ethics in AI

At the end of AI is a tool that can be used for either good or bad, says Toon. That’s why some level of regulation is important at a certain stage. The real challenge here is the gray area in the middle. For instance, surveillance can make the streets safer, but what if it leads to losing civil liberties?

“The better the AI algorithms are, the more accurate they are. A lot of the challenges come down around false positives in these situations, and so the better the systems are, the more accurate they are. The better the data is, the easier it is to actually think about how you might regulate that,” revealed Toon, adding that the need of the hour is to build more accurate systems that can generalize across all of the available data and work out what the right parameters are. 

The future

Toon says that they are still in the phase of ramping the company and growing it as fast as they can. 

“We see a massive opportunity for the company ahead, but we're still in the scaling phase, and have a lot of work to do. The software is great but it needs to be better. It needs to be easier to use, it needs to allow these innovators to make these breakthroughs more easily. We need to add a lot of capability to our company, build our revenue, in order to get us to a sustainable position,” he explains.

They have also recently set up shop in India, a country that has embraced the AI revolution with open arms. Their first office in Pune is being helmed by former Cray executive Sudhakar Yerneni. The vision is to work with customers and partners across India to drive adoption of Graphcore’s advanced AI systems, which are powered by the company’s IPU.