Imagining a world where AI creates architectural drawings or controls robotics to perform medical surgeries may be difficult. However, it was once unfathomable that a computer could beat a human at chess or drive a car autonomously. How far away are we from artificial general intelligence, and what obstacles must be overcome to get us there?

Machine learning (ML) and deep learning (DL) have achieved impressive progress recently, and the success of artificial intelligence (AI) in the medical field has resulted in a significant increase in medical AI applications. Medical AI research aims to build applications that use AI technologies to assist doctors in making medical decisions. AI is used in various medical applications, such as disease diagnosis, surgery, etc. However, medical AI applications face some challenges, including the black-box nature of some AI models.

It’s taken many years, but Grantcharov, now a professor of surgery at Stanford, believes he’s finally developed the technology to make this dream possible: the operating room equivalent of an airplane’s black box. It records everything in the OR via panoramic cameras, microphones, and anaesthesia monitors before using artificial intelligence to help surgeons make sense of the data.

Grantcharov’s company, Surgical Safety Technologies, is not the only one deploying AI to analyze surgeries. Many medical device companies are already in the space—including Medtronic with its Touch Surgery platform, Johnson & Johnson with C-SATS, and Intuitive Surgical with Case Insights.

The OR Black box

Grantcharov’s OR black box is not a box but a tablet, one or two ceiling microphones, and up to four wall-mounted dome cameras that can reportedly analyze more than half a million data points per day per OR. “In three days, we go through the entire Netflix catalogue regarding video processing,” he says.

The black-box platform utilizes a handful of computer vision models and spits out a series of short video clips and a statistics dashboard—like how much blood was lost, which instruments were used, and how many auditory disruptions occurred. The system also identifies and breaks out key segments of the procedure (dissection, resection, and closure) so that instead of having to watch a whole three- or four-hour recording, surgeons can jump to the part of the operation where, for instance, there was significant bleeding or a surgical stapler misfired.

Critically, each person in the recording is rendered anonymous; an algorithm distorts people’s voices and blurs out their faces, transforming them into shadowy, noir-like figures. “For something like this, privacy and confidentiality are critical,” says Grantcharov, who claims the anonymization process is irreversible. “Even though you know whathappened, you can’t use it against an individual.”

Another AI model evaluates performance. For now, this is done primarily by measuring compliance with the surgical safety checklist—a questionnaire that is supposed to be verbally ticked off during every type of surgical operation. 

Each model has taken up to six months to train through a labour-intensive process relying on a team of 12 analysts in Toronto, where the company was started. While many general AI models can be trained by a gig worker who labels everyday items, the surgical models need data annotated by people who know what they’re seeing—either surgeons, in specialized cases, or other labellers who have been properly trained. 

Not error-free

While most algorithms operate nearly perfectly independently, Peter Grantcharov explains that the OR black box is still not fully autonomous. For example, capturing audio through ceiling mikes is difficult, and thus, getting a reliable transcript to document whether every element of the surgical safety checklist was completed is difficult; he estimates that this algorithm has a 15% error rate. 

“It will require a human in the loop,” Peter Grantcharov says, but he gauges that the AI model has made confirming checklist compliance 80% to 90% more efficient. He also emphasizes that the models are constantly being improved.

MIT

Image: Unsplash