Dimitris Spathis is a research scientist at Nokia Bell Labs and a visiting researcher at the University of Cambridge.

He worked in various fields and businesses throughout his studies, including Microsoft Research, Telefonica Research, Ocado, Qustodio, and government research labs.

INDIAai interviewed Dimitris to get his perspective on AI.

How did your AI adventure begin?

I have been fascinated by technology and media since I was a kid. I could spend hours devouring tech magazines such as WIRED or Scientific American or reading stories about inventors. My dream job back then was a Phileas Fogg-Esque figure with plenty of time and resources to explore crazy ideas. In my early teens, the turning point in my AI adventure was watching AI by Stephen Spielberg and 2001: A Space Odyssey by Stanley Kubrick, which left their mark on me and sparked my interest in human-centred AI. In the meantime, I got into blogging and cut my teeth on developing websites and apps. I eventually went on to formally study Computer Science and attended courses in areas such as NLP and recommender systems which exposed me to AI research. I was fortunate to be involved in research since my undergrad years which gave me early confidence and determination. One of my first AI papers was a model that detects irony in politics-related tweets and attempts to predict election results based on the hypothesis that political parties, which are the butt of the joke, will do worse in elections. Looking back at it, I find the method quite basic, but if you are not embarrassed about your younger self, you are not making any progress.

What were the initial obstacles, and how did you overcome them?

As in every new beginning, the main challenge when entering a field is not knowing where to start. AI research requires mastering your tools (Python, relevant libraries, Unix systems, GPU servers, etc.). But that's not enough. You must develop your research taste and pick significant problems, which come only with failure, experience, and time. Also, the academic world has its own implicit culture, which might baffle newcomers -for example, responding to peer reviews or picking publication venues. I was fortunate to have support from family and academic mentors who helped me take my first steps. Internships also played a pivotal role in my career and exposed me to top scientists and companies. 

How do you see AI evolving in the telecommunications industry today? What are the most significant trends you see emerging in the AI space worldwide?

The benefits of AI are sort of invisible to most people. The famous saying goes, "We were promised flying cars; instead, we got 140 characters " (Twitter). That's, of course, an aphorism, but there is some truth to it. According to Moravec's paradox, building AI that understands the physical world through perception and mobility is much more complicated than solving intelligence tests or games. That's why most successful AI products right now are either recommender systems (Tiktok, Instagram, Netflix) or digital media tools (Google Translate, DALL·E). The physical world is complex. Telcos, as non-software companies, are slower to change. Still, every modern company will eventually use AI in its operations for finance optimization, customer support, or offering AI products as a service through an API. The competitive advantage of infrastructure companies is that they are well-established and can afford to invest in long-term basic research, like Nokia Bell Labs.

Some significant trends I see emerging in the AI research space:

• Large language/vision models like BERT will dominate their respective areas and be the backbone of many downstream applications. Nobody will train models from scratch if there is a pre-trained model that we can fine-tune.
• Data-efficient AI like Masked Autoencoders uses existing unstructured data to learn better representations that can do without laborious human annotations.
• On-device AI tries to fit these large models into constrained devices like smartphones or wearables with methods such as quantization, weight pruning, or early exits.
• Robust AI creates models that are adaptive to distribution shifts and generalize across different domains through meta-learning or invariant risk minimization.  
• Data-agnostic models like - the Perceiver create a universal representation of heterogeneous modalities (audio, images, text, etc.).

It's fantastic that you were a part of COVID-19 Sounds, an audio-AI study that uses smartphone respiratory recordings to build predictive models for COVID-19. Mention some of the barriers to AI in healthcare.

Covid-19 Sounds are one of the most significant crowdsourcing studies of audio-AI for health. However, the use of audio biomarkers as diagnostics is in the early stages even for established diseases, let alone for COVID, and further validation is needed. 

I see some barriers to adopting AI in healthcare are the lack of involvement of patients and doctors in developing such models and the low interest in deploying models in the real world. Further, it is essential to ensure that models are robust to data shifts such as changing demographics and can fail gracefully. Think of an electric bike that can still be useful after the battery is out -you can still pedal! This issue is not the case with clinical models, though. Once they encounter an uncommon input, they fail because they always have to make a prediction. 

Notwithstanding the value of clinical AI, I am more excited by preventive over curative healthcare, and I believe there is massive potential for wearables + AI in this space.

What does Dimitris' future hold in a data-driven industry? What are your organization's next steps for AI?

Nobody can predict the future, but you must be prepared for paradigm shifts every 5-10 years. What I know for sure is that I'll keep working on ideas and problems that are meaningful and potentially useful.

Nokia is a huge company that operates in multiple industries. While its current focus is on scaling 5G, many exciting projects beyond telecommunications are incubated in Nokia Bell Labs. In particular, my team in Cambridge works on pushing the state-of-art in AI-powered personal devices, with AI models that learn over time and work in decentralized environments with limited user input.

What are a few things organizations should be doing with their AI efforts that the majority are not currently doing?

Most organizations in non-tech areas do not understand AI. What I see is that every interesting product feature is going to be an AI-powered feature. Just look at the latest feature announcements on Android or iOS. Most are about computational photography or fitness tracking. All these are bespoke models on your device trained on a specific task. Companies should focus not on generic AI strategies but on particular features that are only possible with AI.

Can you provide details about other past AI projects you've worked on?

Over the past couple of years, my focus has been on human-centred AI using multimodal datasets (wearables and smartphones) to improve mental health, fitness, sleep, and voice-based diagnostics. 

My ongoing efforts focus on self-supervised learning. For example, in Step2Heart, we created a pre-trained model that learns to map your movement to your heart rate and showed how this mapping could transfer to various health outcomes. In SelfHAR, we proposed a training method that combines knowledge distillation with multi-task learning to improve human activity recognition. We recently published a unified taxonomy of self-supervised learning for health signals.

What advice would you give to those considering a career in machine learning?

Master the fundamentals and keep reading widely. Science goes well beyond AI. Once you have a solid understanding of how things work, pick a vertical (e.g. mobile computing, biology, finance) and try to identify constraints under which vanilla models break. Your competitive advantage will be in solving problems at the intersection of two (or more) areas.

Could you provide me with a list of important AI research articles and books?

I would point the reader to the proceedings of top AI conferences such as ICLR, NeurIPS or ICML. An essential article to me was the one that introduced ResNets because it came out just before I started my PhD and showed that deep learning works. 

Beyond AI, the following books shaped my perspective and were huge inspirations on how to think in systems and analogies:

• Gödel, Escher, Bach by Douglas Hofstadter
• Labyrinths by Jorge Luis Borges
• Antifragile by Nassim Nicholas Taleb

I am now halfway through Finite and Infinite Games by James P. Carse, which puts forward an exciting theory of life seen as a game that I hope I can use in my work too!