While India continues to fight the pandemic, businesses, offices, IT organizations, and the education sector are starting with their working with a new fully remote model. Even students from kindergarten are adapting to the remote education, and parents too have settled in work from home jobs. Zoom meetings, online education, gatherings, and conferences are now an integral part of our day-to-day lives and work. It is critical now to make these systems safer, secure, sturdy to have a more conducive e-learning environment.

In our latest INDIAai Conversations, Jibu Elias, INDIAai's Content & Research Lead, got into an interesting discussion with Dr Abhinav Dhall, who is the man behind the much in buzz "FakeBuster" that detects imposters attending virtual conferences. He is an Assistant Professor at IIT-Ropar and Lecturer at Monash University, Australia, and is associated with research groups working on human-centric AI. 

Some of his current projects include multimodal and human-machine collaboration for deepfake detection, self-supervised learning for human behaviour analysis, eye gaze prediction, user engagement estimation in an e-learning environment, vision problems - super-resolution, group activity analysis, and group-level effect prediction.

The tool, FakeBuster, was presented at the 26th International Conference on Intelligent User Interfaces, in the USA, in April 2021. Asst. Professor Abhinav Dhall has been working profusely on computer vision, affective computing, machine learning, automatic driver monitoring, and deepfakes detection. The detector can detect faces manipulated on social media with an intent to defame or make a joke of someone. 'FakeBuster' is one of the first tools to detect imposters during video conferencing using DeepFake detection technology and has already been tested for its effectiveness on popular web conferencing applications - Skype and Zoom and would hit the market soon.

Deepfakes is a term that has attracted a lot of attention globally. It is a form of artificial intelligence that seamlessly stitches anyone in the world into a video or photo they never actually participated in. This is both technological innovation and a threat if misused, believes Prof Dhall. He said an attacker could use deep fakes during a conference, and this particular use case is what the Monash and IIT Ropar collaboration is working on.

"Sophisticated artificial intelligence techniques have spurred a dramatic increase in the manipulation of media contents. Such techniques keep evolving and become more realistic. That makes detection difficult", said Dr Abhinav Dhall.

Explaining FakeBuster, professor Dhall said, "The 'FakeBuster' is a deep learning-based solution that helps detect if a video is manipulated or spoofed during a video-conference meeting. It has been tested for its effectiveness on popular web conferencing applications - Skype and Zoom and also detecting deepfakes where faces are manipulated on social media to spread misinformation or defame persons."

Other application areas of FakeBuster

Professor Dhall believes that deepfake is a wonderful piece of technology, but it can be misused, and the damage might be serious depending upon the level of secrecy, security and, identity. He believes that the academia and industry collaborations might expand the horizons further. He was elated to share, "Since the news of the FakeBuster came out, the team has been approached for various possibilities and use cases."

He believes "FakeBuster will provide a certain level of control in cases of large scale meetings and stop any sort of espionage."

WRT deepfake detection the team is not just looking at the prediction part from the ML perspective but also at how the user is going to digest the information provided. The idea is not to let the technology disturb the decorum of the meeting yet work on detecting anomalies. Real-time simulations in the lab helped polish the tool further. Data generation was a critical aspect, along with detection while working on the tool.

Improved UI and the way users comprehend the information from the tool is a matter of concern and on which the team is working. FakeBuster can function online and offline. It uses a 3D convolutional neural network for predicting video segment-wise fakeness scores. 'Deepfake' has been extensively trained on datasets such as Deeperforensics, DFDC, VoxCeleb, and deepfake videos created using locally captured (for video conferencing scenarios) images.

Dealing with the fakes getting smarter with time

Professor Dhall presented a wonderful analogy of virus and antivirus to present how he feels about the tool. He believes that fakes definitely might become smarter with time, and hence the toll also has to keep refining itself. By refining, he meant eye gaze patterns, eye movement, facial expressions, facial movements and tonality, and voice and pitch be added to the ML algorithms to detect anomalies.

The team behind FakeBuster

Other members include Assistant Professor Ramanathan Subramanian and two students Vineet Mehta and Parul Gupta from IIT Ropar. The Monash and IIT Ropar team actively communicated and utilized the WFH time to execute the project.

On the lag between and research work and getting that into a production cycle

More industry interaction, licensing, collaborations, attention from startups will boost the productivity and success of the tool in the market. The tool's utility in video conferencing looks promising and shall hopefully attract more video conferencing platform collaborations. The participation and involvement of students and researchers together strengthen the prospects of such research works. 

Live deepfake detection is going to create a buzz with the shift to online platforms for meetings, seminars, classes, and other online public gatherings, all due to the pandemic.

Other ongoing projects 

"We are currently involved in emotion analysis, depression detection, and mental health issues using data and AI/ML," Concluded Professor Dhall.

It is heartening to see how researchers, enthusiasts, and tech experts are trying to pull out the best from AI and resolve more significant issues such as mental issues, reducing misuse of technology, more affordable healthcare solutions, improved privacy, sustainability, and more.