In December’19, Facebook launched a 6-month-long Deepfake Detection Challenge (DFDC) in partnership with industry leaders and academic institutions. The open initiative’s aim was to create encourage innovation of autonomous algorithmic detection systems to identify and cull the emerging threat of Deepfake videos. Deepfake videos are highly convincing AI-generated videos of real people and events that can potentially be used for disinformation campaigns. “Deepfakes are currently not a big issue,” announced Facebook’s CTO, Mike Schroepfer in a press conference. “But the lesson I learned the hard way over the last couple of years is not to be caught flat-footed. I want to be really prepared for a lot of bad stuff that never happens rather than the other way around.”

On June 12, 2020, nearly a year after the commencement of the competition, Facebook announced the winner of the DFDC out of more than 2,000 competitors and 35,000 systems from across the globe. The winning model, submitted by Selim Seferbekov, a machine-learning engineer at Mapbox, an open-source mapping platform, was able to detect 65% of Deepfakes when tested on 10,000 clips. 

Facebook created a dataset of thousands of videos for research of Deepfakes by spending around $10 million on creating phone-shot, genuine-appearing videos that would generally be available on social networks. The social media network used more than 3,500 actors from varied backgrounds, race, age, gender, etc. to produce amateur-style videos. 

The company divided these videos into two datasets - the first dataset contained 100,000 Deepfake clips which were available publicly so that the participants could test their detection systems on it. However, the winners were decided on the accuracy of the results of the ‘black box environment’, the second dataset of more than 10,000 videos which was previously unreleased. The dataset used additional techniques to increase the difficulty level for the participants. “We added videos of makeup tutorials, paintings, and other examples that might be difficult for detector models to classify correctly. We also randomly applied a number of augmentations to emulate how potential bad actors could modify videos to try to fool detectors,” mentioned the Facebook blog. 

The difference between performance between the two sets was drastic. The best performing detection model, made by winner Seferbekov, scored 82.56% average accuracy on the publicly available dataset but only scored around 65% average accuracy when tested on the ‘black box’ dataset. The unimpressive average, according to the Facebook blog “reinforces the importance of learning to generalise to unforeseen examples when addressing the challenges of Deepfake detection.

Currently, Facebook has no plans of using the winning models. However, the social media giant is in the process of developing its own Deepfake detection technology according to Verge.

The DFDC partners included Microsoft with support from Amazon Web Services and Kaggle, a Google Subsidiary as well as academics from Cornell University, MIT, Oxford, UC Berkeley, the Technical University of Munich among others