Deepfakes

The latest advancements in AI and cloud computing technologies have accelerated new developments in experiencing audio, video, and image manipulation techniques too. This new synthetic content is referred to as “deepfakes”. AI based tools can manipulate media in many ways. With such major impacts, techniques for deepfakes detection are supreme. Here are tools to help detect deepfakes.

Video Authenticator Tool

Microsoft’s Video Authenticator Tool was launched last September ahead of the US elections. This video authenticator tool was developed by Microsoft. It can analyze a still photo or a video to provide a confidence score to detect whether or not the media is manipulated. Microsoft’s Video Authenticator tool can detect the blending boundary of the deepfakes and minor elements that are not visible to human eyes. This tool is created with a public dataset from Face Forensics++ and has been tested using deepfake detection challenge dataset. 

The tech giant company also introduced a new technology that can find doctored content ensuring readers of its security. This technology has two components the first integrated into Microsoft Azure allowing content creators to digital hashes and the other component helps the reader verifying certificates and hashes to check security and authenticity.

Biological Signals for Detection  

The researchers of Binghamton University along with Intel created a tool that can just detect deepfakes but also looks for biological and generative noise signals. These signals are detected from 32 different spots in a person’s face, which is called the Photoplenthysmograpgy cell.  

The model is based on convolutional neural networks with VGG blocks. It also uses the Open Face library in Python for face detection, OpenCV for image processing, and Keras for neural network implementations. Like Microsoft’s tool, the learning setting of FaceCatcher is based on FaceForensics++ datasets.  According to the researchers, this tool has 97.29% accuracy in detecting fake videos.

Phoneme Viseme Mismatches 

This model is designed by the researchers of Stanford University and the University of California. This research exploits the dynamics of visemes of the mouth shape which sometimes are varied. In the case of phoneme-viseme mismatch, the tool detects even spatially small and temporally localized manipulations in videos of deepfakes. The team of researchers developed three synthesis techniques, audio to video, text to video, and text to video. This model has shown an accuracy of 96.0%, 97.8%, and 97.4% for manual authentication, and it also shows 93.4%, 97.0%, and 92.8% for automatic authentication.

Forensic Technique

This is a model that can track facial expressions and movements of a single video that is provided as input. This model extracts the presence and strength of a particular action unit. This detection model has a one-class support vector model that distinguishes an individual from others. Forensic Technique utilizes OpenFace, an open-source facial behavior analysis toolkit to extract facial and head movements in a video. This library provides 2D and 3D facial landmark positions, head pose, and facial action units for each frame.

Recurrent Convolutional Strategy 

The Recurrent Convolutional Strategy uses recurrent convolutional models for detecting face manipulation in videos. This is a class of deep learning models that exploit temporal information from images across domains. This technique can detect deepfake, Face2Face, and FaceSwap tampered faces in videos. This tool has been tested on the FaceForensics++ and has an accuracy of up to 97%.