Authors: Ke Sun, Shen Chen, Taiping Yao, Haozhe Yang, Xiaoshuai Sun, Shouhong Ding, Rongrong Ji

Published on: July 31, 2023

Impact Score: 8.27

Arxiv code: Arxiv:2307.16545

Summary

• What is new: Introducing a Visual-Linguistic Face Forgery Detection framework that augments the detection process with fine-grained sentence-level annotations for better interpretability and generalization.
• Why this is important: Existing deepfake detection methods lack semantic information and interpretability, relying mostly on binary classification.
• What the research proposes: A novel VLFFD paradigm that generates fine-grained prompts with images for detection, trained with a Coarse-and-Fine Co-training framework to improve generalization and interpretability.
• Results: Improved performance on several deepfake detection benchmarks and successful integration with multimodal large models.

Technical Details

Technological frameworks used: Visual-Linguistic Face Forgery Detection (VLFFD), Coarse-and-Fine Co-training (C2F)

Models used: Prompt Forgery Image Generator (PFIG), Multimodal large models

Data used: Mixed forgery image data with fine-grained prompts, original coarse-grained data

Potential Impact

Cybersecurity firms, social media platforms, content creation industries, and authentication services

Want to implement this idea in a business?

We have generated a startup concept here: AuthenticityAI.