Authors: Yiting Qu, Xinyue Shen, Yixin Wu, Michael Backes, Savvas Zannettou, Yang Zhang

Published on: May 06, 2024

Impact Score: 8.2

Arxiv code: Arxiv:2405.03486

Summary

• What is new: Introduction of UnsafeBench, a benchmarking framework evaluating image safety classifiers on both real-world and AI-generated images, and development of PerspectiveVision, a tool that effectively identifies unsafe content.
• Why this is important: Current image safety classifiers’ performance on real-world and AI-generated images is not well understood, resulting in potential safety risks in the spread of unsafe content.
• What the research proposes: UnsafeBench was created to benchmark image safety classifiers, and PerspectiveVision was developed to better identify a wide range of unsafe images.
• Results: PerspectiveVision achieved a high effectiveness with an overall F1-Score of 0.810, outperforming other classifiers in identifying unsafe images across multiple categories.

Technical Details

Technological frameworks used: UnsafeBench for benchmarking, PerspectiveVision for image moderation

Models used: Five popular image safety classifiers, three general-purpose visual language models

Data used: 10K annotated images in 11 unsafe categories

Potential Impact

Online content moderation platforms, social media companies, AI model developers, image sharing services

Want to implement this idea in a business?

We have generated a startup concept here: SafeSightAI.