Apple and Meta Under Scrutiny After AI Deepfake App Promoted on Facebook, Features Politician Lookalike
CUPERTINO, CA & MENLO PARK, CA – May 24, 2024 – Both Apple and Meta have found themselves in hot water after an app capable of generating AI-powered pornographic deepfakes of real individuals was promoted through a Meta advertising campaign, according to a report by WIRED. The app, named Kromix, was subsequently removed from the App Store, and Meta has taken down the offending ad campaign.
The controversy centers around Kromix, which marketed itself as "the AI that men actually use." One particular video advertisement, seen by WIRED, explicitly stated the tool had "no restrictions" and that "all the characters are real people." Alarmingly, this ad featured an AI-generated pornographic deepfake of a woman strongly resembling a prominent female US politician. The ad depicted the woman in front of US flags with the caption "What if she moved?", followed by a wink, and then transitioned to a scene of a woman with the same facial features in a pornographic video.
Apple confirmed that Kromix was pulled from its App Store, emphasizing that the app appeared to have incorporated prohibited content and features after successfully passing the initial App Store review process. In a statement to WIRED, Apple reiterated its stringent policies: "We have always strictly prohibited apps designed to generate, distribute, or consume pornography. ‘Nudification’ apps are against our App Review Guidelines and we aggressively work to identify and remove apps—as well as the developer accounts behind them—that attempt to evade our processes and policies.”
This incident echoes a similar situation just last month when Apple was urged by San Francisco City Attorney David Chiu to remove eight other "nudify" apps from the App Store, highlighting a recurring challenge in content moderation.
Meanwhile, Meta, the parent company of Facebook and Instagram, removed the advertising campaign for Kromix. The campaign reportedly consisted of 32 ads exclusively targeted at male users. This raises significant questions about Meta’s ad review processes and its responsibility in preventing the promotion of harmful and exploitative content.
This isn’t an isolated incident for Apple either. WIRED’s investigation also uncovered problematic ads on Meta platforms linking to another App Store app called MaskAI. This app similarly allowed users to upload photos and then face-swap individuals into sexually explicit scenarios. Apple also removed MaskAI after being contacted by WIRED, indicating a pattern of apps bypassing initial checks only to introduce illicit functionalities later.
The widespread availability of AI tools capable of generating realistic deepfakes presents a growing challenge for platform holders like Apple and Meta. While both companies have policies against such content, these recent incidents demonstrate the difficulty in consistently enforcing these guidelines and the potential for malicious actors to exploit review processes. The ongoing struggle to identify and remove these "nudification" apps underscores the need for continuous vigilance and potentially more robust pre-publication vetting procedures to prevent the proliferation of non-consensual deepfake pornography.
