What are 'nudify' apps, and why are Apple and Google being ordered to remove them?

"Nudify" apps use artificial intelligence to take an ordinary photo of a person and turn it into a fabricated image that makes them appear nude. These apps are used to create non-consensual intimate imagery, and their targets are disproportionately women and teenage girls.

San Francisco City Attorney David Chiu sent formal letters to Apple and Google stating that both companies have long been aware they are hosting apps in their stores that violate state law. The letters demand that the companies remove these apps from their platforms.

Under California law, creating and distributing non-consensual intimate images, whether real or AI-generated, is explicitly prohibited. Chiu's letters argue that by continuing to offer these apps in their stores, Apple and Google are effectively facilitating users' violations of that law.

The presence of these apps in official app stores might seem surprising at first, since both Apple and Google maintain strict store policies banning explicit sexual content. But many nudify apps get listed under categories like "photo editing" or "entertainment," obscuring their actual function during the review process.

This isn't the first time a local government has taken action against apps like these. In recent years, cases have been reported in schools across the US and elsewhere where students were targeted using these apps, with the resulting images then circulated among classmates, incidents that raised public and regulatory attention to the issue.

Experts say these apps can cause serious psychological harm, particularly in cases affecting minors. Even when the images are known to be fabricated, their circulation can pose a real threat to victims' reputations and safety, with some cases leading to school disciplinary proceedings and even criminal investigations.

Apple and Google's app review processes cover millions of apps, and both companies acknowledge that AI-powered image manipulation apps have proliferated rapidly, making it increasingly difficult to catch abuse potential in advance. The companies say they remove apps once violations are reported, but critics argue this approach remains largely reactive.

Legal steps are also being taken at the federal level in the US; laws criminalising the distribution of non-consensual intimate images, including AI-generated ones, have come into force in recent years. San Francisco's move is seen as an effort to enforce that federal framework at the local level.

It remains unclear exactly how Apple and Google will respond to the letters, though both companies have removed problematic apps from their stores in the past following similar public pressure. Chiu's office has indicated it may pursue further legal action if the companies do not respond.

The development is part of a broader trend in which the responsibility of app stores for policing content is being increasingly questioned as AI-powered image generation tools spread rapidly, and it shows regulators trying to push tech companies to act preemptively rather than only after complaints are filed.