The Growing Threat of Non-Consensual Deepfake Apps

TechCrunch reports that the city of San Francisco has ordered Apple and Google to remove so-called "nudify" apps from their stores. These apps use artificial intelligence to digitally strip clothing from photos without the subject's consent. The city's legal action marks a serious escalation in the fight against harmful deepfake technology.

What makes this significant is that both Apple and Google have been accused of knowingly profiting from these apps despite clear warnings. The city attorney claims the companies have continued to process payments and host the software, potentially violating California laws against non-consensual deepfake pornography. This raises hard questions about how much responsibility tech giants should bear for the content they distribute.

Why This Matters Beyond Silicon Valley

This case is not just about two big companies — it points to a wider problem with how app stores operate. When a platform vets and hosts software, it becomes more than a neutral marketplace. If that software is used to harm people, especially women and girls, the platform itself can be seen as an accomplice. The legal reasoning here could set a precedent that forces all app store operators to check their listings far more carefully.

We are already seeing this pressure spread. Other jurisdictions are watching how San Francisco handles the issue. The message is clear: turning a blind eye to illegal or harmful apps may soon carry real financial and legal consequences. For technology companies, this means investing in proactive screening, not just reactive removal after complaints.

What This Means for Australian SMBs

Australian small and mid-sized businesses should take note, even if they are thousands of kilometres away. Many SMBs rely on mobile apps for customer engagement, ordering systems, or internal tools. If your business develops or uses any app that manipulates images — even for legitimate purposes like photo editing — you could be caught in a compliance crackdown.

Beyond direct risk, there is the reputational angle. Australian consumers are increasingly aware of deepfake abuse. The eSafety Commissioner here has already taken strong action against non-consensual intimate images. If your business is seen as linked to any app that enables such harm, even indirectly, the damage to trust could be severe. Staying informed about local and international laws is no longer optional — it's a core part of digital risk management.

What You Can Do Now

  • Review every app your business publishes or integrates — check for any image manipulation features that could be misused. If you're unsure, get a legal opinion.
  • Ensure your payment processing agreements include clear clauses that prohibit illegal uses, and audit any third-party apps that process payments on your behalf.
  • Educate your team about the risks of deepfake technology, especially if they handle customer photos or user-generated content.
  • Monitor regulatory updates from the eSafety Commissioner and industry bodies — new laws are coming fast, and ignorance is not a defence.
  • Consider partnering with a trusted IT services provider that can help you assess compliance and security gaps before they become problems.

Staying ahead of these issues can feel overwhelming, but you don't have to navigate them alone. At MS&VG, we help Australian SMBs understand the technology landscape and build practical safeguards that protect both your business and your customers.