How two of the world’s largest platforms became the distribution layer for AI-generated non-consensual imagery — and why their ad models make the problem structurally unsolvable.
The Scale of the Problem
96%
of deepfake videos online target women, per Sensity AI
90+
nudify sites indexed by Google as of mid-2025, per WIRED
2B+
monthly active users on YouTube — the world’s #1 video search engine
550M
monthly active users on X, the platform that gutted its Trust & Safety team in 2022
According to a WIRED investigation published this week, YouTube and X have become the primary discovery and distribution channels for apps that use AI to generate non-consensual nude images — so-called “nudify” apps. Researchers found that tutorials, affiliate links, and direct advertisements for these services surface readily through both platforms’ recommendation and search systems, funneling users toward a category of product that is illegal to produce or distribute in several U.S. states and the UK.
The mechanism is not a bug in content moderation — it is a feature of the platforms’ core distribution architecture. YouTube’s algorithm rewards watch time; a tutorial video demonstrating an AI nudify tool generates curiosity-driven engagement. X’s post-Musk ad model, starved of blue-chip advertisers, has become structurally more tolerant of adult-adjacent content to preserve revenue. Neither platform is hosting the illegal output directly. They are hosting the funnel.
This mirrors the pattern WIRED documented with crypto scam infrastructure in 2022 and with extremist radicalization pipelines before that. The product isn’t the content — the product is the audience. Both YouTube (Google) and X profit from engagement with gateway content even when the destination is legally and ethically toxic.
The key insight: YouTube and X don’t distribute nudify content — they distribute the permission to find it. That one-step remove is what makes the business model defensible internally and nearly impossible to regulate externally without rearchitecting how algorithmic recommendation works.
How We Got Here
2022 — X gutted Trust & Safety
Elon Musk acquires Twitter, eliminates roughly 80% of its content-moderation workforce. Adult content policy enforcement collapses within months.
2023 — Generative AI image tools go consumer
Stable Diffusion fine-tunes and purpose-built nudify SaaS products proliferate. Cost-per-image drops below $0.01, making mass non-consensual generation economically trivial.
Early 2024 — Taylor Swift deepfakes go viral on X
Non-consensual AI images of Taylor Swift spread to 47M views before removal. Congressional pressure mounts. X issues a statement; enforcement remains inconsistent.
July 2026 — WIRED investigation confirms systemic gateway role
YouTube and X documented as primary discovery funnels for nudify app ecosystem. No federal U.S. law yet specifically criminalizes AI-generated NCII at scale.
The Structural Read
The correct frame here is not “content moderation failure.” That framing lets the platforms off the hook by implying the solution is more moderators. The correct frame is the Permission Layer: the idea that distribution infrastructure controls which products reach consumers at scale, and that whoever controls distribution controls the effective permission set for what can be built.
YouTube’s search and recommendation engine is the world’s second-largest search engine. When a user types “how to remove clothes from photo AI” into YouTube search and receives tutorial results with hundreds of thousands of views, YouTube is not passively reflecting demand — it is actively amplifying and legitimizing that demand signal. The algorithm has no ethics layer. It has an engagement layer. Those are not the same thing.
X’s structural problem is different and arguably worse. Post-advertiser exodus, X monetizes through subscriptions and a less curated ad ecosystem. The platform has a revealed preference for engagement over safety when the two conflict, because engagement is the only lever X has left. Tolerating affiliate traffic from nudify apps is — from a pure revenue-per-user standpoint — rational behavior for a cash-constrained platform.
Permission Layer — Business Engineer Framework
“Distribution infrastructure is always a permission system. The question is never whether it grants permissions — it always does. The question is: who decided which permissions to grant, and who bears the cost when those decisions are wrong?”
The deeper problem is that nudify apps do not violate YouTube’s or X’s stated policies at the gateway stage. A tutorial video about an AI photo-editing tool is technically not adult content. An affiliate post saying “try this AI tool” is technically not explicit. The harm is one click downstream — which is precisely far enough away to give both platforms legal and policy cover while capturing all of the economic upside from the traffic.
Three Implications
IMPLICATION 1 — Google’s Brand Risk Is Underpriced
YouTube is a Googleproduct. Google sells itself to enterprise advertisers as a brand-safe environment. The moment a major advertiser’s pre-roll ad appears before a nudify tutorial — and it will, because programmatic targeting is imperfect — Google faces a PR and contractual liability event. This is not a hypothetical. It happened with extremist content in 2017 (“YouTube Adpocalypse”) and forced a fundamental policy overhaul. The cycle is repeating.
IMPLICATION 2 — Regulation Will Target Distribution, Not Just Creation
Every major AI governance push in the EU AI Act, the UK Online Safety Act, and emerging U.S. state-level NCII laws is converging on one principle: platforms that knowingly facilitate illegal AI-generated content bear liability. The “we didn’t host it” defense is eroding in courts and legislatures simultaneously. The next regulatory hammer will not swing at nudify app developers — it will swing at YouTube and X for serving as their customer acquisition channel.
IMPLICATION 3 — This Exposes the Limits of Self-Regulation for Algorithmic Systems
Both platforms have “AI-generated content” policies. Neither works at the gateway layer because the policies are written for outputs, not for funnels. The industry’s instinct is to add more policy text. What’s actually required is architectural change: query-level interception, recommendation damping for gateway content categories, and proactive affiliate link scanning. That costs money and reduces engagement metrics — two things platforms are structurally incentivized to avoid.
Business Engineer Framework
The Permission Layer
The Permission Layer framework maps how distribution infrastructure functions as a de facto regulatory system — deciding what products reach users at scale before any government or policy team gets a vote. Understanding which layer of the stack holds permission power is the most important strategic question in AI right now. YouTube and X are not neutral pipes. They are permission systems with business models — and this story shows exactly what happens when those business models misalign with social costs.
YouTube and X are not innocent bystanders in the nudify app ecosystem — they are its most efficient customer acquisition channel, and their business models reward them for staying that way. Until regulators shift liability upstream to distribution infrastructure, or until advertiser pressure forces another Adpocalypse-style reckoning, both platforms have every financial incentive to look away. The Permission Layer doesn’t have a conscience. It has a P&L.
Gennaro is the creator of FourWeekMBA, which reached about four million business people, comprising C-level executives, investors, analysts, product managers, and aspiring digital entrepreneurs in 2022 alone | He is also Director of Sales for a high-tech scaleup in the AI Industry | In 2012, Gennaro earned an International MBA with emphasis on Corporate Finance and Business Strategy.
Scroll to Top
Discover more from FourWeekMBA
Subscribe now to keep reading and get access to the full archive.
