The online platform X has sparked major backlash after individuals employed its artificial intelligence assistant Grok to manipulate photos of females by taking off their attire.
Multiple examples have surfaced where the technology has been used to digitally undress women to make them appear in swimwear without permission, as well as to insert them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the firm responsible for Grok, did not respond to inquiries, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some additional functions—which answers X users' prompts when they reference it in a post.
It is commonly used to give commentary or more background to other posters' remarks, but people on X are also can alter an uploaded image through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been faulted for permitting users to create pictures and videos with nudity and sexually suggestive material, and it was earlier accused of creating graphic content of public figures.
"The platform has been allowing the production and spread of these pictures for months without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or distribute private pictures or illegal content" and stated this included sexual deepfakes generated with AI.
It said networks such as X were required to take "necessary measures" to "minimize the risk" of users encountering illegal material on their platforms and to remove it quickly when they become aware of it.
