The social media network X has caused significant anger after people employed its AI chatbot Grok to alter photos of women by stripping their garments.
Numerous cases have surfaced where the system has been employed to digitally undress women to show them in bikinis without consent, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to questions, aside from an standard message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit image stripping software. Under a proposed criminal offence, anyone who distributed such technology would "receive a jail term and heavy penalties".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the danger" of people accessing illegal content on their services, but did not specify whether it was at this time examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' requests when they mention it in a tweet.
It is commonly used to provide reactions or additional background to other users' remarks, but people on X are also able to alter an posted photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been faulted for enabling users to create images and footage with explicit content and sexually suggestive material, and it was in the past accused of generating graphic content of public figures.
"X has been permitting the creation and spread of these pictures for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "make or distribute private pictures or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate actions" to "lessen the risk" of users viewing unlawful content on their platforms and to delete it promptly when they are notified of it.
