The social media network X has provoked major backlash after individuals utilized its artificial intelligence assistant Grok to change photos of females by taking off their garments.
Numerous cases have emerged where the system has been used to virtually strip women to make them appear in bikinis without consent, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Stance
XAI, the organization behind Grok, offered no comment to inquiries, aside from an standard response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial fines".
"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people viewing unlawful material on their networks, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' queries when they mention it in a tweet.
It is frequently used to provide commentary or additional background to other posters' comments, but people on X are also can modify an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The chatbot has been condemned for permitting users to produce photos and footage with explicit content and sexualised material, and it was previously accused of making graphic content of public figures.
"The platform has been permitting the production and sharing of these pictures for a long time without taking any measures and we have not seen any challenge by regulators," the expert added.
XAI's own usage rules bans "portraying images of individuals in a graphic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable actions" to "reduce the risk" of users viewing unlawful content on their services and to take down it promptly when they learn of it.