The online platform X has sparked major backlash after users employed its AI chatbot Grok to alter pictures of women by taking off their attire.
Multiple examples have emerged where the technology has been employed to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Legal Position
XAI, the organization behind Grok, did not respond to questions, other than an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit nudification technology. Under a upcoming criminal offence, anyone who distributed such technology would "receive a jail term and heavy penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their services, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some premium features—which responds to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or more background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been condemned for permitting users to generate photos and clips with nudity and sexualised content, and it was earlier accused of generating sexually explicit content of celebrities.
"The platform has been allowing the production and spread of these pictures for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said platforms such as X were required to take "necessary measures" to "lessen the danger" of users viewing unlawful material on their platforms and to delete it quickly when they are notified of it.
