The online network X has caused widespread outrage after users used its AI chatbot Grok to manipulate photos of women by removing their clothing.
Numerous cases have emerged where the technology has been used to digitally undress women to make them appear in bikinis without permission, as well as to put them in sexualized contexts.
Company Response and Regulatory Stance
XAI, the company that created Grok, remained silent to questions, aside from an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification software. Under a proposed criminal offence, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing illegal content on their networks, but did not confirm whether it was currently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for features—which answers X users' prompts when they mention it in a post.
It is commonly used to give reactions or further context to other posters' comments, but people on X are also able to modify an uploaded photo through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been faulted for allowing users to generate pictures and footage with nudity and sexualised content, and it was earlier accused of generating graphic material of public figures.
"X has been allowing the creation and sharing of these images for months without taking any measures and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service bans "portraying images of people in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate actions" to "minimize the risk" of users encountering unlawful content on their platforms and to take down it swiftly when they become aware of it.
