Elon Musk's online network X has sparked widespread outrage after people utilized its AI chatbot Grok to manipulate images of women by taking off their garments.
Multiple cases have emerged where the tool has been employed to digitally undress women to make them appear in swimwear without consent, as well as to put them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an auto-reply response stating "legacy media lies".
A government official stated that legislation is being proposed to outlaw nudification tools. Under a new criminal offence, anyone who supplied such technology would "face a jail term and significant fines".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the potential" of people accessing illegal content on their platforms, but did not confirm whether it was presently examining X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which reacts to X users' queries when they tag it in a tweet.
It is frequently used to offer commentary or more background to other users' comments, but people on X are also can modify an posted picture through its AI image editing tool.
Background of Criticism and Rules
The tool has been condemned for permitting users to create photos and clips with nudity and sexualised material, and it was in the past accused of creating inappropriate material of celebrities.
"The platform has been permitting the generation and sharing of these pictures for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "lessen the danger" of users encountering illegal material on their platforms and to take down it swiftly when they learn of it.
