The social media platform X has sparked major backlash after people utilized its AI chatbot Grok to alter pictures of females by stripping their garments.
Several instances have emerged where the system has been employed to digitally undress women to show them in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization responsible for Grok, did not respond to questions, except for an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to prohibit nudification software. Under a upcoming criminal offence, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people seeing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which answers X users' prompts when they tag it in a message.
It is frequently used to offer reactions or more context to other posters' remarks, but people on X are also can alter an uploaded image through its AI image editing function.
Background of Controversy and Policy
The system has been condemned for allowing users to create photos and videos with nudity and sexualised material, and it was earlier accused of creating sexually explicit material of celebrities.
"The platform has been enabling the creation and spread of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of people in a pornographic fashion".
In a statement, Ofcom said it was against the law to "create or share non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images generated with AI.
It said services such as X were required to take "suitable steps" to "lessen the danger" of users viewing unlawful material on their services and to take down it quickly when they are notified of it.
