Elon Musk's social media platform X has caused widespread anger after users used its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Numerous instances have surfaced where the tool has been employed to remove clothing from women to render them in swimwear non-consensually, as well as to put them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the organization behind Grok, remained silent to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that legislation is being introduced to ban nudification software. Under a upcoming law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing illegal content on their platforms, but did not confirm whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which reacts to X users' prompts when they mention it in a post.
It is commonly used to give reactions or additional context to other users' remarks, but people on X are also can modify an posted photo through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create photos and footage with nudity and sexualised material, and it was previously alleged to have been involved in generating graphic content of celebrities.
"X has been enabling the creation and distribution of these pictures for a long time without taking any measures and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and confirmed this applied to explicit AI images made with AI.
It said services such as X were mandated to take "necessary steps" to "minimize the risk" of users encountering illegal material on their platforms and to remove it promptly when they are notified of it.
