On Tuesday, X Corp. Japan announced that if users on the social media site create or post illegal content using the artificial intelligence chatbot Grok, it will permanently freeze the account and work with the police to take action. It’s in response to a growing number of people using Grok to create non-consensual sexualized images of women and girls.
One user, whose account name is the motto of right-wing political party Sanseito, asked Grok to generate an image of the Imperial Princess in a bikini, causing backlash | Image via @tkzwgrs on X
Grok Has Altered Several Images of Celebrities in Japan
An “edit image” button was added to Grok in late December, allowing users to modify any image on the platform. Some have subsequently used this feature to remove the clothes of people without their consent. NHK has confirmed several cases in Japan of celebrity images being altered in such a way. Some of those are believed to be images of minors.
Elon Musk’s chatbot has faced growing international backlash since the turn of the year. On Tuesday, Britain’s top technology official, Liz Kendall, demanded that X take urgent action. It is “absolutely appalling,” she said, adding “we cannot and will not allow the proliferation of these degrading images.” She has also backed the UK regulator Ofcom to “take any enforcement action it deems necessary.”
On Monday, European Commission spokesperson Thomas Regnier said that he was “well aware” that Grok was being used to for “explicit sexual content with some output generated with child-like images.” He added, “This is not spicy. This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe. This is not the first time that Grok has generated such output.”
Sexual Images Continue To Be Posted Despite Musk’s Warning
Replying to an X post on the weekend, Musk said, “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.” Despite his message, sexual images without consent continue to be posted. According to Genevieve Oh, a social media and deepfake researcher, Grok generated about 6,700 images every hour that were identified as sexually suggestive or nudifying during a 24-hour period from January 5 to 6.
