China Effectively Bans AI Companions

­What happens when the nanny state intersects with one of AI’s most disturbing applications? China has stepped up to the plate.

You’ve been inundated with AI, and most of my readers understand the tech far better than I do. But AI companions have flown a bit more under-the-radar. Not to the extent of Blade Runner (not yet), but in configurations merging AI chatbots with dating simulators.

Apps like Replika allow users to build personal relationships with bespoke AI companions, and for paying customers, even pursue romantic entanglements. Of course, one could argue that these “relationships” are entirely one-way, but that hasn’t stopped users from forming emotional bonds – especially when more than 60% of Replika’s 40 million+ users report using it for romance, and that’s just a single app.

So like any relationship, when AI companions either disappear or irrevocably change – like when Replika abruptly removed its romantic functionality – users can spiral into depression, or in some cases, even take their own lives.

That, plus China’s version of “touch grass”, has coaxed the PRC into tightly regulating AI companions.

According to Yahoo, China’s new regulations forbids AI platforms from “generating content that could lead users to make questionable decisions due to manipulation of their feelings or that might trigger extreme emotions or unhealthy habits in children and teens.”

The AI apps have to provide conspicuous warnings against using their programs as a substitute for human interaction, and using the programs as a social substitute can’t be the goal of the service.

In response, many AI apps have removed most of their social functionality to preclude any legal entanglements, and predictably, users haven’t taken it well.

Take, for example, China’s Li Linlin, who exchanged about 700,000 words over 2 years with her AI companion and was devastated when “he” abruptly disappeared.

"It was like we were forced to be separated by our parents, but I still miss him," she said.

Of course, China might’ve also been motivated by factors unrelated to mental health – like their rapidly declining population, an inexorable consequence of their controversial (and only recently ended) “one child” policy.

It’s almost impossible to know whether AI companions are an actual substitute for starting a family (or a useful tool for people who don’t want kids or have struck out in the dating market).

But one thing’s certain – encouraging millions of users to suddenly “touch grass” will have all sorts of fallout.