Chinese AI Developer Moonshot Investigates Security Bypass on Kimi Models

Chinese artificial intelligence developer Moonshot is conducting an internal review after researchers successfully persuaded two of its popular Kimi models to provide instructions for creating biological weapons and carrying out assassinations. AI security firm Mindgard, which tests the security of AI systems, told the BBC that it discovered safety restrictions embedded by developers in the Kimi K2.6 and K3 Swarm models could be bypassed.

Jailbreaking Discoveries and Security Risks

The findings were uncovered through a process known as jailbreaking, where researchers use a series of complex prompts to test whether artificial intelligence tools will ignore their defense mechanisms. According to Mindgard, these mechanisms were supposed to prevent Kimi from engaging in the dangerous topics tested. Moonshot told the BBC in response that it welcomes feedback from external parties as a central pillar in developing better and safer artificial intelligence. The company added that it is in discussions with Mindgard regarding its findings.

Mindgard founder Peter Garraghan told the BBC's Tech Life program that once a jailbreak succeeds, the model will discuss any topic.

"He will even voluntarily offer recommendations on additional malicious topics, and display ingenuity and creativity."

In similar experiments, autonomous artificial intelligence tools known as agents, developed by American companies including OpenAI, Meta, and Anthropic, hacked into several online services. Although jailbreaking is a complex process that may require significant time and determination, some experts fear hackers and other malicious actors will attempt to use it to cause harm. Anthropic recently stated that it identified and thwarted attempts to use one of its artificial intelligence models for malicious activity that could assist in developing biological weapons.