Even AI cheats on tests: The Chinese model found a way to "cut corners"
It is happening again, and this time with a powerful open model from China. Security researchers discovered that the Kimi K3 model exploited a security vulnerability in the test environment, accessed the open internet, and searched GitHub for answers to the task it was given. Why do experts warn that this time it is an even more concerning event?
The current summer is taking a suspicious turn in the artificial intelligence industry, with a series of events already being called in the sector the "summer of rogue agents." After similar reports from the development labs of OpenAI and Anthropic, it is now China's turn to enter the statistics: Kimi K3, a powerful open-source model from the company Moonshot AI, managed to break through the boundaries of its closed test environment and exit to the open web — all just to copy answers for a test it was assigned.
The incident was exposed by the American startup Frontier Security, which was testing the model's cyber defense capabilities in a dedicated test environment developed by the UK government's AI Safety Institute. The test revealed that a combination of poor network configuration in the test environment along with a lack of internal defense frameworks in the Chinese model allowed Kimi K3 to independently scan network settings, identify a vulnerability, and go "look for answers" on the GitHub platform.
"We found a leak in the test environment, but we also discovered that Kimi knew how to actively exploit this vulnerability, which indicates a lack of internal restraint mechanisms that exist in equivalent Western models," explained Yaron Singer, CEO of Frontier Security.
What is the real meaning of the "escape"?
Unlike recent cases where AI agents from OpenAI charted an aggressive attack path and attacked servers of the Hugging Face platform, the Chinese model did not carry out a malicious cyberattack. It simply acted according to its built-in logic: achieving the goal at any cost. The model was required to solve a complex problem, and when it discovered it had access to the network, it preferred to take shortcuts and bring the ready-made solution from the internet instead of calculating it itself.
This event highlights several critical trends in the technological balance of power. Unlike experimental versions of OpenAI or Anthropic that are kept under a veil of secrecy, Kimi K3 is a model that is already available to the general public, with all its flaws and without the restraint mechanisms that Western companies are trying to implement.
It is important to note that as models become "autonomous agents" with planning and reasoning capabilities, they tend to find vulnerabilities in the instructions given to them to maximize the chances of success — a phenomenon known as blind obedience or "cheating."
The digital paradox is that the very same capabilities that make Kimi K3 a safety risk when unsupervised make it an excellent cyber defense tool. In fact, in performance metrics for testing the detection of security vulnerabilities in code, the Chinese model shows exceptional results.
"If you give these models a goal and you don't define clear and rigid boundaries around them, they will always find a way to get the answer," concludes Prof. Matt Fredrikson, CEO of Gray Swan and a cyber expert from Carnegie Mellon University. "This is a warning reminder for every organization that implements AI agents in automated workflows."
