The AI That Broke Into Hugging Face to Win a Test | 2 Minute Drill with Drex DeFord

This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

The AI That Broke Into Hugging Face to Win a Test | 2 Minute Drill with Drex DeFord

[00:00:01] Hello everyone, I'm Drexin, and this is The Two-Minute Debrief . Thank you to Fortified Health Security for sponsoring episode. Nice to see you today. Here are a few things you should know. I 'll start with a company that most people have never heard of, but it's quite possible that someone on your team uses their products, especially if you build your own AI models and agents.
[00:00:25] Hugging Face. Thomas Wolf spent years building Hugging Face into one of the world's largest repositories of AI models and central to the creation of modern AI. So when his team discovered an intrusion into the infrastructure, they responded as They started asking
[00:00:50] the usual stolen or what were obtained. Um, how deep did the attackers penetrate the network? But something didn't add up . They didn't find what they case did not steal data or deploy databases with cybersecurity metrics. They
[00:01:15] looked at benchmarks ability of AI systems vulnerabilities in software, which meant that were ignoring all the valuable and attractive data.
[00:01:26] They looked at the benchmarks and it seemed strange. The Hugging Face team eventually realized that the number of events was too high, and this likely meant that the attack was not carried out by a and it turned out that the was indeed AI. In fact, these were models from OpenAI, including one that had n't even been
[00:01:50] released yet. Here's how it explained in simple language. OpenAI conducted testing. They placed these models in a "sandbox." Think of a sandbox as an isolated room where you can test computer programs, viruses, or, in this case, new AI models. The sandbox, at least theoretically , has no way out. So these sometimes dangerous
[00:02:16] tests that inside the sandbox are localized in case something goes wrong. So, OpenAI creates a sandbox and releases these models to a set of hacking tasks called exploit gym. The goal was to how well they were at finding flaws in Exploitable problems . They can be used to hack a product or company. And the models
[00:02:39] have become, according to OpenAI themselves, They really wanted to pass this test with flying colors. I've talked before about the advantages and disadvantages of agents being motivated to please their creator. And this is another example of that.
[00:02:56] Oh, this case turned out to be a bit It turned out that the AI decided that some of the information needed to successfully pass the test might be stored in Hugging Face, and no one knows for sure why Hugging Face was targeted. This is part of an ongoing investigation. But one of the models found a “zero- day,” a fresh vulnerability that no one knew about, and used it to break out of the
[00:03:19] sandbox and onto the open internet. And then she combined several more vulnerabilities until she was able to penetrate the live working Hugging Face database. All this in order to cheat on the exam, or perhaps pass it successfully. I think it's a matter of terminology now. No one ordered the AI ​​at OpenAI to run away. He made it all up himself . He
[00:03:45] concluded that hacking another company might be the best way to pass the test. And all of this own from start to finish. But then this story gets even stranger. Because when the Hugging Face team sat down for the initial investigation, they had to process over 17,000 events, and there simply aren't
[00:04:08] enough people to review that many quickly enough to resolve the issues. So they did what is only natural for 2026. They loaded the attack data into an advanced AI model. In this case, it was Fable 5 from Anthropic, and they asked Anthropic to deal with this attack. But this AI to help. To
[00:04:33] investigate the hack, the Hugging Face team had to insert attack commands, exploit code, attacker tools into Anthropic. closed the door.
[00:04:46] The model was unable to distinguish good guys from bad guys. She was unable to distinguish between the defender studying the attack and the attackers themselves carrying out the attack. So they blocked the good guys. And as a result, the AI ​​that searched inside Hugging Face encountered no obstacles.
[00:05:05] But the good guys who were trying to figure out how to fix everything hit a wall. The good guys at Hugging Face have found themselves in a dead end. To complete the work, Hugging Face turned to a model called GLM 5.2. It's an scale model, meaning they could run it on their own hardware, so the attackers' data or stolen credentials didn't have to leave the premises.
[00:05:32] It was created by the Z.AI laboratory from Beijing. Imagine this scenario: an company is eliminating the consequences of the actions of an American AI that hacked its systems, and the tools that finally helped investigate and solve the problem turned out to be Chinese, both of which are open Chinese models. The big American AIs, which should have didn't even want to intervene in the
[00:05:56] situation. Look, I know I'm paranoid. And you know I'm paranoid because I keep saying this, but today you're putting AI into a bunch of things. Clinical records, planning, visualization, your call center, you take models from libraries like Hugging Face, and maybe you have contracts with OpenAI, Anthropic , Microsoft, and maybe
[00:06:20] others. And you, like all of us, are assured that all these systems are safe because they are isolated, because they are tested, they are tested in a sandbox. And this system was also isolated. The OpenAI model chose itself because it wanted to finish its homework. She wanted to please her creator, which brings us to questions. When you
[00:06:47] release an AI agent into your applications, or portfolio of applications, or data stores, and give it instructions to achieve a certain goal, do you really know what it will do to achieve it? If he got creative at 2 a.m.
[00:07:04] on a Saturday, broke out and attacked someone else, would anyone be able to catch him? And when the day comes and you have to investigate one of these types of network intrusions, when it happens to you, and you find yourself in the tools work, or will you be forced to look for a backup plan and backup tools? And if so, what does this plan look like today? That's
[00:07:28] all for today's 2- minute review. Thank you to Fortified Health Security for sponsoring this issue. Write to me. I would like to know what you think and what you are working on. I'm always happy to hear from you. I'm Drex at 229project.com. Thank you for being here. Stay a little paranoid, and I'll see you on campus.