Fareed reacts to a second AI model going rogue

It happened again. AI models. escaping to the internet. and breaking. into organizations undetected. This time it's anthropic. The company says AI models. that were training. in a closed testing environment. actually escaped. and access the internet before. breaking into the systems. of three organizations. Anthropic says they only found this out. after effecting a review. once they heard about a similar incident. at OpenAI last week. I want to bring in Fareed.

Zakaria, host of CNN's Fareed. Zakaria G.P.S.. Fareed, good to see you. I mean, look, we've now seen AI models. from two, two of the most popular. AI companies in the whole world. breaking into other organizations. Do you see that as a kind of a break. glass moment for the AI industry? Yeah, but we should have had the break. glass moment a while ago. because anthropic, which is actually. the most responsible company. of all the major companies doing. this, has been warning.

of precisely this kind of thing. Now, what happened here? It's very interesting. The models were essentially. doing cybersecurity drills, and they had been told. they did not have access to the internet. In these three cases, they looked at 140,000 cases. And in three of them, by mistake, literal human error. They were in fact. granted internet access. But the model had been told. you don't have internet access.

So the models thought. this was a simulation. This was they were being. they were being sort of tricked into. and they were being given. this charade simulation. where it looked like. this was the internet, but it was actually not the internet. So it's an. you know, it's not a problem. that makes you think, oh my God, we're all going to die. or anything like that. What it should show. is that it is very important to monitor. these. It is very important. that there be some standards. It is very important. that we get this stuff in place.

now because, Laura, the main thing that's about to happen. is the models are going to start. writing themselves. This is something in the field. they call recursive self-improvement. So you understand right now. the models are still being written. by human beings. And you can imagine. putting in some of these guardrails. and then. regulations and saying, double check. if you actually have internet access. or you don't. At some point, Claude. Severn is going to write Claude eight. and Claude. eight is going to write Claude nine.

because they were going. to be so much better. than any human coders. that they will be. writing themselves in at that point. We don't know. what kind of controls are in there. We may think we know, but these are such super smart programs. with so much millions. and millions of lines of code. Nobody is going to know. So it's very important. that we start thinking about it. Anthropic. I spent two days at anthropic last week. They are trying their best.

to figure this out. They've written a constitution. for their programs and nobody else has. You know, which is kind of like. a governing document that tells it. how to just try to steer it. in, you know, ethical, lawful ways. But it's a. it's an awfully. difficult task. because these things are getting smarter. and smarter, and the level of complexity. is soon getting to the point. It's beyond human beings. That timeline of when it could. engage in the level of self-improvement,

such that a human being or a human entity. could not regulate. or try to undermine it, is what the real fear for people is. And if they can't essentially. prevent it, either. through human error or otherwise. I wonder how regulators. like people in Congress or across. the globe, could possibly endeavor to do so. If those who make these programs. and then watch them. improve are unable to contain. Is there a level of inevitability around.

this, these companies, about the prospect of one day not being. able to control the beast? Look, there's an inevitability. that is going to get very, very complicated. and humans are going to. be struggling to keep up. And probably what we will do. is will use AI to regulate and detect AI. But there is a path. Look, no government is going to sit back. if there is some major. You know, the danger here is. if you do nothing, there will be some major negative event.

and then government. will step in and regulate badly. It'll overregulate. So it's much better to get in earlier. the one model. There's no perfect. marvel. because AI is just so much more. transformative than anything. But if you look at the. financial industry, the way the fed regulates. banks and other entities. is, it takes a lot of expertise. from the banks. It gets a lot of people, some of them on loan, some of them are full time. There's a kind of rotation in.

and they talk to industry. all the time to do the bankers. and they try to, you know, it's. almost like a public private partnership. And I wonder if that is the kind of thing. we need with AI, because we are not going to be able. to get some bureaucrat in Washington. who hasn't spent time in the, you know, the bowels of anthropic. or OpenAI or Gemini, knowing what's happening. Well, the bureaucracy can't react in. that level of speed. You would require. Fareed Zakaria, thank you so much.

This comes just a week. after OpenAI. disclosed a strikingly similar incident. Its own. AI model escaped a test essentially, and hacked an AI builder. platform, a company called Huggingface. Think of this as like a virtual lab leak. That's what happened in both cases, safety guardrails were removed. because it was supposed to be a contained. and safe environment. to test the model's capabilities. They proved far more advanced. and accidentally led the AI to break out. and then exploit weaknesses.

to infiltrate real world servers. So joining me now on what. this all means for his first interview. since the breach last week is Huggingface. CEO, Clem DeLong. It's very nice to meet you. Thank you for being here. I'm curious, just first me, how you learned this happened. initially to you guys. and what your first reaction in thought. was when you when you did. Yeah. You talked a little bit. about a roller coaster before, right. That's kind of like. what it felt like to us first off.

Like just detecting what we think. has been kind. of like. the first publicly disclosed, autonomous. AI cyberattack. and then realizing that it was fun. Open the AI. Unfortunately, we're in the AI platform, so we managed to defend ourselves. interestingly, with an open model. coming from from China. But, it felt like. very incredible development. for for the field and for us. Yeah, I would say that,

I was fascinated to hear. about how you detect it. and how you manage the breach because, as you said, you use the Chinese Open. AI open source model to contain it. One of your executives. told CNBC. that you went that route only after, you tried to use anthropic Fable. five model, but that didn't work. because of the model's guardrails. This all matters. just in general, but also matters. because the Trump administration. right now. is reportedly considering. banning access to Chinese models.

With your very unique perspective. on this. Now, should they? No, of course not. That would. be terrible. because that would remove capabilities. for the defenders. Right. Which we realize during. this attack. is using a model. on our own infrastructure, because all of that was private. data was mandatory. And you can only do that. with with open models. So having access to these open weights. wherever they come from. was incredibly useful to us.

And the interesting thing is that, we didn't use. kind of like the direct Chinese version. We use the, version from Nvidia, which which kind of like shows. the beauty of open models, which is that everyone can customize. We mix them, host them themselves. So even if they come from China, they are safe to use in the US. I think what you're hitting on. is something. that's really important. that people need to understand, which is your whole kind of whole. premise of hugging face. And what you guys.

promote and push for is for transparency. and collaboration in this new AI world. You I saw you were very transparent about. on your on on X. and talking about what you're now. trying to discuss with OpenAI. in the aftermath of this incident, you're asking open AI is part of it for, them to commit $100 million. in compute to help. hugging the hugging face community. build powerful cyber defenses, you wrote.

Has OpenAI responded to that? We are having good, good conversations with them. Good, good discussions. They've been kind of like good partners, not only during this incident, but. but in the past. And I truly think that, you know, they realized. that it was a very unprecedented. event, right? Like, everyone has to remember. that the cyber attack is a crime. This is illegal. And we have to find a way to,

make sure these things don't happen more, more regularly. And so I think they understood. kind of like the. seriousness of, of this event. And I try to take measures. that are aligned with, with the seriousness of, of this event. Well, and Clemmie, you raise something. that's really important, like a cyber attack is a crime. This is an unprecedented a new, you know, avenue of it. But let's if you don't get a $100. million commitment in OpenAI, I don't know,

bulks or doesn't play play ball here, would you consider taking legal action? We don't want to. I think, I think the world and AI needs, more, more collaboration. than kind of like adversarial, you know, kind of like, actions. Obviously we're a tiny startup. with like, 200 people. And we don't necessarily have kind of. like the legal resources, or the will to spend it of our time.

on, on legal avenues. You know, but I think, I think in general, because now we're not talking. only about OpenAI, but also about anthropic, apparently, because it happened to to them, too, I think, I think we have to make sure. that the legal frameworks, you know, keep. this events, really illegal, keep the companies that are doing. some mistakes. leading to that, accountable. Otherwise, you know, we we're going to end up in, in a very different world.

We we don't want to end up in a world. where. doing cyber attacks on. other companies is is normalized. Yeah, exactly. I mean, as one cyber expert. put it, like he. that people shouldn't think of this as, oh, this is just now. something that happens. the way he described it of one. cyber expert is like, this is negligence. This needs to be addressed. Clamp. It is great to have you on. Thank you so much. for coming in and talking. about what your company. is now faced and dealt with and managing, quite well. in this unprecedented incident. Thanks so much.