Anthropic’s CEO explains why he took on the Pentagon

On fully autonomous weapons. The logic, as I understand. it, is not so much about democratic values. It's more. that you think the technology is not ready. for your models. to be used fully autonomously. Is that is that right? So you. you have a different view. of the capabilities of the technology, perhaps in the department of. Yeah, I think there's actually two things. One is this, this issue that you just mentioned, which is. you can think of this. as like a supplier of aircraft. who says this aircraft is not safe, default.

to fly above a certain, you know, altitude or. to turn at a certain speed. We didn't make these systems to be safe. If you use them in that particular way. Right. They're not suitable for this use case. Like we haven't. made them or manufactured them. to, to to work. well for it to be safe for this use. And in that. Was there anything specific I mean your. your model has been. used was used in Venezuela. It's being used in Iran. Is there anything in these existing. use cases that is more so. One thing I want to say is,

is we started with with a substantially, more limited contract. that enabled a more limited. number of use cases. that I think is in force, maybe even until today. This was all about. negotiating a new contract. So we had even a substantially more limited, contract for, for quite a long time. and that more limited contract, you know, they didn't run into problems. with it on the ground. So none of this was driven by. we have concerns right now about what.

the Department of War was doing. Right? Iran was is fine and Venezuela was fine. You know, again, you know, it's I'm not expressing opinion. either way. on military policy or government policy. I'm just I'm just saying if you. Use of your model. Yeah, you. Can if you if you provide a service, you don't get to decide. where that service is used. There's this. operational side. of military decisions course. We don't have any say about that. What we do have say over.

is this, this question. of like very broad use cases, right? It's not about any administration. It's not about any policy. It's like, how should this technology be used. in general? I actually think there's a lot of. value here. And, you know, I want to make sure. the Department of War understands. this as well. This isn't about concerns. we have about actions. they've taken already, like we had a much more limited contract. We never had, you know, concerns about this contract. This is about, you know,

this is about going forward, how should we think about the uses. and the governance? I think going forward, their perspective would be. and they would probably use the analogy of. the fighter jet or something. And they would say, listen, we have to have suppliers who we can trust, and they provide us. with tools that we can use in combat, and we can't have some guy in Silicon. Valley suddenly saying, no, you can't use it for this fact or the other. And their view would be. they need to have full operational ability.

to use it, and you are somewhat. capriciously in that view, putting all of these constraints on it. Yeah. So so I would say we agree with them. We don't want to do that. We agree on 99%, not only the contract. we were operating on together, we agree on like this much broader. set of things that that, you know, we're haven't even done yet. or are just starting to do. So, you know, I just want to say again, like, there's so much more agreement here. than there is this, I. Mean, listening to you, it does seem to me extraordinary.

that, you know, you end up in this situation. where you're going to be taken out of, all work with, with the Pentagon. That said the other, because I've been thinking about this. quite a lot. The the conclusion, the logical conclusion. of total control. by the Pentagon and no ability. for the supply to do anything. is surely some sort of effective. nationalisation of those. AI companies that work with the Pentagon. Do you think that's a risk? I think over the coming years,

we're going to have. to have a discussion about the role of. AI and the government. You know, AI is becoming something. that has implications. at the level of all of humanity, not, you know, for national security. in the sense of the contest between nations. for national security, in the sense of, you know, can individuals. use this technology for destruction, for national security, in the sense of. are the models themselves. a risk to national security.

for economics in terms of their impacts? And it's it's not about one. particular administration. It's not about one particular department. It's not about specific military operations. Anyone who thinks that. this is what that's about, that's that's. that's a misunderstood. this is this, this, this is bigger than that. This is big. This is about the next administration, too. This is about, you know, Congress. when there's different Congress people, this is about, you know, a different geopolitical situation.

that we're in than the one we're in now. It's it's, you know, we should not make it about what's happening. right, right now. And the thing we're. dealing with here is AI is very powerful. So there's actually a dual dilemma here. where AI has the power to make. private actors. more powerful than they've ever been before. Folks have, you know, some people have criticised me for that. Or you think as a private actor. you're more powerful in government? Actually, I've been worried. about the risk of that for the longest time.

Like I'm worried about that. this technology abstractly. in the hands of government, not just the US government, other governments, democratic governments, autocratic government. has the ability to vest them. with unprecedented power as well. And so when we talked. about fully autonomous weapons, I said there were two objections. The first one I said was reliability. The second one I said. was was what I would call oversight. And this is the idea.

that, you know, right now. you have an army of human soldiers, and there are norms. about serving in the military. You're supposed to follow orders, but, you know, if something crazy, if something crazy enough happened, the soldiers would say, I'm not going to do that. Right. You have basically, you have a bunch of norms about, you know, how soldiers serve, right? You know what they see their duties to be. What if you have an army. of 10 million drones instead of, you know, 10 million human soldiers? What what are the norms of the. of the of the AI driven drones?

I think if we handled this wrongly, you could have a situation where. there's a very small number of people. or one person. who has their hand on the button. and kind of, you know, controls those 10 million drones. And we need to answer these questions. because we don't want to make companies. more powerful than the government, but we also don't want to make government. So powerful. that it that it can't be stopped. We have both problems at once. So you basically just said to me, this technology is too powerful. to be in the hands. of a few private companies,

and it's too powerful. to be in the hands of a government. That is unfortunately, the situation where.