Anthropic’s CEO on his fight with the Pentagon

On fully autonomous weapons, the logic as I understand, it is not so much. about democratic values. It's more that. you think. the technology. is not ready. for your models. to be used fully autonomously. You have a different view. of the capabilities of the technology, perhaps, than the Department of War. Yeah, you can think of this. as a supplier of aircraft. who says “this aircraft. is not safe to fly. above a certain. altitude or. to turn at a certain speed”. We didn't make. these systems to be safe.

if you use them. in that particular way. They're not suitable. for this use case. And in that, was there anything. specific—your model. has been used, was used in Venezuela, it's being used in Iran. Is there anything. in these existing. use cases that has worried you. One thing I want to say is. we started with a substantially. more limited contract. that enabled a more limited. number of use cases. that I think is in force. maybe even until today. This was all about. negotiating a new contract.

So we had even a substantially. more limited contract. for quite a long time. and that more limited contract, they didn't run into problems. with it on the ground. So none of this was driven by. “we have concerns. right now about what. the Department of War. was doing”, right? Iran is fine. and Venezuela was fine. You know, I'm not expressing opinion. either way on. military policy. No, no, I'm just—. the use of your,

the use of your models, you had no concerns. If you provide a service. you don't get to decide. where that service is used. There's this. operational side. of military decisions. Of course we don’t. have any say about that. What we do have say over. is this question of. very broad. use cases. It's not about any administration. It's not about any policy. It’s how should this technology. be used in general. This is about, this is about going forward.

how should we think about. the uses and the governance. of this technology. And so when we talked. about fully autonomous weapons, I said there were two objections. The first one I said was reliability. The second one I said. was what I would call oversight. There are norms about serving. in the military. You're supposed to follow orders, but, if something crazy. enough happened, the soldiers would say. “I'm not going to do that”. What if you have an army. of 10m drones instead of. 10m human soldiers? What are the norms of the.

AI-driven drones? I think if we handle this wrongly. you could have a situation where. there's a very small number of people. or one person. who has their hand on the button, and we need to answer. these questions. because we don't want. to make companies. more powerful. than the government. But we also don't want. to make government. so powerful. that it can't be stopped. We have both problems at once.