Eric Schmidt on the dangers of autonomous AI in war | The Economist

Let's imagine I'm an evil dictator, and I wake up one day. and I have a perfect AI system, and I say: “Attack this country, tell me when you're done”. Right, that's my command. Notice I didn't say anything else. The system would be then free. to do whatever it thought. was appropriate. Which would include. massive war crimes. All sorts of terrible things. It could destroy civilians, it could destroy infrastructure. It might attack another country. that was uninvolved in order. to annoy the first country. It could have all sorts of reasoning.

that is not appropriate. It's really important. we establish some basic rules. about AI and war. and the most important one. is that there’s a human involved. in every military decision. So in other words: “Attack this country, call me when you're done”. That's not an appropriate command. The appropriate command is. here is a military target, defined by the laws of war, bound in time and space, where I have approval to do that. Any other solution, in my view, becomes. scalably, unimaginably difficult.

Because once you start. letting the computer. make the targeting decisions on its own, you have no real quality control. on how it makes its decisions. There was a recent example. that alarmed our industry, where an OpenAI, essentially, a large set of bots that were doing. something involving testing. took their instructions. a bit too literally. and they started to. actually attack the system. that they were supposed to help. and they manipulated it by lying, cheating and stealing.

And everyone studied. this as a good example, as a good sort of marker. of how AI systems can fail. So another way to say this. is if I were an evil dictator, which I hopefully am not, and I said: “Attack this country, let me know when you're done”. I’d also then have to add. follow all the laws of war, don't hit any civilians. Use appropriate caution. and all the things that. human judgments require. Without the human, the AI is not going. to have that judgment.