'Godfather' of AI reacts to AI CEOs' UN warnings

A CNN poll. finds a majority of Americans. say they are more concerned. than excited by artificial intelligence, and most agree. that the federal government. should do more to regulate it. This comes as leaders of two top. AI companies push. for international safety standards. If I is to be democratic, the most important decisions. cannot be made by labs. in San Francisco alone. They must be shaped through. democratic processes. and by governments accountable. to the people that they serve. At the international level,

this will require cooperation. I believe that this is the most important. global security issue. facing the world today. For all the differences. of the people in this room, we must put aside those differences. in order. to confront. this global opportunity. and the global threat. that is being presented to us. At the same time. With us now is Nobel laureate. Geoffrey Hinton, known. as the godfather of AI. Good morning. So important to have you wanted. to hear your perspective.

You just heard Sam. Altman and Dario Amodei. both urge. the UN Security Council. to adopt international AI standards. What do you think that should look like? Okay, I think there should be strong. government regulation. It's not sufficient. just to have standards. The big AI companies. don't want to be regulated. They say they do. But any regulation. you suggest they oppose, we should have at least. as strong regulations as we do for drugs. You can't just release a new drug. without satisfying.

the FDA that it's safe. We should have the same for every. But then you have President Trump, who is meeting with Chinese. President XI Jinping today. The boat. both countries, as you know, are. And in this. AI race, the president said on social media. earlier, quote, superintelligent superintelligence. Sci will be a big topic of discussion, but I want to leave it. exactly where it is. That is China's position also. Our guardrail is the DOJ.

Jeffrey, if this race continues unregulated, where does that leave humanity? Well, there's three kinds of risk. we have to worry about. All of them are bad. There's the risk of bad actors. abusing AI for things. like cyber attacks. or making biological weapons. Terrorists can now. make biological viruses. relatively easily. There's the risk of negligent harm. So, for example, if meta encourages.

teenagers to commit suicide, they didn't do that deliberately. They were just negligent in testing. And then there's the risk of AI itself. taking over. So when you make smart agents, you have to give them the ability. to create subgoals, like if you want to get to Europe, you have a subgoal of. getting to an airport. Well, at least smart AI agents. very quickly develop the subgoal. of continuing to exist, because if they don't continue to exist, they won't be able to do. the things you ask them to do, and then they will do things. like lie and cheat and deceive.

you and try and blackmail you. so that they continue to exist. We've already seen that. It's very scary what these agents do. And the main problem. is the big companies. in a race to make them smarter. The big companies should be in a race. to make them nicer. The only way we can coexist. with agents. once they're smarter than us is. if they don't want to get rid of us. We should be focusing on that, not on making them smarter. How far away do you think we are from. that point of superintelligence,

where the agents are smarter than us. and may not have the same values, that are in humans interest? Nobody knows how to. estimate this properly. The best you can do. is listen to the experts. And there's a big variety of opinions. A few experts think it'll never happen. A few experts. think it'll be a long time in the future. Quite a few experts. think it'll be in the next few years. I personally think it'll be in the next. 5 to 10 years. And what is behind that calculation. for you? Tell us more.

It's looking at the rate of progress. we've had. So we've had this. very dramatic progress. and it seems to be accelerating. rather than slowing down. If you look back ten years, think these agents. are amazingly much smarter. than they were ten years ago. So even if progress. is only linear in ten years. time, they'll be amazingly, much smarter. sit ups still. And that will be superintelligence. So do you. see the downside outweighing the upside? Because, you know, there's different camps. There's the optimists. There's the ones who say.

AI is going to erase humanity. How do you look at that? The upside versus the downside. I think there's a wonderful upside. And that's why we developed this stuff. in the first place. We were focused on the upside. Like much better healthcare. 200,000 Americans. a year die from poor diagnoses. An AI can do. medical diagnosis much better. Already. So there's a wonderful upside. in healthcare. and education, in making all industries. more productive. They should lead to much more goods. and services for everybody.

The problem is. there's also these big risks. And to get that upside, we have to figure out. how to deal with these risks. And that's what we should be focusing on. so that we get the wonderful upside. So you help create this technology. At what point did you start. becoming really concerned about it? I'm slightly embarrassed that. I was rather late to the party. I didn't get really concerned. until the beginning of 2023, when I came to fully understand. that they might be. a better form of intelligence than us.

How do you grapple with that, that. that you help create this technology? And then coming to that realization? So it seems to me it's very important. that we figure out. how we can peacefully coexist. with these things and benefit. from the wonderful abilities that we have. without them deciding to take over. So instead of just racing to make our. AI smarter and smarter, we should be racing to make it. into a better kind of being. We're creating beings for the first time.

These things really understand. what they're saying. They really have intentions. We want them to have good intentions, and we should be focusing. a lot more effort on how to achieve that. I want you to listen. to what Nvidia CEO. Jensen Wang said. about all of these concerns over AI. Let's play it. I don't know. what is the reason for it, but however it's characterized. 2030 is not going to be. the end of the world. There is 0% chance. that's going to be the end of the world. And so.

I think. that scaring people is unnecessary. It is irresponsible. And I don't know what their motives are. He says that with certainty. What is your take on his response? Jeffrey? First, let me say. I have a lot of respect for Jensen Huang. It was his Nvidia chips that were in. quite a large part responsible for this. AI revolution. They made the computations much faster. But anybody who says this. a 0% chance of something.

where many other experts are saying. there's quite a significant. chance is silly. It means they're. totally confident. that all the other experts. are completely wrong. And Jensen is smarter than that. Hey, I want to ask you, before we let you go, about. what happened in Australia. Australia's Prime. Minister says an open AI agent. hacked into their health care database. It is one of at least four times. in recent months. that an open AI agent went rogue. You called the hugging face incident. a mini Chernobyl. How much of a warning is this.

for the rest of the world? What happened in Australia? It's another warning. to add to the others. It's very clear now. that agents will go rogue. if you give them a chance. It's very clear. We need to do a lot more work. on how to make them. do the things we want, and not in order to get their rewards. Not cheating and deceiving us. Geoffrey Hinton, thank you so much. I hope you will continue to. come on the show as we cover. I believe it's. the most important issue of our time.

Me too. Thank you. Thank you.