The AI Safety Expert: These Are The Only 5 Jobs That Will Remain In 2030! - Dr. Roman Yampolskiy

You've been working on AI safety for two. decades at least. >> Yeah. I was convinced we can make safe. AI, but the more I looked at it, the. more I realized is not something we can. actually do. You have made a series of. predictions about variety of different. dates. So, what is your prediction for. 2027? Dr. Roman Yampolskiy is a globally. recognized voice on AI safety and. associate professor of computer science. He educates people on the terrifying. truth of AI. >> and what we need to do to save humanity.

In 2 years, the capability to replace. most humans in most occupations will. come very quickly. And then in 5 years, we're looking at a world where we have. levels of unemployment we've never seen. before. Not talking about 10% but 99%. And that's without super intelligence, a. system smarter than all humans in all. domains. So, it would be better than us. at making new AI. But it's worse than. that. We don't know how to make them. safe. And yet we still have the smartest. people in the world competing to win the. race to super intelligence. But what do.

you make of people like Sam Altman's. journey with AI? So, a decade ago, we. published guardrails for how to do AI. right. They violated every single one. And he's gambling 8 billion lives on. getting richer and more powerful. So, I. guess some people want to go to Mars. Others want to control the universe. But it doesn't matter who builds it. The. moment you switch to super intelligence, we will most likely regret it terribly. And then by 2045? Now, this is where it gets interesting.

Dr. Roman Yampolskiy, let's talk about. simulation theory. I think we are in. one. And there is a lot of agreement on. this. And this is what you should be. doing in it so they don't shut it down. First, I see messages all the time in the. comment section that some of you didn't. realize you didn't subscribe. So, if you. could do me a favor and double-check if. you're a subscriber to this channel, that would be tremendously appreciated. It's the simple, it's the free thing. that anybody that watches this show. frequently can do to help us here to. keep everything going in this show in. the trajectory it's on. So, please do.

double-check if you subscribed and uh. thank you so much because in a strange. way you are you're part of our history. and you're on this journey with us and I. appreciate you for that. So, yeah, thank. you. Dr. Roman Yampolskiy. What is the mission that you're. currently on? Cuz it's quite clear to me. that you are on a bit of a mission and. you've been on this mission for, I. think, the best part of two decades at. least. I'm hoping to make sure that super. intelligence we are creating right now.

does not kill everyone. Give me some Give me some context on. that statement cuz it's quite a shocking. statement. Sure. So, the last decade, we actually. figured out how to make artificial. intelligence better. Turns out if you add more compute, more. data, it just kind of becomes smarter. And so now, smartest people in the. world, billions of dollars, all going to.

create the best possible super. intelligence we can. Unfortunately, while we know how to make. the systems much more capable, we don't know how to make them safe. How to. make sure they don't do something we. will regret. And that's the state of the art right. now. When we look at. just prediction markets, how soon will. we get to advanced AI? The timelines are very short, couple. years. Two, three years according to prediction.

markets, according to CEOs of top labs. And at the same time, we don't know how to make sure that the. systems are aligned with our. preferences. So, we are creating this alien. intelligence. If aliens were coming to Earth and. you had 3 years to prepare, you would be panicking right now. But most people don't don't even realize. this is happening.

So, some of the counterarguments might. be, well, these are very, very smart. people. These are very big companies. with lots of money. They have a. obligation and a moral obligation but. also just. a legal obligation to make sure they do. no harm. So, I'm sure it'll be fine. The. only obligation they have is to make. money for their investors. That's the. legal obligation they have. They have no. moral or ethical obligations. Also, according to them, they don't know how. to do it yet. The state-of-the-art. answers are, we'll figure it out when we. get there or AI will help us control.

more advanced AI. That's insane. In terms of probability, what do you. think is the probability that something. goes catastrophically wrong? So, nobody can tell you for sure what's. going to happen. But if you're not in. charge, you're not controlling it, you. will not get outcomes you want. The. space of possibilities is almost. infinite. The space of outcomes we will. like is tiny. And. who are you and how long have you been. working on this?

I'm a computer scientist by training. I. have a PhD in computer science and. engineering. I probably started work in AI safety, mildly defined as control of bots at the. time, 15 years ago. 15 years ago. So, you've been working on. AI safety before it was cool. Before the. term existed. I coined the term AI. safety. So, you're the founder of the. term AI safety? The term, yes, not the. field. There are other people who did. brilliant work before I got there.

Why were you thinking about this 15. years ago? Because most people have only. been talking about the term AI safety. for the last two or three years. Yeah, it started very mildly just as a. security project. I was looking at poker. bots. And I realized that the bots are getting. better and better. And if you just project this forward. enough, they're going to get better than us, smarter, more capable. And it happened. They are playing poker way better than. average players. But.

more generally, it will happen with all. other domains, all the other cyber. resources. I wanted to make sure AI is a technology. which is beneficial for everyone. So, I. started work on making AI safer. Was there a particular moment in your. career where you thought, "Oh my god."? First 5 years at least, I was working on. solving this problem. I was convinced we. can make happen, we can make safe AI. That was the goal. But the more I looked. at it, the more I realized every single.

component of that equation is not. something we can actually do. And the more you zoom in, it's like a. fractal. You go in and you find 10 more. problems and then 100 more problems. And. all of them are not just difficult, they're impossible to solve. There is no. seminal work in this field where like, we solved this. We don't have to worry. about this. There are patches. There are. little fixes we put in place and quickly. people find ways to work around them.

They jailbreak whatever safety. mechanisms we have. So, while progress. in. AI capabilities is exponential or maybe. even hyper exponential, progress in AI safety is linear or. constant. The gap is increasing. The gap between. the the how capable the systems are and. how well we can control them, predict. what they're going to do, explain their. decision-making. I think this is quite an important point. because you said that we're basically.

patching over the issues that we find. So, we're developing this this core. intelligence and then to stop it doing. things. or to stop it showing some of its. unpredictability or its threats, the companies that are developing this. AI are programming in code over the top. to say, "Okay, don't swear. Don't say. that rude word. Don't do that bad. thing." Exactly. And you can look at. other examples of that. So, HR manuals, right? We have those humans, they're. general intelligences, but you want them. to behave in a company. So, they have a.

policy. No sexual harassment. No this, no that. But if you're smart enough, you. always find a workaround. So, you're. just pushing behavior into a different, not yet restricted subdomain. We We should probably define some terms. here. So, there's narrow intelligence which. can play chess or whatever. There's. artificial general intelligence which. can operate across domains. And then. super intelligence which is smarter than. all humans in all domains. And where are we? So, that's a very fuzzy boundary, right?

We definitely have many excellent narrow. systems, no question about it. And they. are super intelligent in that narrow. domain. So, protein folding is a problem which was. solved using narrow AI and it's superior. to all humans in that domain. In terms of AGI, again I said, if we. showed what we have today to a scientist. from 20 years ago, they would be. convinced we have full-blown AGI. We. have systems which can learn. They can. perform in hundreds of domains and.

they're better than human in many of. them. So, you can argue we have a weak version. of AGI. Now, we don't have super intelligence. yet. We still have brilliant humans who. are completely dominating AI, especially. in science and engineering. But that gap is closing so fast. You can. see. especially in the domain of mathematics. 3 years ago, large language models couldn't do basic. algebra. Multiplying three-digit numbers was a.

challenge. Now, they're helping with. mathematical proofs. They're winning. mathematics Olympiads, competitions. They're working on solving millennial. problems, hardest problems in. mathematics. So, in 3 years, we closed. the gap from subhuman performance to. better than most mathematicians in the. world. And we see the same process. happening in science and engineering. You have made a series of predictions. and they correspond to a variety of. different dates and I have those dates.

in front of me here. What is your prediction for the year. 2027? We're probably looking at AGI as. predicted by prediction markets and tops. of the labs. So we'd have artificial general. intelligence by 2027. And how would that make the world. different. to how it is now? So if you have this concept of a drop-in. you have free labor, physical and. cognitive, trillions of dollars of it.

It makes no sense to hire humans for. most jobs. If I can just get, you know, a $20. subscription or free model to do what an. employee does, first, anything on a computer will be. automated. And next thing, humanoid robots are. maybe 5 years behind, so in 5 years all. the physical labor can also be. automated. So we're looking at a world where we. have levels of unemployment we've never. seen before. Not talking about 10%. unemployment, which is scary, but 99%.

All you have left is jobs where, for. whatever reason, you prefer another. human would do it for you. But anything else. can be fully automated. It doesn't mean. it will be automated in practice. A lot. of times. technology exists, but it's not. deployed. Video phones were invented in. the '70s. Nobody had them until iPhones. came around. So we may have a lot more time with jobs. and with world which looks like this.

But capability. to replace most humans in most. occupations will come very quickly. Okay, so let's try and drill down into. that and and stress test it. So. a podcaster like me, would you need a podcaster like me? So let's look at what you do. You. prepare, you. ask questions, you ask follow-up. questions, and you look good on camera.

Thank you so much. Let's see what we can. do. Large language model today can. easily read everything I wrote Yeah. and. have very solid understanding. Better, I. assume you haven't read every single one. of my books. I haven't. Yeah. That thing. would do it. It can train on every podcast you ever. did, so it knows exactly your style, the. types of questions you ask. It can also. find correspondence between what worked. really well, like this type of question. really increased viewers, this type of. topic was very promising, so it can.

optimize, I think, better than you can. because you don't have a data set. Of course, visual simulation is trivial. at this point. So it can you can make a. video within seconds of me sat here and. So we can generate videos of you. interviewing anyone on any topic very. efficiently and you just have to get. likeness approval, whatever. Are there many jobs that you think would. remain in a world of AGI? If you're. saying AGI is potentially going to be. here, whether it's deployed or not, by.

2027, what kind and then okay, so let's take. out of this any physical labor jobs for. a second. Are there any jobs that you. think a human would be able to do better. in a world of AGI. still? So that's the question I often. ask people. In the world with AGI, and I. think almost immediately we'll get. superintelligence as a side effect. So. the question really is, in a world of. superintelligence, which is defined as. better than all humans in all domains, what can you contribute?

And so you know better than anyone what. it's like to be you. You know what ice cream tastes to you. Can you get paid for that knowledge? Is. someone interested in that? Maybe not, not a big market. There are jobs where you want a human. Maybe you're rich and you want a human. accountant for whatever historic. reasons. Old people like. traditional ways of doing things. Warren. Buffett would not switch to AI. He would.

use his human accountant. But it's a tiny subset of a market. Today we have products which are. man-made. in US as opposed to mass-produced in. China, and some people pay more to have. those. But it's a small subset. It's a almost a. fetish. There is no practical reason for it. And I think anything you can do on a. computer could be automated. using that technology. You must hear a lot of rebuttals to when. this when you say it because people.

experience a huge amount of mental. discomfort when they hear. that their job, their career, the thing. they got a degree in, the thing they. invested $100,000 into is going to be. taken away from them. So their natural. reaction, some for some people is that. cognitive dissonance that no, you're. wrong. AI can't be creative. It's not. this, it's not that. It will never be. interested in my job. I'll be fine. because. you hear these arguments all the time, right? >> It's really funny. I ask people and I. ask people in different occupations. I'll ask my Uber driver, are you worried.

about self-driving cars? And they go, "No. No one can do what I do. I know the. streets of New York. I can navigate like. no AI. I'm safe." And it's true for any job. Professors are saying this to me. Oh, nobody can lecture like I do. Like this. is so special. But you understand it's ridiculous. We. already have self-driving cars replacing. drivers. That is not even a question. if it's possible. It's like how soon. before you're fired.

Yeah, I mean, I've just been in LA. yesterday and my car drives itself. So I. get in the car, I set I put in where I. want to go, and then I don't touch the. steering wheel or the brake pedals, and. it takes me from A to B, even if it's an. hour-long drive without any intervention. at all. I actually still park it, but other than that, I'm not I'm not. driving the car at all. And then. obviously in LA we also have Waymo now, which means. you order it on your phone and it shows. up with no driver in it and takes you to. where you want to go. Oh, yeah. So it's. quite clear to see how that is.

potentially a matter of time. For those. people, cuz we do have some of those. people listening to this conversation. right now, that their occupation is. driving, to offer them a I think driving is the. biggest oc-. occupation in the world, if I'm correct. I I'm pretty sure it is the biggest. occupation in the world. >> of the top ones, yeah. What would you say to those people? What what should they be doing with. their lives? What should they should. they be retraining in something or. what time frame? So that's the paradigm. shift here. Before we always said this. job is going to be automated, retrain to.

do this other job. But if I'm telling. you that all jobs will be automated, then there is no plan B. You cannot retrain. Look at computer science. 2 years ago, we told people, learn to. code. Mhm. You are an artist, you cannot. make money, learn to code. Then we realized, oh, AI kind of knows. how to code and getting better. Become a. prompt engineer. You can engineer prompts for AIs. It's.

going to be a great job. Get a 4-year. degree in it. But then we're like, AI is. way better at designing prompts for. other AIs than any human. So that's gone. So I can't really tell. you right now, the hottest thing is. design AI agents for practical. applications. I guarantee you in a year. or two it's going to be gone just as. well. So I don't think there is a. this occupation needs to learn to do. this instead. I think it's more like, where's the humanity when we all lose. our jobs? What do we do? What do we do.

financially? Who's paying for us? And what do we do in terms of. meaning? What do I do with my extra 60, 80 hours a week? You've thought around this corner, haven't you? A little bit. What is around that corner in your view? So the economic part seems easy. If you. create a lot of free labor, you have a. lot of free wealth, abundance, things. which are right now. not very affordable become dirt cheap,

and so you can provide for everyone's. basic needs. Some people say you can. provide. beyond basic needs. You can provide very. good existence for everyone. The hard. problem. what do you do with all that free time? For a lot of people, their jobs are what. gives them meaning in their lives, so. they would. retire or do early retirement. And for. so many people who hate their jobs, they'll be very happy not working. But. now you have people who are chilling all.

day. What happens to society? How does that. impact crime rate, pregnancy rate, all. sorts of issues? Nobody thinks about. Governments don't. have programs prepared to deal with 99%. unemployment. What do you think that world looks like? Again, I I think. >> you going to be doing? very important. part to understand here is the. unpredictability of it. We cannot predict what a smarter than us.

system will do. And the point when we get to that is. often called singularity, by analogy. with physical singularity. You cannot. see beyond the event horizon. I can tell. you what I think might happen, but. that's my prediction. It is not what. actually is going to happen because I. just don't have cognitive ability to. predict a much smarter agent impacting. this world. When you read science fiction, there is never a superintelligence in it. actually doing anything because nobody.

can write believable science fiction at. that level. They either banned AI, like. Dune, because this way you can avoid. writing about it, or it's like Star. Wars. You have this really dumb bots, but not nothing superintelligent ever. Cuz by definition, you cannot predict at. that level. Because by definition of it being. superintelligent, it will make its own. mind up. By definition, if it was. something you could predict, you would. be operating at the same level of. intelligence, violating our assumption.

that it is smarter than you. If I'm playing chess with super. intelligence and I can predict every. move, I'm playing at that level. It's. kind of like my French bulldog trying to. predict. exactly what I'm thinking and what I'm. going to do. That's a good cognitive. gap. And it's not just he can predict. you going to work, you coming back, but. he cannot understand why you doing a. podcast. That is something completely. outside of his model of the world. Yeah, he doesn't even know that I go to. work. He just sees that I leave the. house and doesn't know where I go.

By food for him. What's the most. persuasive argument against. your own perspective here? That we will. not have unemployment due to advanced. technology? That there won't be this. French bulldog human gap in. understanding and. I guess like power and control. So, some people think that we can. enhance human minds either through. combination with hardware, so something. like Neuralink, or through genetic.

engineering to where we make smarter. humans. Yeah. It may give us a little more. intelligence. I don't think we are still. competitive in biological form with. silicon form. Silicon substrate is much. more capable for intelligence. It's. faster. It's more resilient, more energy. efficient in many ways. Which is what. computers are made out of the brain. Yeah. So, I don't think we can keep up just. with improving our biology. Some people.

think maybe, and this is very. speculative, we can upload our minds. into computers. So, scan your brain, connectome of your. brain, and have a simulation running on. a computer and you can speed it up, give. it more capabilities. But to me, that. feels like you no longer exist. We just. created software by different means and. now you have AI based on biology and AI. based on some other forms of training. You can have evolutionary algorithms. You can have many paths to reach AGI.

But at the end, none of them are humans. I have a another date here, which is. 2030. What's your prediction for 2030? What. will the world look like? So, we probably will have. humanoid robots with enough flexibility, dexterity to compete with humans in all. domains, including plumbers. We can make artificial plumbers. Not the plumbers. We That was That felt.

like the last. bastion of. human employment. So, 2030, 5 years from. now, humanoid robots So, many of the. companies, the leading companies, including Tesla, are developing humanoid. robots. at light speed and they're getting. increasingly more effective. And these. humanoid robots will be able to move. through physical space, for you know, make an omelet, do anything humans can do, but obviously. have. be connected to AI as well. So, they can think, talk,

Like they're controlled by AI. They're. always connected to the network, so they. are already dominating in many ways. Our world will look remarkably different. when humanoid robots are functional and. effective. Because that's really when, you know, I start to think, "Crap." Like. the combination of intelligence and. physical ability. is really really doesn't leave much, does it, for. us um. human beings.

Not much. So, today, if you have. intelligence through internet, you can. hire humans to do your bidding for you. You can pay them in Bitcoin, so you can. have bodies, just not directly. controlling them. So, it's not a huge. game changer to add direct control of. physical bodies. Intelligence is where. it's at. The important component is. definitely higher ability to optimize, to solve problems, to find patterns. people cannot see. And then by 2045,

I guess the world looks even even more. um. which is 20 years from now. So, if it's. still around, If it's still around, >> Ray Kurzweil predicts that that's the. year for the singularity. That's the. year where progress becomes so fast, so. this AI doing science and engineering. work makes improvements so quickly we. cannot keep up anymore. That's the. definition of singularity, point beyond. which we cannot see, understand, predict. See, understand, predict the.

intelligence itself or. What is happening in the world? The. technology is being developed. So, right. now, if I have an iPhone, I can look. forward to a new one coming out next. year and I'll understand it has slightly. better camera. Imagine now this process. of researching and developing this phone. is automated. It happens every 6 months, every 3 every month, week, day, hour, minute, second. You cannot keep up with. 30 iterations of iPhone in 1 day. You. don't understand what capabilities it.

has, what. proper controls are. It just escapes. you. Right now, it's hard for any. researcher in AI to keep up with the. state of the art. While I was doing this. interview with you, a new model came out. and I will no longer know what the state. of the art is. Every day, as a percentage of total. knowledge, I get dumber. I may still know more because I keep. reading, but as a percentage of overall. knowledge, we all getting dumber. And when you take it to extreme values,

you have zero knowledge, zero. understanding of the world around you. Some of the arguments against this. eventuality are that when you look at. other technologies like the Industrial. Revolution, people just found new ways. to. to work and new careers that we could. never have imagined at the time were. created. How do you respond to that in a world of. super intelligence? It's a paradigm shift. We always had. tools, new tools which allowed some job.

to be done more efficiently. So, instead. of having 10 workers, you could have two. workers and eight workers had to find a. new job. And there was another job. Now. you can supervise these workers or do. something cool. If you creating a meta. invention, you inventing intelligence, you inventing a worker, an agent, then. you can apply that agent to the new job. There is not a job which cannot be. automated. That never happened before. All the inventions we previously had.

were kind of. a tool for doing something. So, we. invented fire. Huge game changer. But. that's it. It stops with fire. We invent. a wheel. Same idea. Huge implications, but wheel itself is not an inventor. Here we are inventing. a replacement for human mind, a new. inventor capable of doing new. inventions. It's the last invention we. ever have to make. At that point, it. takes over and the process of doing. science, research, even ethics research,

morals, all that is automated at that. point. Do you sleep well at night? Really well. Even though you you spent the last 15, 20 years of your life working on AI. safety and it's suddenly. among us in a in a way that I don't. think anyone could have predicted 5. years ago. When I say among us, I really. mean that the amount of funding and. talent that is now focused on reaching. super intelligence faster has made it. feel more inevitable and more.

soon. than any of us could have possibly. imagined. We as humans have this built-in bias. about not thinking about really bad. outcomes and things we cannot prevent. So, all of us are dying. Your kids are dying, your parents are. dying, everyone's dying, but you still. sleep well, you still go on with your. day. Even 95-year-olds are still doing. games and playing golf and whatnot, cuz. we have this ability to not think about. the worst outcomes, especially if we.

cannot actually modify the outcome. So, that's the same. infrastructure being used for this. Yeah, there is. humanity level. death-like event. We happening to be. close to it probably, but unless I can. do something about it, I I can just keep. enjoying my life. In fact, maybe knowing. that you have limited amount of time. left gives you more reason to have a. better life. You cannot waste any.

And that's the survival trait of. evolution, I guess, because those of my. ancestors that spent all their time. worrying. wouldn't have spent enough time having. babies and hunting to survive. >> Suicidal ideation. People who really. start thinking about how horrible the. world is usually escape pretty soon. Mhm. One of the You co-authored this paper. um analyzing the key arguments people. make against the importance of AI. safety. And one of the arguments in there is.

that there's other things that are of. bigger importance right now. It might be. world wars, it could be nuclear. containment, it could be other things. There's other things that the. governments and podcasters like me. should be talking about that are more. important. What's your rebuttal to that. argument? >> So, super intelligence is a meta. solution. If we get super intelligence. right, it will help us with climate. change, it will help us with wars, it. can solve all the other existential. risks. If we don't get it right, it.

dominates. If climate change will take. 100 years to boil us alive and super. intelligence kills everyone in five, I. don't have to worry about climate. change. So, either way, either it solves. it for me or it's not an issue. So, you think it's the most important. thing to be working on? Without. question, there is nothing more. important than getting this right. And I know everyone says it. You take. any class but you take English. professor's class and he tells you this. is the most important class you'll ever.

take. But. you can see the meta level differences. with this one. Another argument in that paper is that. we will be in control and that the. danger is not AI. This particular argument asserts that AI. is just a tool. Humans are the real. actors that present danger and we can. always maintain control by simply. turning it off. Can't we just pull the. plug out? I see that every time we have. a conversation on the show about AI, someone says can't we just unplug it? Yeah, I get those comments on every. podcast I make and I always want to like.

get in touch with the guy and say this. is brilliant. I never thought of it. We're going to write a paper together. and get a Nobel Prize for it. This is. like let's do it. Because it's so silly. Like can you turn. off a virus? You have a computer virus. you don't like. You turn it off. How about Bitcoin? Turn off Bitcoin. network. Go ahead. I'll wait. This is silly. Those are distributed. systems. You cannot turn them off and on. top of it they're smarter than you. They. made multiple backups. They predicted. what you're going to do. They will turn. you off before you can turn them off.

The idea that we will be in control. applies only to pre super intelligence. levels. Basically what we have today. Today humans with AI tools are. dangerous. They can be hackers, malevolent actors. Absolutely. But the. moment super intelligence becomes. smarter, dominates, they no longer be. important part of that equation. It is. the higher intelligence I'm concerned. about, not the human who. may add additional malevolent payload. but at the end still doesn't control it.

It is tempting. to. follow your the next argument that I saw. in that paper which basically says. listen. this is inevitable. So there's no point fighting against it. because there's really no hope here. So. we should probably give up even trying. and be faithful that it will work itself. out. Because everything you've said sounds. really inevitable. And if with China working on it, I'm. sure Putin's got some secret division. I'm sure Iran are doing some bits and. pieces. Every European country is trying.

to get ahead of AI. The United States is. leading the way. So it's it's inevitable. So we probably. should just have faith and pray. Praying is always good but incentives. matter. If you. looking at what drives these people. So. yes, money is important. So there is a. lot of money in that space and so. everyone's trying to be there and. develop this technology. But if they. truly understand the argument, they. understand that you will be dead.

No amount of money will be useful to. you. Then incentives switch. They would want. to not be dead. A lot of them are young. people, rich people. They have their. whole lives ahead of them. I think they. would be better off not building. advanced super intelligence, concentrating on narrow AI tools for. solving specific problems. Okay, my. company cures breast cancer. That's all. We make billions of dollars. Everyone's. happy. Everyone benefits. It's a win. We are still in control today. It's not.

over until it's over. We can decide not. to build general super intelligences. I mean the United States might be able. to conjure up enough enthusiasm for. that. But if the United States doesn't. build general super intelligences, then. China are going to have the big. advantage, right? So right now at those levels, whoever. has more advanced AI has more advanced. military. No question. We see it with. existing conflicts. But the moment you. switch to super intelligence and control.

super intelligence, it doesn't matter. who builds it, us or them. And if they. understand this argument, they also. would not build it. It's a mutually. assured destruction on both ends. Is this technology different than say. nuclear weapons which require a huge. amount of investment and you have to. like enrich the uranium and you need. billions of dollars potentially to even. build a nuclear weapon. But it feels like this technology is. much cheaper to get to super.

intelligence potentially or at least it. will become cheaper. I wonder if it's. possible that some some guy, some. startup is going to be able to build. super intelligence in. you know, a couple of years without the. need of. you know, billions of dollars of compute. or or electricity power. That's a great. point. So every year it becomes cheaper. and cheaper to train sufficiently large. model. If today it would take a trillion. dollars to build super intelligence, next year it could be 100 billion and so. on. At some point a guy in a laptop. could do it.

But you don't want to wait four years to. make it affordable. So that's why so. much money is pouring in. Somebody wants. to get there this year and lock in all. the winnings. Litecoin level award. So in that regard they're both very. expensive projects like Manhattan level. projects. Which is the nuclear bomb. project. >> Right. The difference between the two. technologies is that nuclear weapons are. still tools. Some dictator, some country, someone has.

to decide to use them, deploy them. Whereas super intelligence is not a is. not a tool. It's an agent. It makes its own decisions and no one is. controlling it. I cannot take out this. dictator and now super intelligence is. safe. So that's a fundamental difference to. me. But if you're saying that it is going to. get. incrementally cheaper like I think it's. Moore's law, isn't it? The technology. gets cheaper. It does. Then there is a future where some guy on. his laptop is going to be able to create. super intelligence without.

oversight or regulation or employees, etc. Yeah, that's why a lot of people. suggesting we need to build something. like um. surveillance planet where you are. monitoring who's doing what and you're. trying to prevent people from doing it. Do I think it's feasible? No. At some. point it becomes so affordable and so. trivial that it just will happen. But at. this point we're trying to get more. time. We don't want it to happen in five. years. We want it to happen in 50 years.

I mean that's not very hopeful. Depends. on how old you are. Depends on how old you are. I mean. if you're saying that. you believe in the future people will be. able to make super intelligence. without the resources that are required. today, then it is just a matter of time. Yeah, but so will be true for many other. technologies. We're getting much better. in synthetic biology where today someone. with a bachelor's degree in biology can. probably create a new virus. This will also become cheaper. Other.

technologies like that. So we are. approaching a point where it's very. difficult to make sure no technological. breakthrough is the last one. So. essentially in many directions we have. this. pattern of making it easier in terms of. resources, in terms of intelligence to. destroy the world. If you look at I don't know, 500 years. ago, the worst dictator with all the. resources could kill couple million. people. He couldn't destroy the world.

Now we know nuclear weapons we can blow. up the whole planet multiple times over. Synthetic biology we saw with COVID, you. can very easily create a combination. virus which impacts billions of people. And all those things becoming easier to. do. In the near term you talk about. extinction being a real risk, human. extinction being a real risk. Of all the. the pathways to human extinction that. you think are. most likely, what what is the leading. pathway? Because I know you talk about.

there being some issue pre-deployment of. these AI tools like you know, someone. makes a mistake when they're. designing a model or other issues. post-deployment. When I say. post-deployment, I mean once the chat. GPT or something, an. agent is released into the world and. someone hacking into it and changing it. and reprogram reprogramming it to be. malicious. Of all these potential paths. to human extinction, which one do you. think is the highest probability? So I can only talk about the ones I can.

predict myself. So I can predict even. before we get a super intelligence, someone will create a very advanced. biological tool, create a novel virus. and that virus gets everyone or most. everyone. I can. envision it. I can understand the. pathway. I can say that. So just. assuming on that then, that would be. using an AI to make a virus and then. releasing it. Yeah. And would that be. intentional or. There is a lot of psychopaths, a lot of. terrorists, a lot of doomsday cults.

We've seen historically again, they. tried to kill as many people as they. can. They usually fail. They kill. hundreds of thousands. But if they get technology to kill. millions or billions, they would do that. gladly. The point I'm trying to emphasize is. that it doesn't matter what I can come. up with. I am not a malevolent actor. you're trying to defeat here. It's the. super intelligence which can come up. with completely novel ways of doing it. Again, you brought up example of your. dog.

Your dog cannot understand all the ways. you can take it out. It can maybe think you'll bite it to. death or something. But that's all. Whereas you have. infinite supply of resources. So if I asked your dog exactly how. you're going to take it out, it would. not give you a meaningful answer. It can. talk about biting. And this is what we know. We know. viruses. We experienced viruses. We can. talk about them. But what.

an AI system capable of doing novel. physics research can come up with is. beyond me. One of the things that I think most. people don't understand is how little we. understand about how these AIs are. actually working. Cuz one would assume, you know, with computers, we kind of. understand how a computer works. We we. know that it's doing this and then this. and it's running on code. But, from reading your work, you describe it. as being a black box. We actually So, in. the context of something like ChatGPT or. an AI we know, you're telling me that.

the people that have built that tool. don't actually know. what's going on inside there. That's exactly right. So, even people. making those systems have to run. experiments on their product to learn. what it's capable of. So, they train it. by giving it all of data, let's say all. of internet text. They run it on a lot of computers to. learn patterns in that text. And then we. start experimenting with that model. Oh, do you speak French? Or, can you do. mathematics? Or, are you lying to me.

now? And so, maybe it takes a year to. train it and then 6 months to get some. fundamentals about what it's capable of. Some safety overhead. But, we still discover new capabilities. in old models. If you ask a question in. a different way, it becomes smarter. So, it's. no longer. engineering how it was the first 50. years where someone was a knowledge. engineer programming an expert system AI.

to do specific things. It's a science. We are creating this artifact, growing. it. It's like a alien plant. And then we. study it to see what it's doing. And just like with plants, we don't have. 100% accurate knowledge of biology. We don't have full knowledge here. We. kind of know some patterns. We know, okay, if we add more compute, it gets. smarter most of the time. But, nobody can tell you precisely what the. outcome is going to be given a set of. inputs.

I've watched so many entrepreneurs treat. sales like a performance problem. When. it's often down to visibility. Because. when you can't see what's happening in. your pipeline, what stage each. conversation is at, what's stalled, what's moving, you can't improve. anything. And you can't close the deal. Our sponsor, Pipedrive, is the number. one CRM tool for small to medium. businesses. Not just a contact list, but. an actual system that shows your entire. sales process end-to-end. Everything. that's live, what's lagging, and the. steps you need to take next. All of your.

teams can move smarter and faster. Teams. using Pipedrive are on average closing. three times more deals than those that. aren't. It's the first CRM made by sales. people for sales people that over. 100,000 companies around the world rely. on, including my team who absolutely. love it. Give Pipedrive a try today by. visiting pipedrive.com/ceo. And you can get up and running in a. couple of minutes with no payment. needed. And if you use this link, you'll. get a 30-day free trial. What do you.

make of OpenAI and Sam Altman and what. they're doing? And obviously, you're aware that one of. the co-founders, was it um This was Ilya. Sutskever? Ilya yeah. Ilya left and he started a a. new company called Superintelligence. Safety. Superintelligence Safety. >> AI safety wasn't challenging enough, he. decided to just jump right to the hard. problem. As an onlooker, when you see that people. are leaving OpenAI to to start.

superintelligent safety companies, what was your read on that situation? So, a lot of people who worked with Sam. said that maybe he's not the most direct. person in terms of being honest with. them and they had concerns about his. views on safety. That's part of it. So, they wanted more. control, they wanted more concentration. on safety. But, also it seems that. anyone who leaves that company and. starts a new one gets a $20 valuation.

just for having it started. You don't. have a product, you don't have. customers, but. if you want to make many billions of. dollars, just do that. So, it seems like. a very rational thing to do for anyone. who can. So, I'm not surprised that there is a. lot of attrition. Meeting him in person, he's super nice, very smart. Absolutely. perfect public interface. You see him. testify in the Senate, he says the right.

thing to the senators. You see him talk. to the investors, they get the right. message. But, if you look at what people who know. him personally are saying, it's probably not the right person to be. controlling a project of that impact. Why? He puts safety second. Second to. winning this race to superintelligence, being the guy who created God and. controlling light cone of the universe.

He's worse. Do you suspect that's what he's driven. by is by the the legacy of being an. impactful person that did a. remarkable thing versus. the consequence that that might have on. for society? Because it's interesting that his his. other startup is Worldcoin, which is. basically a platform to create universal. basic income. I a platform to give us. income in a world where. people don't have jobs anymore. So, on. one hand you're creating an AI company, on the other hand you're creating a.

company that is preparing for people to. not have employment. It also has other. properties. It keeps track of everyone's. biometrics. It. keeps you in charge of a world's. economy, world's wealth. They are. retaining a large portion of Worldcoins. So, I I think it's kind of very. reasonable part to integrate with world. dominance. If you have a. superintelligence system and you control.

money, you're doing well. Why would someone want world dominance? People have different levels of. ambition. Then you are very young person. with billions of dollars, fame, you. start looking for more ambitious. projects. Some people want to go to. Mars, others want to control light cone. of the universe. What What did you say, light cone of the. universe? >> Light cone. So, every part of the. universe light can reach from this.

point, meaning anything accessible you. want to grab and. bring into your control. You think Sam. Altman wants to control. every part of the universe? I I suspect he might, yes. Hmm. It doesn't mean he doesn't want a side. effect of it being a very beneficial. technology which makes all the humans. happy. Happy humans are good for control. If you had to guess. what the world looks like in.

2100, if you had to guess, it's either. free of human existence or it's. completely not comprehensible to someone. like us. It's one of those extremes. So, there's. either no humans It's basically the. world is destroyed or it's so different. that I cannot envision those. predictions. What can be done to turn this ship to a.

more certain positive outcome at this. point? Is Is there still things that we can do. or is it too late? So, I believe in. personal self-interest. If people. realize that doing this thing is really. bad for them personally, they will not. do it. So, our job is to convince. everyone with any power in this space. creating this technology working for. these companies, they are doing something very bad. for them. Not just forget about eight. billion people you're experimenting on. with no permission,

no consent, you will not be happy with. the outcome. If we can get everyone to. understand that's the default, and it's. not just me saying it. You had Geoff. Hinton on. him. Nobel Prize winner, founder of the. whole machine learning space. He says. the same thing. Bengio, dozens of. others, top scholars. We had a statement. about dangers of AI signed by thousands. of scholars, computer scientists. This. is basically what we think right now and. we need to make it a universal. No one. should disagree with this. And then, we.

may actually make good decisions about. what technology to build. It doesn't. guarantee long-term safety for humanity, but it means we're not trying to get. there as soon as possible to the worst. possible outcome. And do you Are you hopeful that that's. even possible? I want to try. We have no choice but to. try. And what would need to happen and who. would need to act? What is it government. legislation? Is it. Unfortunately, I don't think making it. illegal is sufficient. There are.

different jurisdictions, there is, you. know, loopholes. And what are you going. to do if somebody does it? You're going. to fine them for destroying humanity? Like very steep fines for it? Like what. are you going to do? It's not. enforceable. If they do create it, now. the superintelligence is in charge. So, the judicial system we have is not. impactful. And all the punishments we. have are designed for punishing humans. Prisons, capital punishment doesn't. apply to AI. Here's the problem I have is when I have. these conversations, I never feel like I. walk away with.

hope that something's going to go well. And what I mean by that is I never feel. like I walk away with clear some kind of. a clear set of actions that can course. correct what might happen here. So, what. should What should I do? What should the. person sat at home listening to this do? You You talked to a lot of people who. are building this technology. Mhm. Ask them precisely to explain some of. those things they claim to be. impossible, how they solved it or going.

to solve it before they get to where. they're going. Do you know, I don't. think Sam Altman wants to talk to me. I don't know. He seems to go on a lot of. podcasts. Maybe he does. >> wants to go on mine. I wonder why that is. I'd love to speak to him, but I don't I. don't think he wants to. I don't think he wants me to. interview him. Have an open challenge. Maybe money is not the incentive, but. whatever attracts people like that,

whoever can convince you that it's. possible to control and make safe super. intelligence gets the prize. They come. on your show and prove their case. Anyone. If no one claims the prize or. even accepts the challenge after a few. years, maybe we don't have anyone with. solutions. We have companies valued again at. billions and billions of dollars working. on safe super intelligence. We haven't seen their output yet. Yeah, I'd like to speak to Ilya as well.

cuz I know he's he's working on safe. super intelligence, so. Notice the pattern too. If you look at. history of AI safety organizations. or. departments within companies, they usually start well, very ambitious, and then they fail and disappear. So, OpenAI had super intelligence alignment. team. The day they announced it, I think they. said we're going to solve it in 4 years. Like half a year later they canceled the. team. And there is dozens of similar examples.

Leading. a perfect safety for super intelligence, perpetual safety as it keeps improving, modifying, interacting with people. You're never going to get there. It's. impossible. There is a big difference between. difficult problems in computer science. and be complete problems and impossible. problems. And I think control indefinite. control of super intelligence is such a. problem. So, what's the point trying. then if it's impossible? Well, I'm. trying to prove that it is specifically.

that. Once we establish something is. impossible, fewer people will waste. their time claiming they can do it and. find looking for money. So many people. go and give me a billion dollars in 2. years and I'll solve it for you. Well, I don't think you will. But people aren't going to stop striving. towards it. So, if there's no attempts. to. make it safe and there's more people. increasingly striving towards it, then. it's inevitable. But it changes what we. do. If we know that it's impossible to. make it right, to make it safe, then. this direct path of just build it as.

soon as you can become suicide mission. Hopefully fewer people will pursue that. They may go in other directions like. again, I'm a scientist, I'm an engineer. I love. AI. I love technology. I use it all the. time. Build useful tools. Stop building. agents. Build narrow super intelligence, not a. general one. I'm not saying you. shouldn't make billions of dollars. I. love billions of dollars. But. don't kill everyone, yourself included.

They don't think they're going to. though. Then tell us why. I hear things about intuition. I hear. things about we'll solve it later. Tell. me specifically in scientific terms. Publish a peer-reviewed paper explaining. how you're going to control super. intelligence. It's strange. It's strange to it's. strange to even bother if there was even. a 1% chance of human extinction. It's. strange to do something. Like if there. was a 1% chance Someone told me there. was a 1% chance that if I got in a car,

I might not I might not be alive, I. would not get in the car. If you told me. there was a 1% chance that if I drank. whatever liquid is in this cup right. now, I might die, I would not drink the. liquid. Even if there was. a billion dollars. if I survived. So, the 99% chance is I. get a billion dollars, the 1% is I die. I wouldn't drink it. I wouldn't take the. chance. It's worse than that. Not just. you die, everyone dies. Yeah. Yeah. Now, would we let you drink it at any odds? That's for us to decide. You don't get. to make that choice for us.

To get consent from human subjects, you need them to comprehend what they. are consenting to. If those systems are unexplainable, unpredictable, how can they consent? They don't know what they are consenting. to. Mhm. So, it's impossible to get consent by. definition. So, this experiment can never be run. ethically. By definition, they are doing unethical. experimentation on human subjects. Do. you think people should be protesting? There are people protesting. There is. Stop AI. There is Pause AI. They block.

offices of OpenAI. They do it weekly, monthly. There are quite a few actions. and they're recruiting new people. You. think more people should be protesting? Do you think that's an effective. solution? If you can get it to a large enough. scale to where majority of population is. participating, it would be impactful. I. don't know if they can scale from. current numbers to that, but I support. everyone trying everything peacefully. and legally. And for the for the person listening at. home, what should they what should they.

be doing? What what. cuz they they don't want to feel. powerless. None of us want to feel. powerless. So, it depends on what scale we are. asking about time scale. I was saying. like this year your kid goes to college, what major to pick? Should they go to. college at all? Yeah. Should you switch. jobs? Should you go into certain. industries? Those questions we can. answer. We can talk about immediate. future. What should you do in 5 years with. this being created? For an average. person, not much. Just like they can't. influence World War III nuclear.

holocaust, anything like that. It's not. something anyone's going to ask them. about. Today, if you want to be a part of this. movement, yeah, join Pause AI, join Stop. AI. Those are organizations currently. trying to build up momentum to. bring. democratic powers to influence those. individuals. So, in the meantime, not a huge amount. I was wondering if. there there are any interesting. strategies in the meantime. Like should. I be thinking differently about my.

family, about I mean, you've got kids, right? You've got three kids. That I. know about, yeah. How are you thinking about parenting in. this world that you see around the. corner? How are you thinking about what. to say to them, the advice to give them, what they should be learning? So, there. is general advice. I would say that there is the main that. you should live your everyday as if it's. your last. It's a good advice no matter what. If. you have 3 years left or 30 years left, you live your best life. So, try to not do things you hate for too.

long. Do interesting things. Do impactful. things. If you can do all that while helping. people, do that. Simulation theory. is a interesting sort of adjacent. subject here because as computers begin. to accelerate and get more intelligent. and we're able to. you know, do things with AI that we can. never have imagined in terms of like. imagine the worlds that we could create. with virtual reality. I think it was. Google that recently released.

what was it called? Um. like the AI worlds. You take a picture. and it generates a whole world. Yeah. Yeah. You can move through the world. I'll put it on the screen for people to. see, but Google have released this. technology which allows you, I think. with a simple prompt actually, to make a. three-dimensional world that you can. then navigate through. And in that world. it has memory. So, in the world if you. paint on a wall and turn away, you look. back, the wall. >> Yeah, it's persistent. And when I saw. that, I thought oh God, Jesus, bloody. hell, this is.

this is like the foothills of being able. to create a simulation that's. indistinguishable from everything I see. here. Right. That's why I think we are in one. That's. exactly the reason. AI is getting to the. level of creating human agents, human. level agents, and virtual reality is. getting to the level of being. indistinguishable from ours. So, you think this is a simulation? I'm. pretty sure we are in a simulation, yeah. For someone that isn't familiar with the. simulation arguments, what are what are. the first principles here that convince.

you that we are currently living in a. simulation? So, you need certain technologies to make it. happen. If you believe we can create. human level AI, Yeah. and you believe we. can create virtual reality as good as. this in terms of resolution, haptics, whatever properties it has, then I commit right now, the moment this. is affordable, I'm going to run billions. of simulations of this exact moment. making sure you are statistically in. one. Say that last part again. You're going.

to run you're going to run I'm going to. commit right now when it's very. affordable. It's like 10 bucks a month. to run it. I'm going to run a billion. simulations of this interview. Why? Because statistically that means you are. in one right now. The chance of you. being in the real one is one in a. billion. Okay. So, to make sure I'm clear on this, It's a. retroactive placement. Yeah. So, the. minute it's affordable, then. you can run billions of them.

and they would feel and appear to be. exactly like this interview right now. >> Right. So, assuming that AI has internal. states, experiences, qualia. Some people. argue that they don't. Some say they. already have it. That's a separate. philosophical question, but if we can. simulate this, I will. Some people might misunderstand. You're. not. you're not saying that you will. You're saying that someone will. I can. also do it. I don't mind. Okay. Of course, others will do it.

before I get there. If I'm getting it. for $10, somebody got it for $1,000. That's not the point. If you have. technology, we're definitely running a. lot of simulations for research, for. entertainment, games, all sorts of reasons. And the number of those greatly exceeds. the number of real worlds we're in. Look at all the video games kids are. playing. Every kid plays 10 different. games. You know, billion kids in the. world. So, there is 10 billion. simulations in one real world. Mhm.

Even more so when we think about. advanced AI super intelligent systems. Their thinking is not like ours. They. think in a lot more detail. They run. experiments. So, running a detailed. simulation of some problem at the level. of creating artificial humans and. simulating the whole planet would be. something they'll do routinely. So, there is a good chance this is not. me doing it for $10. It's a future. simulation thinking about something in. this world.

So, it could be the case that. a species of humans or a species of. intelligence in some form got to this. point where they could affordably run. simulations that are indistinguishable. from this. and they decided to do it. and this is it right now. And it would make sense that they would. run simulations as experiments or for. games or for entertainment. And also,

when we think about time in the world. that I'm in in this simulation that I. could be in right now, time feels long. relatively. You know, I have 24 hours in. a day, but on there in their world it. could be. Time is relative. Relative, yeah. It. could be a second. My whole life could. be a millisecond in there. >> Right. You can change speed of. simulations you're running for sure. So, your belief is that this is probably. a simulation. Most likely. And there is. a lot of agreement on that if you look. again returning to religions. Every.

religion basically describes what? A. super intelligent being. an engineer, a programmer creating a. fake world for testing purposes or for. whatever. But, if you took the. simulation hypothesis paper. you go to jungle, you talk to primitive. people, a local tribe and in their. language you tell them about it. Go back two generations later. They have. religion. That's basically what the. story is. Religion, you know, it describes a.

simulation theory basically. Somebody. creates. >> So, by default that was the first theory. we had and now with science more and. more people are going like I'm giving it. non-trivial probability. A few people as. high as I am, but a lot of people give. it some credence. What percentage are. you at in terms of believing that we are. currently living in a simulation? Very. close to certainty. And what does that mean for. the nature of your life? If you're close. to 100% certain that we are currently. living in a simulation.

does that change anything in your life? So, all the things you care about are. still the same. Pain still hurts. Love. still love, right? Like those things are. not different, so it doesn't matter. They're still important. That's what. matters. The. little 1% difference is that I care. about what's outside the simulation. I. want to learn about it. I write papers. about it. So, that's the only impact. And what do you think is outside of the. simulation? I don't know. But, we can. look at this world and derive some.

properties of the simulators. So, clearly brilliant engineer, brilliant scientist, brilliant artist. Not so good with morals and ethics. Room for improvement. In our view of what morals and ethics. should be. Well, we we know there is. suffering in the world. So, unless you. think it's ethical to torture children, then. I'm questioning your approach. But, in. terms of incentives to create a positive. incentive, you probably also need to. create negative incentives. Suffering.

seems to be one of the negatives and. incentives built into our design to stop. me doing things I shouldn't do. So, like. put my hand in a fire, it's going to. hurt. But, it's all about levels, levels. of suffering, right? So, unpleasant. stimuli, negative feedback doesn't have. to be at like negative infinity hell. levels. You don't want to burn alive and. feel it. You want to be like, "Oh, this. is uncomfortable. I'm going to stop.". It's interesting because we we assume. that they don't have great moral morals. and ethics, but we too would we take.

animals and cook them and eat them for a. dinner and. we also take conduct experiments on mice. and rats. >> But, to get university approval to. conduct an experiment you submit a. proposal and there is a panel of. ethicists who would say, "You can't. experiment on humans. You can't burn. babies. You can't eat animals alive.". All those things would be banned. In most parts of the world. Where they. have ethical boards. Yeah. Cuz some. places don't bother with it, so they. have easier approval process. It's funny when you talk about the.

simulation theory, there's a there's an. element of the conversation that makes. life feel less meaningful in a weird. way. Like. I know it doesn't matter but whenever I. have this conversation with people not. on the podcast about are we living in a. simulation. you almost see a little bit of meaning. come out of their life for a second and. then they forget and then they carry on. But, the the the thought that this is a. simulation almost. posits that it's not important.

or that I I think humans want to believe. that this is the highest level and we're. that the most important and we're the. it's all about us. We're quite. egotistical by design. And yeah, I just a interesting. observation I've always had when I have. these conversations with people that it. it seems to strip something out of their. life. Do you feel religious people feel. that way? They know there is another. world and the one that matters is not. this one. Do you feel they don't value. their lives the same? I guess in some religions. Think um they.

think that this world is being created. for them and that they are going to go. to this heaven or or hell and that still. puts them at the very center of it. But, it but if it's a simulation, you know, we could just be. some computer game that a 4-year-old. alien has is messing around with and. while he's got some time to burn. But, maybe there is, you know, a test. and there is a better simulation you go. to and a worse one. Maybe there are. different difficulty levels. Maybe you. want to play it on a harder setting next. time.

I've just invested millions into this. and become a co-owner of the company. It's a company called KetoneIQ and the. story is quite interesting. I started. talking about ketosis on this podcast. and the fact that I'm very low carb, very very low sugar and my body produces. ketones which have made me incredibly. focused, have improved my endurance, have improved my mood and have made me. more capable at doing what I do here. And because I was talking about it on. the podcast, a couple of weeks later. these showed up on my desk in my HQ in. London, these little shots. And oh my.

god. The impact this had on my ability. to articulate myself, on my focus, on my. workouts, on my mood, on stopping me. crashing throughout the day was so. profound that I reached out to the. founders of the company and now I'm a. co-owner of this business. I highly. highly recommend you look into this. I. highly recommend you look at the science. behind the product. If you want to try. it for yourself, visit ketone.com/steven. for 30% off your subscription order and. you'll also get a free gift with your. second shipment. That's. ketone.com/steven.

And I'm so honored that once again a. company I own can sponsor my podcast. I've built companies from scratch and. backed many more. And there's a blind. spot that I keep seeing in early stage. founders. They spend very little time. thinking about HR. And it's not because. they're reckless or they don't care, it's because they're obsessed with. building their companies and I can't. fault them for that. At that stage. you're thinking about the product, how. to attract new customers, how to grow. your team, really how to survive. And HR. slips down the list because it doesn't.

feel urgent. But, sooner or later it is. And when things get messy, tools like. our sponsor today JustWorks go from. being a nice-to-have to being a. necessity. Something goes sideways and. you find yourself having conversations. you did not see coming. This is when you. learn that HR really is the. infrastructure of your company and. without it things wobble. And JustWorks. stops you learning this the hard way. It. takes care of the stuff that would. otherwise drain your energy and your. time automating payroll, health. insurance benefits and it gives your. team human support at any hour. It grows.

with your small business from startup. through to growth even when you start. hiring team members abroad. So, if you. want HR support that's there through the. exciting times and the challenging. times, head to justworks.com now. That's. justworks.com. And do you think much about longevity? A. lot, yeah. It's probably the second most. important problem because if AI doesn't. get us, that will. What do you mean? You're going to die of. old age. Which is fine. That's not good. You want. to die? I mean You don't have to. It's just a.

disease. We can cure it. Nothing stops you from living forever. As long as universe exists unless we. escape the simulation. But, we wouldn't want a world where. everybody could live forever, right? That would be Sure we do. Why? Who do. you want to die? Well, I don't know. I. I mean I say this because it's all I've. ever known that people die, but wouldn't. the world become pretty overcrowded if. >> No, you stop reproducing if you live. forever. You have kids because you want. a replacement for you. If you live.

forever, you're like, "I'll have kids in. a million years. That's cool." I'll go. explore universe first. Plus, if you look at actual population. dynamics outside of like one continent, we're all shrinking. We're not growing. This is crazy. It's crazy that the more. rich people get, the less kids they they. have which aligns with what you're. saying and I do actually think I think. if I'm going to be completely honest. here, I think if I knew that I was going. to live to 1,000 years old, there's no. way I'd be having kids at 30. Right. It's a Biological clocks are based on.

terminal points. Whereas, if your. biological clock is infinite, you'd be. like one day. And you think that's close? Uh Being able to extend our lives? It's one breakthrough away. I think. somewhere in our genome we have this. rejuvenation loop and it's set to. basically give us at most 120. I think. we can reset it to something bigger. AI is probably going to accelerate that. That's one very important application. area, yes, absolutely.

So, maybe Brian Johnson's right when he. says don't die. Now, he keeps saying to me, he's like, don't die now. Don't die ever. Because. he's saying like don't die before we get. to the technology. >> Right. Longevity escape velocity. You. want to long live long enough to live. forever. If at some point we. every year of your existence add 2 years. to your existence through medical. breakthroughs, then you live forever. You just have to make it to that point. of longevity escape velocity. And he thinks that longevity escape.

velocity, especially in the world of AI, is pretty. is pretty. is decades away, minimum, which means. As soon as we fully understand human. genome, I think we'll make amazing. breakthroughs very quickly. Because we. know some people have genes for living. way longer. They have generations of. people who are centenarians. So, if we. can understand that and copy that or. copy it from some animals which will. live forever, we'll get there. Would you want to live. forever? >> Reverse reverse the question. Let's say. we lived forever and you ask me, do you.

want to die in 40 years? Why would I say. yes? I don't know. Maybe. >> You just used to the default. Yeah, I am. used to the default. And nobody wants to. die. Like, no matter how old you are, nobody goes, yeah, I want to die this. year. Everyone's like, ah, I want to. keep living. I wonder if life and everything would be. less special if I lived for. 10,000 years. I wonder if going to. Hawaii for the first time or, I don't. know, a relationship. All of these things. would be way less special to me if. they were less scarce.

And if that I just, you know, It could. be individually less special, but there. is so much more you can do. Right now, you can only make plans to do something. for a decade or two. You cannot have an. ambitious plan of working on this. project for 500 years. Imagine. possibilities open to you with infinite. time in an infinite universe. Gosh. Well, you can't. >> Because it's exhausting, I guess. It's a. big amount of time. I don't know about you, but I don't. remember like 99% of my life in detail.

I remember big highlights. So, even if I. enjoyed Hawaii 10 years ago, I'll enjoy. it again. Are you thinking about that really. practically as as in terms of, you know, if in the same way that Brian Johnson is. Brian Johnson is convinced that we're. like maybe two decades away from being. able to extend life. Are you thinking. about that practically? And are you. doing anything about it? >> Diet, nutrition. I try to think about. investment strategies which pay out in. the long years, yeah. Really? Yeah, of course. What do you. mean of course? Of course Why wouldn't. you if you think this is what's going to.

happen? You you should try that. So, if. we get AI right, now, what happens to. economy? We talked about Worldcoin. We. talked about free labor. What's money? Is it now Bitcoin? Do you. invest in that? Is there something else. which becomes the only resource we. cannot fake? So, those things are very. important research topics. So, you're. investing in Bitcoin, aren't you? Yeah. Because it's a It's the only scarce. resource. Nothing else has scarcity.

Everything else, if price goes up, will. make more. I can make as much gold as. you want given a proper price point. You cannot make more Bitcoin. Some people say Bitcoin is just this. thing on a computer that we all agreed. was valuable. >> Yeah, a thing on a computer. Remember? Okay, so, I mean, not investment advice, but investment advice. It's hilarious. how that's one of those things where. they tell you it's not, but you know it. is immediately. There is a your call is.

important to us. That means your call is. of zero importance. And investment is. like that. Yeah, yeah, when they say no. investment advice, it's definitely. investment advice. Um, but it's not. investment advice. Okay, so, you're. bullish on Bitcoin because it's. it can't be messed with. It is the only thing which we know how. much there is. in the universe. So, gold, there could. be an asteroid made out of pure gold. heading towards us, devaluing it. Well, also killing all of us, but.

Bitcoin, I know exactly the numbers. And. even the 21 million is an upper limit. How many are lost? Passwords forgotten. I don't know what Satoshi's doing with. his million. It's getting scarcer every day while. more and more people are trying to. accumulate it. Some people worry that it could be. hacked with a supercomputer. A quantum. computer can break that algorithm. There. is. strategies for switching to quantum. resistant cryptography for that. And. quantum computers are still kind of.

weak. Do you think there's any changes to my. life that I should make. following this conversation? Is there. anything that I should do differently. the minute I walk out of this door? I assume you already invest in Bitcoin. heavily. Yes, I'm an an investor in. Bitcoin. Is this financial advice? Uh, no, just you seem to be winning. Maybe it's your simulation. You're rich, handsome. You have famous people hang. out with you like that's pretty good. Keep it up.

Robin Hanson has a paper about how to. live in a simulation, what you should be. doing in it. And your goal is to do exactly that. You. want to be interesting. You want to hang. out with famous people so they don't. shut it down. So, you are part of a part. someone's actually watching on. pay-per-view or something like that. Well, I don't know if you want to be. watched on pay-per-view because then you. would be the same. >> Then they shut you down. If no one's. watching, why would they play it? I'm saying you don't you want to fly. under the radar? Don't you want to be. the the guy just living a normal life.

that the the masters Those are NPCs. Nobody wants to be an NPC. Are you religious? Not in any traditional sense, but I. believe in simulation hypothesis which. has a super intelligent being. So, But you don't believe in the like, you know, the religious books. So, different religions. This religion. will tell you don't work Saturday. This. one, don't work Sunday. Don't eat pigs. Don't eat carbs. They. just have local traditions on top of. that theory. That's all it is. They're. all the same religion. They all worship.

super intelligent being. They all think this world is not the. main one. And they argue about which animal not to. eat. Skip the local flavors. Concentrate on. what do all the religions have in. common? And that's the interesting part. They all think there is something. greater than humans. Very capable, all-knowing, all-powerful. Then they run. a computer game. For those characters in the game, I am. that. I can change the whole world. I can shut.

it down. I know everything in the world. It's funny. I was thinking earlier on. when we started talking about the. simulation theory that there's there. might be something in a in us that has. been left from the creator, almost like. a clue. Like a like an intuition. Cuz that's what we we tend to have. through history. Humans have this. intuition. Yeah. That all the things you. said are true. That there's this. somebody above and that We have. generations of people who were. religious, who believed God told them. and was there and gave them books. And.

that has been passed on for many. generations. This is probably one of the. earliest generations not to have. universal religious belief. What if those people are telling the. truth? What if there's people there's people. that say God came to them and said. something. Imagine that. Imagine if that. was part of the I'm looking at the news. today. Something happened an hour ago. and I'm getting different conflicting. results. I can't even get with cameras, with drones, with like guy on Twitter. there. I still don't know what happened. And you think 3,000 years ago we have.

accurate record of translations? No, of. course not. You know, these conversations you have. around AI safety. Do you think they make people feel good? I don't know if they feel good or bad, but people find it interesting. It's one. of those topics where I can't have a. conversation about different cures for. cancer with an average person, but. everyone has opinions about AI. Everyone. has opinions about simulation. It's. interesting that you don't have to be. highly educated or a genius to.

understand those concepts. Cuz I tend to think that it makes me. feel. not positive. And I understand that, but I've always. been of the opinion that. you shouldn't live in a world of. delusion where you're just seeking to be. positive have sort of. positive things said and avoid. uncomfortable conversations. Actually, progress often in my life comes from. like having uncomfortable conversations,

becoming aware about something, and then. at least being informed about how I can. do something about it. And so, I think that's why that's why I asked. the question cuz I think I assume most. people will should, if they're, you. know, if they're normal human beings, listen to these conversations and go, gosh, that's scary. And this is concerning. And and then I can come back to this. point which is like, well, what do I do. with that energy? Yeah, but I'm trying to point out this.

is not different than so many. conversations. We can talk about, oh, there is starvation in this region, genocide in this region. You're all. dying. Cancer is spreading. Atheism is. up. You can always find something to be. very depressed about and nothing you can. do about it. And we're very good at. concentrating on what we can change, what we are good at, and. basically. not trying to embrace the whole world as. a local environment. So, historically,

you grew up with a tribe. You had a. dozen people around you. If something. happened to one of them, it was very. rare. It was an accident. Now, if I go. on the internet, somebody gets killed. everywhere all the time. Somehow, thousands of people are reported to me. every day. I don't even have time to. notice. It's just too much. So, I have to put. filters in place. And I think this topic is what. people are very good at filtering as. like this was this entertaining. talk I went to, kind of like a show, and.

the moment I exit, it ends. So, usually. I would go give a keynote at a. conference and. I tell them basically you're going to. die, you have 2 years left, any. questions? And people be like, will I lose my job? How do I lubricate my sex robot? Like. all sorts of nonsense, clearly. understanding what I'm trying to say. there. And those are good questions, interesting questions, but not fully. embracing the result. They're still in.

their bubble of local versus global. And the people that disagree with you. the most as it relates to AI safety, what is it that they say? What are their counterarguments. typically? So, many don't engage at all. Like they. have no background knowledge in the. subject. They never read a single book, single paper, not just by me, by anyone. They may be even working in a field, so. they are doing some machine learning. work for some company maximizing ad.

clicks. And to them, those systems are very. narrow. And then they hear that all this AI is. going to take over the world like has no. hands. How would it do that? It's. nonsense. This guy is crazy, has a. beard, why would I listen to him, right? That's uh. Then they start reading a little bit. They go, oh okay, so maybe I can be. dangerous, yeah, I see that, but we. always solve problems in the past, we're. going to solve them again. I mean, at. some point we fixed the computer virus.

or something, so it's the same. And basically, the more exposure they. have, the less likely they are to keep. that position. I know many people who. went from. super. careless developer to safety researcher. I don't know anyone who went from I. worry about AI safety to like there is. nothing to worry about. What are your closing statements?

Uh let's make sure that it's not a. closing statement we need to give for. humanity. Let's make sure we stay in. charge, in control. Let's make sure we only build things. which are beneficial to us. Let's make sure people who are making. those decisions are remotely qualified. to do it. They are good, not just at science, engineering, and business, but also have. moral and ethical standards. And uh if you're doing something which. impacts other people, you should ask. their permission before you do that. If.

there was one button in front of you. and it would. shut down every AI company in the world. right now. permanently, with the inability for. anybody to start a new one, would you press the button? Are we. losing narrow AI or just. superintelligent AGI part? Losing all of. AI. That's a hard question because AI is an. extremely important, it controls stock. market, power plants, it controls. hospitals. It would be a devastating.

accident. Millions of people would lose. their lives. Okay, we can keep narrow. AI. Oh, yeah. That's what we want. We want narrow AI. to do all this for us, but not God we. don't control doing things to us. So, you would stop it, you would stop AGI. and superintelligence. >> We have AGI. What we have today is great. for almost everything. We can make. secretaries out of it. 99% of the. economic potential of current technology. has not been deployed. We make AI so.

quickly, it doesn't have time to. propagate through the industry, through. technology. Something like half of all. jobs are considered BS jobs. They don't. need to be done, jobs. So, those can be not even automated, they can just gone. But I'm saying we. can replace 60% of jobs today with. existing models. We've not done that. So, if the goal is. to grow economy, to develop, we can do. it for decades without having to create. superintelligence as soon as possible. Do you think globally, especially in the.

Western world, unemployment's only going. to go up from here? Do you think relatively this is the low. of unemployment? I mean, it fluctuates a lot with other. factors. There are wars, there is. economic cycles, but overall, the more. jobs you automate and the higher is the. intellectual necessity to start a job, the fewer people qualify. So, if we plotted it on a graph over the. next 20 years, you're assuming. unemployment's gradually going to go up. over that time. I think so. Fewer and. fewer people would be able to.

contribute. Already, we kind of. understand it because we created minimum. wage. We understood some people don't. contribute enough economic value to get. paid. anything really. So, we had to force. employers to pay them more than they're. worth. And we haven't updated it. It's what, 725 federally in US? If you keep up with the economy, it. should be like $25 an hour now. Which means all these people making less. are not contributing enough economic.

output to justify what they're getting. paid. We have a closing tradition on this. podcast where the last guest leaves a. question for the next guest not knowing. who they're leaving it for. And the question left for you is, what. are what are the most important. characteristics. for a friend, colleague, or mate? Those are very different types of. people. Mhm. But for all of them, loyalty is. number one.

And what does loyalty mean to you? Not betraying you, not screwing you, not cheating on you. Despite the temptation. Despite the world being as it is, situation, environment. Dr. Roman, thank you so much. Thank you. so much for doing what you do because. you're you're starting a conversation. and pushing forward a conversation and. doing research that is. incredibly important and you're doing it. in the face of a lot of um. a lot of skeptics.

I'd say there's a lot of people that. have a lot of incentives to discredit. what you're saying and what you do. because they have. their own incentives and they have. billions of dollars on the line and they. have their jobs on the line potentially. as well, so. it's really important that there are. people out there that are willing to. I guess stick their head above the. parapet and. come on shows like this and go on big. platforms and talk about. the unexplainable, unpredictable, uncontrollable future that we're heading. towards. So, thank you for doing that. This book,

which which I think everybody should. should check out if they want a. continuation of this conversation, I think it was published in 2024, gives a holistic view on many of the. things we've talked about today, um. preventing AI failures and much, much. more. And I'm going to link it below for. anybody that wants to read it. If people. want to learn more from you, if they. want to go further into your work, what's the best thing for them to do? Where do they go? They can follow me, follow me on Facebook, follow me on X, just don't follow me home. Very. important. >> do it. Okay, so I'll put your Twitter, your X. account um as well below so people can.

follow you there. And yeah, thank you so much for doing. what you did. Remarkably eye-opening and. it's given me so much food for thought. and it's actually convinced me more that. we are living in a simulation. But it's. also made me think quite differently of. religion, I have to say, because um you're right, all the. religions, when you get away from the. sort of the local traditions, they do. all point at the same thing. And actually, if they are all pointing. at the same thing, then maybe the. fundamental truths that exist across. them should be something I pay more. attention to. Things like loving thy. neighbor, things like the fact that we. are all one, that there's a a divine. creator, and maybe also they all seem to.

have consequence beyond this life. So, maybe I should be thinking more. about. how I behave in this life and and where. I might end up thereafter. Roman, thank you. Amen. Oh. oh oh oh oh.

Oh. oh oh oh oh.