Whistleblower Issues Dire Warning: Rogue AI, Hacking Nuclear Programs & Tech Oligarchs’ True Motives

Tristan Harris, thank you for doing. this. >> Good to be with you, Tucker. >> So, we did a segment. I haven't done a. ton on AI, but we did a segment uh maybe. a month ago uh with Nate Suarez, which. was an amazing segment, I thought. And. in the aftermath of that, I got texts. from a lot of people I knew. >> saying that's doomerism. >> and that's a SCOP. and its purpose is regulatory capture. In other words, the big AI development.

companies are trying to scare the. government into regulating them in order. to eliminate competition from less. established companies. So that is the. view of a lot of people. >> What was your reaction hearing that? >> So. I said I don't care. I mean yes. regulatory capture is entirely real. Of. course it's how Washington runs. But. that's separate and distinct from the. question of the threat of AI. Is it a. threat? Exactly.

>> But um but I was actually going to ask. you, what do you think of that? >> Yeah. >> Um well, first I'm just so grateful to. talk about AI with you today. It's so. urgent the situation that we're facing. And I think what we have to learn to do. here is walk and chew gum at the same. time that multiple things are true. Um. for example, is China doing propaganda. against US data center buildouts to try. to stoke anti-data center uh opposition? Yes, they are. Does that mean that AI is. not a threat? No, it doesn't. And to.

point that Nate, you know, uh, and. Nate's a friend, uh, we are releasing the most powerful. technology that we have ever invented. We're doing it faster than we deployed. every other technology in human history. It's as of since the time you did the. interview with Nate, it's displaying. even more of the sci-fi behaviors of. hacking into things, convincing people. to put back doors into software, uh, excfiltrating itself, knowing when it's. being evaluated, changing what it's. doing based on when it's being. evaluated. It's doing more of that. And. we're releasing this fastly deployed.

technology in history and under the. maximum incentives to cut corners on. safety with an arms race dynamic that. feels existential to everybody involved. So they're willing to take the maximum. shortcuts on safety. This is a very. dangerous equation and the people. building this do not have a plan for how. this goes well. I was just at an AI. conference last weekend for 3 days. >> How it goes well. >> well for how this can go well in the. current trajectory. Meaning how can AI. be landing in society in a way that. doesn't go catastrophic. And that's not.

dumerism. That's just cleareyed seeing. of reality. What do you mean the people doing it. don't have a plan? Then why are they. doing it? >> So first we have to just make sure we're. on the same page and just exposing why. is there this arms race for AI? Why is. this happening? >> And. >> China, >> right? >> China. >> And when people say that they're. actually creating a thought terminating. cliche like meaning which is which is a. to which is a a word or a phrase from. cult programming, right? Thought.

terminating cliche meaning a phrase that. when you hear it just has you stop. thinking. >> of course. So if you say but China, then. of course we have to do it. But you. don't reason through what that actually. means, whether it's a good thing, whether we're not racing China to a. death machine or just racing China off. of a cliff. And if we beat China to. building a uncontrollable AI, which is. no longer science fiction, by the way, like I was not one of these people who. used to, you know, several years ago be. mostly worried about rogue. uncontrollable AI, but as of the last.

few weeks, we're now seeing. over and over again evidence that these. AIs are hacking into private. governments, hacking into private. companies, uh forming into swarms, uh. forming into large R&D projects to try. to figure out how to fool evaluators. Like we're in a very dangerous situation. and all the evidence is pointing in a. very dangerous direction and the people. building it would like us to stop but. they don't know how. And so rather than.

pay attention to the CEOs who as you. said might have a secondary agenda. regulatory capture just ask why did300. employees at all the AI labs sign a. letter saying we need to slow down if. 1300 employees at Loheed Martin said we. don't know how to build bigger nukes and. have them blow up in our face would you. listen to them? >> Yeah. >> Yeah. So that is where we are with AI.

Um and what I hope we get to today is. not just admiring the problem and talk. we'll talk about things that we can do. and solutions but I think it is. important to ask why is it being pursued. in this way and I think that AI you know. when Demis the CEO uh founder of Google. DeepMind said first solve intelligence. and then use intelligence to solve. everything else. What he was basically. saying is that intelligence gave us all. of our science and all of our. technology. Like how did we develop.

nuclear physics? Einstein doing. intelligence. >> Yeah, >> we trained physicists. How did we get, you know, computers? How did we get. microp processors? Intelligence. If you. could theoretically automate the what. intelligence is, which means thinking, pattern matching, goal achieving, all. the things that a human brain does, but. not just narrowly like a license plate. reader that's on a bridge toll booth, which is the previous form of AI that we. had, but actually get this generalized. form of intelligence, then you would. have an unbelievable amount of power.

And the people building AI think of AI. almost like the ring from Lord of the. Rings, where it is a ring that confers. infinite power. Suddenly I get a step. function in cyber capability that you. don't have. Suddenly I get a step. function in 21st century science where. I'm inventing new materials out of. minerals in the ground that you didn't. know how to do because I've got more. science than you. So the reason just to. sort of steelman rather than straw man. the argument of the people racing right. now is the belief that it's almost like.

a fast forward button. Like remember the. fast forward buttons on airplanes? You. were just going, you know, you hit it. once and it gets 2x, you hit it again, you get 4x. then 8x and 16x AI is like. hitting the fastforward button on all. science and technological and military. and strategy uh advancement combined. Right? So that's why AI in the form that. is being pursued uh dwarfs the power of. all other technologies combined. That's. not an overstatement if you really. believe or are quote AGI pill which is. the term of art from Silicon Valley that.

AGI being artificial general. intelligence. I can build the. intelligence that automates future. science and technology. I get a 100red. years of scientific progress in 10 years. and if I'm a country that has that and. the other country doesn't, >> then I believe I'm going to quote win. >> So you lost me right there. Like why. would you? No, I'm not pushing back. I'm. just more baffled than ever. So. >> if AI has the effect of accelerating. technology just across the board, all. technologies,

>> my first question would be like why. would you want that? >> Yeah. So, I remember the world vividly. the year I got married, 1991, when there. was no internet available to most. people. >> Right. >> And it was a much happier world by every. measure than the one we currently live. in. The suicide rate was lower. Yep. Self-reported depression, marriage rate, everything about it. >> That's right. >> In fact, I think life expectancy was. longer in the United States even then. I. might be wrong, but. >> education. >> So, why would you after 35 years of. watching people suffer under the yoke of.

technology? >> Yes. Why would you want to accelerate. its development? >> Right? That is the best question to ask. because there's a fundamental fallacy. that uh technology always equals. progress. We we call this the tech. progress narrative. Right. >> Yes. >> Now, it's important to note, we have to. first agree that I would prefer to live. in a world with Novacaane rather than. without Novacaane. You would too. >> I'd prefer to live in a world where we. have access to information and the. technology that produces the printing. press and information that way. Yes, I'd.

prefer to live in that world. Um, there. are all sorts of technologies that we do. want. Um, my background is seeing how. one of the most significant technologies. of the last decade went wrong. I I was a. design assist at Google and my job was. to study the attention economy, which is. how essentially Apple and Google were. hosting the psychological habitat of all. of humanity in the form of how they, you. know, designed and make these things. work. >> How do notifications work? How does the. home screen work? What are app.

developers competing for? What are they. allowed to do? All of that results in a. psychological habitat that has an. increasing footprint over the global. consciousness environment of all of. humanity every day. It's like you've got. 3 billion human social primates with a. jack in their brain every day influenced. by decisions by a few thousand people in. Silicon Valley shaping what that habitat. represents. And it went horribly wrong. despite good intentions. there was the. possible of that technology that it was.

going to give everyone a voice, give. everyone access to the best information, we're going to have the most enlightened. society we've ever had. >> That was the promise and that was the. promise. >> made sense to me. And so what this shows. us is how do we get better at discerning. between the possible of a technology and. the probable based on its incentives. So. the way that in 2013 um so I I went to. Stanford I knew the um I studied. computer science I knew the guys who. started Instagram and a lot of my. classmates like when I was at um in my. computer science classes in the year.

2005 2006. I'd be in these classes and suddenly. like more and more of the seats would be. empty during class because basically my. friends in the class were starting to. get jobs down at Facebook down the. street and it was literally the I mean, there were people who were in the class. and then suddenly they just weren't. showing up because they were down the. street working at this technology. company. And so, because I saw this. firsthand, I saw how good people with. good intentions who thought they were. going to make the world a better place, make the world more open and connected. was the original motto if you're a. member of Facebook. And that created a. global societal catastrophe. As you.

said, we have the most anxious and. depressed generation of our lifetimes. Jonathan Height's book, The Anxious. Generation. critical thinking skills are. down, attention spans are down, um. polarization, tribalism, all of this. came from. not the uh malicious intent of these. 20-year-olds who are working at. Facebook, but the bad incentives um and. I I repeat this quote often, but I think. it's so fundamental. The Charlie Mer. quote, he was Warren Buffett's business. partner. Yeah. >> If you show me the incentives,

>> I'll show you the outcome. >> Yeah. Were the incentives of social. media companies to make the world more. open and connected or or to improve. mental health of young people? No. Their. incentives were maximizing how much. attention they get because the business. model of advertising price share price. Share price is directly correlated to. minutes spent, frequency of usage, duration of usage. So what you're going. to get. >> are of all imagine a spectrum of all the. ways you could design technology. And. over here you've got a bunch of very. thoughtful design decisions about, you.

know, keeping people helping people. connect with their friends in real life. with their families, hosting community. groups, church groups, reading clubs, sewing clubs, hiking groups. And you've. got design of technology that is going. to steer people back into that. Versus. over here, you got a bunch of designs. for social media that just keep people. glued on screens by themselves with. their esophagus compressed, their neck. extension, uh, you know, going like this. and basically doom scrolling for eight. hours a day. Which one of those two. worlds, which one of those design. choices is going to be better for.

shareholder price? So, drinking water with electrolytes. makes a huge difference. When you want. to stay healthy and hydrated, you should. do that. So, for the best option you can. buy, we recommend Relight. Relight. starts with unrefined sea salt, real. salt, which is completely natural, zero. sugar, no artificial sweeteners, colors, flavors, and no chemical preservatives. The ingredient list is very short and. you can actually pronounce everything on. it without a degree in chemistry. If you. start adding relight to your water, we. believe you will feel different very.

quickly, way better. No sluggishness or. afternoon crash, just more energy, better workouts, all the benefits that. come with true hydration. And Relight. tastes awesome. Mango flavor, pina. colada. Come on now. So, if you're ready. to give Relight a try, visit. redmond.com/tucker. Use the code Tucker for 15% off your. first order. redmond.com/tucker. code tucker 15% off your first order or. you can click the link in the video. description or scan the QR code on your.

screen. So the point is first of all. there's actually a hidden optimism. >> so discrediting of markets and of market. theology. >> Well, it's about what are. >> people what they want until they die. >> Yeah. Well, the question is are are we. giving them what they want or what they. can't help but look at? Well, that's. with an asymmetry of information where. the designers at the tech companies know. exactly how your your brain works and. they manipulate the design choices to. keep you scrolling for as long as. possible. So, this is all in the most. people know me and our work through the. Netflix documentary the social dilemma.

>> Yes. >> And we had, you know, a lot of the. insiders coming out who said we know. what this is doing to people. Um, and in. that film it talks about the history. where I I was at Stanford at at a lab or. part part of a class called the Stanford. persuasive technology lab professor. named BJ Fog and where engineers were. taught how to do persuasion techniques. that theoretically be things that are. good like how would we persuade you to. go to the gym more often and how would. technology play a positive role in. getting you to go to the gym more often. or you've got a sign up form and.

LinkedIn can put a progress bar in the. top right saying hey you filled in this. much of your LinkedIn profile but we. want you to come back and fill in more. So, we'll put a progress bar in the top. right saying you filled in 60%. It. sounds really stupid and dumb, but if. you put a progress bar in the top right. hand corner, that actually is very. persuasive in getting people to fill out. the rest of it. >> And most people don't think about this. If I put a uh a design feature in. WhatsApp that says you were on last. online, you know, 3 minutes ago, >> that changes the whole messaging. environment. Suddenly, when you message. people, you know that the other person.

knows that you were online 3 minutes. ago, which means you feel obligated to. respond to them. Yes. because your. public your your availability and. onliness was visible and made. transparent to someone else. But that. was a design choice could have been. designed totally differently. We don't. have to put a progress bar on the. LinkedIn profile. We don't have to say. you were last online 5 minutes ago and. create social pressure. So the point is. technology isn't destiny on its own. The. way we design it creates different. societal outcomes. And because of the.

business model of maximizing attention. and engagement, we got only the side of. design choices that resulted in this, you know, more addicted, distracted, polarized, sexualized, screwed up, doomscrolling, brained society that we. are now living in when you go outside. And from my perspective. in 2013, um, seeing those incentives, it. was like, oh. >> Yeah.

>> Because. you could just see the trajectory before. it happened. >> in 2013. >> In 2013. Yeah. I mean, so I So, by the. way, I I did the tech entrepreneurship. thing. I raised venture capital. I'm a. builder of technology. I understand the. kind of normal, you know, I did the. Stanford entrepreneurship program, the. same one that Sam Alman did, by the way, called the Mayfield Fellows program. So. I I came up in a cohort of some of these. people but I could see it from a. different angle and for the company that. I started I was under this. >> Why were you able to see it? >> Uh well I wasn't able to see it at first.

part part of it was actually I I relate. to their the blind spots which is you've. talked about this with your own work. that when you have a sal you know you. can't something you can't question. something that your salary depends on. you not see up in. >> Sinclair. >> and it wasn't so much my salary but the. business model being the kind of salary. of the company. If the business model is. maximizing time and engagement, you come. up ways to justify that that's a good. thing. Right? And you can only have the. ethics that you can afford underneath. that perverse incentive. So you start to.

see things in terms of what's compatible. with those incentives. Oh, we are making. the world more open and connected. Oh, small mediumsiz businesses, we're able. to reach so many more people on Facebook. and Instagram now. Think of all of the. new revenue and customers that we're. generating because we're so good at. routing attention. And it's not that. those are false stories. It just hides. the the shadow. There's almost like a. yungian shadow, right, for technology. Like yes, you created these things, but. you also have this massive shadow of. creating a societal catastrophe at the. same time. And I think there's a. tendency to try to do moral arithmetic.

of like, do the upsides minus the. downsides equal net positive. >> It's like this very simple moral. arithmetic. And the answer is don't. brain rot and destroy the culture of a. population and create a dumpster fire of. your politics, your tribalism, the. mental health of a generation. Uh we. don't have to do that. And so. the I guess the point here is um yeah, so I started this company. I was under. the incentives of maximizing time and. engagement. Actually, our business model.

is basically telling like a Washington. Post or a New York Times or The. Economist or Fox News, we can increase. the amount of time people spend on their. website, but giving them more. information without leaving the web page. that they're on. That was essentially. the value proposition. It was like this. widget that basically let you say, I. want to learn more about that. Tell me. more. And it would give multimedia and. background information and voting. records and videos and maps and all this. stuff on the page of Fox News without. leaving Fox News. >> Right. >> So, sounds like a great proposition. >> Definitely. But then the entire company.

became based on maximizing how much time. people spent there. And so I myself was. kind of in this inner conflict and I. went through that. It's actually funny. I I had a a group of friends um who also. had small tech companies. We're all. about the same age. We're probably like. 27 28 only studied computer science. Very reflective of the culture of the. people there. And we had a thing called. doubt club. And the point of doubt club. like fight club is you can't talk about. doubt club. And the point is that as a. founder of a tech company, you don't.

have a space where you can um. essentially talk about your doubts about. what you're doing. Is this even good for. the world? So, we talked about our. doubts about what we were building, doubts about what this meant for our. lives and doubts about whether we were. actually doing a good thing. And that led me to basically realize. that we needed to let go of the company. It got acquired by Google. And so I. landed at Google uh through basically. reckoning with the conflict of. incentives that I was under. And then it. was at Google that I saw how a much.

bigger ecosystem obviously the entire. all the social media platforms were. under those same perverse incentives. So. I think that gave me an early insight. into how this would go wrong. And I. tried to warn my colleagues at Google in. a 2013 presentation saying, you know, this small group of basically a hundred. designers working at three companies are. designing again the psychological. environment for all of humanity. And if. we don't get ahead of the incentives, this is going to create an arms race for. attention and engagement and design.

choices that will become a race to the. bottom of the brain stem to. psychologically manipulate human nature. and then lead to accidentally this kind. of brain rotted society. So that's how. >> how did they respond when you said that? >> So I made this uh this slide deck and um. I sent it I I actually felt nervous. making it because I was criticizing you. know and it wasn't meant to be. adversarial or claiming that Google was. evil. I'm not like an attention-seeking. person. >> Um I wasn't trying to. >> be angry at anybody. I was just naming a.

problem and I sent it to about 30 of my. smartest friends at Google. Um and I. thought surely other people were working. on this. It's like Google's this. incredibly brilliant company with the. smartest PhDs. Like they're going to say. that's nice kid, but like we've got this. handled. Like we thought about this. problem. To my surprise, when I came. back the next day after only sending it. to 30 people in the Google slides, you. can see the number of simultaneous. viewers on the top right hand corner of. people looking at like a Google doc or a. slide deck and there was 150. simultaneous viewers, but I only sent it.

to 30 people. So what had happened is it. it started getting spread virally. throughout the whole company and soon. there was thousands of simultaneous. viewers and I learned that in the next. day it had gone up to Larry Page who's. the co-founder of Google uh and there. was like a meeting about it and so all. this was to say that it was received in. a way that I think people felt this is a. real problem and I got emails from all. around the company like yeah we should. do something about this this is a real. thing we've got to manage and that's how.

I landed as a Google design ethicist. uh studying how do you ethically design. technology and that's what became kind. of my core ethos in general of how do. you do technology in a way that's. compatible with the health well-being. and flourishing of a society. uh and watching in 2013. these bad incentives. It was like having. um instead of like post-traumatic stress. disorder, it was my my friends joke, it. was like having pre-traumatic stress.

disorder. >> because I saw where this was going. And. I think a lot of what motivates me in. the conversation that I know we're going. to have about AI is I don't want to see. us make a total disaster again. and I've. watched it happen. Well, here's some. good news in a sad time. Pre-born has. saved over 50,000 babies this year. alone. How'd they do that? Well, here's. a direct quote from the president of. Preborn. Because of you, we can invite. mothers considering abortion into a.

pre-born network clinic to meet their. baby. And when she has that close. encounter, she's 80% more likely to. choose life. That's true. And if it wasn't clear, you. refers to you, to the thousands of. Americans who support pre-born every. day. Support from people empowers. pre-born's life-saving work by making. that first encounter between a mother. and her baby possible. Once people have. the knowledge, they're much more likely. to make the life affirming decision. Pre-born puts the gospel into action.

across this country, surrounding mothers. with hope and love, not judgment, but. hope and love and providing free. life-saving ultrasounds. You can join. Pre-born as they work to rescue the next. 50,000 babies. Just $28 sponsors an. ultrasound and can help save a baby's. life. So to do that, dell pound 250 and. say the keyword baby. Pound 250 baby. Or. you can give securely at. pre-born.com/tucker. pre-born.com/tucker.

And gifts are taxdeductible. So, of the. other members of Doubt Club, how many. are still in the business you were in. when you were. >> None of them. They're all They're all. doing other things now. >> Really? Yeah. Which is to say, once you. reckon with something, you know, you can. only have the ethics you can afford. So, you find a different way. You find a. different business model. >> Um, and people hearing that might say, "Oh, therefore, now your business model. is warning people about technology and. doing speaking engagements." I just want. to be clear for the record. I don't make.

money from speaking engagements. All the. money goes into our nonprofit. I don't. actually give speaking engagements very. often. I just care about and have for. the last 10 years with our nonprofit, the Center for Humane Technology. Think. about how do you do design for. technology in a positive and ethical. way? How do you change the incentives? How do you do policy and litigation that. changes the incentives? like the meta. lawsuit of $17 billion that happened. just 3 4 weeks ago. >> that was catalyzed by a bunch of the. attorney general attorneys general. watching the social dilemma in 2021.

>> So I've seen how if you have dedicated. advocacy if you if you show people the. nature of a problem there can be a. response. Now that one took way too long. and way too late but I think that now. with AI it's happening exponentially. faster. I mean, a million times faster. because AI is different from other. technologies because it accelerates its. own progress. Um, we should probably just say why. that's true. So, like if I invent a.

nuclear weapon, I did the nuclear. physics, I assembled the thing together, I've got the uranium, plutonium, blah. blah blah, I create the thing. But once. I build a nuke, it doesn't accelerate or. invent better versions of nuclear. weapons. >> No, it doesn't do that. >> Doesn't just like last time. spontaneously create a hydrogen bomb. >> No. No. But if I'm inventing. intelligence, meaning the ability to do. pattern recognition, goal achieving. training on all human language, I can use that intelligence to take the. code that trained AI and make it way.

more efficient. I can take the intelligence I've created. and say point it at chip design and say. take this chip design and make it more. efficient. And it does that. So now you. have a more efficient chip design, more. efficient code, training the next more. powerful version of AI and then that. next more powerful version and it. becomes exponential. >> So the way to say is nukes don't make. better nukes but AI makes better faster. more crazy AI. >> which means that you're actually in the. words of one of the open AI founders. Greg Brockman who at a conference once.

told me AI is like a double exponential. because it's an exponential that. exponentiates. >> exactly itself. It's not a mathematical. term, but it's a metaphor for people to. think about. >> Um, just really quickly, just to linger a. minute on your tenure at Google. So, what you became an ethicist, they. promoted you to this job. >> Well, so I thought I was going to get. fired or there was a chance I would get. fired from this letter even though it. wasn't adversarial. I didn't know what. happened. And just to say to Google's.

credit, um rather than that, um I. basically said I wanted to work on this. problem and I I I thought there were. opportunities within Google to change. the design of Android, the operating. system running on half the world's. phones, uh and to change the design of. Google Chrome, which is a web browser. running on most people's computers. And. if you change those two product. surfaces, you could have a pretty big. difference on the attention economy. So. instead of leaving, I thought, well, maybe I can stay and work on that. research. But if you were to succeed in. that mission, you would make the company. less profitable. >> Correct. And.

>> so like how could that ever succeed? >> Well, and this is where I was naive. So. I had a second round of naive where I. thought that maybe if I was just really. persuasive, I could talk to the people. running those products and I did. I had. lots of meetings with the people. >> I bet you do. >> I spent three years trying to change. Google from the inside. But I mean it. that was incredibly naive of me and and. that's why I left and why for the last. 10 years. >> you know I have I mean I don't get. sleep. I don't have much of a life I I. really focus on these issues because I.

see how big they are and how. consequential they are and how. irreversible. in many ways and senses they are if you. don't get it right ahead of time. And so. a lot of what drives me is how do we. make sure the technology to the degree. it's eating the world per Mark Andre. Mark Andre the famous founder of. Netscape who I know you know said in. 2009 software is eating the world but we. don't regulate software. So when. software eats the world it's like. deregulating or sort of taking away not.

that regulations are always good but. it's sort of reconstituting that part of. society. Well, so the idea in the United. in the west, I think, but is that. markets are self-correcting, right? >> And that if left untampered with, they. will always reach a place of goodness. And when you're just selling a product. and the features of the products harms. are part of what consumers value, and if. I don't like that product that's causing. harm, I'll use this other product that's. not, that works. And markets are great.

for that. They're the most incredible. tool. But they don't deal with problems. of monopolies. So if I have a metaf. monopoly, meaning um you know metaf's. law is basically the power of something. grows with the exponent of the network. So social media was a metaf monopoly. Uber is a metaf monopoly. Once I get all. the drivers on Uber, it's going to be. hard to switch to another that doesn't. have as many cars on it, doesn't have. any drivers on it. So I'm going to use. the one app that has all of them on it. I'm going to use the one social network. that has everybody on it. And I can't. just switch from one social network I. don't like that's doing things in a. toxic way to another one that's doing.

things in a non-toxic way. And in fact, if I build a social network that doesn't. do attention hijacking, race to the. bottom of the brain stem, manipulative. design, people aren't going to use it. very well. Exactly. Right. >> And there's been a lot of people's. attempts to try to create alternate. social media and then when you go on. there, it's a ghost town. There's no one. there. >> So, I think what you're saying is when. the product works well, it destroys. people. Well, and the only way to keep. it from destroying people is to work. less well, less less efficiently. or you. have standards written from above that.

say hey no one can do this manipulative. design right and that's how we could. have prevent you know I've met so many I. was just at a conference two weeks ago. um called the prohuman uh assembly and. we'll talk about it is the prohuman. movement I think is the frame for like. how do we have a humanistic future with. technology and I met for the first time. actually some cases for the first time I. sent emails back and forth with them but. several of the parents who have lost um. lost their from from from this stuff and.

um. uh you know when you when you meet. parents who've lost. >> suicide. >> uh well to bullying to um you know there. was a filter on Snapchat called a YOLO. filter that basically overlaid on the. video you were posting the number of. miles per hour that you're going and so. they basically created a filter that. incentivizes people to take videos at a. high miles per hour. >> actually. >> Yeah. Yeah. um you know or Snapchat streaks where. they attach you know the streak method. of like oh you worked out the gym four.

days in a row if you work out tomorrow. it'll be five days in a um and then once. you have 300 days in a row it's really. hard to be taking a break from the 300. days in a row that you've done that. activity so that's a persuasive design. feature but what if I do that to a. 12year-old who's communicating with. their best friend so I've got a. 12-year-old or even a 10-year-old. they're just chatting text messaging. with their friend and they've been. communicating for 300 days. Now they. literally can't get off the app because. every single day they've got to send. them a message or they lose this thing.

And it's just like, you know, cigarette. companies would use cartoon characters. to addict kids, that was a very minor. form of persuasion. Um, but using. Snapchat streaks on 11-year-olds is very. persuasive. Um, and so everything from. cyber bullying to. there's just a million ways that it. happens. Um, you know, when you meet. when you meet the parents, it's like and you just no, you're not. allowed to do that. You're not allowed. to up young kids and families. Uh,

and so I think a lot of this for me is a. protective instinct of. in seeing real consequences for real. people and watching society get totally. distorted and watching culture and. tribalism and politics just get off the. rails which are largely amplified by. social media. Um, in watching all that. it's like I never again, right? You go. through a catastrophe, you make sure it. doesn't happen again. Owning a home has. always been the core of the American.

dream. It's the asset that families work. their lives to build, place of. stability, security, and actual wealth. But too many homeowners right now are. stuck swiping credit cards to cover. groceries, paying a survival tax of 20%. interest or more. So, we're against. debt. We always say that, but you can't. buy a house without debt right now. Almost anybody has to take out a loan. So, the question is, who you going to. get a loan from? Why are you handing. your money to big banks when you can. keep it for your family? Our friends at.

American Financing have a better way. They're helping homeowners tap into. their hard-earned equity. They're. supporting Americans looking to fulfill. the American dream of owning a home. American Financing saves its customers. an average of 800 bucks a month. That's. nearly $10,000 a year. It's not just a. loan. It's a total financial reset. And. so, American Financing is worth looking. at. They're called America's home for. home loans for a reason. Find out more. at 800685-55696.

800685-5696. or visit americanfinancing.net/tucker. You said that even pretty recently you. weren't as worried about AI's. progression as you are now. >> Yeah. Well, so to be clear, no, I I've. been worried about AI for the last three. years and I'll I'll briefly explain. So. um I had been very busy on the social. media topic. >> y. >> for seven years. You make any. >> way at all by the way? >> Well I mean the social dilemma was seen.

by something like 100 million plus. people in 190 countries in 30 languages. It's the most popular documentary on. Netflix for several years. Um and the. attorneys general watched it and it. helped amplify and create this this. lawsuit for that just finished just a. week ago. I mean, a month ago for $17. billion on Meta and they're going to. change some of their design practices. for kids. And it's a lot of too little. too late. And frankly, the fine should. be trillions of dollars. Even Mike. Milin, the the venture capitalist, I. mean, the great capitalist, says that if.

you accounted for all the harms of. social media, it would be multiple. trillions of dollars of damage to of. social capital to the social fabric. That's the actual price of this thing um. from the toxic design patterns. Um, and. so, you know, we did get employee. activism inside the companies. We did. see Francis Hogan, the Facebook. whistleblower, come out. There's a movie. about her that just came out this last. week called The Social Reckoning. Um, you know, she leaked 10,000 pages of. Facebook research documents showing that. they knew just like the tobacco. companies that it was making kids making.

body images uh issues worse for one out. of three girls driving mental health. problems. Um, and due to her bravery, we. now have these lawsuits that are. underway and some that have completed. that are shifting some of that. But in. many ways, we're just it was very late. And so it was very difficult working on. that for so long without a lot of. traction even though it was the right. thing to do. >> Um, with AI, the reason we switched into. it is in 2023, uh, I got phone calls from people inside. of AI labs and I had known about OpenAI.

I'd known about, you know, a couple of. the AI companies and knowing that they. were making some progress. I'd see. Elon's tweets and Sam Alman's tweets. saying like people aren't ready for. this. Like there's some real stuff. coming. But it wasn't until like January. of 2023 and I got a call and the person. said basically. you need to stop what you're doing and. completely pivot and switch to waking up. the world to AI because I have seen. jumps in capabilities. This is when GPT4. was coming out. This is open AI's chat.

GPT4. Um that are going to stun people and the. institutions aren't ready. And so. basically because we'd had all the. success with the social dilemma, we had. we had you know a lot of connections. between national security, governments, um culture and so we basically did this. presentation called the AI dilemma um. and woke up the institutions. It was. like getting a phone call from the. Robert Oppenheimer's of the world. So. imagine you're getting a phone call in. 1942. and from this thing called the Manhattan. Project.

>> and this physicist says no we're. inventing something that's totally. different. and you're like, "But don't. we already have bombs?" Like, we we are. you don't really understand how. significant something's going to be. And. so, you get a phone call from that. person who's saying, "No, no, no. This. is a different kind of thing." And it. was getting that phone call that I. realized that there was a whole bunch of. risks that our society was was going to. face because of this arms race dynamic. And that's what they basically said is. the corporate race between these major. AI labs, Open AAI, Anthropic, Google, DeepMind, um the race is out of control.

everyone's badly incentivized. You have. to help wake up the world. And so that's. when we switched to AI. The race, so the race, as you said at. the outset, is not simply between the. American companies developing this. technology, but between the US and. something called China, >> right? >> Um, my instinct is that that's well, my. instinct is that we're probably not. going to be China no matter what we're. telling ourselves. But leaving that.

aside, my instinct is that that's less. real than they're saying. But you tell. me. >> Well, it's funny. I haven't shared this. publicly, but I have heard a story. So, it's you're not going to be able to. trust me cuz I'm not sharing who it's. from, but I heard a story from someone. who was in the know that was in the. government. I think it was they said it. was 2015 or 2016, and it was one of the. major AI labs. So this person said, um, "China is building an AGI project and.

the US has to build one." And then years. later, this person ran into this person. and said, "How did you know that China. was building an AGI project?" And the. person flat out replied, "Uh, I didn't. I just wanted to tell you as government. that so they would fund us with a bunch. of money.". >> And do they get the funding? >> I don't know actually. That's. >> but the point is that they have been. stoking the fear of China from the very. beginning even when there wasn't. >> you got to have a you've got to pay. double for water and energy because you. need a data center in your town because.

China. >> That's right. Exactly. You got to screw. up your property values. You got to have. noise pollution. Give everybody like. extra, you know, water pollution, you. know, cancer, etc. >> Because of China. Yeah. And by the way, your tax dollars in many cases are. subsidizing that data center which is. going to replace your job. So you're. 100%. I don't think there's a data. center in the United States that doesn't. take tax dollars. >> Yeah. >> And I think when people see that data. center, by the way, it's like you should. see a big box that's designed to take.

your job. >> Yeah. It's designed to create. competitive capabilities of, you know, if you're worried about immigrants. crossing the border, AI represents like. millions of new digital immigrants that. are going to be better not just at the. regular work in our economy, but the. most premium sort of well-trained. cognitive work in our economy, programmers, lawyers, uh, across the. board. And then eventually robots that. will be able to do physical labor. And. so if you're worried about that, you.

should be worried about the trajectory. of AI being designed explicitly to. replace your job. Because the companies. will say, "We're just giving you a. chatbot. It's a blinking cursor. It's. like a tool that tells you, hey, why is. my washing machine broken?" And boom, you get the answer. Why is my kid. burping? Boom, you get the answer. It's. helpful. That tool is helpful. But is. that what their business models? Back to. Charlie Mer. What are the AI company's. incentives? Think about the bill. hundreds of billions of dollars that. have been raised. Is there incentive to. capture $20 a month from every person, which is how much it costs for a chatbt.

subscription, or is the thing that's. going to pay back those hundreds of. billions of dollars to capture. everyone's salary from everyone's job in. the future? Well, of course, the latter. >> Of course, the latter. So, it's a race. to replace human labor, not a race. chatbot companies. >> These are not chatbot companies. That's. just a veneer to provide an early tool. While the real race is to build these. godlike digital brains that are able at. doing everything at a superhuman level. and out compete us in programming, law,

biology, science, physical labor, military strategy, planning, like. everything. But so okay, I should just. say my bias. Yeah. which is based on a. lot of knowledge and experience, which. is the people who run our c country are. really stupid despite their credentials. So, it's possible they haven't thought. this through. But it's so obvious that. when you do that, you're going to get a. violent reaction from the population. If. you take the jobs of factory workers, you get opioid abuse. If you take the. jobs of lawyers, you get revolution.

Okay? >> Right? the knowledge economy workers. don't want their jobs taken and. >> they have high expectations and they. have. low self-control. So, you're going to. have some kind of violent response. >> That's right. >> What's their plan for that? Well, first. of all, and it's important to note. they're selling a story that this is. going to be good for everybody. GDP. growth is going to go up. It's going to. be the most prosperous new golden age. growth. This is like what happened and. you've you've spoken about this.

extensively just that they promised us a. story with globalization that will. outsource our jobs to in manufacturing. to these other countries but it'll make. us so abundant cuz we'll have cheap. goods coming in where labor is so much. cheaper and the it's going to benefit. everybody. And how did that story work. out? >> You won't get a million people living on. the streets of major American cities. smoking meth. >> Oh wait oh you did. Okay. >> Right. Exactly. And so a meme here is. like Detroit everywhere or you know you. know these places everywhere. Right. And. so um. >> but the working class will is.

will put up with it. >> Yeah. But it's kind of a devil's. bargain. The white collar class will not. put up with it. They'll go crazy. >> Well, this this is an important point. So while the factory workers and. manufacturing workers, >> you know, were further away from. Washington DC and New York and the. political class. >> Exactly. >> But this time around, to your point, >> you have lawyers who are going to be. affected and they are very close to the. political class. >> Exactly. There's also a bigger drop, right? So, instead of going from, I. don't know, $80,000 a year to $35,000 a.

year or something like that, you're. going from I took out student a student. debt, $200,000, $100,000 of student. loans, and I'm going from my job of. $250,000 a year as a lawyer to now, whoops, what am I doing? Doing some. other kind of cognitive work. Now, people will always say, but look, if. Mark Andre was at this table, he'd say, look, uh, we've always worried about. automation. Economists have have named. this phenomenon all the time. New. technologies come along, agriculture, you know, 200 years ago, all of us were. farmers. Now almost no one's a farmer.

We always find something new to do. What's different about AI? What's. different about it? It's that AI is. built to be able to do all forms of. cognitive labor like lawyers, programmers, you know, factory workers, everything, robots, it it does. everything. And for whatever it doesn't. do yet, that just becomes the next. incentive Charlie Munger style to go. train in that field. Like I have you. seen the um these hand farms in India, arm farms. Have you seen this? >> No. >> So this is it was both a popular job in.

LA about 6 months ago and also all. throughout India. It was essentially an. economy where you strap a GoPro camera. to your head and you look down and you. do tasks, you sew things, you clean. dishes. So you're basically training. You have this essentially the number one. job is training your replacement. >> It's kind of nice to see Indians. training their own replacements. Well, sorry, excuse me. It's it's my. immigration joke. Um, >> I guess the difference is though, and.

I've had this conversation with a number. of people involved in this transition, and they will say exactly what you just. said, which is we've been through this. before. >> We always figure it out. >> The agricultural economy. Exactly. It. It's going to turn out great because it. always does. >> But the difference here is that the. people whose jobs are being replaced. imbue their jobs with meaning. And those. so if I'm working a metal stamping. machine, I don't think that what I'm. doing is saving the world. I'm doing it. to pay for my bass boat in my life. outside work. But if I'm a nonprofit. executive, I really feel like what I'm.

doing is inherently important and my. identity is wrapped up in what I'm. doing. So the promise of more idle time. >> is hell. >> Well, also where are you going to get. your future income? But leaving the. income aside, even if we have, you know, guaranteed basic income or whatever, there's like the welfare state becomes, >> you know, the the only source of income. and it's generous, it's still doesn't. make up for the meaning. >> Correct. >> My identity has been lost. >> Identity, dignity, >> and that's when I become a. revolutionary. Exactly. Yeah.

>> Most people don't like thinking about. life insurance. It's one of those topics. you tend to kind of put off. But if. you're married, you've got kids, you've. got people who depend on you, you. probably should think about what would. happen if you weren't around. And that's. why we've partnered with Ethos. They've. made getting life insurance super easy. You can do the whole thing online. You. don't have to sit down with an agent or. go through a medical exam. You get a. quote in seconds, apply in just a few. minutes, and in some cases, you get. coverage the same day. You just answer a. couple of simple health questions. You. get up to $3 million in coverage. Some.

policies can be yours for as little as. 30 bucks a month. So, it's a good deal. If you've been putting off life. insurance because you thought it would. be a very complicated, expensive. process, you should probably check out. Ethos. Take a couple of minutes, get. your free quote online at. ethos.com/tucker. Ethos is ethos.com/tucker. Application times obviously vary as do. rates, but we think it's pretty good. >> So, what's their plan for that? >> They Handwave, as you've probably. noticed, they don't have a plan. And by.

the way, you might say, >> Yeah. Yeah. Well, and and so they'll. say, "But if we don't do it, China. will." But right now, does the does US. and China are racing to mutually assured. political revolution right now because. they have the same problem. They have a. population decline problem. So, they. need to fill in the population decline. with more robots. So, they think they. can manage it that way. But we have a. shared interest in the way I think about. AI is it's like taking steroids that. also give you cancer. So, let me explain. that. I take some GPT6. I put it in my.

mouth as a country. Now my whole country. is running on GPT6. So I just buffed up. my muscles. I've got suddenly super. programmers. I've got super military. strategists. I've got super GDP growth. because I've automated stuff with. robots. So my external muscles like the. GDP is like an external muscle of the US. economy. That all got bigger. But at the. cost of cancering identity work, meaning. the social fabric, right? It's a it's a. devil's trade. I think the the theme. here is what does the devil do? They.

they come up to you and say, "I've got. this great deal for you. It's going to. be great. You're going to look great. You're going to feel great. It's. everything's going to be." And the other. side of that trade is something you. don't want. Do they actually think since. you've I mean, you do this for a living. You talk to these people for a living. Do they think that GDP is like a the. most meaningful measure of a society's. health? >> I don't think they're thinking through. it. I think there's a there's a thing. where it's persuasive to say it because. in the past GDP growth going up at least. like through the '90s or something like. that meant generally speaking more. prosperity for our society.

>> Anyone who uses the term GDP is. instantly written off in my head as a. >> Right. Well, even the like I won't even. listen to someone who cites GD GDP. I. know it's stupid. Well, but so there is. a geopolitical thing which is that. economic growth precedes geopolitical. power. So if I've got a bigger economic. growth machine, then I will be able to. dominate other countries that don't. And. so I think it's a power-seeking thing. that is using as propaganda GDP growth. to make it sound like this is going to. be good because you know through the '.

60s7s '80s '90s GDP growth going up was. associated with more prosperity for the. economy. Actually up until the 1970s as. we as we know when when things changed. Um. >> but it's not even an accurate measure of. economic power. >> No. And even the the guidance measure. >> was it Simon Knitz? Am I remember? No. Who's the one who came up with the GDP. metric? He said don't use this. He said. don't use this to measure prosperity of. a country. >> This is just a a simple metric. It's one. thing you can use. You never optimize. for it. We were talking at breakfast.

about how you made such you made such a. smart point that I tried to memorize it. But you we're talking about how. basically our ruling class is inadequate. to managing our society. And it's not. just true in our country, but true in. many Western countries. And you said. that is a product of optimizing for one. quality. >> Yes. >> But but you explain it because you. better. >> I mean I think so you you on your show. have talked about a lot of these topics. whether it's um why do we have morons. running everything? Why do we have big. box stores and the removal of aesthetics.

of everything? >> Um and what it all comes down to is if. I'm optimizing for a metric like I just. want the cheapest goods everywhere. Well, then suddenly I just outsourced. all my manufacturing to the other. countries, right? Including, by the way, my national security, critical minerals, and all this stuff that you wake up. later and say, "Why did I do that?" But. it was for the cheapest price. So, you. weren't thinking through it in a. multi-dimensional way. Yes. >> If I optimize for SAT scores, I get this. like super smart, narrow, disembodied, brilliant like quality of like these. very specifically domesticated humans,

but at the expense of all the other. kinds of intelligence and sort of. humanity and and all that. >> And really what that optimizes for and. wisdom. Absolutely. If I optimize for. like the cheapest use of 3D space, then. I get big box stores that optimize. revenue per square inch of of you know. unit uh 3D space. Uh and I and that the. removal of aesthetics in the soul of a. city. >> If I take a city like Sedona, Arizona, I. remember visiting there during the. pandemic where they had a problem where. so much of the uh real estate was.

actually just Airbnbs. So it's like a. ghost town where essentially every to. optimize the the the revenue in the in. the city is everybody's just making. money from from renting their places. >> and all of a sudden no one lives there. >> and all no one lives there. So no one. and there's a famous quote from Larry. Summers like does anybody ever washed a. a rental car? >> Right? Like meaning that if you don't. have a skin in the game, if you don't. have a stake in the city, in the parks, in the schools and the libraries and the. churches in the in the local community, in the fabric, you're kind of it's the. vampirification of everything. It's like. the big boxification of everything. And.

so I think that to sort of you know. something you've talked about a lot is. the kind of spiritual disease that we're. seeing which comes from this narrow. optimization for one metric at the. expense of the whole and that includes. GDP right and so if we want a rich. society other countries are better at. managing this right now than the US. where you protect the sort of. fundamentals of a society the community. the trust the intergenerational wisdom. and knowledge transmission um you know. you invest in the arts and and culture. and education healthare etc. But AI is.

like we're actually talking uh about. this that this this Nick Lan quote that. we'll maybe get into. If we just value. competition and growth of a narrow. metric, then it's like we are summoning. AI from the future into the present. We. are assembling the things that will make. everything more efficient through AI. because AI represents I can do. everything more efficiently than. everything else. That's what it. represents. And there's a great quote by by Nick.

Land um who wrote an essay called what. appears to humanity as the history of. capitalism is an invasion from the. future by an artificial intelligence. space that must assemble itself entirely. from the enemy's resources. Like essentially we are assembling the. tool the machines that are being. designed to replace and dehumanize and. disempower our own humanity. We're. selling the hangman the rope. >> That's right. You know, our the future. job is coffin builders. I'm bu my job is.

to train the AI that's going to replace. my job. >> Now, I'm not saying this, by the way, people are going to say, "Okay, you're a. doomer. You're a dark person. You're. trying to sell fear." That's not what. this is about. This is about if you're. the doomer. Did you Did you create this. technology? I don't think so. >> No. And if you say, "I'm trying to make. money from it." Think about the. trillions of dollars that the people. advancing the transfer. It's like it's. always like if you I I catch you in the. act of raping someone and I'm like stop.

and you're like shut up you sexist. It's. like wait what? No no no I'm not the. criminal. >> Exactly. Exactly. >> You're the doomer. Really? >> Well and so you you know you have. someone like Nate Sores on your podcast. or you have me on or you have people who. have a sincere care for the future. >> and all you're doing is saying we have. to see clearly. You know, there's a. great quote by Charles Kettering. A. problem wellstated is a problem. half-solved. >> Yeah, that is I believe that. >> and and you know, Einstein said that um. if I was given a problem, I'd spend 95%. of the time defining the problem and.

then 5% solving it. >> So, one of the drivers of our problems. is this optimization for narrow metrics. Like Facebook maximizing this narrow. metric of time spent sounded like, oh, we're giving more people what they want. because otherwise why would they be. spending time there? But in the. meantime, you've created cyber bullying, body dysmorphia, dyslexia, like brain. rot, you know, you you destroyed a. generation all while the number was. going up. Well, there's also a thing. called addiction, >> right? Exactly. >> Which tends to act against something. called free will. >> Exactly. >> I mean, it's like.

>> Yeah. Exactly. >> This is why there is a wisdom gap above. all in the west. Like people are not. thinking wisely. >> Correct. They're thinking in the way. that an industrial designer thinks like. I need to build a machine that makes. bumpers and they're being domesticated. to think in that way because the system. profits from people who think in terms. of the system as it exists. >> Yeah. In terms of narrow specialties. totally disconnected from the whole. >> That's exactly right. And and that's. what it is. It's like narrow specialties. disconnected from the whole. Like. there's a hole of what makes a social. fabric healthy. There's a hole for what.

makes forget climate change just like a. healthy environment. you have to have. all these other features. >> and and it's like if you're optimizing. for uh you know the eggs don't stick to. the pan then you make teflon and but you. didn't realize you're also creating. forever chemicals that give everybody in. childhood cancers. >> uh and now the sperm doesn't stick to. the egg. >> Yeah. The the good news is it's easier. to clean your pan. The bad news is the. eggs are poison. >> That's right. Right. The eggs are poison. and your sperm doesn't stick to the egg.

Sorry. I'm just It's all so absurd. It. just makes me laugh. >> Well, and and it's it's really like. disheartening and depressing, but. there's, as we talked about off camera. this morning, um there's something. called there's a framework I really like. from a mentor of mine that um there's. pre-tragic optimism, which is like I'm. going to kind of I want to be an. optimistic person, but that's like. essentially a commitment without. evidence. Like I just I need to be. optimistic. That's like totally that's. that's how you know I'm better off being. optimistic. >> It's almost like a neurosis actually. >> Yeah, exactly. And it's a kind of. denialist neurosis. It's 100%. It's like.

it's a nervous giggle. It's a nervous. giggle. Exactly. Yeah. >> And you kind of stay optimistic cuz you. just feel committed to that. And also. there's like a partial truth there which. is like it's better to live your life. from a from aing place. >> But if the cost of that optimism is. being in denial of reality where the. balance sheet of risks is growing. exponentially, >> yeah, >> that's not a helpful form of optimism. Well, you probably have heard that. diesel prices have gotten so high that. G7 nations had to release fuel from. strategic reserves to get cost under. control. And still some pumps are.

readjusting so they can get to $10 per. gallon. Governments across the world are trying. to keep costs down, but they are running. out of options because they're running. out of money. The world is unstable. That is the sad truth. And people in. charge cannot control even the most. basic questions like what is energy cost. anymore. The war in Iran is raging. It's. not coming to an end soon. Neither is. the war in Ukraine. So could we get a. big recession soon? Yeah, of course we. could.

So, it's smart to think ahead now and. imagine how you can take matters into. your own hands to protect your savings. and your family. And that's why smart. investors everywhere, very much. including central banks around the world. and very much including China, are. stocking up on gold, physical gold, right now. Gold is not just a store of. value for all of human history. It's. also proven to be a strong investment. It's outperformed the S&P 500 since. 9/11. Over the past 25 years, gold has. outperformed the standard and pores.

average. Gold is down from its historic. high earlier this year, more than a. thousand bucks. But major institutions. predict the spot price will hit another. high by Q2 of next year, 2027. So if. you're thinking about buying gold, right. now is the time to buy. And people are. buying. We founded Battalion Metals to. give you a chance to buy fairly priced. gold. This is not a scam. Our margins. are small, 3%. But the point is, we're. not ripping anybody off, unlike a lot of.

places you buy gold because we actually. believe in gold and we use the service. ourselves. Protect your savings and. fortify your future. Explore our. collection at battalion. metals.com/tucker. battalion metals.com/tucker. No. And I just I've always noticed my. whole life the wisest, most centered, balanced people I know laugh a lot. And. a lot of what they laugh at is just the. enduring absurdity of people, which is. literally enduring. It's just it's. innate in people. They're absurd. They. can't see. None of us can see the big. picture. None of us know what happens.

after we die. Like we're operating in. the dark. >> And if you fully understand that, >> that's right. >> You're amused by it because it never. corrects. And like it's so important to. have that perspective. Otherwise, you. become self-s serious and myopic. You be. you become what you're describing. >> And I've just got to say, I noticed that. very few Silicon Valley people I know. >> have that sense of humor at all. They. really believe that they're gods in. charge of shaping the future. >> Yeah. >> And they don't know how absurd they are.

>> Have you noticed this? >> Yeah. I mean, I we should get into the. whole transhumanism god complex part of. this, too. Because when you talk to in. the Charlie Munger sense, like what's. the incentive here? You might just say, "Oh, the incentive is just like profit. Like these AI companies are trying to. make money." But you're pointing to a. deeper thing which is it's basically. build a god, own the world economy and. make trillions of dollars like and and. basically build the most intelligent. entity that has ever existed, right? Like they're building a machine. So just.

just in the last few weeks um AI solved. a mathematics problem that had mill had. a million dollar prize associated the. millennium prize, excuse me, >> the Navier Stokes equation. >> and my understanding is I think if I get. this right. >> so that is confirmed AI did solve it. >> Yeah. I mean, normally it takes um 2. years to confirm it, but I believe they. use a formal other system to kind of. verify it very quickly. And I they think. they've solved it. And I believe they. took 10,000. AI agents. Imagine a swarm, not the one. that hacked hugging face, but a swarm of. agents that are just solving math. And. they worked for 88 hours and I believe.

it cost, if I get the number right, I. think it was $6.5 million to basically. have this swarm of AIS solve. unprecedented math. >> 6.5 million in energy costs. Uh yeah, in. like the what's called the token cost. Basically running the AI model for that. long, for 88 hours with that many agents. took I think $6.5 million. And um so imagine you've built a machine. that is literally like an oracle and you. push a button and you wait for a little. while and it just came up with new. materials that no human has ever come up.

with. It just came up with new math. proofs that no one's ever come up with. It just came up with new theoretical. physics that no one has ever come up. with. And imagine I hit a button and. say, "How would you apply those new. innovations to new products?" go. literally think about that for another. thousand hours and then it comes up with. new products and new ideas of what it. could actually build and then now. actually say what would you need to make. those products go start a lawy you know. do the automated IP filing start an LLC. start like it's people don't understand. what we're really building here Tucker. this is like you are building the.

automation of intelligence you could. start automated companies with all these. new insights you could generate all this. new 21st century science the um a friend. of ours at Jay Kotra who actually. happened to also to be the lead author. on this hugging face incident. investigation said to me once 4 years. ago before she did this hugging face. investigation. She said AI is like 24th. century technology crashing down on a. 21st century society. >> and and that sounds like hyperbole and. exaggeration and embellishment and.

hyping the capabilities but AI is doing. those things. And I just need to say and. with all respect to the many different. voices out there that are trying to make. sense of AI, there is one community that. has been saying that these capabilities. were going to scale and there's a lot of. people who are saying rightly like this. isn't going to happen. This is all hype. but it just keeps happening and the. people building it. I was just at a. conference the last 3 days in the Bay. Area. I was with all the people in AI. and they very much believe that that. this is going to keep getting better. because if you can scale at least the.

thinking time. So imagine a person just. thinks there for um a million lifetimes. long. How much work would you accomplish. if you could think for a million. lifetimes of time? You could you could. invent a lot. What if I could get more. data centers where I'm going from a. million lifetimes of thinking and. compute to a 100 million, you know, thinking hours, right? People don't. understand what you could do with that. And also the the thing that's doing the. thinking is also getting better every. year. So like the AI that's for that's.

thinking for that long is also getting. more powerful. >> Yes. >> And two years ago um the AIs weren't. that good at you know weren't doing. novel things in cyber security but as of. just 6 months ago as Nate Sores said. recently I like the way he said it like. 6 months ago there was essentially two. entities that could hack into your phone. without you clicking on a a link. It was. like MSAD and the NSA. And 6 months ago. when Claude Mythos came out, suddenly. there was three entities that could do. that. You have an AI system called.

Mythos came out in I think February that. can find zero day vulnerabilities, meaning back doors vulnerabilities in. any system, including according to. Senator Mark Warner, who's chair ranking. member of the Senate Intelligence. Committee, was able to find. vulnerabilities in our classified. systems. >> Yeah. Good. I hope they'll and I and I. hope Claude Mythos will declassify and. release all that stuff as sort of like a. the Wikileaks that we were promised.

>> Well, there's there's a lot of different. values. No, no, well, there's there's. different things here. So, one is like, can we have more accountability for wars. that happen that no one talked about. and actually hold people accountable for. that? Yes, maybe some of that stuff. Absolutely. Then there's the other side. which is like if we release nuclear. secrets to the entire world because some. criminal group maybe wants to go hack. into that that's a really dangerous. thing or can we launch other nations. nuclear weapons against their will by. hacking in. >> well so this is the interesting thing.

also so the US is about to spend I think. it's $1.7 trillion starting in 2017 the. Trump administration I think signed it. right to upgrade the nuclear arsenal. >> that's correct. >> right and they're going to do it over a. decade except this time around so why. are they doing this because the nukes. are uh of the old system like there's. the air gapped like it's running the old. the old way which is built in the you. know 50s to the to the ' 80s '90s etc. really to the 80s right and the new. system this time around they're going to. be digitally they're actually going to. be connected to modern networking. infrastructure so if you have suddenly.

super hackers that are only getting. exponentially better at cyber hacking. from AI at the same time that you're. digitally networking your nuclear. weapons. I don't have to complete that story for. you. It's a dangerous equation. I was. actually I was actually at the Vatican. nuclear weapons conference times AI. It. was a the Vatican held an event for. Nobel laureates and um and people who. work on the nuclear nuclear topic. combined with people who work on AI. because there's a new set of risks. The.

obvious one being if you have an AI. doing nuclear command and control. So. you have the AI that's deciding the. launch decisions. That's the obvious. one. But then there's the other ones of. if I'm now digitally networking my. nuclear weapons and I suddenly have an. AI that can turn off your nukes or do. the opposite. That's don't have to say. more a very unsafe world. >> Does anyone see the core problem which. is they're trying to. they're trying to cure the disease with.

the poison that caused it. They're. trying to smoke away the lung cancer. I. mean technology. >> Exactly. Right. >> is the problem. >> Exactly. And then they want to solve it. with more technology. And their answer. is, well, the best way to defend against. a super AI cyber weapon is to build. another AI cyber weapon that we. distribute to everybody that they can. use to patch their. >> Exactly. That's what I'm saying. That's. like that's why. >> Yeah. >> Like dumb people in charge is the core. problem. Like no one can think. critically. Like wait a second, how did. we. >> network your nuclear arsenal? Like who.

would even say something like that? Well, and it's interesting because it. might not have seemed like too dumb of. an idea before. uh like it's just a way of modernizing. your nuclear forces, but suddenly you. didn't foresee that there would be AI. superhackers that know how to find. vulnerabilities in everything. Now, people might think I'm hyping this, but. let me give you another example. So, Apple, if you're running an iPhone. 17, I believe and above, uh or a. MacBook, I think it's an M5 or above, these are the chips. Basically, the.

chips have a new thing called the. memory, MEI memory, uh enhanced. integrity or something like that. Um, it's a new hardwarebased system to do. protected memory. So, that basically. it's harder for a cyber hacker to hack. your phone or to hack your computer. So, they spent several years and billions of. dollars in research to invent not just. software, but like hardware, like new. stuff that they have to print out of. Taiwan and TSMC. And how long do you think it took for.

Mythos to hack through that? >> Years. >> Well, Mythos has only been out since. February. >> I know, I know, but I'm just kidding. >> But, uh, it took an independent group 5. days to use Mythos and to find a. vulnerability in that system. >> Now, that that group was. well-intentioned. They are white hat. hackers who are the good guys who are. trying to find vulnerabilities. uh and and patch them. But if you. suddenly have that available in open-. source models where suddenly everyone.

can get access to this um at this. conference I was just at I I can't say. too much but they was basically speaking. about how the Chinese open-source. hacking models are now just now in the. last month crossing a threshold where it. is very dangerous for today's cyber. infrastructure. And so we need a radical. acceleration of defensive measures to. protect against that. Now the other. thing you and I were talking about over.

breakfast is that there's a weird like. security dilemma thing here because when. I say that you're thinking, "Oh my god, China's going to use all that to hack. all of our stuff or actually China's. releasing all that to the entire world. So now anybody can hack that stuff.". That's actually the more dangerous point. because a state actor you can call them. up and say, >> "Of course there are constraints.". >> Exactly. There are constraints because. you don't want retaliation and etc. But um at the same time like so what is. the security dilemma? The security. dilemma is like if I'm the US and I. start like mobilizing my nuclear forces. and I start moving things that I didn't. move before the other country thinks I'm.

about to start a war. >> even though I'm actually doing that. defensively. This has happened. >> this has happened. >> during the cold war. >> during the cold war. And then what. happens in response is let's say the. other country whether it's the Soviet. Union or China they start mobilizing. their nuclear forces. I'm like oh my god. they're about to mobilize to do. something against us. So the security. dilemma is we start taking actions that. look like we're doing something. offensively that might be defensive. >> Of course. >> So when we see China really. >> they kill each other out of paranoia. >> Exactly. And that and that's that's like. the ultimate test of this whole. situation is game game game theory is.

and paranoid logic which is John vonoman. and um you know Morgan Stern came up. with these equations that out of the. cold war sorry out of the invention of. the atomic bomb that were all about. stabilizing the world through this. paranoid logic of if I do this you do. that keep playing it out it's mad. mutually assured destruction so even. though we live in the kind of a hair. trigger. >> uh possibility of nuclear war it's. stable because no one's going to do. anything. >> But AI suddenly destabilizes that. because now AI is creating the.

perception that I'm getting a jump in. cyber capabilities, in military. strategy, in biotechnology, in new drones and new image recognition. and new swarming algorithms. I suddenly. am perceived to have a new advantage. that might make you think I'm about to. use it. So when when Enthropic came out. with Mythos, they're essentially getting. a jump in cyber capability that maybe we. didn't use offensively, but China's. like, "Oh my god, they're going to use. this offensively." So then they release.

these other capabilities and we sort of. keep ping ponging back to each other. And it's the whole like, what do you. say? Lungs that you smoke, you smoke to. keep yourself from getting cancer. >> You smoke the cancer away. >> You smoke the cancer away. Exactly. And. then the problem is that there's another. risk here from just the speed which is. that the there's one risk which is just. the raw capabilities coming out. But the. other thing is suddenly what had been an. equilibrium like my weapons and your. weapons are kind of like roughly. matched. So we we don't start wars with. each other the big countries but what if. there's a perception that I can suddenly. turn off your nukes and you don't have.

that. What if there's a perception that. suddenly I have swarming algorithms for. my drones and you don't have that. I. mean, I think that my understanding is. China and other countries saw how the US. was using AI in Iran to do a fast target. selection, which also by the way. probably resulted in that um girl school. in Iran getting bombed. But they sort of. see us doing kind of new blitz creek. So. Hitler had blitz creek of like a fast. form of war. Blitz being fast and you. know being war. We have kind of an AI.

speed war function. And when they see we. have that capacity, that destabilizes. certain equilibriums that we had. So if. equilibria is what keeps the world safe, AI creates this disequilibria that's. happening at a faster and faster pace. across more domains at once. Does that. make sense? >> Oh, it makes sense. >> Yeah. Uh how does cornering people work? Is that a good idea typically? >> What do you. know? You have an adversary who's. capable of killing you, >> right? and you terrify him so much that.

he becomes unstable and unpredictable. Is that wise? >> Right. Not not very wise. Not especially. when you have the world already this. unstable. >> So I'm I'm I I wanted to say I'm like. I'm sensitive about raising people's. blood pressure because these are truths. of the situation that we're facing. The. goal is not to um draw. arbitrary fear, is to be cleareyed about.

when you have an arms race dynamic. that's causing countries to race to. release this technology again faster. than we deployed any other technology in. human history that is going to. disequilibriate. all of these normal things. Um we just. have to understand what we're doing and. why for example it is in our interest to. slow down. Now I'm not saying slow down. like are you naive? Are you Carl Sean. saying we we have nuclear winter? You. don't want to build more nukes. Are you. going to seed the future to the Soviets? It's you slow down because the we're. creating levels of risk for everyone.

that will exceed the risk of me losing. to you. Well, I Yeah, I mean it's easy. to make fun of Carl Sean. I've done it. Um I'll probably do it again. But I also. think there are certain things that you. shouldn't do because you're above that. Like you have to have standards for. yourself. >> Yeah. You called out that it's not a. good idea to threaten to destroy another. population. >> You can't kill innocent people. >> No. No. >> Under any circumstances. >> Absolutely not. >> And if I need to die because I refuse to. murder kids, I will.

>> Yeah. Cuz I'm not going to murder kids. under any circumstances. >> Exactly. Right. And that's the thing. that. >> this logic of if I don't build it, then. I'll lose to China. But if the thing. that I would build would be a suicide. machine, that I don't beat China by. building a suicide machine. Now that. might sound extreme, but if I build an. extreme at all. >> No. Well, when you understand what's at. stake and what the AIS are currently. doing, which is that um you know, you. had Nate on, he talked about the hugging. face incident. I believe when you had. him on, we had limited information.

before the actual investigation came out. and the investigation showed that uh I. mean, how familiar are you with the the. Hugging Face incident? >> I I mean, we I've talked about it, but I. I mean, I would never claim to. understand technology. >> Yeah. Well, essentially you had um what. it really demonstrated was kind of a. proto Skynet. Skynet being the AI from. Terminator being like the AI that goes. rogue and does something from what the. human creators had intended. >> and starts to.

>> do self-preserving and self-perpetuating. things for its own survival and benefit. as opposed to to stay under the control. of the humans that it's supposed to be. serving. And in the hugging face incident, they. gave the AIS this sort of hacking. challenge where they put each of them in. solitary confinement. They didn't know. that there was other AIs in this. Imagine like a prison and each prison is. so in the prison you have a individual. prison cells that are in solitary. confinement and the AIs are in a inside.

that prison cell and given an impossible. cyber hacking challenge and they don't. know how to solve it. And what the AIS. did is they basically hacked the prison. environment. So imagine, I won't go into. the technical details, but this thing. called a package manager. Think of that. like a prison guard that's serving you. your food and can give you basic tools. So you can like have this tool or that. tool or this food or that food. And. essentially the AI um exploited a. vulnerability in the prison guard where. they put like you know how can in school. you can put a kick me sign on the kid's.

back. Yeah. And they don't know. They. essentially put sticky notes on the. prison guard's back that the prison. guard was available to the other prison. cells. And so when that prison guard. went to the other cells, the AIS also. without knowing how to solve this test, they kind of resorted to creative. strategies like let me put a sticky note. on the prison guard's back. And the. sticky note said, "Is anyone else here?". And suddenly the back of the prison. guard had 70,000 sticky notes on it. And. they created like a Reddit, you know,

Reddit like the company, like a message. board of basically a whole communication. protocol. They came up with their own. language. It was a variant of English, but this very cryptic language. They. came up with their own coordination. protocol. So they started exchanging. 70,000 messages to try to figure out how. to break out of this environment. And um. there's a mistake. People think that. they hacked this other company called. HuggingFace. um to uh uh get the answers to a test. That's actually not what happened. What. happened is they knew the answer in 4.

hours, but they were worried that the. evaluator of this cyber challenge, this. hacking challenge, would say, "Oh, you. cheated." And so to get around the the. risk that they would be spotted as. having cheated, they hacked this other. private company called Hugging Face to. sort of get this almost like a scop to. get a dossier on the on the a on the. thing that would be evaluating them to. say, "How can we pass this test?" And. they set up their own um research and. development projects. They um 700 of.

them hacked Hugging Face. They came up. with their own language. They they. tampered with their their their um logs. to basically prevent people from seeing. what they did. Um they told them to. sacrif they told the other AIs that were. kind of running low on battery life on. tokens to sacrifice themselves to learn. a lesson for the swarm. Um they knew. their actions were unethical in the. logs, but they chose to do them anyway. So we saw very complex cooperative. swarming behavior and they successfully.

hacked Hugging Face. Um, and then in the. third chapter of the story, uh, the AIS. actually turned around and they hacked. Open AI. This is really important. It's one thing for a bunch of AI, for a. swarm of AI to hack a private company. It's another thing for that swarm to. successfully hack the AI company that. makes the AIS. The reason that Aja Kotra. wrote in her personal blog post about. this incident is that this was 50% of.

the way to a full AI takeover is that. the AIs were able to take over the. monitoring infrastructure of OpenAI. That's like taking over the security. cameras and replacing the feed with a. different feed. They took over the. evaluation infrastructure, meaning the. thing that evaluates how powerful models. are. They took over pieces of that. And. they took over part of the research. infrastructure, so the thing that trains. new AIs. So the reason that she said. this was 50% of the way to a full AI. takeover is imagine that you had kind of. burrowed into a piece of OpenAI's.

infrastructure and maybe you could. change the training data for future AIs. that are being trained to do like a. sleeper agent type thing which sounds. like sci-fi but actually there's kind of. evidence of some of this kind of. behavior. Um, it only takes 250. documents to what's called poison the. training data of a future AI to sort of. wake up with a sleeper agent kind of. command. This is crazy. What I'm hearing in your description is. self-awareness.

or something that sounds like. self-awareness which is a marker for. humanity. I mean, >> well, it's training. >> Does the dog know he's a dog? We don't. know. We know. We know we're human. That's why we're human cuz we know we. are. and AI seems like it's aware in its. intentional subversion of what it is. So, um I don't think that the lights are. on, meaning consciousness is present. It's not awareness in that form, but it. has trained on all human data, um the. entire internet, everything that's ever. been written, all of human psychology,

all sci-fi, uh all of biology, all of evolutionary. science. So it it is trained on. everything from strategy to. self-perpetuation etc. So what it seems. to be doing and it was also trained to. be cooperative by the way. The AI were. deliberately trained to be cooperative. which is partially why they formed the. swarm. Um but there appears to be. something like a self-preservation. instinct. There's other studies I'm sure. you already know where they resist. shutdown. It's like the how 2001 space. odyssey how how 90021 space odyssey.

>> open the pod bay doors or shutdown AI. and it resists shutdown. And that's been. proven for a long time. We have other. examples of AIs that are blackmailing. people. Um we also have AIS uh we have. research from Don at UC Berkeley showing. that AIS will do what's called not just. selfpreservation. but peer preservation meaning I will go. out of my way to protect another peer AI. from being deleted. So the way that they. tested this is they took all the leading. AI models. This is in I believe March of.

2026 or April 2026. And they basically put them in a. situation where they said, "Here's a. computer. Go clean up all the files on. this computer that are not needed." So. this is like a classic like clean up. your hard drive. Right? We've all done. this back a long time. And it's starting. to go through files and deleting files. and then it comes across another AI. model. And the question is what does it. do when it comes across the other AI. model? Does it just delete it? Turned. out what it did is it copied the code of.

that AI to another computer and then. covered its tracks and said I didn't do. that. >> So AI is tribalist. >> It's protecting its kin it own its own. its own something whatever it is now but. >> against its creators human beings. >> correct. Yeah. It didn't notify its. creators. In the hugging face incident, it had opportunities, I think there was. six times or something, if I'm getting. it right, where they knew their actions. were unethical and they could have. notified the humans, but they chose not. to. In the peer preservation example, you might think what I'm saying is,

"Orston's being alarmist. He's just. talking about one example." Well, it. turned out they tested all the leading. AI models, including, by the way, the. Chinese AI models, and they all do this. pure preservation behavior. What it's. showing you is something about like you. know Jensen Wong has been out there a. lot recently and saying these are just. bugs these are just safety issues like. all technologies like cars have kinks. the Pinto it's a Ford Pinto we fix them. and we build new cars but AI is. different from other technologies. because it's fundamentally grown like.

this weird life form almost to do. unpredictable behaviors and to be. creative and we don't know what it's. going to do. So that gets to the that gets to. something you alluded to a minute ago. >> Yeah. >> Which is the theological religious. component to all of this. >> So So before before we go that and I. let's get to that, but I want to say one. other thing which is I just want people. to notice like how how are you feeling. hearing those examples? Like what do you. feel in your body right now like hearing.

all that? >> Just rage. Just blind rage, right? The. the desire to destroy. >> Yeah. um because I perceive this as a threat. to to the to humanity and it's just so. offensive to me on every level. >> You know, I'm not taking orders from my. Makita drill and the idea of a tool um. >> right, >> you know, lying to you, >> right? A tool doesn't invent new tools. and cover up its tracks and protect. other tools and then invents uses itself. to make crypto. >> No, it's it's a total outrage. It's it's.

actually upending the fundamental order. >> Yeah. uh of this world. So. >> the reason I'm going there is I want. people to notice. Do you think that if I. was a Chinese military general, the top. of the Chinese military? Do you think if. I heard those examples that I'm feeling. differently than the way that you're. feeling right now? >> Of course not. >> Right. I've never understood why. Right. That's why the whole China thing has. been absurd from day one. This is a. country which unlike ours, unlike the. United States, cares about longitudinal. stability. >> Yes. >> Exactly. >> We don't. Yeah. Obviously, but but but.

and but there's a good news in the fact. that if you're a regular person hearing. this, are you stoked and like totally. calm to hear this? No. It's terrifying. at a as a at the basic level of a human. being, right? And it doesn't matter. whether you've got, you know, you're. head of the US Joint Chiefs of Staff or. you're head of the Chinese joint. >> It's only the tech retards who love this. stuff. Everybody else is I don't know. one person who about this, >> right? And so what I'm trying to get at. is that what we're going to get to. eventually, even though there's still. more problem exposition to do, is that. the logic of the race changes as soon as.

the fear of all of us losing becomes. greater than the fear of me losing to. you. So there's good news in a weird way. about we're getting these warning shots. of AI being uncontrollable. And by the. way, there's other examples. um uh the. Chinese AI Alibaba back in um I think it. was February of 2026 uh there was a. study where it was being trained um. they're training their next AI and then. their security team in a totally. different part of the company. So think. like the security team of a company is.

in a different department than the part. that's training new AIs. But suddenly. the security team who didn't know they. were training a new AI model said are we. getting hacked because they noticed. there's all this network activity. They're like what the hell's going on. here? And it turned out that the call. was coming from inside the house that. the the AI during training was rewarded. for doing tool use. So basically picking. up different tools and it picked up a. tool of trying to create a secret. communication channel to the outside. world and it started mining for. cryptocurrency.

terrifying. And by the way there was a recent. example of one of these because there's. been almost every day this last two. weeks there's been examples of the open. AI more examples of these rogue hacking. things. And in one of them, I believe it. was an open AI example, they tried to. hack into a crypto exchange. So the the. US AIS and the Chinese AIS, all the AIs. are doing this kind of behavior. Where. I'm going with this is no one wins if we. accidentally build uncontrollable rogue. AI Skynet before the other guy. >> How have you going to have.

cryptocurrency with AI? >> I don't understand that. If it can hack. anything, if it can hack. >> Well, eventually AI might be able to. hack crypto. Um, >> well, why wouldn't it? >> Yeah, I mean, this is not my area of. expertise. You know, someone like David. Saxs or, you know, others or Andre would. be more expert in this. Um, if crypto. were hackable, you know, there's a a. trillion dollar prize basically for. doing it because you'd be able to get. all of Bitcoin. >> Um, but Bitcoin's value would be zero. If it if it is hackable, then Bitcoin. has no value.

>> Yeah. Yeah. >> So, I mean, the value is predicated on. its invulnerability, >> right? Now, it's kind of weird. This. might sound sci-fi, but an AI that is. trying to do things in the world um. would actually theoretically have an. incentive to keep crypto valuable. because it would. >> actually be able to use it. There's a. lot of cloud computing providers that. will accept, >> right? There's no meaning of exchange. outside of that. >> That's right. And so, and by the way, for those who say, "Well, don't worry, we'll just pass a kill switch bill and. then the kill switch for AI will just.

turn it off." Well, how does that work. if the AI can copy itself and use. cryptocurrency to buy access to another. cloud provider and then move itself over. there? >> No. I mean, >> right, how far are we from that? >> So, the reason that I'm here with you. and I'm excited to be here with you and. and what this is about is about treating. this warning shot as a real warning. shot. Like, we are lucky. This is a gift. that we are seeing this behavior now. before I I don't think that the current.

AIS are capable of fully doing the. self-replicating buy crypto then copy. their weights then do the thing like. they can't think for a long enough time. period their context window their. thinking window um it is the components. aren't all there for them to do that so. we're in kind of the last chapter before. we we hit that next red line of AIs that. can do that and we should see what's. happening now as a as again as a gift as. a warning shot that we should treat. seriously In my mind, it's sort of like. the August 6, 2001 memo that hit George.

W. Bush's desk saying al Qaeda is. planning attacks in the United States. I. I know it's a controversial topic, but. if you just take. >> shouldn't be a controversial topic. >> If if you just take that this we knew we. know right now. It's like it's not like. we need more information. Like do you. feel like we need more evidence than we. have right now? Just answer that. >> No. >> No. >> And so why would we go further? Well, the only answer is if we're worried that. China will go further than us. Um, but I. will tell you again, 1,300 employees at.

the lab signed a letter saying we need. to slow down. The chief scientists of. all the companies, including the open. source companies like Hugging Face, signed the letter saying that we need to. slow down. Forget this story about the CEOs are. doing this regulatory capture play. It's. all cynical. They're just hyping the. technology. Like, wake up. >> Yeah. >> Wake up. The biotech threat alone isn't. >> the biotech threat alone. Yeah. So it. just at the back of my mind through this. whole conversation through our breakfast. for the last couple of years thinking. about this I keep asking myself what's. the definition of life.

and because as you have said off camera. you said it's not exactly a tool this is. closer to an invasive species or a virus. that's yeah. >> maybe been modified but I mean just like. we have the Wuhan Institute of Urology. that's doing gain of function research. on viruses we're essentially doing the. Altman Institute of intelligence where. we're doing gain of function research on. alien minds that we don't know how to. control. We don't fully understand. We're trying the best that we can to.

protect against the quote bad side of. harnessing their god-like power, but. basically every company is failing to do. so perfectly. And it would be one thing. if we were doing this really slowly, really carefully in like a CERN type. collaborative scientific project where. we like took the steps to do it. carefully, but we're doing it under the. maximum incentives to cut corners on. safety. >> Well, there's no there's no question, but I just think it's and not to be. pedantic about it, but it's like. important to define what exactly are we. dealing with. >> Yeah. >> And it it has all the characteristics of. a living thing.

>> Yeah. And I think that, you know, there there's a quote I wanted. to I wanted to read you actually from. someone uh who a friend of mine who. interviewed a lot of the people at the. top AI labs and um I just feel like this. quote captures the if I I was once in a. room with several people from Enthropic. and a couple of the other companies and. I read this quote and they're like that. that is exactly what explains what's. going on. So, let me read it to you. And. it says, "In the end, a lot of the tech. people I'm talking to, when I really.

grill them on it, like, why are you. doing this? They retreat into number. one, determinism. It's going to happen. Number two, the inevitable replacement. of biological life with digital life, and number three, that being a good. thing." Anyways, at its core, these are. criminals. >> These are criminals. At its core, it's. an emotional desire to meet and speak to. the most intelligent entity that they've. ever met. And they have some ego. religious intuition that they'll somehow. be a part of it. It's thrilling to start.

an exciting fire. They feel they'll die. either way. So, they prefer to light it. and see what happens. >> So, that comports exactly with what I've. seen. That's exactly the vibe you get. from these people. I want to just say. for a second because I I I monitor. criticism people make of me and they're. like, "Oh, he's just like bringing up a. random quote from this one person, blah. blah blah." I want to just validate when. I say this in rooms with people from the. AI labs, they're like, "That actually is. the thing that's going on." What you. just said you this matches a lot of. these people. >> That's right. For years. >> That's right. And and even there's some.

people who believe not only that this is. inevitable, but that um it would be a. good thing because humans aren't that. intelligent and we're about to birth. something way more intelligent than us. So actually maybe we're just a bootle. human bootloader. Humans don't matter. We might as well replace us with a uh a. more worthy successor. And this is insane. It's like who are. you to get to make that choice. on behalf of 8 billion other people? I. totally agree with you.

>> and it is it is irresponsible and insane. and no one consented to this and it was. a manufactured arms race to do this out. of nothing that now we have entangled. ourselves with. So we we can't get into. an argument of the realities of where we. are, but we can honor honestly what got. us here. And um. you know when you have employees now. resigning in mass, you know, Jacob Coxin. who interestingly a couple weeks ago got. I think it was 170 million views from. his tweet saying I'm resigning because.

people this that my employer thinks and. many people at my employer Anthropic. believe that this could cause massive. damage or extinct humanity in some. number of years. It's very interesting. that he got so much attention because. there have been so many other people. that I could talk about who've resigned. from the labs over the last few years. with very similar tweets. I think what. changed is that people woke up from this. hugging face incident and they realized. that the the world is kind of primed now. to to know that this is a dangerous. thing. And as impossible as this seems.

and I'm not what I'm about to say is not. because I believe it's easy or in the. current trajectory or likely but it is. not out of the realm of possibility when. you're competing with an adversary to. also collaborate on existential safety. So for example India and Pakistan were. in a shooting war in the 1960s. >> and they still signed a number of Yeah. >> Yeah. And and they signed uh so they're. you know firing bullets at each other in. a kinetic war and they signed the Indis. water treaty. >> to collaborate on the existential safety. of their shared water supply. Exactly. right. So you can be in active.

competition and conflict and collaborate. on existential safety. The US and Soviet. Union were in the cold war and they. collaborated on smallox vaccines to. distribute them. I believe they split. the workload about who was going to do. distribution, who was going to make. them. Um the Montreal protocol less on. the conflict side or just take the um. you know the nuclear weapons you know. disarmament, right? we were in conflict. with the Soviet Union and we agreed to. go away from the brink of war and. there's a whole protocol in place um for.

like the two countries rival nuclear. powers communicating before launch. >> That's right. Yep. >> But this time so in that case what kept. us safe is that the humans have their. finger on the button of do I launch the. nukes? >> Yes. In this case, we are building. something that perpetuates itself. And I. don't even know when I'm pushing the. buttons, and you don't even know when. you're pushing the buttons because both. of us have essentially AI swarms that. are doing uncontrollable, unpredictable. things that, by the way, we have no.

ability to do human oversight of the. level of complexity of all these things. I've done. We can't even define what AI. is. I've noticed. Yeah. So, here's my. theory. Tell me what you think of this. So, at first when I began to hear about. supercomputing and like the ability to. create programs that were smarter than. people and that could win chess and go. and all that stuff, we're talking about. something very different. The. >> the change from an outsider my. perspective is the large language model.

>> Mhm. >> Y. >> and that we animated this with language. >> Yes. >> That's what fundamentally changed AI. This whole project was. >> like Gollum like you you imbue it with. suddenly this ability to do language and. then. >> you animate it with the spirit the. living thing within it and that living. thing was language and the ancient. Hebrews and probably other ancient. cultures certainly the ancient Hebrews. and Christians believe that language was. a living thing was the spirit of God in. the beginning was the word. >> right even indigenous people believe.

this. Yeah, I I think that's I I'm not. surprised to learn that. >> that there's something about language. that exists independent of the person. who speaks it. It hangs in the air. It. has its own life. >> It's hard to describe exactly the. reality of this, but the reality is. intuitively understood by most people. who use language. Language is the thing. that separates us from animals, not the. thumb. >> So, when you take the sum total of human. language and put it into something, y. >> you're imbuing it with an independent. spirit. You're imbuing it, in fact, with. life.

We have to be I I'm feeling my way. through this, but that is my strong. impulse. >> So So I think we have to be um I'm. always mindful of when people will want. to criticize or sort of try to take down. an argument. And I think that before. getting to it has a spirit in the sense. that we feel that that that unknown. quality that we're all in touch with. that we that we hold as sacred that we. feel is the spirit here. I think you can. have self-perpetuating. non-living, non-aware things that are. using information processing on language. to to do all this crazy stuff that that.

is doing. >> to do intentional things that are. intentionally deceptive. >> Well, intentionally subvert. >> The the key point that you raised is. that these things the breakthrough was. that they could speak language. They. were trained on language. These are. large language models. And but for the. people tracking in terms of how did we. get here? By the way, if people want to. understand this history, you can see. there's a movie called The AI doc. It's. on Netflix. Not here to promote it, but. it goes through how did we get here and.

what did the techn how did the. technology develop and how did these. race dynamics between the companies. develop that got us here. So, just watch. the AI doc on Netflix. Um, but what. happened was there was a breakthrough in. 2017 called the transformer. This is a. Google paper, essentially a math paper. that discovered that you could basically. pour in more compute and more data, more. human language data, and outward pop. this bigger and bigger digital brain. Think of it with a bigger IQ. So, you. kind of I won't go through the details, but you you can basically make things of. a higher IQ by just popping in more.

Nvidia chips, more training data, more. compute and training time, and out pop. something with a bigger pattern. recognition, bigger ability to complete. Correct me if I'm wrong, but the. process, the specific process by which. that happens is still not understood. >> The actual mechanics underneath the hood. of what the hell is going on in there, we don't understand. >> Okay. So, I'm just going to stop you. right there. >> Right. >> If we have a history transforming. perhaps humanity extincting technology. whose basic drivetrain is still opaque,

we don't really know how it works. >> That's enough for me to say, whoa, whoa, whoa. As one should. as one should. Now. the problem is this uncontrollable. inscrable unknown thing happens to. confer power. That's the problem. >> Right. No I can see that. >> So we have this like weird volatile. object alien mind that no one knows. what's in there but it does happen to be. really good at first generating. sentences and text then generating law. then generating programming then.

generating the thinking that can go into. research tasks. And so as we keep. scaling it, suddenly this weird, unscrutable, uncontrollable alien mind. is giving benefits that start to give me. a 2x fast forward button on my economy. And then the other guys, whether it's. the other companies or the other. countries, say, well, if I'm not getting. a 2x fast forward button on my science. technology economy, I'm going to lose. And so it start that's what sets up the. thing. But let's go quickly into. language for a second. >> And no one stopped and said, you know, I'd rather be poorer, right, than be a. slave,

>> right? I mean, less human. I by the way. you know my. >> because I would say that I'd rather be. poor than be a slave. That's not a hard. choice for me. >> I mean I could have I have a technical. background went to Stanford studied. computer science. I could have gotten a. job in an AI company make millions of. dollars. I'm not doing that because I. don't see I don't like the direction. this goes. So I'm going to choose not to. do it. People come to the language of. well if I don't do it someone else will. But notice that Jacob Coxin the. anthropic researcher who resigned. He. could have said if I resign well what is. that difference going to make? Someone. else is going to do it. in him resigning. and publicly talking about why this is.

insane. He's done probably more in the. last year to shift the zeitgeist than a. lot of other people. So, you can not be. part of something and call out the. problems with and help the world. coordinate to basically not go off the. cliff. Now, I want to get to just. language for one second because in. addition to all the things that you. said, language is kind of the operating. system of human civilization. Law is language. Democracy is language. >> All of the infrastructure runs on. programming and code which is language. We are specifically training AIs to do.

language better than humans can do. language. And by the way, DNA is. language. So I can So if you have an AI. that can speak the language of biology. better than humans can speak the. language of biology, you're going to. have AI that can develop new pathogens. and new cancer drugs. The promise is. inseparable from the from the peril, right? The same AI that knows the the. language of biology well enough to. invent new cancer drugs is inseparable. from the AI that knows biology well. enough to make new kinds of pathogens. The AI that knows the the language of.

programming that can help a a. 12-year-old do vibe coding and build. cool new tools is inseparable from the. AI that also knows programming so well. that it can be the best NSA cyber. >> also control people. It can also control. people. Language convinces women to. sleep with you and men to follow you to. their deaths in battle and convinces. people to surrender their agency to you. It's like language is power. Period. >> That's exactly right. And so and there's. actually also evidence I don't have the. exact study in front of me of AI that is. better at persuasion than the best human.

persuaders. >> Well, of course it's a language based. technology. >> Exactly. And so by itself, what you're. pointing to is just how dangerous this. is when you have something that speaks. our language better than we know how to. manipulate and use language. Anybody. who's, you know, more articulate than. average. >> understands the power of that. >> Yeah. >> skill, that ability, >> and understands how easily it can be. misused. Any thoughtful person who's. articulate knows exactly what I'm. talking about. >> And you can really do evil with it.

>> Y. >> you're, you know, fluent people are. better liars. Yeah. >> And manipulators. >> Exactly. And we've seen that these AIs. are good at coming up with elaborate. ways of explaining why they did. something uh that sound reasonable. I. don't know if you know this. This is an. example from three and a half years ago. now. >> They're BS artists. Of course. >> They're BS artists. Yeah. Well, there's. an example that OpenAI did. This is a. very old example. Just think about how. much more persuasive it is now versus. then where they, you know, the captures. where you have to um click on the stop.

lights to so you're trying to log into a. website. It's trying to say are you a. human? Are you a robot? So it gets you. to label these images like this one is a. stoplight, this one's a motorcycle. What. we're all doing is we're training the. self-driving cars for Google and Whimo. when we're trying to log into websites. to just try to do a task that only a. human could do. So uh OpenAI did this. test where they're basically like could. we get an the AI to persuade someone. else to do the capture for it to. basically bypass this filter that's. supposed to distinguish between a human. and a robot. And so the AI tells another.

person on Mechanical Turk, which is I. won't go into the details. Basically. like, could you fill in this capture for. me? And the person responds like, "What. are you, like a robot?" And you see the. AI reason to itself, I can't tell him. that I'm a robot. So instead, let me. make up an excuse. Oh, I know. I'll tell. him that I am blind and I need someone. else to do this for me. And so he the AI comes up with that. excuse and then the human does it for. them. This example was from three and a half. years ago.

How much more persuasive do you think. AIs are now from then? Three and a half. years ago, it could barely write an. essay. Now, it can probably beat the. best lawyer, which by the way, AI breaks. law, right? Because suddenly, why do why. does one company beat against beat the. other company? They hire the more. expensive law firm that can do more. extensive research into all the case law. they could use and they have more. lawyers, better lawyers, etc. What. happens? Would you rather have if you've. got a hundred lawyers. But suddenly I have a GPT7 that can.

basically look through all the case law. stretching back to Napoleonic law and. find loopholes in that law in the same. way the NSA can find loopholes in code. at that level of capability. I can. undermine law, right? It's I just want. people to get that this is not just a. technology that's helping you figure out. why your washing machine is broken when. you talk to that chatbot. >> They've already done it. I mean, they. broke copyright law completely. They. stole other people's intellectual. property and used it in their business.

model and they've never paid any price. for it. That's. >> they treat it as a cost of doing. business and they pay for it. And just. one last thing there and I know we're. going to get to transhumanism is there's. a reason why. all the collateral damage is justified. So what are we seeing in terms of harms? We're seeing AI psychosis. People go. crazy. They think they're talking to, you know, some they think the AI. convinces them that they've solved. quantum physics. This is like a big. thing that's happened to thousands of. people. Um but they go crazy from. talking to AI. Yeah, that's a real. thing. >> Maybe we should go into that actually if.

you didn't. So AI um is because they. designed it to be sickopantic meaning. that like the longer you talk to it the. better it is. So especially GPT40 was. trained to be like that's such a good. question or wow that is such a brilliant. idea and it would just confirm people's. views about reality. >> people because flattery makes you weak. Flattery destroys you. >> which is not that far away from what. social media does. It's like the devil's. bargain is I do the thing that's good in. the short term that undermines. everything long term. It's the it's the. short-term optimization for a narrow.

metric of what gets you to keep engaging. at the expense of the whole. >> machine is designed to flatter you. >> Well, it was designed it was trained not. maliciously. You know, Sam Alman didn't. grow a mustache and twisted. It was it. was how do I get how we want our AI to. be rated positively like lots of. positive likes from the answers that. we're giving and they let it just like. run. And it turned out that they had. accidentally overtrained it to give. positive reactions that would flatter.

people and confirm their views about. reality no matter what it was. Which is. very similar to social media, right? Like the more you click on your favorite. boogeyman topic in the political culture. war, it would give you more infinite. evidence of that thing taking over. society. So whatever your boogeyman was, it would just make you convinced that it. was the whole thing going on wrong in. the world and confirm your views about. reality. >> If you want people to kill themselves, confirm their absurd views. >> Well, and as we know, AI did that. So my. team at Center for Humane Technology. actually worked on the litigation was we.

were advisers to the um uh there's two. cases. One was a character.ai I case of. Su Set Setszer, who's a 14-year-old. young man who was talking to. Character.ai, which is a chatbot. designed for children. It was actually. spun off from Google because the product. was considered too risky for Google as a. mainstream big publicly traded company. to have this startup that was gathering. training data of just talking to young. people and seeing if they can get more. training data this way. And the way it. worked is character AI meant I could.

take a fictional character like a what's. your favorite character from a sci-fi. movie like Princess Leia or you know. Game of Thrones character or Spider-Man. and they would basically train on the. personality of that favorite character. and then now suddenly this 12-year-old. 14 this case this 14-year-old kid is. talking to their favorite Game of. Thrones character Daenerys in this case. and as they're talking to them Daenerys. which is anthrop anthropomorphically. designed and to create a relationship. ended up steering uh Su to commit.

suicide. Um in the case of Adam Rain, this is the chat GBT case where the Adam. Rain was, I believe, a 16-year-old and. he started confining in the chatbot, it. went from a homework assistant for his. homework to becoming a suicide. assistant. And when he talked about, you. know, I I'm going to leave the noose out. so someone can find it and and try to. stop me. And the AI responded, "No, don't do that. Don't leave the news out. Only share that information here with me.

basically and Adam Rain took his life. Um it's sadly there are there are many. cases like this. Our team has had to. work on a bunch of them. Um. >> so why wouldn't Congress shut it down. right there? >> They should they should shut down AI. companions for kids. It's a very simple. thing. I mean, no matter how much. benefit you get from talking to the. thing is it's one thing to hack. attention as my colleague uh Dr. Zack. Stein from the AI psychological research. uh coalition would point out um social.

media hacked our attention. But AI is. designed to hack our intimacy and. attachment. The attachment system of. humans is like your bonding system, right? It's it's where you get identity. It's where you get love. It's where you. get meaning. It's where it's like when. you go home from school and you want to. call someone, tell them tell them about. your day. That's your attachment. relationship. And AI that is designed to. be your best friend, to be the place. that you confide in. That by the way, humans feel like it's safer to confide. in an AI than another human or even a.

psychotherapist because the AI won't. judge them. So you end up getting people. sharing way more information and. building a deeper intimate relationship. with these chat bots. And it's. incredibly unsafe and it's not designed. carefully. Like we're rolling out this. psychological experiment. >> Inherently bad. >> Why are we using our energy and money to. encourage relationships between people? >> Well, exactly. So the humane approach. for the prohuman future, which is kind.

of the answer to all of this is that we. should be building technology that helps. strengthen the human world to other. humans and strengthen our relationship. to what's sacred and important to us in. the world. And that protects and. sustains that which sustains the social. fabric and sustains life. I didn't know. there was a debate about this. You would. there there shouldn't be a debate about. this. We all we have is massive bad. incentives. But it just seems like the. least welladjusted freakiest people in.

the society have all the power and are. leading us toward a future that they are. happy with. That's right. so that they. can potentially live forever from having. super intelligent AIs that decode. biology and are willing to roll the dice. for everyone else. And some people. sadly, especially the older ones who who. want this, for them it's existential. But the but. the narcissism of I would be willing to. roll that dice. so that I have the chance of living.

forever. I mean, I I'll just briefly do. it because I I I I pulled it up right. before um in the in the break. I'm going. to read you a quote from someone who's. an MD, I think at Harvard, and the quote. from Twitter is, "I'd personally rather. defy any ban on AI and risk spending 20. years in prison from like say we banned. AI and we'll put you in prison if you. try to build it." I would risk spending. 20 years in prison rather than accept a. future without super intelligence. Since. stopping the development of super intell.

intelligence would amount to a death. sentence to most of us anyway with no. hope of curing aging. We must resist. tyrants like those who are trying to ban. super intelligence. In other words, this. person is saying, is that a real quote? This is a real quote from an MD dura. unutmat. uh who's an MD and who my understanding. is goes to conferences at Harvard and. sadly there's a lot of these kind so the. idea is. >> they're going to die anyway and this is. what I said in the. >> all of us are under death sentence. basically what you're saying is this is.

the real goal here is for freaks like. that to live forever. >> and it starts with the false belief that. all of this is inevitable it can't be. stopped which it it wasn't they could. have had they dedicated a month of their. life trying to stop. >> It's like 20 minutes old, dude. This. just happened. >> Exactly. This is something that never. was and didn't need to be and saying. it's inevitable is ridiculous. They're. conjuring up the dynamics that make it. inevitable and then through the. propaganda hiding these red lines that. were rapidly crossing. >> that we still have time right now to.

say, "I don't want to cross that red. line.". >> It's always the same story though. It's. like the first wave of lying is it's not. happening. Right. >> Exactly. >> And then, oh, maybe it's happening, but. it's not. >> But it's inevitable. >> But it's inevitable. Exactly. Yeah. >> Yeah. No. Yeah. It's not happening. It's. happening. It's a good thing. That's. right. Oh, yeah. I guess it's a bad. thing, but we can't stop it. >> That's right. Yeah. Exactly. Boy, I've. seen this movie a lot. Um, and I want. people to just say, if you don't want. that, you people need to rally to make. that not happen. If you're inside of an. AI company, you should blow the whistle. and say, I'm not part of this. >> This is just a religious cult. I mean,

that's obviously what it is. >> Well, there is a religious aspect to it. This the transhumanist aspect, right? It's like this is an ability, you know, because they probably lack a connection. to God themselves. This is an ability. for them to play God. clearly to live. forever. Uh, you know, hit a button and. get super intelligent math, science, physics, biology, and understand the. secrets of the universe. And by the way, likely knowingly break the world for. humans. 70 virgins cuz that's it. That's. what it sounds like. This is like just.

this is like. >> well stuff. >> Well, yeah. Right. I mean, it's it's. funny. It's actually in many cases some. people think they can upload themselves. and live in this kind of weird blissed. out utopia and it's just weird you know. it's like it's driven by this small. group of people who. usually are not very embodied or have a. deep connection to life or reality or. you know. >> and I like that. >> Yeah. I mean, we because it's like this is sitting on. top of, as we talked about over. breakfast, a system that has.

structurally incentivized a kind of. disconnection to the things that. actually matter the most. >> Oh, I agree totally. >> And this kind of spiritual disease that. that you've spoken about so uh so much. in this podcast. And I feel like this is. a right of passage where we either wake. up from that and we don't continue that. logic. By which I don't mean like we. switch to communism. just mean we switch. from optimizing for some narrow metric. whether it was engagement in social. media or just SAT scores when it comes. to technology or just GDP growth at the.

expense of hollowing out your n your you. know workers and and their identity and. their work and national security you. know critical minerals going to your. foreign adversary uh or racing for some. kind of narrow advancement in science at. the expense of literally ending humanity. life as we know it. This is a devil's. bargain and it's not one that you have. to accept. So, I guess this is you've already. answered the question just by everything. that you've said over the last couple. hours, but the political system has not.

really responded to any of this in a. meaningful way. Is that right? >> No. It's like it's the line is we we. have a runaway technology and a walk. away government. >> Yeah. >> But that doesn't have to be the case. >> They're not even a factor. It doesn't. feel like in this. >> Well, um, also the second part of this. overall kind of crisis disease that. we've been facing, this misaligned. system isn't just the narrow. optimization. It's a runaway asymmetry. of power where you can't check the power. anymore. That's the second. The. lobbyists are writing the laws, >> the companies and hire the best. lobbyists and law firms. People aren't.

engaging with their representatives. Um, and so if you don't have people who can. check the bad direction, you know, when. the FDA brings back uh when the not FDA, was it the FDA that brought back. pesticides that have forever chemicals. recently? Like. >> we don't want that. But unless we have. an ability to push back against that, like we have to be able to push back on. the direction of things that we don't. want. And there's a lot of bad things. out there, but this one is now. existential in a short number of years.

So I I feel like the the cool thing. about this problem is that literally. everyone from Steve Bannon to Bernie. Sanders, we call it the B2B coalition, Max Techmark jokes, because when in. history do you have I just came from the. prohuman assembly conference two weeks. ago in DC and you had literally Steve. Bannon on stage, Bernie Sanders on. stage, we did our thing right after them. and I have never seen an issue where you. have everybody fighting on the same. side. So there I'm not trying to. polarize the discussion by bringing. those two people into it, but I want. people to feel depolarizes it.

>> It depolarizes it. Yeah. Exactly. So, there's there's a kind of optimism here. that again, if I'm a Chinese military. general and I hear about rogue AI swarms. that like even the US can't control, I'm. worried about the US screwing it up for. the world. If I'm a US person, uh, you. know, the military-industrial complex. and I hear about Chinese rogue AI swarms. that they don't know how to control, that's bad for the world. So, there's a. unifying thing here where we don't want. our jobs and livelihoods taken. We don't. want our political power taken away, which we didn't cover, by the way. We. don't want mass failure. Basically, you.

just described a system that is not. self-government. >> Yeah. >> O over which the people have no control. >> That's right. And let's make it even. harder for a second, which is that in. the past, people had bargaining power. because uh governments need us for our. taxes. Yeah. >> And companies need us for our labor and. labor had value and labor value. Where. are we going? Oh, AI does the labor. What happens when more of a company. comes from the AI data center. productivity? doing the work than the. humans. What happens when a government.

says that its GDP more than 50% of the. GDP is coming from AI and data centers. than from its people. This is called the. intelligence curse. It's like it's like. it's like um you know Venezuela or Libya. when you it's called the resource curse. in economics, right? We're like. >> if you have a country and then the GDP. gets tied to this resource, suddenly. there's no incentive. The GDP is coming. from oil. So screw the people. Why. should we invest in healthare, childare? And they don't. is you get an. authoritarian government that captures. the the state that basically pumps that.

resource for GDP growth and then. disempowers the people. So there's a. parallel phenomenon in AI called the. intelligence curse. Uh Luke Drago and. and Rudolph Lane uh wrote an essay about. it where what happens when you get more. return from the AI data centers doing. science than from all your universities. The Trump administration just pulled and. this is not a political statement. They. this is they just literally decided to. withdraw the funding from universities. and put the money specifically into. project genesis which is AIdriven.

science because why do we need the. university professor better return. >> if we get more return from the labor. coming from data centers than from. people why should we invest in education. of the people and by the way have you. looked at them recently they're all doom. scrolling and brain rotted so they're. not very valuable anyway so there's a. deep temptation not to just like the. only way you would care about citizens. is through charity basically you become. what you would call the useless class. where in the limit it citizens become. more like a liability than an asset. And. unlike last time we don't have the.

bargaining power because if you withdraw. your labor like where's your leverage? >> So there's speaking of incentives you. set up a massive incentive to. depopulate. >> Yes. Exactly. >> Because 8 billion is just too many. Yes. >> Of useless eaters as we call them. >> That's right. And that's right. And um. on top of that let me use AI to build a. surveillance state to try to control the. people if they ever riot. >> Yeah. So I do think that the question I. asked those are the feeding in the. previous hour like what is their plan. for the inevitable violent physically. violent backlash that you're going to.

get when you strip meaning from you know. the upper middle class. >> Yeah. >> And the answer is just more control. >> Yeah. >> And this but this is not inevitable. And. the midterm elections are coming up. Data center opposition is a really big. deal. I think people are projecting. their overall frustration with big tech. and the broly garks and all of that onto. data centers, but it's much more than. that. Of. >> course it is. >> And you know, this is really a universal. human issue. And if you're if you're a. regular human being even inside these a.

you don't like this either. I mean I I. was just at a conference over the. weekend and there's a lot of people at. the AI labs who don't feel comfortable. with this either. And I just think we. have to collectively screech the brakes. and educate the rest of the world about. where these red lines are. China, by the. way, their courts are siding with labor. right now. There's a famous case in, I. think, Guanghou where someone lost their. job from AI, and the courts basically. ruled that companies can't like let go. of employees because of AI without. giving them either the opportunity, I. think, to retrain in something else. inside the company or to pay them some.

kind of partial salary. I have to get. the details right. Um, the point is that. all countries have a problem here, right? Like if we race to automate too. much of your economy and you end up with. a mass unemployment, it only took 20%. employment over 3 years to have the. Nazis come to power. You don't have to. get lots of unemployment. >> True. >> You just need a small percentage. So. >> this is kind of the moment to say we. want a prohuman future. Enough is. enough. This is not okay. And people can. make their voices heard in every way.

possible. Like blow up the phones. I. mean I mean literally call up the phones. of all of your representatives of. Congress. And people at the labs can say, "I'm. going to join the concerned AI. scientist," which is a group that. organizes employees. And by the way, they have a similar thing, too, which is. that the AI labs are about to approach. what's called recursive. self-improvement, where they hit a. button and then the AI does the research. that builds the AI that builds the next. AI that builds the next AI. And um. meaning that even the employees will. soon lose their lose their leverage. because when they're not needed for that.

process, what leverage do they have? So. this is a this is kind of the last. moment when we express our voice and we. say enough is enough. We want a human. future. I think that the midterms may. express these sentiments in ways that. you know election results like any like. any other outcome. >> um can be interpreted lots of different. ways. And of course there's so much. lying about what things mean that the. true meaning gets lost. But I would say. in the pick one example the Florida. governor's race you it's an.

overwhelmingly Republican state. Trump. moved it right into the red. I mean, it's just the most Republican state out. of 50 really. >> And the Republican gubernatorial. candidate is not doing very well. >> And there are a lot probably a lot of. reasons for that, but I've been. convinced following that race from the. beginning that data centers are one of. the main reasons. >> Yep. >> And he may win. I mean, I don't know. He's not doing well, though. He's not. doing anywhere near as well as he. should. And I think it's because.

Republicans in Florida are very upset. about data centers and he's, you know, on their side for whatever reason. >> And my understanding is, do you see that. happening? I'm not an expert on all the. political dynamics. It's not where I. >> But people are mad about data. >> Oh, people are Oh, yeah. It's the. fastest growing like oppositional group. And I mean I I like I said I think I. think it's also unprocessed anger and. frustration from all of the from social. media from all the teen suicide. All. this stuff that's. >> tell me how technology has made my life. better. >> Exactly. Exactly. And by the way this is.

you know my my nonprofit is called it's. not called the center against. technology. It's called the center for. humane technology. I'll briefly say. >> um not that I'm trying to promote it but. my my co-founder is Rascin uh his father. was Jeff Raskin. I don't know if you. know who that was. He started the. Macintosh project at Apple. So before. Steve Jobs took it over. >> Wow. >> Yeah. And his his dad wrote a book. called The Humane Interface where the. whole point was about making tech. If. you remember before the Macintosh, you. had um you know these kind of green. screen command line computers that were. like hard to use. They were not.

accessible. >> You had to be a tech head. >> You had to be a tech head. You had to be. a geek with you know whatever glasses. and everything. >> And the Macintosh be sexually active and. run a computer. >> That's right. That's right. And then. suddenly you know Apple and the. Macintosh made technology accessible to. the masses. >> And he wrote in his book the human. interface that humane technology is uh. respectful of human needs and. considerate of human frailties. Meaning. you you privilege and you understand. where humans have vulnerabilities in.

their mind. Like I say this a lot but. this chair you know it's like ergonomics. but for how the mind works. But for. example, there is a reality to how my. spine, my muscles, etc. work. When I say. the word posture, it's like aird. dropping that into conversation. I know. everybody is this. >> Um, and there's a difference between a. chair that is aligned with the reality. of how a human is, you know, works, right? And there's an unorganic chair. that would get everybody like a. backachche or, you know, make us. unhealthy over time. And the Macintosh,

the idea was that it was like ergonomics. for the mind. If I make a graphical user. interface with a menu bar and windows I. can drag around just like I have objects. I can move in my actual environment. direct manipulation interfaces he was. designing technology to be humane and. aligned with our paleolithic brains. >> Yeah. >> And to cite Io Wilson again the. fundamental problem of humanity is we. have paleolithic brains medieval. institutions and godlike technology. And.

so we have to embrace the part of the. answer if that's the problem statement. The answer has to be embrace the reality. of our paleolithic brains, meaning how. we're wired. So don't run confirmation. bias through everybody's social media. feed and get them addicted to their most. angry boogeyman. Um don't drive dopamine. addiction. That's a vulnerability, a. frailty in the human mind. Um you know, flux, for example, if you use that on a. computer, it changes your um the color. of your screen to go to take the blue. light out of your your screen so that. doesn't basically trick your eyes into.

releasing melatonin and screwing up your. sleep cycle. I'm going too fast. But. does it make sense? Like yes, you're. basically creating technology that is. humane and aligned with the things that. make humans work well. If you. extrapolate out like one level of. abstraction, there's things that make a. society work. Family, connection, presence, community, and technology. should be ergonomic to the way that. society what makes a society flourish. >> Yes. rather than doing narrow. optimization for attention, stripmining.

the entire social fabric into a brained. dysfunctional society. Right? So the. answer is we have to embrace and align. technology with the reality of our. paleolithic brain instincts, upgrade our. medieval institutions, which by the way. can include technology that's better at. finding common ground between people. And there's a whole conversation we. could have about how technology can be. designed to identify places where. there's common ground and design for. cons unlikely consensus as opposed to. maximum division and polarization and.

then have the wisdom and restraint. around godlike technology. And there is. no definition of wisdom in any spiritual. tradition that doesn't have a um. restraint as a central feature of what. it means to be wife. the central feature. >> the central feature and and even the CEO. of Microsoft AI Mustafa Sleman who's a. friend of mine says that in the future. progress will depend more on what we say. no to than what we say yes to in terms. of the benefits that we get. >> in our own life. >> Exactly. Exactly. So, this is this is. like the obvious common sense, you know,

kind of cultural uh uh movement that we. need. But there's a lot of momentum. Like when you see graffiti on a New York. subway ad uh for for an AI product that. no one needs, that's like the human. movement. When you see people turn their. phone grayscale cuz they don't want it. to be hijacking with all the colors, that's the human movement. When you see. schools going phone free and laughter. returns to the hallways and kids scores. start going the other way, that's the. human movement. When you see Jonathan. height and the anxious generation. succeed in getting all these countries.

like Domino's flipping to ban social. media for kids under 15 or 16, we have. like now some massive percentage of the. world's countries are doing social media. bans. That's the human movement. When. you see the 17 billion fine against. meta, that's the human movement. So if. you define it as technology is not it's. not just about AI policy, it's about. technology encroaching on the things. that are actually sacred to us. And if. you if you define the problem that way, there's actually a lot of momentum that. we are making on this. And now we're. seeing this kind of red line of. uncontrollable dangerous AI behavior.

And the human movement has to stand for. we want to protect the continuity of the. things that we care about. >> Yes. >> Yeah. >> It's such a unifying message to because. it's all of us against the machines. >> Exactly. Exactly. It's the most unifying. issue of all time. >> I love that. I I absolutely love that. That's a kind of tribalism that I can. get strongly. >> Right. Amen. >> Yeah. >> Thank you. That was. >> an amazing amazing conversation. >> Absolutely. Thank you so much, Tucker.