Military AI: Anthropic, Open AI, & the Future of Warfare | The Weekly Show with Jon Stewart

PAUL SCHARRE: Conceptually, the idea would be, who's choosing the targets? If a human chooses the targets, then you'd. say the human is in the loop, the humans. making that decision. If the AI is choosing it, or the AI is recommending. that humans aren't really paying any attention, then you'd say, well, the machine is doing that. And so one way to look at this would be after the fact, something gets blown up, and said, well, who said it was a good idea to blow this thing up? If the answer is all the humans are like, uh, I didn't do it well, right, that's not a great outcome. It's not a great outcome.

JON STEWART: I assume that will generally be the answer. [THEME MUSIC]. Hello, everybody. My name is Jon Stewart. Welcome to The Weekly Show Podcast. We got a banger for you today. As you know, the world is hurtling. in no small measure towards its utter and complete destruction. And there's a new wrinkle in the destruction of our world. And that is that a lot of the weaponry that we seem. to be deploying at the various places around the world.

are being controlled by not necessarily. autonomous, but large language model; AI, Anthropic, OpenAI. The same people that bring you Claude and ChatGPT and help you. break up with your boyfriend or girlfriend using a rhyming. scheme that Drake would use, that's also. being used to target and destroy our enemies. And it is incredibly chilling. And just recently, a huge controversy broke out.

into the open when one AI company, Anthropic, drew a line. and said, we shall not allow our product. to be used in this way. And then another AI company there, what do you call them there, the OpenAI went, we will. That's cool with us. But it's a lot more nuanced than that. It turns out there may not be heroes and villains. in this story, but we are going to discuss it.

all today in an episode entitled. How are We All Going to Die? And when exactly is it going to be happening? And we have two experts in the field of AI. and how it is utilized, and especially. within a military context. We have with us Dr. Sarah Shoker and Paul Scharre, and let's just get to that. [THEME MUSIC].

Ladies and gentlemen, we are delighted to welcome today. on our continuing episode of I Think We're All Going to Die. Our guests today are experts in the field of how we. are probably all going to die. Dr. Sarah Shoker, who is a senior research scholar. at the University of California at Berkeley, and Paul Scharre, Executive Vice President for the Center. for a New American Security and author of Four Battlegrounds--. Power in the Age of Artificial Intelligence.

Thank you both for joining us here today. Sarah, you worked in AI. Explain just very briefly, your area of expertise as we. move forward. - Yeah. Sure. So I used to be the lead of the geopolitics team at OpenAI. That was a research team, and we focused. on a portfolio of topics relating to AI. and international stability. And currently in my role at Berkeley, I focus on new testing and evaluation methods.

for generative AI models and their potential impact. on warfare and military AI integration. JON STEWART: Very, very apropos for today. And Paul, for you as well, where. do you stand on studying AI, on military and AI, what's your background with that? PAUL SCHARRE: So I've got about 25 years of experience. in the national security field. I was an army ranger, did a couple tours. in Iraq and Afghanistan, and then. I worked for a while in the Pentagon. as a civilian policy analyst.

I actually led the group that drafted the Pentagon's. policy on autonomous weapons, which is still in effect today. And then for the last 12 years or so, I've been at the Center. for new American Security, researching and writing. on this topic, trying to understand how is AI changing. warfare, and how do we avoid some of the bad scenarios. you're talking about? JON STEWART: So this is perfect because I think. it brings in the perspective of, Paul, you've you've been in the military, you worked in the Pentagon. You understand the ins and outs, Sarah, you've been at the companies that.

are developing these products. So let's just start for the basics. And I'm going to say this for my audience. Obviously, I understand how AI is used in the military. But Paul, very briefly, how does the military utilize AI, and how is that different from their general practices? PAUL SCHARRE: So I mean, it's not really. the military is using it like any new technology. that they're going to try to find ways to be more effective,

more efficient, much like they use. computers and computer software and computer networks today. So the military doesn't necessarily. see this as something special or different, but really, a productivity tool, just like I think a lot of people. might use a large language model. JON STEWART: An optimizer. PAUL SCHARRE: An optimizer. Or you're just optimizing for something. a little bit different. JON STEWART: Yes. Slightly. But so as you were working at OpenAI, when they. talk about optimizing, are they developing at these companies?

Are they particularly developing for military. or is the technology that they're using just. being utilized by military? SARAH SHOKER: Yeah. So generative AI models are both dual use. and also general purpose. They're dual use in the sense that they. can be used for both civilian and military purposes, for good and bad. But there are also general purpose in that they. apply to a variety of domains. So these are models that can be used in legal applications.

for software engineering tasks as therapy bots, we now know some people use them as. So they're not trained for particular use in the military. But you know, nevertheless, the military, I think, has been a keen adopter in the last year. I think I'd also be remiss if I didn't add that even. though most consumers now primarily interact. with AI, probably through these generative AI chat bots,

AI is, in fact, a toolbox of methods. It is not exclusive to large language. models or generative AI. And the military uses a variety of different AI techniques, such as, for example, machine vision, which is responsible for object recognition, facial recognition. JON STEWART: So this is not just in the way I might use it, where I would go on and go. I'm thinking of visiting the Jersey Shore, recommend.

five different things. And then the AI will say, boy, that sounds like a great trip. because my eye is relentlessly positive, much to my chagrin. And then it'll list me a few other things. They're not just using it in that regard, they're using the other tools of AI, which I guess. would be optimizing for anything. from targeting to maybe supply chain or any of that.

Is that correct, Paul? PAUL SCHARRE: Yes. You can think about maybe three different types of AI. One is something that's been around for decades. It's really like handcrafted software written by humans. Good example of this would be a commercial airline autopilot. We kind of don't think of that as AI anymore, but once upon a time, it certainly was. Military has a lot of things like that in radars and sensors. and fighter aircraft, that kind of thing. JON STEWART: So already autonomous workings. for some of their machinery.

PAUL SCHARRE: Maybe bounded autonomy, I would say. Like, there's lots of missiles that once you let that thing. go, it's not coming back. But the autonomy is pretty bounded in what it can do. Then you've got machine learning systems that might. have a narrow application. So they're doing computer vision, as Sarah was talking about. Military uses these to analyze satellite images, analyze drone video feeds. The military is collecting more intelligence. than it can possibly put human eyeballs on. There just aren't enough human analysts. But AI can help you then, you look through these images.

and find targets and identify things of interest. And then there are large language models, which are these sort of like much more general purpose. text kind of machines, where you can feed in lots of data. You can have it analyze things. You can combine text and images and other types of data. And that's newer. And the military is also starting to use that as well. JON STEWART: In the public's eye, because I want to see if I can fill. in the gap between what the public may view this as. and what the reality is.

In the public's eye, it is Skynet. It is you know, robots, titanium robots that. can regenerate themselves, that are walking autonomously. over crushed human skulls, and just. firing what appear to be phasers at all kinds. of different things. And you're saying, actually, it's. the same shit that we're all using, like at the office, for the most part.

PAUL SCHARRE: I mean, for the most part, somewhat. different applications. But, I mean, it's the same types of things. And look, a lot of what the military does, to be fair, are back end functions. Right? It's logistics, it's personnel management. JON STEWART: Administrative and bureaucratic. PAUL SCHARRE: It's administrative. JON STEWART: Yeah. PAUL SCHARRE: That's like 95% of what the military does. Now, there's a different component, that is actually. battlefield capabilities, but a lot of the military use cases. are kind of mundane. JON STEWART: So let's get to that, because that's really where it appears this new controversy.

is, which is the battlefield. The controversy appears to be, and this began when Anthropic. had drawn two red lines. The red line being that there can be no just autonomous. kill chains. A person has to be in the kill chain, and that the AI cannot be used for general surveillance. on the American public, or gross surveillance. on the American public. Sarah, is that understanding of the controversy correct,

are those the two lines that are drawn? SARAH SHOKER: So I make a slight adjustment there, which is that they specified Autonomous weapon systems, not kill chains in particular. JON STEWART: What's the difference there? Tell me the difference there. SARAH SHOKER: Yeah. So an autonomous weapon system, according to the US definition, and it's important that I'm noting that it is, in fact, the US definition because different governments define. autonomous weapon systems differently. Are weapons that can select and engage a target.

without human intervention. A human can be in the loop, but it's not required. These weapon systems can function. without human supervision. The language that's used in the DoD directive 3000.09. is appropriate levels of human judgment, and Anthropic's position was that they. don't believe the models are sufficiently reliable, I agree.

And that for autonomous weapon systems, they need a human in the loop, which is essentially already. US policy. JON STEWART: So the US policy is, the human. is in the loop, meaning. So let's walk through a scenario. just to understand a little bit of what we're talking about. Let's say, the AI is used to analyze satellite. imagery and different targets.

A human will then get the results--. a human wrote the program, I'm assuming, to analyze it. A human will then get the results of this data that. has been analyzed, make their selections, and then give an OK to launch certain weapons. that may, in and of themselves, be autonomous. Meaning, they'll guide themselves. to wherever that target is.

And is that a minimalist description of how. this might all go, Paul? PAUL SCHARRE: Yeah. And I think that's right. I think conceptually, the idea would be, who's choosing the targets? If a human chooses the targets, then you'd. say the human is in the loop. The humans making that decision. If the AI is choosing it, or the AI is recommending. that humans aren't really paying any attention, then you'd say, well, the machine is doing that. And so one way to look at this would be after the fact, something gets blown up and said, well,

who said it was a good idea to blow this thing up? If the answer is all the humans are like, I didn't do it. Well, that's not a great outcome. Not a great outcome. I assume that will generally be the answer. But like right now, I think we're probably. in the case of certainly I have no reason. to think otherwise where the humans are the ones making. those decisions. Now, the AI might be helping to process information, helping to even maybe prioritize targets for people, but the debate between the Pentagon Anthropic.

is a potential debate about where. things might go in the future. I don't think actually it's a debate at the moment. about using a large language model to like, autonomously. make these life and death decisions on the battlefield, and then people aren't paying any attention. JON STEWART: Is it that we're nervous that the computer will. just decide on its own, or that it. will be wrong when it targets? So let's talk about Iran for a second. Describe how a situation like that goes wrong. and where the checks and balances are for that.

SARAH SHOKER: So Claude in the Maven Smart System. JON STEWART: OK. Let me back you up real quick. You said the use of Claude in the Maven--. SARAH SHOKER: Smart System. I can define my terms. JON STEWART: I love the fact that it's named after something. you could name your cat. Hey, Claude. All right. So Claude is what? SARAH SHOKER: So Claude is the name. that Anthropic gives to its flagship models, which is then used in the Maven Smart System.

This is an AI-enabled decision support system. that does a variety of things, including some of the tasks. that Paul mentioned, like, helping speed up efficiencies. in logistics, but has also been responsible for targeting. in Iran. We now have confirmation there as well. And if you know, public reporting is anything to go. by in Bloomberg and The Wall Street Journal and others,

the first day that the production of 1,000 targets. in Iran has largely been credited to the MSS, the Maven Smart System. JON STEWART: Now, who makes Maven Smart System? SARAH SHOKER: Palantir does. JON STEWART: Oh. Did you guys just feel the room get colder? Oh. The hair is-- all right. So Claude, who is made by Anthropic, and that is more of an interface. that we are accustomed to using,

what is its role in feeding information to the Maven Smart. System, which is, I believe, a system we. are less accustomed to using, and is maybe. a little less transparent? So tell us how that operates. SARAH SHOKER: Yeah. So the Maven Smart System has been in use for several. years now. The integration of Claude is, I think, relatively recent, I believe, in the last year because Anthropic. was able to gain access, go through the certifications.

to gain access to the government's. classified networks. As far as we can tell, Claude, right now. has been used in targeting. And again, according to public reporting, it seems that it has been used for target selection, and then also target prioritization. The Maven Smart System itself is designed to pull. in different data sources. So from sensors, satellites, and such. And Claude then makes those disparate data more.

readable to the human analyst. So it boosts efficiency in that way. But it does-- reading between the lines a little bit, it does also seem to offload a little bit of human autonomy. and decision-making as well, when. it comes to that target selection. and prioritization process. JON STEWART: Quite frankly, when you brought up 1,000. targets, because I have no context, I. don't know what I don't know. So I don't know if that's an unrealistic amount of targets.

I don't know if that's--. I'm understanding that there are target-rich environments, there are target-poor ones. Is 1,000 in a day, I don't know how they count it, is that an unusual figure? SARAH SHOKER: Oh, yes. I believe CENTCOM said that it was 2X. the number of targets and the 2003 shock. and awe campaign in Iraq. JON STEWART: So 500 targets in a day was shock and awe.

And this was 1,000. Now, I think we have to also take into account Trump math, which generally is like, this is. the biggest crowd ever to see an inauguration and it wasn't. So how much of that is do you think is Trump math, and how much of that is an astonishingly high figure? SARAH SHOKER: I mean, it's being reported by Bloomberg, The Wall Street Journal, and The Washington Post, all without an asterisk.

JON STEWART: Rags. Rags. SARAH SHOKER: I mean, they're all taking it at face value, and it's acting as though it is seemingly plausible. So there is no indication yet at this point. that it's not accurate. [THEME MUSIC]. JON STEWART: Stop paying for too much wireless just 'cause, I don't know, that's just what I do. It's how it's always been. That's just my company. Mint exists purely to fix that. Same coverage, same speed, just without the inflated price tag.

You can change your coverage, people. Mint is the premium wireless you expect, you know, your unlimited talk, your unlimited text, your data, but at a fraction of what others charge. And for a limited time, you can get 50%. off three-month, six-month, 12 month plans. of unlimited premium wireless. The only thing keeping you from doing it is inertia, laziness. Don't be the beanbag chair. Be the-- I'm trying to think of something--. energetic pogo stick?

Be the pogo stick? Eh, that's probably not right. Bring your own phone number. Activate with eSyM in minutes. Start saving immediately. No long-term contracts, no hassle. With a seven-day money-back guarantee and customer. satisfaction ratings in the mid 90s, Mint makes it easy to try and see why people don't go back. Ready to stop paying more than you have to? New customers can make the switch today, and for a limited time, get unlimited premium. wireless just $15 a month. Switch now at mintmobile.com/tws.

That's mintmobile.com//tws. [MUSIC MUSIC]. So you might say--. Paul-- let's say I'm working in the military. You work there, and you've been researching this. Hey, Claude, I'm looking to take out all the radar. installations in Iran. Where would I do that, and how quickly could I get it done? And then Claude would interface with Maven, which.

has all the data that it's gathered from, I'm assuming satellites? And then it's translating that data that they understand. through whatever intel they've gotten, and they're going to place it into real-world menus. of what you could target. Would that be accurate? PAUL SCHARRE: Yeah, so let me explain what we know. and then what we could speculate. reasonably about because--.

JON STEWART: What we know and what we do not know. PAUL SCHARRE: Well, we're gonna take a stab at it. So-- JON STEWART: All right. PAUL SCHARRE: We know that Anthropic's. AI 2, Claude, is deployed on US military classified networks. It's integrated through the Maven smart system, which collects intelligence from different sources. And it's been used by the US military. in real-world operations, including. the operation against Venezuelan president Maduro. and operations in Iran. And there's been some public reporting that's. been used, as Sarah was talking about in, target generation and prioritization.

Like, exactly how? We don't know. So now I'm gonna speculate about what. might that look like. JON STEWART: Speculation alert for Paul. PAUL SCHARRE: Yeah. So that could look like where you're talking to an AI. tool saying, hey, help me plan this vacation. to the Jersey Shore. There's somebody-- there's an Intel analyst or a targeting. analyst who's going to these tools, and instead of having to manually go through all. of this data that we have of where are the radars. and what is the imagery of them, queries it in natural language. Hey, develop me, for example, a prioritization of all.

of the radars that have already been hit. and what the current battle damage assessment is of them. How much have they been destroyed, or are they--. do we hit them again for a follow-on strike, how much. of them have not been hit yet. And let's put all that in a list, put it in a database. Let's prioritize it, and then let's. match it to weapons that would be needed. to take out these radars. Different types of radars might need different weapons. And then let's match that to available aircraft to help.

build a strike package that would eventually. go to an aircraft gets a set of targets and weapons. that are assigned to that target. And so, like, the technology is of being used. throughout that chain to make it just easier for people. to access and process this information. JON STEWART: So we would be doing that anyway. It would just take longer. PAUL SCHARRE: That's right. That's right. Now we're talking about basically replacing the things. which humans are doing with machines, speeding it up, making it a lot faster.

The US military set thousands of targets in Iran. Having the ability to process that information. at machine speed is very valuable for the military. JON STEWART: And then because it's Claude, you could say, and now give it to me. like you're Ernest Hemingway, and then it would give you. the targets, in short, taciturn-- it would just be. very terse and go all there. So Sarah, where does the controversy--. are we kidding ourselves, then, that there is a line? What is the controversy, and how does it break down.

What is Anthropic's argument here against-- what are they--. Paul was saying earlier, it's really about the future. As it stands right now, what is the controversy. SARAH SHOKER: So I think--. the controversy, in itself, is a little mystifying. because it sounds like the contract negotiations. went South due to some, shall we say, strong personality clashes.

If you look at the contracts between OpenAI and Anthropic, they're actually relatively similar, if not the same. They've essentially agreed-- both companies have essentially. agreed to both red lines. JON STEWART: The contract that they have with the DOD. or with Palantir? SARAH SHOKER: Ah, so that's actually-- we. don't actually know about that yet. JON STEWART: [LAUGHS]. SARAH SHOKER: So stay tuned. JON STEWART: All right. Let's speculate some more, people. All right. SARAH SHOKER: Yes, stay tuned.

It's not clear what model, now, Palantir might use. or if they'll have an array of different models. that they can choose from. JON STEWART: So who makes the contract? Does Palantir subcontract to Anthropic or OpenAI? Or does DOD-- who is the leading role in integrating. these companies together? SARAH SHOKER: It's not unheard of, in fact, pretty common for companies to come together and actually.

combine resources to create a product, especially. for defense purposes. You know, for instance, the DIU trial and the DAWG trial--. that's the Defense Innovation Unit. and also the Defense Autonomous Warfare. Group-- have a call for building, essentially, attritable drones. And they've issued that-- they've issued that call--. that call to industry. And industry has, in fact-- and companies. have in fact responded to that call.

by combining resources and submitting-- and submitting. joint proposals. So it's not unheard of four companies. to come into contact with one another. and then to approach the Pentagon. JON STEWART: So they'll do that together. Palantir and Anthropic or Palantir and OpenAI. will get together and say, we've developed this package. Using our product makes it more readable for humans. Your product makes it more-- and so they'll bring it to DOD, and they'll-- so the $200-million.

contract that Anthropic had--. Paul, do you know what that--. they had a contract with DOD. What was that-- what was that for, and for how long? PAUL SCHARRE: Yeah, I think-- so this. is where-- some of the details we don't really know. We know that they have an ongoing contract. with DOD to deploy their AI tools on classified networks. We know they're being used through the Maven smart system. But a lot of these details of, like-- we don't normally get, when defense contractors are working with the government--.

in fact, the, like, silver lining to this whole thing. is the only reason a lot of these details are coming out. is because this whole relationship. blew up between Anthropic and the Pentagon. Otherwise, normally, they would have. some deal about what the tools could and couldn't do. We would never know. And so that's, like--. think it's unfortunate, actually, that this sort of feud has spilled over between Anthropic. and the Pentagon. But it is really the only reason. that we have this kind of insight, which is even still. pretty limited on exactly what the terms.

of use of these contracts are. JON STEWART: How opaque are these military contracts? I know the--. DOD-- it's the only government agency that's never. passed an internal audit. But how opaque are these? And the $200 million that they use--. is that over a five-year period just to use their products. on their classified networks? PAUL SCHARRE: Yeah, I'm not sure that we know--. JON STEWART: Oh. PAUL SCHARRE: --unless Sarah's seen more details than I have. Yeah. SARAH SHOKER: Even as an employee,

I do not have access to contract details. It's very tinted in a lot of these companies. and on a need-to-know basis. JON STEWART: Is $200 million an--. I mean, to me, that's an enormous figure. You know, you're talking about--. the Pentagon budget, in total, obviously dwarfs that. $1 trillion, now, as they're pushing forward. It's an enormous amount of money. Do they have it with--. do they have that with different companies? PAUL SCHARRE: I mean, it's a lot of money.

for, like, a normal person. It's not a lot of money for either the Pentagon. or for these AI companies. They're all dealing in billions and billions of dollars. JON STEWART: Oh, so this is just walking around money. They're just throwing around a little walking around money. to Anthropic, OpenAI. PAUL SCHARRE: It's not quite money under the couch cushions, but, like, it's not a massive amount of money. And the direct cost to Anthropic. of losing this contract is not substantial to them, relative to, like, the scale of AI investment. that's happening right now in the AI sector.

JON STEWART: How much of the contracts. for OpenAI and Anthropic are consumer-based? In other words, I pay $11.95 to get your latest model. And how much of it is corporate-based. and defense-based? Do you guys have a sense of that? SARAH SHOKER: Yeah, I mean, I think OpenAI is, right now. for 2026, projected to generate about $25 billion. in annualized revenue. The majority of that is coming from subscriptions.

to its models. I think Anthropic is in a similar ballpark, where they're. on track to generate, I think, about $19 billion. in annualized revenue. Anthropic is a little different from OpenAI. in that it has prioritized enterprise. contracts earlier on. But there is-- I think OpenAI's strategy-- and this is public--. has been targeted towards generating more enterprise.

contracts in the future. But I do think that the majority. are still coming from individual. consumers, developers. JON STEWART: So the reason I bring. that up is it does mean-- 'cause. we're talking about they're opaque and they're tinted. But it does mean that the consumer has. some influence here, in that the government. is not their sole benefactor. It really is individuals, and that in a case like this--. so in the case of Anthropic, OpenAI--. we'll just go with that.

Anthropic says, I'm drawing a moral line, whether that moral line is an actual line. or it's already been traversed by whoever knows. is a real moral line or not. And OpenAI says, I agree with Anthropic, and we are drawing the moral line here, autonomous weaponry and mass surveillance. Anthropic loses the $200-million contract, and that same night, OpenAI announces, hey, we just.

signed a big deal with DOD. How real is that moral line that Anthropic drew? And how real is the backlash against OpenAI for suddenly. appearing to have turned around and said, oh, they won't do it? OK, we'll do it? PAUL SCHARRE: I mean, look, the backlash is real, and it's happened from some AI scientists. Anthropic vaulted, after this controversy, right to the top of the chart in terms.

of downloads in the App Store. So I think that's happening. The dollar amounts for both these companies are relatively. marginal compared to all of the other non-defense investment. The bigger risk for Anthropic is gonna be actions. that the government is already taking against the company, labeling them as supply chain risk. and going after them in that way, which would designate. other defense contractors saying they can't use. Anthropic's AI tools in the furtherance of their defense.

contracts, and then other steps the US government. might take to retaliate against the company. They talked about using the Defense. Production Act to seize control of their AI models, for example. So those are probably, like, the bigger risk. It's not so much the dollar amount of the contract. JON STEWART: And Sarah, was it a real line? And it appeared to an outside observer. that OpenAI immediately reversed their moral position, given what what you guys are both saying.

is a very small amount of money, comparatively, for their bottom line. SARAH SHOKER: I'm not sure if there was an actual reversal. I think that--. I mean, I do think that the military usage policies that. are often designed by these companies. are meant to preserve optionality for its leadership. There was a lot of backlash that--. I saw it in real time. A lot of the AI community still congregates on Twitter. And OpenAI hosted an ask me anything on Twitter in response.

to that backlash, which I think illustrates the fact. that the public can act as a pressure. point on these companies. But what we ended up seeing as a result of that AMA was not. necessarily an alteration to their previous policy, but adding more language to explain their already existing. position, which, in practice, again, doesn't seem to be all that different from Anthropic.

But I think the communication strategy is maybe--. is maybe a little different. I mean, I don't know if it's the cultural fascination. with the so-called great men of history, but I really would resist any kind. of narrative that tries to identify a hero and a villain. in this story. I'm not necessarily sure that those. are appropriate roles for either Anthropic or OpenAI. But, you know, to Paul's point, I. think part of the sympathy that's been directed.

at Anthropic is because they have been the target. of government overreach. And so I think it's possible to hold two ideas in one. hand here, which is that Anthropic. has been unfairly targeted. But at the same time, these two red lines. that have been identified by both companies. are probably inadequate, and the public does not. actually have to accept those two. red lines as the threshold--. you know, threshold of risk.

JON STEWART: Imagine if you had some kind. of, like, rewards program. You know what I'm talking about, like a miles. program, et cetera? But it's a rewards program that you. pay rent through and then earn points for travel, dining, shopping, et cetera. 2026-- if you're still paying rent without Bilt? Come on, brother. It's a loyalty program for renters. that rewards you for your biggest monthly expense, which. is rent. With Bilt, every rent payment earns you points. You can redeem them. Flights, hotels, Lyft rides, Amazon purchases.

So much more. It's a loyalty program that you pay. when you pay rent that helps you get out. of the place you're renting for, like, a week and go. have fun. And by the way, Bilt members can earn. points on mortgage payments. You know what I'm talking about, whether you got a house, whether you got an apartment, whether you're. sharing it with three of your knucklehead friends. I don't know what your life is. I don't know what you're doing. You can even redeem Bilt points towards your next rent credit,

or even a down payment on a home. It's simple. Pay rent is better with Bilt. What--. you're paying rent anyway. Get something for it. So join the loyalty program for renters at joinbilt.com/tws. That's J-O-I-N-B-I-L-T dot com slash tw. Make sure to use our URL so they know we sent you. But are we kidding ourselves, Paul, in that-- look, if we think through history, there's.

no human advancement that hasn't almost immediately been. sought by the military for advantage, whether that advancement is sonic. or chemical or biological. You know Sarah mentioned two departments over at Defense. that I would pretty much assume nobody who's listening to this. has ever heard of. I think we've all heard of DARPA. But there are development groups, I'm assuming-- they said when they went into Venezuela, they used the Havana syndrome liquidation, new weapon that,

like, you point it at people, and their insides melt. Like, we are-- throughout history, any advancement. that a human can think of, their military wing. is immediately going to try and utilize for some advantage, no? PAUL SCHARRE: Yeah, but look, two of the examples. you gave there, chemical, and biological-- we. do have regulations on how they're used. We have conventions banning chemical. and biological weapons. JON STEWART: But people still use them.

PAUL SCHARRE: People still use them, but not everyone. And they've been sort of--. by many states, they've been treated. as unacceptable weapons. Now, you get some pariahs. You get some outliers. You get people like Saddam Hussein or Bashar al-Assad. who are gonna use them still. But most states have given up those kinds of weapons. And I think it's better that they have. So the question with AI is not actually, are we gonna use AI in the military? None of these companies are saying, don't use AI in the military. The question is, should there be any rules,

and if so, who sets those rules? Because, like, the crazy thing about the dispute. about autonomous weapons is, as near as I can tell you, no one is actually saying, we're. gonna use a large language model as an autonomous. weapon today. That'd be crazy. If you have a large language model writing an email for you, you better fact check that email because they. do weird things sometimes. The question is, who gets to set the rules? And the Pentagon's answer is, we get to set the rules. We don't want these companies dictating to us. And these companies and many of the scientists working there--.

they have a lot of discomfort about how. the technology might be used going forward in the military. JON STEWART: Sarah, when he says about who sets the rules, is it the company or is it the military-- so we also--. and I've read about this group. They're called Congress. We don't hear much from them. It's this group of generally older white men. who, once they're past retirement age, enter into the legislative house.

Is Congress-- are they utterly rudderless here? Are they just overmatched? Do they have any role to play? What can we expect, and what should we expect from them? SARAH SHOKER: Hmm. From--. JON STEWART: [LAUGHS] That wasn't optimistic. SARAH SHOKER: [LAUGHS] Let's start small. Start asking questions. [LAUGHS] I am somewhat sympathetic to this idea that.

private AI companies cannot be setting. the rules in foreign policy. But one of the issues that I see today--. and I think this does track with a role, potentially, for Congress as well--. is that AI companies are, in fact, influencing foreign policy. It. May not always be through the back end and through. their contracts through the Pentagon, but they're certainly donating significant sums to lobbying. efforts and tying those donations to US-China.

tech competition and arguing--. and arguing that that a low or no--regulatory. environment is a requirement to, quote unquote, "beat China.". And they're supporting, potentially, political campaigns that agree with that perspective. And so this conversation is, in fact, coming for Congress, and they probably better be equipped, at the very least. And I actually think Paul may even be a better person.

to speak on this in particular, since he is, in fact in DC, and I would be curious to hear from him. what the general reaction has been. from Congress on this issue. But I can say that AI researchers, typically, are very keen to discuss their work. And I've, in fact, never met a keener bunch of people who. are willing to talk about the risks and opportunities related. to AI models. So you can always send them an email. They do-- yeah, I think they're pretty eager to have.

those conversations. JON STEWART: Paul, what--. so what say you down in Washington? PAUL SCHARRE: Yeah, I mean, look, I'm here in Washington now. I can see the White House out of my office window here. I'm not gonna pretend things are super. functional in Washington. But I think we have seen government engagement. on some of these issues. And there are a lot of tools that Congress can use. to have oversight of the military and the. intelligence communities. One is passing legislation, which.

may or may not be the right answer in some cases. On the domestic mass surveillance stuff, maybe, on the autonomous weapons, maybe not. We might want to maintain some flexibility there. But there's other things. Congress could hold hearings. Congress can--. JON STEWART: Yes, they could. That is correct. PAUL SCHARRE: They could get people. from the executive branch come in. and brief them and say, hey, what are you doing with AI? And if you want to keep it classified, Congress can do classified briefings. to educate them about what's going on inside the military. Congress can use tools like procurement and acquisitions.

Congress has the money. They are the ones that are allocating. money to the military and intelligence community. And so that is a tool that Congress absolutely. does use already to fund some projects and not fund others. And so there's a variety of tools. that Congress has, potentially, to influence these things. And I think the model of, who should be setting the rules--. maybe it's our democratically elected. representatives-- that is probably the right approach. JON STEWART: Well, that's what I was thinking. But to Sarah's point, you know, look,

these guys have more money than anyone. Right now, the money is in AI. Now, obviously, they're using a lot of those billions. to build data centers that we have no idea. where those are all going. But $25 million here, $25 million there. Elon Musk puts $350 million into political campaigns. The amount of money that's flowing from the tech sector. is like nothing we've ever seen before.

Do you think that's had the effect that maybe the AI. companies want, which is--. to regulate us would be-- they've. portrayed it as national security risk. They've portrayed it as it would. cause us to lose to China. Has that been effective? Or is it that they're overwhelmed by not. really understanding the nuts and bolts of AI? PAUL SCHARRE: You mean Congress not understanding. the nuts and bolts of AI. JON STEWART: Congress. That's right.

PAUL SCHARRE: I've actually been super. impressed when I speak with--. I mean, you can always find video clips online. of some congressmen who are not understanding something, but I've been super impressed-- JON STEWART: That's right. I would use them on the show. PAUL SCHARRE: Yeah, you know, like, OK. But I think-- like, I've been impressed. when I speak with members of Congress and their staffs, how knowledgeable many of them are about the technology. and what it can do and its limitations. So I think there's always work to be. done in terms of improving tech literacy in Washington. But I think some of the bigger challenges--. just sort of getting over the hurdles in passing legislation.

and getting agreement, whether that's around federal. regulation of AI or data privacy or social. media or other types of--. that's actually really hard for Washington to do, to pass legislation on these kinds of issues. JON STEWART: Sarah, you know, you spoke to this earlier. It's this great-- here's why I'm very nervous. I've met a couple of these folks, and they do not seem particularly. enamored with humans.

I don't want to say, outright misanthropic. But Peter Thiel was asked, famously, in a conversation, you know, should humans continue, and, he paused, I think for a pretty considerable amount. of time before he went, like, well, you know-- and transhumanism. I once asked Sam Altman about the disruption. that AI's gonna cause to our workforce. and that small amount of time in which. it's going to cause it.

The question was five minutes long, and he just went, we'll be OK. How concerned are you with these great men and how great. they actually are, and what is their connection. to-- do they understand the damage that they also can do, or are they megalomaniacs? SARAH SHOKER: Well, I mean, I can't look. into anyone's heart and mind. But I would say that if they're able to cause harm,

it's only because they are powered by immense wealth. and the high valuations of these companies, and also by institutions that allow. for corporate donations and excessive individual donations. as well. So they're, essentially, enabled. by our current institutional structures. In terms of whether these companies. discuss the downsides--. I mean, I joined in 2021. I left in 2025.

There was a period where I think that was dominant topic--. you know, topic of discussion. Are these tools actually gonna increase productivity? Are they gonna replace tasks? Are they gonna replace workers? Can they enable the proliferation of, potentially, weapons of mass destruction. And there was testing and evaluations that began to try. and answer those questions. So I think, certainly, the researchers at these companies.

have tried to make a concerted effort. But these companies are also complex organizations, and there are always factions that are butting heads. Some people do prefer a low to no-regulatory approach. They don't want to see state legislation. They prefer everything at the federal level. And then there are some who are at these companies. who are actually quite supportive. of state-level legislation. So it really depends. I mean, I think of OpenAI and Anthropic. and, frankly, other companies is often.

going through eras where certain factions. win out over others. And that's what ends up setting the--. the cultural mood of the company. JON STEWART: Do they understand the weight. of what they're making? I can't help but go back to Oppenheimer. And when you have something that looks like it could. be extermination-level type technology, positive and negative--. I mean, if we split the atom one way, we get energy that can power the world.

If we split it this way, you can blow it up, and we all know which one we tried first. And it felt like the people who were making that weapon did it. under the crucible of the Nazis, and so they developed it with this idea. that, well, if the Germans get it, we're all done for. But it was clear that they at least felt the burden of that.

Paul, in your experience, are they. feeling the burden of this? Because what Sarah's talking about is, well, they did go through all that testing. We don't really know what the results of it was. And they seem to have gotten past that reservation. PAUL SCHARRE: I mean, the AI scientists and engineers that I. speak with, particularly those in the frontier labs, are very concerned about AI risk. They, I think, understand better than anybody, actually, the downsides of the technology, the way that it. could be abused, the way that it could just do strange things.

that might be surprising. I think one of the challenges here is there are incentives. for the companies to move fast, to ship their products, because there's this sort of perception of a winner-take-all. dynamic in the marketplace that we have seen. in other tech industries, in operating systems, handsets, in--. JON STEWART: And arms race. PAUL SCHARRE: Well, yeah, I mean, in a way, right? A sort of commercial race to dominate the marketplace. And that does drive incentives. And these companies need a-- they need a lot of money. to build the data centers to train the AI.

So I do think the individuals take it seriously. And I think some of the companies--. I mean, if you look at what Anthropic just did, I mean, they sort of stuck to their guns. on this decision in a way that is gonna. be costly for the company. How costly? I think we just don't know. But they decided to do that. So I do think the companies take. these issues pretty seriously. SARAH SHOKER: And if I could also add, I mean, just at the risk of potentially misspeaking, the testing and evaluations that were done. and are continuing-- continue to be done at these companies--.

they are often released publicly. But, you know, of course, in certain areas. like, you know CBRN-- so that's chemical, biological, radiological, and nuclear testing, and then also cyber, there are greater restrictions placed, placed around what. can be shared with the public. But there are even reports, summary reports about what. that testing looks like. And then a lot of the benchmarks. that are used by AI industry are, in fact, publicly available.

It just so happens that testing and evaluation. of these large language models is still in a relatively. nascent phase, and it's not always clear. what the best way to test these models are if what we're. trying to do is use them as proxies. for social impact or risk. JON STEWART: If you remember the movie War Games--. and it was the first sort of dystopian. look at what would happen when computers take over. It was the Matthew Broderick movie from when I was a kid. It was about a nuclear war game gone wrong,

and the computer just started launching. nuclear weapons at all the different countries. And at the very end, the computer said, the only way to win is not to play. With AI, apparently, it was more. apt to launch nuclear war than humans or standard computers. What do you know about that testing? And is that apocryphal, or is that-- did that really happen?

SARAH SHOKER: I mean, it did really happen. I think a variety of researchers. at academic institutions have now managed. to replicate the findings. The models have a tendency to escalate more aggressively. than humans would. And it's not really clear why the models do that. One theory is that in the training data, a.k.a. the intranet, political scientists have a tendency. to study wartime escalation rather than de-escalation, so that may influence how the models respond to these war.

game-type simulations. But, I mean, that in itself is, of course, a cautionary tale. around using these models for approving the use of force. or for decision making, or frankly, even for war. gaming and simulations. JON STEWART: Is it possible, Paul, that AI, because of how adept it is at creating these targets. and all these other things--. that it actually made going into Iran more appealing,

that before the age of AI, we might've. been more circumspect about the type of attack. that we launched? Are we seeing barriers to military action. fall because of how quickly these models--. can they bring a sense of false confidence? PAUL SCHARRE: I mean, I don't think, today, that's true. Like, I don't think AI was a factor in President. Trump making this decision.

I think it was based, in large part, on the US strike against Iran last summer, against the enrichment program being very successful. and limited, and then the raid against--. to grab Maduro being very successful. and limited and this sort of, like, OK, having a couple--. perception of having a couple wins under his belt. I'm using the military. It seems to be effective. JON STEWART: No downside. Sure, yeah. PAUL SCHARRE: Right? So I think those are probably bigger factors. I think what you're describing could be a risk going forward. So one way this could be a risk is some of the things. that militaries count and try to calculate when they measure.

military power are things that you could see. and you can count. You can count how many tanks somebody. has, how many airplanes, how many ships. Then there are some things that matter. a lot that are hard to count. We see this unfolding in the war in Ukraine. The morale of the troops on the battlefield. The Ukrainians are fighting for their homeland. The Russians are conscripts. They don't want to be there. The leadership, the quality of the unit, cohesion-- those things matter a lot, but they're really hard to measure. So one possibility going forward. is you could see a world where as more and more military power.

gets embedded into software and data and AI, it's kind of hard to measure that. It's like, well, we have this AI, and it's amazing, and it's wonderful, and ours must be great. And there's this-- it becomes harder. for militaries and countries to gauge what their relative level. of power is. And you might see more miscalculation. You might see countries assuming, well, we have this wonderful technology, and we can win, and the war will be over quickly, and we'll all be home, and that turns out not to be true. Countries have made this mistake before.

That's what happened in World War I, right? JON STEWART: Yes. Well, we've made it quite a few times. PAUL SCHARRE: Humans have done this. So I think that is a possibility that could happen, but we're not there today. JON STEWART: Sarah, has anybody studied the confidence--. there's a certain thing in bars. Like, there's a beer courage. You get a couple of shots in. You get a couple of beers. And you're like, it turns out I'm a tremendous MMA fighter, and I think I'm gonna-- you get a weird. confidence from alcohol.

I find you get a weird confidence when you use AI. When you use those models, you tend to be much more assured. in your decision making because you. feel like you have this kind of infallible being behind you. Has anybody studied AI confidence in decision making? 'Cause I feel it when I use it for the mundane tasks.

that I do. SARAH SHOKER: I'm not sure if I've seen anything like that, but that's a really interesting--. that's a really interesting point. I mean, I think what you're referring to--. I've heard some people talk about chatbots or, frankly, any type of statistical analysis. that's used to make decision making as applying. this mathematical veneer. It makes us feel better because it's, therefore, objective,

and it removes the human qualitative or subjective. element to it. You know, the issue that I just keep going back to. is, of course, that these models are not always. going to be reliable because they are, in fact, statistical prediction machines. I mean, they're useful, don't get me wrong, but they're not-- they are inevitably going to output. something that is incorrect. And so being able to keep appropriate human judgment, and to create a system in such a way. that people do not abandon their critical thinking skills.

is a very important facet, I think, to any type of human-machine teaming that we're seeing today. in military AI integration. JON STEWART: Is that something the military's concerned. with, Paul? Because in looking at it from, like--. let's say from an educational standpoint. There's been a lot of studies that show that when kids start. using this, their ability to do that, to think critically, to reason and all that, falls, that it becomes this crutch that, when utilized, you no longer develop those kinds of skills.

and ways of thinking. Does this become a crutch for the military to use/. And the second part of that question is, are we ignoring this whole other area, which is, hey, Claude, or hey, Maven, whatever it is, design me five nerve agents that the world. has never seen before. You know, is that another usage that we're not-- so far, we're only talking about chain of command.

Is there a whole other area we're not. even really thinking about? PAUL SCHARRE: Yeah, well, that is certainly. a risk, the potential for AI to enable biological weapons. and to maybe even lower the barrier. to countries, to non-state groups, to terrorists to do so. Maybe not today, but that's a concern down the road. I think in terms of military usage, I think the military's actually pretty keenly aware of--. for people in uniform, they understand the responsibility. that they have.

If they're gonna launch this missile, they own where that missile goes. And I think there's a couple of concerns. One would be making sure that they really. understand this AI system. Like, what is it gonna do? Is it gonna do something strange? Is it gonna fail? How's that going to work? Ensuring that there's human responsibility. and accountability, I think, is actually quite. important to the military. That's part of the military ethos. But it's challenging for a lot of these AI systems. because it's not like a traditional computer program, where, OK, there's an accident. You go back and you say, oh, this is the line of code.

that caused the problem. Now the answer's embedded in this massive neural network. with billions of connections. And you're like, well, why did it do that? I don't know. And so it gets into these issues of trying to evaluate. the model's performance. What are some conditions in which it might. be biased in certain ways? They tend towards sycophancy, towards, basically, telling you the answer that it. thinks you want to hear. Well, that could really be a problem. in some national security applications--. JON STEWART: Sure. PAUL SCHARRE: --'cause you're an intel analyst, and then you're, like, asking some questions.

And it's like, well, this is what I think you want to hear. So--. JON STEWART: That was Napoleon's whole issue. They were like, sure, boss, Waterloo. What a great idea. You should go there. Now, this is going to sound ridiculous, but does it do, like, what it does with us, which is, would you like me to give you a 10-day bombing plan? Would you like me to add in other targets that. may seem ancillary but might have military-- like-- is it. that casual when it's describing.

what it wants to do next, and how quickly does it do that? SARAH SHOKER: I have never used the Maven smart system, and so I don't actually know what the personality. of the chatbot is. JON STEWART: Or is that what they use Claude for? SARAH SHOKER: Yeah, I mean, you bring up an interesting point, though, in that these models can. be fine-tuned with different personalities. to be either more acquiescing, less acquiescing. We know that users, of course, like. to be fawned over a little bit, but it's possible.

that it's not presenting information in the most. neutral way out there. We just don't know publicly, I don't think. JON STEWART: Right. Do you know, Paul? PAUL SCHARRE: No, I don't know. It's an interesting question. I think one way to think about these models is. they're sort of role playing. They're playing a role that's in their training data. And then that can be fine-tuned by additional training. that they get from the companies. And so that's why you get this sort. of personality-- different personalities. among the different models.

So it's an interesting question of, like, the ones. that the military's using or the intelligence community--. what are they sort of trained on, and are there hidden biases that might be kind of subtle. that are hard to detect? I mean, that's, I think, a difficult problem. JON STEWART: Or not so hard. And I just got a chilling feeling that they're. training it on the Hegseth. And so they plug something in, and the model just pops back, hell yeah! Let's do this! [MUSIC PLAYING].

I told you about my invention, the Crumple. The Crumple-- it is a topographical blanket for dogs, but not the same topography at each time. Every time you throw it on the ground, it changes its topography. It is an amusement park for your dog. to find a place of comfort and warmth, but also with interest. It's not the same old, oh, right, this is where I put my right paw, and this is where I curl my butt.

No, it changes every time. It gives them-- it's like visiting-- it's like EPCOT. It's an EPCOT Center blanket for the dog. And have I started this business yet? I have not. That's right. To the great dismay and disappointment of our audience. and maybe humanity writ large, I have not. started my Crumble business. And I'm gonna tell you why. It's too hard. It's nerve-wracking. I don't know how to do this.

It's daunting. But you know, you got Shopify here. It makes it easy for people. You can get started with-- they got. a design studio, hundreds of templates. Help you build an online store. They can match your style. They also have 24-hour customer service support, world-class. expertise in everything. You can manage inventory, international shipping, processing, returns. It's the commerce platform behind millions of business. around the world, and 10% of all e-commerce. in the United States. It's time to turn those what ifs into [CHA-CHING].

with Shopify today. Sign up for $1-per-month trila today at Shopify.com/tws. Go to Shopify.com/tws. That's Shopify.com/tws. [MUSIC PLAYING]. PAUL SCHARRE: So these are, like, I think some really difficult problems with the technology. that we've got to find ways to work through to use it in ways. that are safe and effective. And I don't think there are easy answers. I think the technology has some strange and new challenges.

associated with it. JON STEWART: Sarah, you strike me as having a really balanced. but also nuanced view of this. What keeps you up at night? Is there something about this that you think about as. particularly challenging? SARAH SHOKER: Yeah, well, there are many challenges. Let me see if I can narrow them down. JON STEWART: All right. All right. Or throw them all out there, and we'll. go through them one by one. SARAH SHOKER: Well, I mean, I think about the challenge. related to global governance. I mean, for over a decade now, over 90-plus member states have.

been meeting at the United Nations to discuss regulating. or the possibility of regulating or even introducing. a treaty instrument that would regulate lethal. autonomous weapons systems. But because of the nature of the forum at which. these discussions are taking place--. it's a consensus-based body. It's at the convention on certain conventional weapons. It's very unlikely that a treaty-based instrument is. even possible in this space. I mean, you can think about how hard.

it is to pick a restaurant with you and your five friends. Now, imagine that you have 90-plus governments trying. to decide on regulating--. JON STEWART: Or pick a restaurant. that could kill all of us. But now, how have they been able to do it--. why can't they use the model that they. used for atomic weapons? SARAH SHOKER: Oh, I see. Well, so-- [SIGHS] I mean, I guess there. are a few reasons for that. So the Convention on Certain Conventional Weapons--. it's really in the name. It is talking about conventional weapons.

And autonomous weapons-- the conversation around them. has really focused on trying to preserve. meaningful human control, to discuss whether that's even. possible, whether they can actually. discriminate between combatants and civilians. And if they can, in fact, discriminate between combatants. and civilians to an extent, then. they technically could be legal under international. humanitarian law, but militaries would still. need to abide by the existing international legal order.

and international humanitarian legal principles. And the good thing about this particular--. about this particular forum is that though, you know, regulation with teeth is probably off the agenda, most states have been able to--. have consented and reaffirmed the norms. around international humanitarian law as applying. to autonomous weapon systems. So that's, I think, also a silver lining as well. JON STEWART: Has anybody kind of gotten it right?

And Paul, I'll ask you because maybe you see ways through this. from being in Washington. But, you know, has the European Union. done a better job with this? Has any governing body? Has any international body? Is there any pathway here that you. see that could help establish at least. the beginning of guardrails. PAUL SCHARRE: I think that, actually, the best avenue we. have is starting at the level of AI hardware. and then building guardrails domestically, eventually.

globally, from the ground up. JON STEWART: Explain the difference between hardware. and the software. PAUL SCHARRE: Right, so the thing about these AI systems. that it's kind of amazing is they require massive amounts. of computing power to train the most capable models. and to deploy them at scale. Now, you can make smaller models that you can deploy. on a laptop, for example, or some other kind. of edge device, smartphones, but they're not as capable. But the most advanced ones are gonna be really big. They're going to have to run in the cloud.

They're going to need really advanced chips. And to deploy them at scale as a society, you're going to need a lot of these really advanced chips. Well, these chips are made in one place on Earth. JON STEWART: Taiwan. PAUL SCHARRE: Taiwan. Now, that does not, on the face of it, seem great that it's an island 100 miles off the Coast. of China that China has pledged to absorb. by force if necessary. But--. JON STEWART: It is considered a drawback, I think. PAUL SCHARRE: It's not the best geographic position. However, these fabs that TSMC has in Taiwan, where the most advanced chips are made,

depend on technology from three countries. in the world, Japan, the Netherlands, and the United States. And without that technology, they cannot. make these advanced chips. And so that starting-- at the hardware level, that actually is, like, a really. narrow choke point to begin to then control the technology. JON STEWART: So the deal we just made with UAE. to give them the chips--. the previous concern had been that they would. then sell the chips to China. Did that just blow a hole in the net?

PAUL SCHARRE: Well, I mean, the bigger question. is, like, what does the global diffusion of this hardware. look like? At the tail end of the Biden administration, literally the last week when they were in office, they dropped this, like, very complicated rule called. diffusion rule that basically would take US export controls. on the most advanced chips to China, which we've. had for several years now. It started under the first Trump administration. And expand that globally.

And it's a kind of tiered system. where, depending on which country you were, you could get so many chips. It was a little complicated. The Trump administration threw that out the window. But I do think that, like, the chips themselves. are a way that we can begin to shape who gets access. to the hardware, who can build the data centers, because they need these chips to do it, and that's a hook for guardrails. So you could say, all right, you want to buy. all these advanced chips? I want to see your domestic regulation surrounding, making. sure that people aren't gonna use these chips. to make a biological weapon.

JON STEWART: Like we did it with enriching. uranium and the things that you would. need to be able to do that. PAUL SCHARRE: That's actually not a bad analogy, right? And we're-- OK, you can get uranium. for peaceful civilian nuclear purposes, not to make a bomb--. JON STEWART: Not to enrich it to that level. PAUL SCHARRE: And we found ways to separate those two. So, like, the idea would be the same thing. You can use these chips for peaceful uses, basically, most. everything, but you can't use it to make, like, an offensive cyber weapon, for example, and put some guardrails around how the technology's used.

JON STEWART: Right, and inspections. Sarah, is there any fear that, like, by the time. we figure this all out, quantum computing is the new standard. and that's pushed us--. so by the time we figure out, OK, these three chips. are crucial to any ability to do that, and then somebody else comes in and says, actually, that's not state of the art anymore--. are we moving so quickly that suddenly. quantum computing is the power that's necessary to.

drive these? And that's a whole different can of worms. SARAH SHOKER: I think you're now learning, in real time, that AI researchers aren't necessarily. experts in quantum computing. And I am the worst person to answer that question. [LAUGHTER]. JON STEWART: 'Cause-- the reason why I bring it up. is I just read an article about it, and I have no idea what it is.

Someone was describing that actually, quantum computing is. going to be wildly preferable to large language models, and I was unable to understand the difference. Knowing that you're not experts in this, is there a sort of remedial version of what. the difference might be? Is it significant? Paul, do you have any idea about this? PAUL SCHARRE: Yeah, I think so. So we're seeing some progress in quantum computing. I don't think it's going to change this picture in AI.

for a couple of reasons. One, quantum computing will become. valuable over time for some very niche. kinds of computation, but not necessarily everything, and I don't think what large language. models or other large neural networks are doing today. It's also, like, the case that-- we're not. seeing in quantum computing this kind. of really rapid exponential growth that we're seeing in AI. So right now, the price performance,

the performance per dollar of AI chips. is doubling about every two years. It's, like, really growing very, very quickly. JON STEWART: That's the productivity of it? PAUL SCHARRE: That's, like, the efficiency of it, right? And so-- so that's really powerful. That's what's allowing this massive growth in edge. It's one of the factors. Data and better algorithms are a factor, too. We're not seeing that kind of exponential growth. in quantum computing. It's really hard science. It's, like, difficult physics. It's much more traditional science, where people. are making incremental gains.

I think we're gonna continue to see progress. But I'm a skeptic that we're going to see this, like, transformative leap ahead in quantum computing. and say, the next five 10 years the way. that we're seeing with AI right now. JON STEWART: So in summation, the drama. that we're seeing between Anthropic and OpenAI--. that's really the soap opera story, and there's not necessarily a lot of there there. It's the general competition between these companies that.

are gonna try and establish primacy. in the realm of AI models. Military application is just one element of the revenue. streams that they're pulling in there. The real-- where you guys are really looking at. is that interface between who are we. gonna end up trusting more, the humans that are developing. the AI models, the humans that are.

running and integrating the AI models, or the models themselves? Would that be where the real tension is going to play out, or the humans that are trying to regulate. and control this incredibly fast-moving industry? SARAH SHOKER: I mean, I think it's fair, but I would just add that it's not only just gonna.

be safety through the technical stack or only safety. through the law or safety through regulation and policy. It is truly going to be an all-of-society effort, and, in part, because AI, again-- general purpose, and it can be used across a variety of applications. So a one-size-fits all approach to safety. is probably not going to work. JON STEWART: Is it akin to the battle against climate change? And if so, we haven't done a great job there, so does that give us a pathway not to follow?

SARAH SHOKER: I mean, I think any pathway. towards AI governance is going to be through cooperation. And I don't want to be overly cynical here, and so I'll try and draw on a positive-- a positive example. JON STEWART: No. Go full cynical. SARAH SHOKER: I'm gonna go--. I'm gonna give you one positive example, just one. JON STEWART: Yeah, come on. SARAH SHOKER: There's plenty of cynicism. JON STEWART: Come on, Sarah. Hit us. SARAH SHOKER: So under the previous administration, they launched the Political Declaration on Military.

Use of AI and Autonomy. And that was a voluntary declaration. with principles and norms, and around 60 countries. signed on to it. And in that declaration, it really. centered international humanitarian law. and also civilian protection. Those conversations can resume. Those diplomatic conversations can resume. Really, what's stopping right now is political will. And that process can, in fact, happen alongside the existing.

UN processes as well. So there isn't really a way out of this that doesn't involve. talking a lot to other people, but there is. something there to build on. JON STEWART: Is the cynical version. of that that international norms and rules seem to be. in disfavor with the current--. I guess what you would call large-power politics. that seem to be playing out? Would that've been-- is that your downside?

SARAH SHOKER: Yeah, I mean, I think. that's probably fair, but--. JON STEWART: We're dancing around lots of things. SARAH SHOKER: It is. But, you know, at the same time, people can continue to demand this through Congress. We mentioned Congress earlier. I see a role here, potentially--. JON STEWART: [LAUGHS]. SARAH SHOKER: --if they want to do something, if they have some free time. JON STEWART: I'm counting on your students, Doctor. I'm counting on your students at Berkeley.

to be able to come up with a--. SARAH SHOKER: They're great. JON STEWART: Yeah, a way through it. Paul, what keeps you up at night, and give us a nice balance between cynicism and optimism. on the way forward that you see. PAUL SCHARRE: Yeah, look, I think. the reality is this technology is gonna. introduce a lot of challenges. How is it used by the military? What are some of the risks in cybersecurity? We talked a little bit about the risks of AI. empowering biological weapons. I mean, there's a lot of risk in the technology. And that's just in, like, the national security space,

not to mention things like job dislocation. I think my takeaway from this fight between Anthropic. and the Pentagon is that these decisions are. too important to be left up to any one of these entities. on their own, for-profit companies. or the government deciding on its own. I think we all have a stake in this world. that we're living in, not just on some of the civilian uses, but even military ones. So, OK, we're not the ones building the killer robots, but if people build them, we're going to live in that world.

We do have a stake in what that looks like. And so there's democratically elected representatives. All of us, your listeners, you know, have a role to play in weighing in in this debate. And if there's a silver lining of this controversy we've seen. over the last couple of weeks, it's. that what would've been a private conversation. is now happening publicly. Kind of messy. A lot of personalities involved on all sides. But it's airing this issue, and then. we're all sort of debating, well, what.

should be these red lines? Hold on a second. That's a good conversation to have, and I'm encouraged that we're having that discussion. JON STEWART: Fantastic. I thank you both, Dr. Sarah Shoker, Senior Research Scholar. at University of California, Berkeley, and Paul Scharre, the Executive Vice President of the Center for New American. Security and author of Four Battlegrounds, Power in the Age of Artificial Intelligence, and also wrote Army of None, which--. I think we can all imagine what that's about. Guys, thank you so much for joining us on this. PAUL SCHARRE: Thank you. Thanks for this discussion. It's been great.

SARAH SHOKER: Thank you for having me on. [MUSIC PLAYING]. JON STEWART: Should I--. did I take the wrong--. should I not be calmer? GILLIAN SPEARE: Yeah, my hair is still on fire. JON STEWART: Still on fire. GILLIAN SPEARE: Sorry to say. LAUREN WALKER: It did not calm me. JON STEWART: Did it help at all that they were still putting it. through a process, that they were-- they still wanted. to filter the problem of AI through international. cooperation or legislative process or other government.

incentives for that, rather than saying, look, we're at one second to doomsday; somebody's got to step in? BRITTANY MEHMEDOVIC: Mm-hmm. GILLIAN SPEARE: I think I was kind of calmed by the idea. that, like, we have these models for, like, other sort. of disarmament that have worked, like what you said. about, like, nuclear weapons, like, the nuclear arms deals, but also, like, the Iran deal that we started--.

JON STEWART: Or the one he used was. biological weapons and chemical weapons, that they create frameworks. GILLIAN SPEARE: Yeah, yeah, exactly. Like, I think that that was encouraging, to think, like, we have these frameworks that we could look at as models. and, like, this isn't totally uncharted territory. And then I think I'm just reminded. that we're not doing that. So that's where the nerves come back in. LAUREN WALKER: I also think freaking people out too much. is not conducive to getting them to act, as we've seen with climate change. I think it's really hampered people's ability to organize.

So I did appreciate that. I also really appreciated-- this is just a personal thing. But over the weekend, I did notice a lot of people framing. Anthropic as the good guys, which I thought. was really odd considering all of the reporting. coming out about these Iran strikes, about the Maduro capture. And I really appreciated--. JON STEWART: That's already been used, yeah. LAUREN WALKER: Yeah, and I really appreciated just that we. had someone who's worked at one of these companies, like, breaking down that it's not a binary, that there's so many considerations for these people. to make. And as you've said, they're not perfect actors.

Everyone makes mistakes. The technology itself makes mistakes. So I just appreciated that nuance. JON STEWART: I also like that what they talked about was. in terms of the usage, it really is, in some ways, a kind of cousin of the way that we use it, in that It's just collating data more quickly. and spitting out those pleasantly formatted, you know--. GILLIAN SPEARE: Yeah, that did not make me feel better.

JON STEWART: Here's five great places you could bomb. BRITTANY MEHMEDOVIC: But Jon, how do you use AI? JON STEWART: Oh, like, I'll go in to AI and be like, OK, I want to find the best, like-- who's got the best. pizza in blah, blah, blah? Like, generally, I use it for, like, those types of recreational-- like, I want to try this sport. You know, what's the stuff I might need? How would it be hard to get into it? Like, that sort of shit. And it's effective. You know, here's five places you. could go to get started with paddle.

tennis, that kind of shit. GILLIAN SPEARE: And then the government asks, what's. the best pizza, and then bombs those places, and I don't know that that makes me feel better--. JON STEWART: Oh, God. GILLIAN SPEARE: --you know? JON STEWART: No, but here's-- so here's why, though. Here's what I'm going to say. So in the same way that I look at autonomous cars as, like, dystopian, almost everything I've read about it. is that it would make it safer--. that human error is actually at a higher. fraction than the other.

Now, obviously, letting it just make decisions on its own. without any kind of interaction makes me uncomfortable. But I guess the point is like, how great are we, actually? GILLIAN SPEAR: At driving? Not good. [CHUCKLES]. JON STEWART: We bombed shit randomly before computers. ever happened. Like, what was our track record on bombing? Like, not so fucking great. Like, we dropped two atomic weapons on Japan.

Would the computer do worse than that? Like, that's my only point is, like, are we elevating humanity to a higher status. than we've earned? GILLIAN SPEAR: I think that the issue is that it makes doing. these things so much faster, so maybe it would have dropped. five atomic bombs on Japan. I don't know. JON STEWART: [LAUGHS] JON STEWART: But if we were to look at the charts, that seems to be the way that it would go. LAUREN WALKER: Also, in the Waymo case, there was reporting recently that people in the Philippines.

were intervening. We're just not there yet. JON STEWART: Oh, really. OK. Yeah, I didn't know that. Yeah, I'm assuming that. I guess what I was saying is, sometimes. in the battle between man and machine, we tend to look at man a little bit. more favorably than maybe man has earned. But I absolutely get that. And again, to that point, one of my biggest fears about AI. continues to be what appear to be. the pathological personalities of the people.

that run those companies. BRITTANY MEHMEDOVIC: Same. Yeah. LAUREN WALKER: I was thinking about that. in terms of the attitudes and the personalities. of these chatbots when you were talking about that. in the conversation and just remembering-- like six. months ago, though, Grok or whatever company. is above Grok for Elon. JON STEWART: Elon-- yeah, yeah. LAUREN WALKER: Yeah. Made a contract with the government. for like $0.42 for like a year and a half, they could integrate Grok into government.

And apparently there's like posters around DOD. with Hegseth's AI-generated mug saying, "We want you to use AI.". Like, they really want to get government. hooked on their product. And I just imagine, like, someone in government being. like, to Gillian's point a little bit--. like, OK, there's flooding in Texas-- what do we do? And they're like, well, Hitler is the best person. to deal with this, you know? JON STEWART: You're thinking they. contracted with MechaHitler as opposed to just normal Grok.

No, you're right. And those guys manipulate algorithms, and they are ideologues. They have a-- a lot of them are transhumanist. Like, they are leading us down a path that is not favorable, I think. GILLIAN SPEAR: Yeah. When Sarah said, I don't know the personality of Maven, like, my stomach dropped. I was like, oh my God, we can't be. talking about the personality of weapons. That's so dark. BRITTANY MEHMEDOVIC: Or when she talked about Sam Altman's. heart and mind, I was like, does he.

have either of those things? JON STEWART: Right, hearts or minds. But it is like, I don't know the personality. of the Palantir-generated war autonomy system. GILLIAN SPEAR: I don't know the personality of the Tomahawk. missile, but that's--. JON STEWART: Some wild shit, man, and not going away. But I loved how measured they were, and I loved how they helped us through there. Brittany, what do the people have for us this week? BRITTANY MEHMEDOVIC: Sure. Jon, we're still going to get Greenland, right? JON STEWART: Oh, I think we already have it. We've already won. Like everything else in the Trump administration,

we've already-- not only-- it's like with the Iran war, we've won and we're doing more. It's, we are Schrödinger's country. We exist in all different-- we have Greenland and don't have. it at the same time, but they respect our. unique and unparalleled power. And so absolutely, we have it and don't have it, and could do whatever we want with it. and won't, because of our largesse and--.

I don't know. [LAUGHTER]. LAUREN WALKER: It's like how the Iran. war is almost complete, but also could go on for. as long as it takes. GILLIAN SPEAR: We live in this middle space, yeah. JON STEWART: Almost complete and never done. And we are going to only stop at unconditional surrender, and we've already stopped. We are Schrödinger's country, and it is only. the beholder that determines where. we are on the existence plane. GILLIAN SPEAR: We obliterated the nuclear program, but they're one day away from it.

BRITTANY MEHMEDOVIC: Hard to keep up, guys. JON STEWART: Very hard to keep up. Is that it for them? BRITTANY MEHMEDOVIC: One more. JON STEWART: One more. BRITTANY MEHMEDOVIC: Jon, why does everyone. ask you where to get pizza? JON STEWART: Because I am considered. one of the world's leading--. and this is recognized around the world--. any of the larger pizza conglomerates, the pizza--. they recognize-- you know what? I think because of that rant I did on deep dish pizza. in Chicago. I think that's the only-- oh, and we.

did something on Trump eating it with a knife and fork. BRITTANY MEHMEDOVIC: OK. JON STEWART: And so those two things--. there is no real accreditation, other than Portnoy's. rating system, for pizza. So oftentimes, non-experts are elevated--. BRITTANY MEHMEDOVIC: Sure. JON STEWART: --to that position. GILLIAN SPEAR: I mean, little do they know. you're just asking the AI. LAUREN WALKER: I was about to say, you revealed earlier. JON STEWART: Can I tell you the truth? Like, my world there is so small I go.

to Joe's on Carmine's if I want a slice. and I go to John's on Bleecker if I want a pie. And that's kind of my world-- like, as you guys know me, my world is small. GILLIAN SPEAR: You know what you like. JON STEWART: I am not a man who is out there. It's the same clothes. I have eaten-- I shouldn't even be telling you guys this. I eat the same lunch every day when I go in. to work at The Daily Show, and I've done it since I've been. back-- the exact same lunch. BRITTANY MEHMEDOVIC: Well, what is it?

JON STEWART: I'm embarrassed to say. GILLIAN SPEAR: Oh, no, come on. LAUREN WALKER: Was it, like, girl lunch? JON STEWART: What's a-- what's a girl lunch? LAUREN WALKER: It's girl dinner. Just like little bits of everything. I'm just very curious-- trying to prompt you. [LAUGHTER] BRITTANY MEHMEDOVIC: Yeah. We are not ending the podcast until you tell us. JON STEWART: That's what a girl lunch is, is little bits of everything? LAUREN WALKER: Yeah, you don't have to really cook. JON STEWART: Yeah, no--. I order-- I don't make it. Let me just be very clear. When I go to work at The Daily Show, I don't cook.

I call out, and I get a bean and cheese tostada. GILLIAN SPEAR: OK. We have to stop talking about lunch during these recordings. [LAUGHTER]. JON STEWART: With all that setup--. I know that that was a bit of a letdown in terms of--. I should probably be more particular, like, "I get every day the same thing, a quarter of a lime spritzed lightly on steamed cod.". It's a bean and cheese tostada.

And the only difference is it comes with jalapeno, and I generally say no jalapeno. And I've done it every time for three years. BRITTANY MEHMEDOVIC: It's very Jennifer Aniston of you. JON STEWART: Is it really? Does she get a tostada? Is she a tostada lady? BRITTANY MEHMEDOVIC: Well--. LAUREN WALKER: No, definitely not. GILLIAN SPEAR: Does she look like a tostada lady? LAUREN WALKER: She doesn't look like a carb lady. BRITTANY MEHMEDOVIC: Not a tostada. But when they were doing Friends, she would eat the same lunch every day. JON STEWART: Oh, is that true? Now, what did she get? BRITTANY MEHMEDOVIC: It was like a chef salad.

I actually know exactly what it is, and I'm not going to tell you because I don't want. to look like a crazy person. JON STEWART: I am the Jennifer Aniston of late night. I think people have always-- but you guys know that about--. on my 50th birthday, The Daily Show bought me--. we had one of those staff all-hands. meetings down in the studio, and they. had a box sitting on a table. And I opened the box, and I pulled out--. it was a T-shirt, a long johns shirt, khaki pants, hiking. boots, and a thing-- and it was exactly. what I was wearing that day.

And I was flattered and humiliated. all in the same moment. But I am a creature of very lame habits. But I hope-- man, what information they got today. Very, very, very nice. Lovely-- a lovely program. Thrilling and chilling and nerve. wracking and all those different things. Brittany, how do they keep in touch with us? BRITTANY MEHMEDOVIC: Twitter, we are @WeeklyShowPod. Instagram, Threads, TikTok, Bluesky, we are @WeeklyShowPodcast.

And you can like, subscribe, and comment. on our YouTube channel, The Weekly Show with Jon Stewart. JON STEWART: Beautiful. As always, guys, thank you guys so. much for the incredible preparation. you did on this episode. Lead producer Lauren Walker, producer Brittany Mehmedovic, producer Gillian Spear, video editor and engineer. Rob Vitolo, audio editor and engineer Nicole Boyce, and our executive producers, Chris McShane and Caity Gray. We will see you next time. [VOCALIZING] The Weekly Show with Jon Stewart. is a Comedy Central podcast that's produced by Paramount.

Audio and Busboy Productions. [MUSIC PLAYING].