Dmitri Dolgov: Waymo and the Future of Self-Driving Cars | Lex Fridman Podcast #147

the following is a conversation with. dimitri dalgov. the cto of waymo which is an autonomous. driving company that started. as google's self-driving car project in. 2009 and became waymo in 2016. dimitri was there all along waymo is. currently leading in the fully. autonomous vehicle space. in that they actually have an at-scale. deployment of. publicly accessible autonomous vehicles. driving passengers around. with no safety driver with nobody.

in the driver's seat this to me is an. incredible. accomplishment of engineering on one of. the most difficult. and exciting artificial intelligence. challenges of the 21st century. quick mention of a sponsor followed by. some thoughts related to the episode. thank you to trial labs a company that. helps businesses. apply machine learning to solve real. world problems. blinkist an app i use for reading. through summaries of books. better help online therapy with a.

licensed professional. and cash app the app i use to send money. to friends. please check out the sponsors in the. description to get a discount and to. support this podcast. as a side note let me say that. autonomous and semi-autonomous driving. was the focus of my work at mit and it's. a problem space that i find. fascinating and full of open questions. from both a robotics. and a human psychology perspective. there's quite a bit. that i could say here about my. experiences in academia on this topic.

that revealed to me let's say the. less admirable size of human beings but. i choose to focus on the positive on. solutions. i'm brilliant engineers like dimitri and. the team at waymo. who work tirelessly to innovate and to. build amazing technology. that will define our future because of. dimitri. and others like him i'm excited for this. future. and who knows perhaps i too will help. contribute something of value.

to it if you enjoy this thing subscribe. on youtube review it with five stars and. up a podcast. follow on spotify support on patreon or. connect with me on twitter. at lex friedman and now here's my. conversation. with dmitry dolgov when did you first. fall in love with. robotics or even computer science more. in general computer science first. at a fairly young age and robotics. happened much later. um i i think my first.

interesting introduction to computers. was. in the late 80s uh. when we got our first computer i think. was an. uh an ibm i think ibm it remember those. things that had like a turbo button in. the front you'd press it and you know. make the thing go faster did they. already have floppy disks. yeah yeah yeah like the the five point. four inch ones i think there's a bigger. inch so good when something then five.

inches and three inches. yeah i think that was the five i don't i. maybe that was before that was the giant. plates and i didn't get that. uh but it was definitely not the not the. three inch ones. uh anyway so that that you know we got. that uh. computer i spent the first a few. months just you know playing video games. uh as you would expect. i got bored of that so i started messing. around. and trying to figure out how to make the. thing. do other stuff got into.

exploring know programming and a couple. of years. later i got to a point where um i. actually wrote a game. a lot of games and a game developer a. japanese game developer. actually offered to buy it for me for. you know a few hundred bucks but you. know for for a kid. yeah in russia that's a big deal that's. a big deal yeah uh i did not take the. deal. wow integrity yeah i i instead uh. stupidity yes that was not the most. acute financial move that i made in my. life you know looking back at it now.

uh i instead put it well you know i had. a reason i i put it online. uh it was what did you call it back in. the days it was a freeware i think. right it was not open source but you. could upload the binaries you put the. game online and the idea was that you. know people like it and then they you. know. contribute and they send you little. donations right so i did my quick math. of like you know my of course you know. thousands and millions of people are. going to play my game send me a couple. of bucks a piece you know. should definitely do that as i said not. not the best. remember what language it was what.

programming it was about. which what pascal pascal and they had a. graphical component. so that text based yeah yeah it was uh. like. i think 320 by 200 whatever it was. i think that kind of the earlier that's. the cga resolution right. and i actually think the reason why this. company wanted to buy it is not like the. fancy graphics or the implementation. it was maybe the idea uh of my actual. game. the idea of the game okay well one of. the things. it's so funny i used to play this game.

called golden axe. and the simplicity of the graphics. and something about the simplicity of. the music like. it still haunts me i don't know if. that's a childhood thing i don't know if. that's the same thing for call of duty. these days for young kids. but i still think that the. simple one the games are simple that. simple purity makes for. like allows your imagination to take. over and thereby creating a more magical.

experience. like now with better and better graphics. it feels like. your imagination doesn't get to uh. create worlds. which is kind of interesting um it could. be just an. old man on a porch like waving at kids. these days that have no respect but. i still think that graphics almost get. in the way of the experience. i don't know flippy bird yeah. i don't know if the imagination gets. closed i don't. yeah but that that's more about games.

that up like that's more like tetris. world where they. optimally masterfully. like create a fun. short-term dopamine experience versus. i'm more referring to like. role-playing games where there's like a. story you can live in it for. months or years um like uh. there's an elder scroll series which is. probably my favorite set of games. that was a magical experience and then. the graphics were terrible. the characters were all randomly.

generated but they're i don't know. that's. it pulls you in there's a story it's. like an. interactive version of an elder scrolls. tolkien world and you get to live in it. i don't know. i miss it it's one of the things that. suck about being an. adult is there's no you have to live in. the real world as opposed to. the elder scrolls world you know. whatever brings you joy right. minecraft right minecraft is a great. example you create like it's not the. fancy graphics but it's the.

creation of your own worlds yeah that. one is crazy. you know one of the pitches for being a. parent that people tell me. is that you can like use the excuse of. parenting to. to go back into the video game world and. like. like that's like you know father-son. father-daughter time. but really you just get to play video. games with your kids so anyway. at that time did you have any ridiculous. ambitious dreams of. where as a creator you might go as an.

engineer did you. what did you think of yourself as as an. engineer as a tinkerer or did you want. to be like an astronaut. or something like that you know i'm. tempted to make something up about you. know robots. uh engineering or you know mysteries of. the universe but. that's not the actual memory that pops. into my mind uh when you when you ask me. about childhood dreams so i'll actually. share the real thing. uh when i was. maybe four or five years old i you know. as well do i. thought about you know what i wanted to.

do when i grow up and i had this dream. of being. a traffic control cop you know they. don't have those today's i think but you. know. back in the 80s and you know in russia i. you probably are familiar with that legs. they had these. uh you know police officers they would. stand in the middle of an intersection. all day. and they would have their like striped. black and white batons that they would. use to you know control the flow of. traffic. and you know for whatever reason i was. strangely infatuated with this whole.

process and like that that was my dream. uh that's what i wanted to do when i. grew up and you know my. parents uh both physics profs by the way. i think we're. you know a little concerned uh with that. level of ambition coming from their. child. yeah uh you know that age well that it's. an interesting i don't know if. you can relate but i very much love that. idea. i have a ocd nature that i think lends. itself. very close to the engineering mindset. which is you want to kind of.

optimize you know solve a problem by. create creating an automated. solution like like a set of rules the. set of rules you could follow. and then thereby make it ultra efficient. i don't know if that's. it was it of that nature i certainly. have that there's like fact. like simcity and factory building games. all those kinds of things. kind of speak to that engineering. mindset or did you just like the uniform. i think it was more of the latter i. think it was the uniform and.

you know the the striped baton that made. cars go. in the right direction but i guess you. know it is. i did end up uh i guess uh you know. working on the transportation industry. one way or another no uniform though but. that's right. maybe it was my you know deep inner. infatuation with. the you know traffic control batons that. led to this. career okay what uh when did you. when was the leap from programming to.

robotics that happened later that was. after grad school. uh after and actually the most. self-driving cars was i think my first. real hands-on introduction to robotics. but i i never. really had that much hands-on experience. in school and training i you know worked. on applied math. and physics then in college i did more. of abstract computer science. and it was after grad school that i. really got. involved in robotics which was actually. self-driving cars and.

you know that was a big big flip what uh. well grad school. so i went to grad school in michigan and. then i did a postdoc at stanford uh. which is. that was the postdoc where i got to play. with celebrating cars. yeah so we'll return there let's go back. to uh to moscow so i. you know for episode 100 i talked to my. dad and also i grew up with my dad. i guess uh. so i had to put up with him for many. years and uh. he he went to the fistiach.

or mipt it's weird to say in english. because i've heard all this in russian. moscow institute of physics and. technology and to me that was like. i met some super interesting as a child. i met some super interesting characters. it felt to me like the greatest. university in the world. the most elite university in the world. and just the the people. that i met that came out of there were. like. not only brilliant but also special.

humans. it seems like that place really tested. the soul. uh both like in terms of technically and. like spiritually. so that could be just the. romanticization of that place i'm not. sure but so maybe you can speak to it. but did is it correct to say that you. spent some time. at fistia yeah that's right six years i. got my bachelor's and master's and. physics and math there and it's actually. interesting because my. my dad actually both my parents uh went.

there and. i think all the stories that i heard. like just like you alex. uh growing up about the place and you. know how interesting and special and. magical it was i think that was a. significant maybe the main reason. uh i wanted to go there uh for college. uh. enough so that i actually went back to. russia. from the us i graduated high school in. the us um you went. back there i went back there yeah that. wow exactly the reaction. most of my peers in college had but you.

know. perhaps a little bit stronger that like. you know point me out as this crazy kid. were your parents supportive of that. yeah yeah i came to your previous. question. they uh they supported me and you know. letting me. kind of pursue my passions and the you. know things that. that's a bold move wow what was it like. there it was interesting you know. definitely. fairly hardcore on the fundamentals of. math. and physics and you know lots of good. memories. from you know from those times so okay. so stanford how did you get into. autonomous vehicles.

i had the great fortune. and great honor to join stanford's darpa. urban challenge team. in 2006 there this was a third in the. sequence of the. darpa challenges their two grand. challenges. prior to that and then in 2007 they held. the darpa. urban challenge so you know i was doing. my postdoc i had i joined. the team and uh worked. on motion planning uh for you know.

that competition so okay so for people. who might not know i know. from from a certain perspective. autonomous vehicles is a funny world. in a certain circle of people everybody. knows everything and then a certain. circle. uh nobody knows anything in terms of. general public. so it's interesting it's it's a good. question what to talk about but. i do think that the urban challenge is. worth revisiting it's a fun little. challenge. one that in first it like sparked so.

much so many incredible. minds to focus on one of the hardest. problems of our time. in artificial intelligence so that's. it's a success from a perspective of a. single little challenge. but can you talk about like what did the. challenge involve. so were there pedestrians were there. other cars. what was the goal uh who was on the team. how long did it take any fun fun sort of. specs sure so the way the. the challenge was constructed in just a.

little bit of background and as i. mentioned this was the. third uh competition in that series the. first two uh were the grand challenge. called the grand challenge the goal. there was to just. drive in a completely static environment. you know you had to drive in the desert. uh. that was very successful so then darpa. followed. with what they called the urban. challenge where the goal was to have you. know build vehicles that could operate. in more dynamic environments and share. them with other vehicles there were no. pedestrians. there but what darpa did is they took.

over an abandoned. air force base and it was kind of like a. little fake city. that they built out there and they had. a bunch of uh robots uh you know cars. that were autonomous uh in there all at. the same time uh mixed in. with other vehicles driven by. professional uh drivers. and each car had a mission and so. there's a crude map that they received. at the beginning and they had a mission. and go you know here and then there and.

over here. um and they kind of all were sharing. this environment at the same time they. interact to interact with each other. they had to interact with the human. drivers. there's this very first very rudimentary. um version of uh a self-driving car. that you know could operate on and on. yeah in an environment you know shared. with other dynamic actors. that as you said you're really in many. ways. you know kick started this whole. industry okay so. who was on the team and how did you do i.

forget. uh we came in second uh. perhaps that was my contribution to the. team i think the stanford team came in. first in the darpa challenge uh but then. i joined the team and you know. you were the one with the bug in the. code i mean do you have sort of memories. of some particularly challenging things. or. you know one of the cool things it's not. a you know this isn't a product this. isn't the thing that. uh you know it there's you have a little. bit more freedom to experiment so you. can take risks and there's.

uh so you can make mistakes uh so is. there interesting mistakes. is there interesting challenges that. stand out to you or some like taught you. um a good technical lesson or a good. philosophical lesson. from that time yeah uh you know. definitely definitely a very. memorable time not really a challenge. but like one of the. most vivid memories that i have from the. time. and i think that was actually one of the. days that. really got me hooked on this whole field.

was. the first time i got to run my software. on the car and i was working on a part. of. our planning algorithm uh that had to. navigate in parking lots so it's you. know something that you know called free. space. motion planning so the very first. version of that uh. you know we tried on the car it was on. stanford's campus uh. in the middle of the night and you know. i had this little you know course. constructed with cones. uh in the middle of a parking lot so. we're there like 3 a.m you know by the. time we got the code to.

you know you know compile and turn over. and you know it drove like i actually. did something quite reasonable. and you know it was of course very buggy. at the time. and had all kinds of problems but it was. pretty darn magical i remember going. back. and you know later at night trying to. fall asleep and just. being unable to fall asleep for you know. the rest of the night uh. just my mind was blown and yeah that. that's what i've been you know doing.

ever since for more than a decade. uh in terms of challenges and uh you. know. interesting memories like on the day of. the competition i it was been pretty. nerve-wracking. i remember standing there with mike. montemerlo who was. the software lead and wrote most of the. code i think i did one little part. of the planner mike you know incredibly. that you know. pretty much the rest of it uh with with. you know a bunch of other incredible. people. but i remember standing on the day of. the competition uh. you know watching the car you know with. mike and your cars are.

completely empty right they're all there. lined up in the beginning of the race. and then you know darpa sends them you. know on their mission one by one. so they leave and like you just they. have these sirens. they all had their different silence. silence right each iron had its own. personality if you will so you know off. they go and you don't see them you just. kind of and then every once in a while. they you know come a little bit closer. to where. the audience is and you can kind of hear. you know the sound of your car and you. know it seems to be moving along so that. you know gives you hope. and then you know it goes away and you.

can't hear it for too long you start. getting anxious right just a little bit. like you know sending your kids to. college and like you know. kind of you invested in them you hope. you you you you you build it properly. but. like it's still anxiety-inducing uh so. that was. an incredibly uh fun few days. in terms of you know bugs as you. mentioned you know one. that was my bug that caused us the loss. of the first place. is still a debate that you occasionally. have with people on the cmu team scene. you came first. i should mention uh that you haven't.

heard of them. but yeah no it's something you know it's. a small school it's it's. really a glitch that you know they. happen to succeed at something robotics. related very scenic though. most people go there for the scenery um. yeah. that's right it's a beautiful campus. unlike stanford so for people yeah. that's true i like stanford for people. who don't know cemu is one of the great. robotics and sort of artificial. intelligence universities in the world. cmu carnegie mellon university okay. sorry go ahead. good good psa so in the part.

that i contributed to which was. navigating parking lots. and the way you know that part of the. mission worked is. yeah you in a parking lot you would get. from darpa. an outline of the map you can get this. you know giant polygon. that defined the perimeter of the. parking lot uh and there would be an. entrance and you know so maybe. multiple entries or access to it and. then you would get a goal. within that open space xy.

you know heading where the car had to. park it had no information about the. optical selling obstacles that. the car might encounter there so it had. to navigate uh kind of completely free. space. from the entrance to the parking lot. into. that parking space and then uh. once parked there it had to. exit the parking lot while of course. encountering and reasoning about all the. obstacles that it encounters in real. time. so uh. our interpretation or at least my. interpretation of the rules was that you.

had to reverse. out of the parking spot and that's what. our cars did even if there's no optical. in front. that's not what seam used car did and it. just kind of. drove right through so there's still a. debate and of course you know if you. stop and then reverse out and go out the. different way that cost you some time. right so there's still. a debate whether you know it was my poor. implementation that cost us extra time. or whether. it was you know cmu violating. an important rule of the competition and. you know i have my own uh opinion here.

in terms of other bugs and like i i have. to apologize to mike montemerla. uh for sharing this on air but it is. actually uh one of the more memorable. ones. uh and it's something that's kind of. become a bit of a. a metaphor had a label in the industry. uh since then i think. at least in some circles it's called the. victory circle or victory lap. um and uh our cars. did that so in one of the missions in. the urban challenge in one of the. courses uh there was this big. oval right by the start and finish of.

the race so darpa had. a lot of the missions would finish kind. of in that same location. and it was pretty cool because you could. see the cars come by and kind of finish. that part lag of the trip without that. leg of the mission and then you know go. on. and you know finish the rest of it uh. and other vehicles would you know come. hit. their waypoint and you know exit the. oval and. off they would go our car in the hand. which hit the checkpoint. and then it would do an extra lap around. the awful and only then you know.

leave and go on its merry way so over. the course of you know the full day it. accumulated. uh some extra time and the problem was. that we had a bug where. it wouldn't you know start reasoning. about the next waypoint and plan around. to get to that next point until it hit. the previous one and in that particular. case by the time you hit the. that that one it was too late for us to. consider the next one and kind of. make a lane change so that every time it. would do like an extra lap so. that's the the stanford victory lap. oh there's there's i feel like there's. something philosophically profound in.

there somehow but uh. i mean ultimately everybody is a winner. in that kind of competition. and it led to sort of famously to the. creation of uh. google self-driving car project and now. waymo. so can we uh give an overview of how is. way more born. how's the google self-driving car. project born what's the what is the. mission. what is the hope what is it is the. engineering kind of uh set of milestones.

that it seeks to accomplish there's a. lot of questions in there. uh yeah i think you're right it kind of. the. urban challenge and the upper and. previous darpa grand challenges uh kind. of. led i think to a very large you know. degree to that next step and you know. larry and sergey um. uh larry page and sergey brin uh uh. google hunter scores. uh uh saw that competition and believed. in the technology. so now the google self-driving car. project was born. you know at that time and we started in.

2009 it was a pretty small. group of us about a dozen people who. came together. uh to to work on on this project at. google. at that time we saw an. you know that incredible early result. in the darpa urban challenge i think. we're all incredibly excited. about where we got to and we believed in. the future of the technology but we. still had a very. rudimentary understanding of the problem.

space so the. first goal of this project in 2009 was. to really. better understand what we're up against. and you know with that. goal in mind when we started the project. we created a few milestones for. ourselves. that maximized learnings. well the two milestones were you know uh. one was to drive a hundred thousand. miles. in autonomous mode which was at that. time you know orders of magnitude that. more than anybody has ever done and the.

second milestone was to drive. 10 routes uh each one was 100 miles long. they were specifically chosen to become. extra spicy you know extra complicated. and sample the full complexity. of the that that domain. um and you had to drive each one. from beginning to end with no. intervention no human intervention so. you get to the beginning of the course. uh you. you press the the button that include. engage in autonomy.

and you had to you know go for 100 miles. you know beginning to end. uh with no interventions um and. it sampled again the full complexity of. driving conditions. some were on freeways we had one route. that went all through all the freeways. and all the. bridges in the bay area you know we had. some that went around lake tahoe and. kind of mountainous. roads we had some that drove through. dense urban. um environments like in downtown palo. alto. and through san francisco so it was.

incredibly. uh interesting uh to work on. and it uh it took us. just under two years about a year and a. half a little bit more. to finish both of these milestones and. in that process. uh yeah hey it was an incredible amount. of fun probably the most. fun i had in my professional career and. because you're just learning so much you. are you know the goal here is to learn. and prototype you're not yet starting to. build a production system right so you. just.

you were you know this is when you're. kind of you know working 24 7 and you're. hacking things together and you also. don't know. how hard this is i mean it's the point. like. so i mean that's an ambitious if i put. myself in that mindset even still. that's a really ambitious set of goals. like just those two. picking picking 10 different. difficult spicy challenges. and then having zero interventions so. like not saying gradually we're going to.

like you know over a period of 10 years. we're going to have a bunch of roots and. gradually reduce the number of. interventions. you know would that literally says like. by as soon as possible we want to have. zero. and on hard roads so like to me if i was. facing that. it's unclear that whether that takes two. years or whether that takes 20 years. i mean under two i guess that speaks to. a really big.

difference between doing something once. and having a prototype. uh where you are going after you know. learning about the problem. versus how you go about engineering a. product. that you know where you look at uh you. know you. properly do evaluation you look at. metrics you you know drive down and. you're confident that you can do that at. home. and i guess that's the you know why it. took a dozen people. uh you know 16 months or a little bit. more than that. uh back in 2009 and 2010.

and with the technology of you know the. more than a decade ago. that amount of time to achieve that. milestone. of 10 routes 100 miles each and no. interventions. and you know it. took us a little bit longer to get to. you know a full. driverless product that customers use. that's another really important moment. is there some. memories of technical lessons or just. one like what did you learn about the.

problem of driving from that experience. i mean we could we can now talk about. like what you learned from. modern day waymo but i feel like you may. have learned some profound things. in those early days even more so. because it feels like what waymo is now. is to trying to. you know how to do scale how to make. sure you create a product how to make. sure it's like safety and all those. things which is. all fascinating challenges but like you. were facing the more fundamental. philosophical problem of driving in.

those early days like. what the hell is driving as an. autonomous. or maybe i'm again romanticizing it but. is it. is there uh is there some valuable. lessons you picked up over there. at those two years uh a ton. the most important one is probably that. we believe that it's doable and we've. gotten. uh far enough into the problem that uh.

you know. we had a i think only a glimpse of the. true complexity uh. of the the domain yeah it's a little bit. like you know climbing a mountain where. you kind of. see the next peak and you think that's. kind of the summit but then you get to. that and you kind of see that that this. is just. the start of the journey uh but we've. tried. we've sampled enough of the problem. space and we've made. enough rapid uh success even you know. with technology. of 2009 2010 that it gave us confidence. to then you know pursue this as.

a real product so okay so the next step. you mentioned the the milestones that. you had in the. in those two years what are the next. milestones that then led to the creation. of waymo and beyond. now it was a really interesting journey. and. waymo came a little bit later uh. then you know we completed those. milestones in 2010. that was the pivot when we decided to. focus. on actually building a product yeah.

using this technology. uh the initial couple years after that. we were focused. on a freeway you know what you would. call a driver assist. uh maybe an l3 driver assist uh. program then around 2013 we've learned. enough uh about the space and the. thought. more deeply about you know the. product that we wanted to build that we. pivoted uh we pivoted towards. of this vision of you know building a.

driver. and deploying it fully driverless. vehicles without a person and that. that's the path that we've been. on since then and uh very it was exactly. the right decision for us. so there was a moment where you also. considered like what is the right. trajectory here. what is the right role of automation in. the in the task of driving there's still. it wasn't from the early days obviously. you want to go fully autonomous. from the early days it was not i think. it was in 20 around 2013 maybe. that we've that became very clear.

and we made that pivot and it also. became very clear uh. and that it's even the way you go. building. a driver assist system is you know. fundamentally different from how you go. building a fully driverless vehicle so. you know we've. uh pivoted towards the ladder. and that's what we've been working on. ever since and. so that was around 2013 then. there's sequence of really meaningful. for us really important. defining milestones since then in.

the 2015. we had our first. actually the world's first fully. driverless. trade on uh public roads it was. in a custom-built vehicle that we had we. must have seen this we called them the. firefly that you know. funny-looking marshmallow looking thing. um and we. put a passenger uh his name was steve. mann a great. uh friend of our project from the early.

days uh the the man happens to be. uh blind so we put him in that vehicle. uh the car had no steering wheel no. pedals it was an uncontrolled. environment. um you know no you know lead or chase. cars no police escorts um. and uh you know we did that trip a few. times in austin texas. so that was a really big milestone well. that was in austin yeah cool. okay um and you know we only but at that. time we're only it took a tremendous. amount of engineering it took a. tremendous amount of validation. uh to get to that point uh but you know.

we. only did it a few times i only did that. it was a fixed route. it was not kind of a controlled. environment but it was a fixed route and. we only did a few times. uh then uh in. uh 2016 uh end of 2016 beginning of 2017. is. when we founded waymo uh the company. that's when. we kind of that was the next phase of. the project where. i wanted uh we believed in kind of the. commercial. uh vision of this technology and it made.

sense to create an independent entity. you know within that. alphabet umbrella to pursue uh this. product. at scale beyond that in 2017 later in. 2017. was another really a huge. step for us really big milestone where. we started it was october. of 2017. where when we. started regular uh driverless. operations on public roads that first. day of operations we drove.

uh in one day and that first day 100. miles and. you know driverless fashion and then. we've the most the most important thing. about that milestone was not that you. know 100 miles in one day. but that it was the start of kind of. regular ongoing. driverless operations can we say. driverless it means no driver. that's exactly right so on that first. day we actually had a mix and. up uh in some uh we didn't want to like. you know be on youtube on twitter that. same day so in. uh and many of the rides we had somebody.

in the driver's seat but they could not. disengage like the car. it's not disengaged but actually on that. first day. uh some of the miles were driven and. just completely. empty driver's seat and this is the key. distinction that i think people don't. realize. it's you know that oftentimes when you. talk about autonomous vehicles. you're there's often a driver in the. seat that's ready to. uh to take over uh what's called a. safety driver.

and then waymo is really one of the. only companies that i'm aware of or at. least as like boldly and. carefully and all and all that is. actually has. cases and now we'll talk about more and. more where there is. literally no driver so that that's. another. the the interesting case of where the. driver is not supposed to disengage. that's like a. nice middle ground if they're still. there but they're not supposed to. disengage. but really there's the case when there's. no.

okay there's something magical about. there being nobody in the driver's seat. like just like to me you mentioned um. the first time you wrote some code for. free space navigation of the parking lot. that was like a magical moment. to me just sort of an as an observer of. robots. the first magical moment is. seeing an autonomous vehicle turn like. make a. left turn like apply.

sufficient torque to the steering wheel. to where like there's a lot of rotation. and for some reason and there's nobody. in the driver's seat. for some reason that that communicates. that here's a being with power. that makes a decision there's something. about like the steering wheel because. we perhaps romanticize the notion of the. steering wheel it's so essential to the. our conception. our 20th century conception of a car and. it. turning the steering wheel with nobody.

in driver's seat. that to me i think maybe to others. it's really powerful like this thing is. in control. and then there's this leap of trust that. you give like i'm gonna put my life. in the hands of this thing that's in. control so in that sense. when there's no but no driver in the. driver's seat. that's a magical moment for robots so i. i'm i gotten a chance to uh last year to. take a ride. in in a waymo vehicle and that that was. the magical moment there's like.

nobody in the driver's seat it's it's. like the little details you would think. it doesn't matter whether it's a driver. or not. but like if there's no driver and the. steering wheel is turning on its own. i don't know that's magical it's. absolutely magical. like i you've taken many of these rights. in a completely empty car. no human in the car pulls up you know. you call it on your cell phone it pulls. up you get in. it takes you on its way there's nobody. uh in the car. but you right that's something called.

you know fully driverless. our rider only mode of operation. uh yeah it it is. magical it is uh transformative this is. what we hear from our. uh writers it really changes your. experience and not like that that really. is what unlocks. the real potential of this technology uh. but you know. coming back to our journey uh you know. that was 2017 when we started. uh truly driverless operations then in.

2018 we've launched our. public commercial service that we call. waymo one. in phoenix in 2019 we started. offering truly driverless rider only. rights. to our early writer population. of users and then you know 2020. has also been a pretty interesting year. uh one of the first ones. less about technology but more about the.

maturing and the growth of. waymo as a company we raised our. first round of external financing uh. this year you know we were part of. alphabet so obviously we have. access to you know significant resources. but as kind of on the journey of waymo. maturing as a company it made sense for. us to you know partially go externally. uh uh and in this round so you know we. raised uh. about 3.2 billion dollars uh with from. you know that round. uh we've also you know uh started.

putting. our fifth generation of our driver our. hardware. uh uh that is on the new vehicle but. it's also a qualitatively different. set of uh self-driving hardware uh. that's all. uh that is now on the jlr pace so that. was a very. important step for us the hardware specs. fifth generation. i think it'd be fun to maybe i apologize. if i'm interrupting but. maybe talk about maybe the generations.

with a focus on what we're talking about. in the fifth generation in terms of. hardware specs. like what's on this car sure so we. separated out the actual car. that we are driving from the. self-driving hardware we put on it. um right now we have so this is as i. mentioned the fifth generation and we've. gone. through we started you know building our. own. hardware you know many many years ago. and. that firefly vehicle also had. the hardware suite that was mostly.

designed engineered and built in-house. lidars are of one of the more important. components that we design and build from. the ground up. uh so on the fifth generation uh of. our uh drivers uh of our driving. hardware that we're. switching to right now uh we have. uh as with previous generations in terms. of sensing we have. lidars cameras and radars and when you. have. a pretty beefy computer that processes.

all that information and makes you know. decisions. in real time on on board the car uh so. in all of the and it's really a. qualitative uh jump forward in terms of. the capabilities. and uh the various parameters and the. specs of the hardware. compared to what we had before and. compared to what you can kind of get off. the. of the shelf in the market today meaning. from fifth to fourth or from fifth to. first definitely from uh first to fifth. but also from the other. world's dumbest question definitely.

definitely from fourth to fifth okay. as well as uh uh there's the. the last step is a big step forward so. everything's in-house. so like lidar's built in house and. and cameras are built in-house uh you. know it's. different you know we work with partners. there are some components uh. that you know we get from our. manufacturing and you know supply chain. partners. uh what exactly is in-house is a bit. different if you. we we do a lot of you know custom uh.

design on. all of our sensing materials sliders. radars cameras. you know exactly there's lighters are. almost exclusively in-house and some of. the technologies that we have some of. the fundamental technologies there. are completely unique uh to weima uh. that is also largely true about radars. and cameras it's a little bit more of a. a mix in terms of what we do ourselves. versus what we get from. uh partners is there something uh super. sexy about the computer that you can.

mention that's not top secret. like uh for people who enjoy computers. for i mean uh so there's there's a lot. of. machine learning involved but there's a. lot of just basic compute there's. you have to uh probably do a lot of. signal processing on all the different. sensors you have to integrate everything. has to be in real time there's probably. some kind of redundancy type of. situation. is there something interesting you can. say about the computer for the people. who love.

hardware it does have all of the. characteristics all the properties that. you just mentioned. uh redundancy uh very beefy compute. for general processing as well as you. know inference. and ml models it is some of the more. sensitive stuff that you know i don't. want to get into for ip reasons but. yeah it can be shared a little bit. uh in terms of the specs of the sensors. that we have on the car you know we. actually shared some videos of. what our lighter seas lighters.

see in the world we have 29 cameras we. have. five lighters we have six raiders on. these vehicles. and you can kind of get a feel for the. amount of data that they're producing. that all has to be processed in real. time. uh to you know do perception to do. complex reasoning. and kind of gives you some idea of how. beefy those computers are but i don't. want to get into specifics of exactly. how we build them. okay well let me try some more questions. that you can't get into the specifics of. like gpu wise. is that something you can get into you. know i know that google works.

with tpus and so on i mean for machine. learning folks. it's kind of interesting or is there no. how do i ask it uh i've been talking to. people. in the government about ufos and they. don't answer any questions so this is. this is how i feel right now asking. about gpus. [Laughter]. but is there something interesting they. could reveal. or is it just you know uh yeah or would. leave it up to our imagination some of. the some of the compute is there any.

i guess is there any fun trickery like i. talked to. chris lattner for a second time and he. was a key. person about tpus and there's a lot of. fun stuff going on in. google in terms of uh. hardware that optimizes for machine. learning is there something. you can reveal in terms of how much you. mentioned customization how much. customization there. is for hardware for machine learning. purposes. i'm going to be like that government you. know you that guy uh personally. audio foes.

i i guess i you know will say that it's. really compute. is really important uh we have. very data hungry and compute hungry ml. models. of all over uh our stack and this is. where. you know both being part of alphabet as. well as designing our own sensors and. the entire hardware suite together. where on one hand you get access to like. really rich uh raw sensor.

data that you can pipe from your sensors. uh. into your compute platform yeah and. build like build the whole pipe. from sensor raw sensor data to the big. compute as then have the massive compute. to process all that data. and this is where we're finding that. having a lot of control. of that that hardware part of the stack. is. really advantageous one of the. fascinating magical places to me. again might not be able to speak to the. details but. is the it is the other compute which is.

like you know this we're just talking. about a single car. but the you know. the driving experience is a source of a. lot of fascinating data. and you have a huge amount of data. coming in on the car on the car. and you know the infrastructure of. storing some of that data. to then train or to analyze or so on. that's a fascinating like piece of it. that that i understand a single car i. don't understand how you pull it all. together in a nice way.

is that something that you could speak. to in terms of the challenges of um. of seeing the network of cars and then. bringing the data back. and analyzing things that weren't that. like like edge cases of driving be able. to learn on them to improve the system. to. to see where things going wrong with. where things went right and analyze all. that kind of stuff. is there something interesting there in. the from an engineering perspective. oh there's an incredible uh. amount of really interesting work that's.

happening there both. in the you know the real time operation. of the fleet of cars. and the information that they exchange. with each other in real time. to make better decisions as well uh. as on the kind of the off board. component where you have to deal with. massive amounts of data. for training your ml models evaluating. the male models. for simulating the entire system and for. you know evaluating your entire system. and this is where. and being part of alphabet has been once.

again been tremendously. uh advantageous because we consume an. incredible amount of you know compute. for ml infrastructure we build a lot of. custom frameworks to you know get good. at you know. on data mining uh finding the. interesting edge cases for training and. for evaluation of the system. for both training and evaluating some. components and you know sub. uh parts of the system and various ml. models as well as the. uh evaluating the entire system and. simulation okay that first piece that.

you mentioned that. cars communicating to each other. essentially i mean through perhaps. through a centralized point. but what uh that's fascinating too how. much does that help you like if you. imagine. like you know right now the number of. way more vehicles is. whatever x i don't know if you can talk. to what that number. but it's it's not in the hundreds of. millions yet. and imagine if the whole world is way. more vehicles. uh like that changes potentially the.

power of connectivity like the more cars. you have. i guess actually if you look at phoenix. because there's enough vehicles. there's enough when there's like some. level of density. you can start to probably do some really. interesting stuff with. the fact that cars can negotiate can. be uh can communicate with each other. and thereby make decisions. is there something interesting there. that you can talk to about like how does. that help with the driving problem. from as compared to just a single car.

solving the driving problem by itself. uh yeah it's it's a spectrum i uh first. to say that yeah. it's it helps uh and it helps in various. ways but it's not required. uh right now the way we build our system. engaged cars can operate independently. they can operate with no connectivity uh. so i think it is important that you know. you have. a fully uh autonomous you know fully. capable. uh driver uh that. computerized driver that each car has.

then you know they do. share information and they share. information in real time it really. really helps right so the way we. do this today is uh you know whenever. one car. encounters something interesting in the. world whether it might be an accident or. a new construction zone that information. immediately gets. uh you know uploaded over the air and is. propagated to the rest of the fleet. so and that's kind of how we think about. maps as priors. in terms of the knowledge of our drivers.

of our fleet of drivers that is. distributed across the fleet and it's. updated. in real time so that's one use case. you know you can imagine as the you know. the the density. of these vehicles go up that they can. exchange more information. in terms of what they're planning to do. uh and uh start. uh influencing how they interact with. each other uh as well as you know. potentially. sharing some observations right to help.

with if you have enough density of these. vehicles where you know one car might be. seeing something that another is. relevant to another car. that is very dynamic you know it's not. part of kind of you're updating your. static prior. of the map of the world but it's more of. a dynamic information that could be. relevant to the. decisions that another cars make in real. time so you can see them exchanging that. information. and you can build on that but again i i. see that as. an advantage but it's you know not a. requirement. so what about the human in the loop so. uh when i got a chance to drive with a.

ride in a waymo you know there's. customer service. [Laughter]. so like is somebody that's able to. dynamically. like tune in and uh help you out. what uh what role does the human play in. that picture that's a fascinating like. you know the idea of teleoperation be. able to remotely control a vehicle. so here what we're talking about is like.

like frictionless uh like a human being. able to in a. in a frictionless way sort of help you. out i don't know if they're able to. actually control the vehicle. is that something you could talk to uh. yes okay uh to be clear. we don't do teleportation i'm going to. believe in teleoperation. for rare reasons that's not what we have. on our cars. we do as you mentioned have you know. version of you know customer support. uh you know we call it live health in. fact we find it that it's very. uh important for our rider experience. especially.

if it's your first trip you've never. been in a fully driverless rider only. way more vehicle you get in there's. nobody there. right so you can imagine having all. kinds of you know questions in your head. like how this thing works. so we've put a lot of thought into kind. of guiding our. our writers our customers through that. experience especially for the first time. they get some information on the phone. uh if the fully driverless vehicle. is used to service their trip uh when. you get into the car. we have an in-car you know screen and.

audio that kind of guides them and. explains. uh what to expect they also have a. button. that they can push that will connect. them to. you know a real life human being that. they can talk to. all right about this whole process so. that's one aspect of it um there is. i should mention that there is uh. another function that uh humans provide. uh to our. cars but it's not tele operation you can. think of it a little bit more like you. know fleet assistance. kind of like you know traffic control uh. that that you have.

where our cars again they're responsible. on their own for making all of the. decisions all the driving decisions that. don't require connectivity. they you know anything that is safety or. latency critical. uh is done you know purely autonomously. by. on board uh our on onboard system uh but. there are situations where you know if. connectivity is available. uh can a car encounters a particularly. challenging situation you can imagine. like a super hairy. uh scene of an accident uh the cars will. do their best. they will recognize that it's an off.

nominal situation they. will you know do their best to come up. you know with the right interpretation. the best course of action in that. scenario but if connectivity is. available they can ask. for confirmation from you know here mode. human. assistant to kind of confirm those. actions and perhaps. provide a little bit of kind of. contextual information and guidance. so october 8th was when you're talking. about the. was weimar launched the the.

the fully self the public version. of its fully driverless that's right. term i think. service in phoenix is that october 8th. that's right. it was the introduction of fully. driverless rider only vehicles into our. you know public waymo one service okay. so that's that's amazing. so it's like anybody can get into waymo. in phoenix. oh that's right yeah so we previously. had. early people in our early writer program. uh. taking fully driverless rides in phoenix.

and. uh just uh this a little while ago we. opened on october 8th we opened. that mode of operation to the public so. i can you know download the app and you. know go on the right. there is uh a lot more demand right now. uh for that service and then we have. capacity uh so we're kind of. uh managing that but that's exactly the. way you described it yeah well that's. interesting so there's more demand than. you can. you can handle like what uh. what has been uh reception so far like.

what. i mean okay so you know that's this is. a product right that's a whole other. discussion of like how compelling of a. product it is. great but it's also like one of the most. kind of transformational technologies of. the 21st century. so there it's also like a tourist. attraction. like it's fun to you know to be a part. of it. so it'd be interesting to see like what. do people say what do people. uh what have been the feedback so far. you know still early days but so far the.

feedback has been. uh incredible uh incredibly positive. they you know we asked them for feedback. during the ride we asked them for. feedback. uh after the ride as part of their trip. you know we asked them some questions we. asked them to. you know rate the performance of our. driver uh most. by far you know most of our drivers give. us five stars. in our app uh which is uh absolutely. great to see and yeah that's and we're. they're also giving us feedback on you. know things we can improve. uh and you know that's one of the main. reasons we're doing this is phoenix and.

you know over the last couple of years. and every day today uh we are just. learning. a tremendous amount of new stuff from. our users there's there's no. substitute for actually doing the real. thing. actually having a fully driverless. product out there in the field with you. know users. uh that are actually paying us money to. get from point a to point b. so this is a legitimate like that's a. paid service that's right. and the idea is you use the app to go. from point a to point b. and then what what are the a's what are.

the what's the freedom of the. of the starting and ending places it's. an area of geography where that service. is enabled it's a. you know decent size of geography of. territory it's actually larger than you. know the. size of san francisco uh and you know. within that. you have you know full freedom of you. know selecting where you want to go. you know of course there's some and you. on your app you get a map. you tell the car where you want to be. picked up. you know where you want you know the car.

to pull over and pick you up and then. you tell it where you want to be dropped. off. all right and of course there's some. exclusions right you want to be you know. you uh. where in terms of where the car is. allowed to pull over right so. you know that you can't do but you know. besides that uh it's amazing. it's not like a fixed just would be very. i guess i don't know maybe that's what's. the question behind your question. but it's not a you know preset set of uh. yeah. so within the geographic constraints. with that within that area anywhere else. it can be you can be picked up and. dropped off anywhere that's right. and you know people use them on like all.

kinds of trips they. we have and we have an incredible. spectrum of riders we i think the. youngest. actually have car seats them and we have. you know people taking their kids and. rides i think the youngest. riders we had on cars are one or two. years old you know and the full spectrum. of use cases people you can take them to. you know schools uh to you know go. grocery store shopping. to restaurants to bars you know run. errands you know go shopping et cetera. et cetera you can go to your office. right. like the full spectrum of use cases and. uh people. gonna use them in their daily lives to.

get around. uh and we see all kinds of you know. really. interesting uh use cases and that that. that's providing us. incredibly valuable experience that. we then you know use to improve our. product so as somebody who's been on. done a few long rants. with joe rogan and others about the. toxicity of the internet and the. comments. and the negativity in the comments i'm. fascinated by feedback i. i believe that most people are.

good and kind and intelligent and can. provide. like even in disagreement really. fascinating ideas so. on a product side it's fascinating to me. like how do you get. the richest possible user feedback. like to improve what's what are the. channels that. you use to measure because like you're. you're no longer that's one of the. magical things about autonomous vehicles. is it's not like it's frictionless.

interaction with the human so like you. don't get to. you know it's just giving a ride so like. how do you get feedback from people to. in order to improve. uh yeah uh great question various. mechanisms uh so as part of the. normal flow we ask people for feedback. they as the car is driving around. we have on the phone and in the car and. to have a touchscreen. in the car you can actually click some. buttons and provide uh. real-time feedback on how the car is. doing and how the car is handling a. particular situation.

you know both positive and negative so. that's one channel uh we have as we. discussed customer support or live help. where you know if a customer wants to. has a question. uh uh or he has some sort of concern. they can. talk to a person in real time so that. that is another mechanism that gives us. feedback uh at the end of a trip you. know we also ask them. how things went they give us comments. and you know star rating. and you know if it's uh we also you know. ask them. to explain what you know one one.

well and you know what could be improved. and uh. we we have uh our writers providing you. know very rich. uh feedback there a lot the large. fraction is. uh very passionate and very excited. about this technology so we get really. good feedback. uh we also run uxr studies right you. know specific. and that are kind of more you know go. more in depth and we'll run both kind of. lateral and longitudinal studies um. where we have. you know deeper engagement uh with our.

customers you know we have our. user experience research team tracking. over time and testing is about longitude. no it's cool. that's that's exactly right and you know. that's another really valuable uh. feedback uh source of feedback and you. we're just covering a tremendous amount. right. uh people go grocery stroping and they. like want to load. you know 20 bags of groceries in our. cars and like that that's one workflow. that you maybe don't you know think. about uh. you know getting just right when you're. building the driverless product.

i have people like you know who uh. bike as part of their trip so they you. know bike somewhere then they get on our. cars they take a. part their bike they load into our. vehicle then go and that's you know how. they. you know where we want to pull over and. how that you know uh get in. and get out um uh process works uh. provides. very uh useful feedback in terms of you. know what makes a good. uh pickup and drop-off location uh we. get really valuable feedback. and in fact we had to um uh do some.

really interesting work with. high definition maps and uh thinking. about. walking directions if you imagine you're. in a store right in some giant space. and then you know you want to be picked. up somewhere like if you just drop. a pin in the current location which is. maybe in the middle of a shopping mall. like what's the best. location for the car to come pick you up. and you can have simple heuristics where. you just kind of take your you know your. cleaning distance uh and find the. nearest. uh spot where the car can't pull over. that's closest to you but oftentimes. that's not the most convenient one you. know i have many anecdotes where that.

heuristic. breaks in horrible ways i one example uh. that yeah i often mention is somebody. wanted to be. you know uh dropped off uh and. phoenix uh and you know we car picked a. location. uh that was close the closest to their. you know where the pin was dropped. on the map in terms of you know latitude. and longitude but it happened to be. on the other side of a parking lot that. had this row of cacti. and poor person had to like walk all.

around the parking lot to get to where. they wanted to be. in 110 degree heat so that you know that. was about so then you know we took all. take all of these um all that feedback. from our users and. uh incorporate it into our system and. yeah and improve it. yeah i feel like that's like requires. agi to solve the problem. of like when you're which is a very. common case when you're in a big space. of some kind. like apartment building it doesn't. matter it's not some large space. and then you call the like the waymo. from there.

right like so and you whatever it. doesn't matter right your. vehicle and like where is the pin. supposed to drop i feel like that's i. you don't think i think that requires a. gi i'm gonna. in order okay the alternative. which i think the google search engine. has taught. is like there's something really. valuable about. the perhaps slightly dumb answer but a. really powerful one which is like. what was done in the past by others like.

what was the choice made by others. that seems to be like in terms of google. search when you have like billions of. searches. you can you could see which like when. they recommend. what you might possibly mean they. suggest based on. not some machine learning thing which. they also do but like on what. was successful for others in the past. and finding a thing that they were happy. with. is that integrated at all with waymo. like. what what pickups worked for others it.

is. i i think you're exactly right so. there's uh real it's an interesting. problem. uh naive solutions uh. have uh interesting failure modes. uh so there's definitely. lots of things that can be done to. improve. uh and both learning. from you know what works what doesn't. work in actual heal from you know. getting richer data and. getting more information about the. environment and you know.

richer maps but you're absolutely right. that there's something and there's some. properties of solutions that. uh in terms of the effect that they have. on users so much much much much better. than others right unpredictability. and understandability is important so. you can have maybe something that is not. quite as optimal but. is very natural and predictable to the. user and kind of works the same way. all the time and that matters that. matters a lot for the user experience. and but you know to get to the basics. the pretty fundamental property.

is that the car actually arrives. where you told it right like you can. always you know change it see it on the. map and you can move it around if you. don't like it and. but like that property that the car. actually shows up reliably. yeah is critical which you know where uh. compared to some of the human uh driven. yes. analogs i think you know you can have. more unpredictability. it's actually uh the fact uh if if i. have uh. might do a little bit of a detail here. uh i think the fact that it's.

you know your phone and the cars two. computers talking to each other uh can. lead to some. really interesting things we can do in. terms of the user interfaces. both in terms of function uh like the. car actually shows up. exactly where you told it uh you want it. to be but also some you know really. interesting things on the user interface. right as the car is driving as you. you know call it and it's on the way to. come and pick you up and of course you. get the position of the car. and the route on the map uh but and they. actually follow that route of course. uh but it can also share some really. interesting information about what it's.

doing. so uh you know our cars uh as. they are coming to pick you up if it's. come if a car is coming up to a stop. sign. it will actually show you that like it's. there sitting because it's at a stop. sign or a traffic light it'll show you. that it's got you know sitting at a red. light. so you know they're like little things. uh right uh but. it i find those little touch uh touches. uh really interesting really magical. and it's just you know little things. like that that you can do to kind of. delight your users. you know this makes me think of um.

there's some products that i just love. like. there's a there's a company called rev. uh rev.com where i like for this. podcast for example i can drag and drop. a video. and then they do all the captioning. uh it's humans doing the captioning but. they connect you good they they. automatic. automate everything of connecting you to. the humans and they do the captioning. and transcription it's all effortless. and like i remember when i first started.

using them it was like. life is good like because it was so. painful to. to figure that out earlier uh the same. thing with uh. something called izotope rx this company. i use for cleaning up audio like. the sound cleanup they do it's like drag. and drop and it just cleans everything. up. very nicely uh another experience like. that i had with amazon one click. purchase first time i mean other places. do that now but. just the effortlessness of purchasing.

making it frictionless. it kind of communicates to me like i'm a. fan of design i'm a fan of products. that you can just create a really. pleasant experience. the simplicity of it the elegance just. makes you fall in love with it. so on the do you think about this kind. of stuff i mean. that's exactly what we've been talking. about it's like the little details that. somehow make you fall in love with the. product. is that we went from like urban. challenge days where.

where love was not part of the. conversation probably. and to to this point where there's uh. where there's human beings and you want. them to fall in love with the experience. is that something you're trying to. optimize for trying to think about like. how do you. how do you create experience that people. love absolutely. i think that's the vision is removing. any friction or complexity. from getting our.

users our writers to where they want to. go and. making that as simple as possible and. then you know beyond that. on just transportation making you know. things and you know goods. get to their destination as seamlessly. as possible and talked about. you know a drag and drop experience. where you kind of express your intent. and then. you know it just magically happens and. for our riders that's. what we're trying to get to is you. download an app and you can. click and car shows up it's the same car. it's very predictable it's.

a safe and high quality experience and. then. it gets you in a very reliable very. convenient. uh frictionless. way to where you want to be and along. the journey. i think we also want to like do a little. things to delight. our users like the ride-sharing. companies because they don't control the. experience i think. they can't make people fall in love. necessarily with the experience.

or maybe they haven't put in the effort. but. i think it if i would just speak to the. ride-sharing experience i currently have. it's just very it's just very convenient. but there's a lot of room for like. falling in love with it. like we can speak to sort of car. companies car companies do this well you. can fall in love with a car. right and be like a loyal car person. like whatever. like i like bad ass hot rods i guess 69. corvette.

and at this point you know you can't. really. cars are so owning a car is so 20th. century man. but is there something about the waymo. experience where you hope that people. will fall in love with because that. is that part of it or is it part of. is it just about making a convenient. ride. not ride sharing i don't know what the. right term is but just the convenient. eight to be. autonomous um transport. or like do you want them to fall in love.

with waymo so maybe elaborate a little. bit. i mean almost like from a business. perspective i'm curious. like how. do you want to be in the background. invisible or do you want to be. uh like a source of joy that's. in very much in the foreground i want to. provide the best most enjoyable. transportation solution. uh and that means.

building it building our product and. building our service in a way. that people do uh kind of use. in a very. seamless frictionless way in their in. their day-to-day lives. and i think that does mean uh you know. in some way falling in love. in that product right just kind of. becomes part of your routine i. uh it comes down my mind to. safety predictability of the experience.

and um. privacy i think aspects of it right our. cars you get the same car you get very. predictable behavior. and that that is important and if you're. going to use it in your daily life. privacy and when you're in a car you can. do other things you're spending a bunch. just another space where you're spending. a significant. part of your life right so not having to. share it with. other people who you don't want to share. it with i think is uh.

a very nice property uh maybe you want. to take a phone call or. do something else in the vehicle um and. you know. safety on the quality of the driving as. well as the physical safety. of you know not having so you know to. share that. ride is you know important to a lot of. people. what about the idea that when when. there's. somebody like a human driving and they. do a rolling stop on a stop sign like. sometimes like.

you know you get an uber a lift or. whatever like human driver. and you know they can be a little bit. aggressive. as as drivers it feels like there is um. not all aggression is bad uh. now that may be a wrong again 20th. century conception of driving. maybe it's possible to create a driving. experience like. if you're in the back busy doing. something maybe. aggression is not a good thing it's a. very different kind of experience. perhaps. but it feels like in order to navigate.

this world. you need to uh how do i. uh phrase this you need to kind of bend. the rules a little bit or at least like. test the rules. i don't know what language politicians. use to discuss this but uh. whatever language they use you like. flirt with the rules i don't know. but like you uh you sort of. uh have a bit of an aggressive way of. driving. that asserts your presence in this world.

thereby making other vehicles and people. respect your presence. and thereby allowing you to sort of. navigate through intersections in a. timely fashion. i don't know if any of that made sense. but like how. does that fit into the experience of. driving autonomously is that. a lot of sales this is you're hitting a. very important point of. a number of behavioral components and. parameters that make your driving.

feel you know assertive and natural and. comfortable predictable. um now our cars will follow rules right. they will do the safest thing possible. in all situations let you know be clear. on that. uh but if you think of really really. you know good drivers just you know. think about you know professional limo. drivers right they will follow the rules. they're very very smooth uh and yet. they're very efficient uh and but. they're assertive. uh they're comfortable for the people in. the vehicle.

they're predictable for the uh other. people outside the vehicle that they. share the environment with. and that that's the kind of driver that. we want to build and you think. if maybe there's a sport analogy there. right yeah you can. do in very many sports the. true professionals are very efficient in. their movements. right they don't do like you know hectic. uh. flailing right they're you know smooth. and. precise right and they get the best. results so that's the kind of driver.

that we want to build. in terms of you know aggressiveness yeah. you can like you know roll through the. stop signs you can do crazy lane changes. uh it typically doesn't get you to your. destination faster typically not the. safest or most predictable uh. very most comfortable thing to do and uh. but there is. a way to do both and that that that. that's what we're doing we're trying to. build a driver that is. uh safe comfortable smooth. and predictable yeah that's a really. interesting distinction i think in the. early days of autonomous vehicles.

the vehicles felt cautious as opposed to. efficient. and and still probably but when i rode. in the waymo i mean there was. it was it was quite assertive it moved. pretty quickly. like um yeah and he's one of the. surprising feelings was that it actually. it went fast and it didn't feel like. awkwardly cautious than autonomous. vehicle like.

like so i've also programmed autonomous. vehicles and everything i've ever. built was felt awkwardly either overly. aggressive. okay especially when it was my code or. uh like awkwardly cautious is the way i. would put it. and the waymo's vehicle felt. like uh assertive and i think efficient. as. like the right terminology here it. wasn't uh. and i also like the professional limo.

driver because we often think like. you know an uber driver or a bus driver. or a taxi. this is the funny thing is people think. that taxi drivers are professionals. they i mean it's it's like that that's. like saying. me i'm a professional walker just. because i've been walking all my life. i think there's an art to it right. and if you take it seriously as an art. form. then there's a certain way that mastery.

looks like. it's interesting to think about what. does mastery look like in driving. and perhaps what we associate with like. aggressiveness. is unnecessary like it's not part of the. experience of driving it's like. unnecessary fluff. that efficiency you could you can be. you can create a good driving experience. within the rules.

that's uh i mean you're the first person. to tell me this so it's it's kind of. interesting i need to think about this. but that's exactly what it felt like. with waymo i kind of had this intuition. maybe it's the russian thing i don't. know. that you have to break the rules in life. to get anywhere. but maybe maybe it's possible that. that's not the case. in driving i have to think about that. but it certainly felt that way on the. streets of phoenix when i was there in. in waymo that. that that that was a very pleasant. experience and it wasn't frustrating in.

that like. come on move already kind of feeling it. wasn't it that wasn't there. yeah i mean that's what that's what. we're going after yeah i don't think you. have to pick one i think. truly good driving and gives you both. efficiency assertiveness but also. comfort and predictability and you know. safety. uh and you know it's that's what. fundamental improvements in the. core capabilities truly unlock and. you can kind of think of it as you know. a precision and recall trade-off you.

have certain capabilities of your model. and then it's very easy when you know. you have some curve of precision and. recoil you can move things around and. you can choose your operating point in. your training of precision versus recall. false positives versus false negatives. right but then and you know you can tune. things on that curve and be kind of more. cautious or more aggressive but then. aggressive is bad or you know cautious. is bad. but true capabilities come from actually. moving the whole curve. up right and then you are kind of on a. very. different plane of those trade-offs and. that that's what you know we're.

trying to do here is to move the whole. curve up before i forget let's talk. about trucks. a little bit uh so i also got a chance. to check out some of the waymo truck. uh trucks i'm not sure if uh we want to. go. too much into that space but it's a. fascinating one so maybe we can mention. at least briefly. you know waymo is also not doing. autonomous trucking and uh how. different like philosophically and. technically is that whole space of. problems.

it's one of our two big. products and uh you know commercial. applications. of our driver right right handling and. deliveries you know. we have waymo one and waymovia moving. people and moving goods. uh you know trucking is an example of. uh moving goods uh we've been uh working. on trucking since 2017. uh it. is uh. a very interesting space and your.

question. how different is it it has this really. nice property that. the first order challenges like the. science. the hard engineering uh whether it's you. know hardware or you know onboard. software or. off-board software all of the you know. systems that you build. for you know training your ml models for. you know evaluating a retirement system. like those fundamentals. carry over the true challenges of. driving perception semantic. understanding.

prediction decision making more planning. evaluation. uh the simulator ml infrastructure those. carry over. i think the data and the application and. kind of the the. domains might be different but the the. most difficult problems. uh all of that carries over between the. domains so. that that's very nice so that's how we. approach it we're kind of. build investing in the core the. technical core. and then there's specialization of and. uh. of that core technology to different.

product lines to different commercial. applications. so on just to tease it apart a little. bit. uh on trucks so starting with the. hardware the configuration of the. sensors. is different right they're different. physically geometrically you know. different vehicles. uh so for example we have two of our. main laser. uh on the trucks on both sides so that. we have you know don't have the blind. spots. uh whereas on the jlr i-pace we have you. know one of it uh. sitting at the very top but the actual.

sensors are uh almost the same. or largely uh the same so all of the. investment. that uh over the years we've put into. building our custom lighters custom. radars and pulling the whole system. together. that carries over very nicely uh then. you know on the perception side. uh the like the fundamental challenges. of. seeing understanding the world whether. it's you know object detection. classification. you know tracking semantic understanding. all that carries over now yes there's. some specialization when you're driving. on freeways.

uh you know range becomes more important. the domain is a little bit different. but again the fundamentals carry over. very very nicely. same and i guess you get into prediction. or decision making. right the fundamentals of what it takes. to. predict what other people are going to. do to. find the long tail to improve your. system in that long tail. of behavior prediction and response that. carries over right and so on and so on. so i mean that's pretty exciting by the. way does waymovia include.

using the the smaller vehicles for. transportation goods that's an. interesting distinction. so let's say there's three interesting. modes of operation so one is moving. humans one is moving goods. and one is like moving nothing zero. occupancy. meaning like you're going to the. destination. your your empty vehicle i mean it's it's. the third is the last wave that's the. entirety of it it's so less you know.

exciting from the commercial perspective. [Laughter]. well i mean in terms of like if you. think about what's inside a vehicle as. it's moving. because it does you know some. significant fraction of the vehicle's. movement has to be. empty i mean it's kind of fascinating. maybe just on that small point is. is there different. control and like policies. that are applied for a zero occupancy.

vehicle so vehicle with nothing in it. or is it just move as if there is a. person inside what was. with uh some subtle differences as a. first order approximation. there are no differences and if you. think about. you know safety and you know comfort and. quality of driving. only part of it you know. has to do with the people or the goods. inside of the vehicle. right but you don't want to be you know.

you want to drive smoothly and as we. discussed. not for the purely funded benefit of you. know whatever you have inside the car. right it's also for the benefit of the. you know people outside kind of. feeding fitting uh naturally and. predictably into the whole environment. right. so you know yes there are some second. order uh things you can do it's gonna. change your route. and you know optimize maybe kind of your. fleet. things at the fleet scale and you would. take into account whether. some of your cars are actually. you know serving a useful trip whether. with people or with goods whereas you.

know other cars. are you know driving completely empty. you know to that next. valuable trip that they're going to. provide but that those are. mostly second order effects okay cool so. phoenix. is uh is an incredible place and what. you've announced in phoenix is uh. it's kind of amazing but you know that's. just like one. city how do you take over the world. uh i mean i'm asking for a friend once.

one step at a time. is that the cartoon pinky in the brain. yeah okay. but you know gradually is a true answer. so i think. the heart of your question is. what can you ask a better question than. i asked they asked a great question. to answer that one i i i'm you know just. gonna. you know phrase it in the terms that i. want to answer perfect. exactly right brilliant please.

you know where are we today and you know. what happens next. uh and what does it take to go beyond. phoenix and was it what does it take. uh to get this technology to. more places and more people around the. world right. so our next. big area of focus is. exactly that larger scale. commercialization and. you know scaling up uh.

if i think about. you know the main and your phoenix gives. us that. platform it gives us that foundation of. upon which we can build them and it's. there are few really. challenging aspects of this whole. problem that you have to pull together. in order to build the technology. in order to deploy it.

uh into the field to go. from a driverless car to. a fleet of cars that are providing a. service. and then all the way to you know. commercialization so. uh and then you know this is what we. have in phoenix we've taken the. technology from. uh a proof point to an actual deployment. and have taken our driver you know from. you know one car to a fleet that can. provide a service um. beyond that if i think about what it.

will take to. scale up and you know deploy in. you know more places with more customers. i. tend to think about uh three. main dimensions three main axes um of. scale. one is the core technology you know the. hardware. and software core capabilities of our. driver. the second dimension is. evaluation and deployment and the third. one.

is the product commercial and. operational excellence so you can talk. you know. a bit about where we are along you know. each one of those three dimensions about. where we are today. and you know what has what will happen. next. um on you know the core technology on. you know the hardware and software and. together comprise our driver. we you know obviously have that. foundation. that is providing fully driverless trips.

to our customers as we speak. in fact and we've learned a tremendous. amount from that so now what we're doing. is we are incorporating all those. lessons. into some pretty fundamental. improvements in our core technology both. on the hardware side and on the software. side. to build a more general more robust. solution. that then will enable us to massively. scale you know beyond phoenix. so on the hardware side.

all of those lessons are now. incorporated into this fifth generation. hardware platform. that is you know uh being deployed right. now. and that's the platform the fourth. generation the thing that we have right. now driving in phoenix. it's good enough to operate operate. fully driverlessly you know. night and day in various speeds and. various conditions. but the fifth generation is the platform. upon which we want to go to massive. scale. we it in turn we've really made.

qualitative improvements in terms of the. capability of the system. the simplicity of the architecture the. reliability of the redundancy. it is designed to be manufacturable at. very large scale and you know provides. the right unit economics. so that's that's the next big step for. us um on the hardware side. that's that's already there for scale. the version five. that's right is that uh coincidence or. should we look into it conspiracy theory. that's the same version as the pixel. phone. is that what's the harder they neither.

confirm okay. all right cool so sorry so that's the. okay that's that axis what else. uh so similarly hardware is a very. discrete. jump but you know similar to the uh that. to how we're making that change from the. fourth generation hardware to the fifth. we're making similar improvements on the. software side. to make it more you know robust and more. general and allow us to kind of. quickly uh scale beyond phoenix so that. that's the first dimension of core. technology the second dimension is. evaluation and deployment. now how do you measure.

your system how do you evaluate it how. do you build the release and deployment. process. where you know with confidence you can. you know regularly release new versions. of your driver. into a fleet how do you get good at it. so that. it is not you know a huge tax on your. researchers and engineers that you know. so you can how do you build all these. you know. processes the frameworks the simulation. the evaluation the data science the. validation. so that you know people can focus on.

improving the system and kind of the. releases. just go out the door and get deployed. across the fleet so we've gotten really. good at that. in phoenix that's been a tremendously. difficult problem. but that's what we have in phoenix right. now that gives us that foundation. and now we're working on kind of. incorporating all the lessons that we've. learned. to make it more efficient to go to new. places you know scale up and just kind. of you know stamp things out. so that's that second dimension of. evaluation and deployment and the third. dimension. is product commercial and operational.

excellence right and again phoenix there. is providing. uh an incredibly valuable platform you. know that's why we're doing things. end-to-end uh in phoenix we're learning. as you know we discussed a little. earlier today. a tremendous amount of really valuable. lessons from our users getting really. incredible feedback. uh and uh we'll continue to iterate on. that and. incorporate all those uh those lessons. into. making our product you know even better.

and more convenient for our users. so you're converting this whole process. of phoenix. in phoenix into uh something that could. be copy and pasted elsewhere. so like uh perhaps you didn't think of. it that way when you were doing the. experimentation phoenix but. so how long did basically. you can correct me but you've i mean. it's still early days but you're taking. the full journey. in phoenix right as you were saying. of like what it takes to basically. automate i mean it's not the entirety of.

phoenix right. but i imagine it can encompass. the entirety of phoenix that's some some. uh near-term date but that's not even. perhaps important like. as long as it's a large enough. geographic area so. what how copy-pastable. is that process currently and. how do like um. you know like when you copy and paste in. in uh in google.

docs i think you know in or in word. you can like apply source formatting or. apply destination formatting. so how when you copy and paste uh the. phoenix. into like say boston. uh how do you apply the destination. formatting. like how much of the core of the entire. process. of bringing an actual public. transportation autonomous transportation.

service to a city. is there in phoenix that you understand. enough to copy and paste into boston. or wherever um so we're not quite there. yet. we're not at a point where we're kind of. massively copy and pasting. all over the place uh but phoenix. what you know we did in phoenix and we. very intentionally have chosen phoenix. as. our first full deployment. uh area you know exactly for that reason. to kind of tease the problem apart. look at each dimension and focus on the.

fundamentals of complexity and. de-risking. you know those dimensions and then. bringing the entire thing together to. get all the way. and force ourselves to learn all those. hard lessons on technology hardware and. software. on the evaluation deployment on you know. operating a service operating a business. using. uh actually you know um serving our. customers. all the way so that we're fully informed. about. the most difficult most important. challenges. to get us to that next step of massive.

copy and pasting. as as you said and uh. [Music]. that's what we're doing right now we're. incorporating all those things that we. learned into. that next system that then will allow us. to kind of copy paste all over the place. and to massively scale to you know more. users and. more locations i mean you know just. talked a little bit about you know what. does that mean along those. different dimensions so on the hardware. side for example again it's that. uh switch from the fourth to the fifth. generation and the fifth generation is. designed to kind of have that property.

can you say what other cities you're. thinking about like i'm thinking about. sorry we're in san francisco now i. thought i want to move to san francisco. but i'm thinking about moving to austin. um i don't know why people are not being. very nice about san francisco. currently for maybe it's a small it's. like maybe it's in vogue right now. but uh austin seems i visited there and. there was uh. i was in a walmart it's funny these.

moments. like turn your life there's this very. nice. woman with kind eyes just like stopped. and said you look so handsome in that. tie honey. to me this has never happened to me in. my life but just the sweetness of this. woman. is something i've never experienced. certainly on the streets of boston but. even in san francisco where people. wouldn't. that's just not how they speak or think. i don't know there's a warmth too to. austin that love.

and since waymo does have a little bit. of a history there. is that a possibility is this your. version of asking the question of like. you know dimitri i know you can't share. your commercial and deployment roadmap. but i'm thinking about moving to should. i cisco austin like in a blink twice if. you think i should move to him. yeah that's true this room you got me we. you know we've been. testing and all over the place i think. we've been testing more in. 25 cities we drive in san francisco we. drive in. you know michigan for snow uh we we are.

doing significant amount of testing in. the bay area including san francisco. which is not like because we're talking. about the very different thing which is. like. a full-on large geographic area. public service uh you can't share any. okay. what about moscow is that when is that. happening. take on yandex i'm not paying attention. to those folks. they're doing you know there's there's a.

lot of fun. i mean maybe as a way of a question. you didn't speak to sort of like. policy or like is there tricky things. with government and so on. like is there other friction. that you've encountered except sort of. technological. friction of solving this very difficult. problem. is there other stuff that you have to. overcome.

when when uh deploying a public service. in a city. that's interesting it's very important. so we we. put significant effort in uh. creating those partnerships and you know. those relationships. with governments at all levels you know. local governments municipalities you. know state level federal level uh we've. been engaged in very deep conversations. from the earliest days of our. you know projects uh whenever at all of. these levels you know whenever we go.

to test uh or you know operate in a. new area you know we always lead with. with a conversation with the local. officials and but the result of that. that investment is that no. it's not challenges we have to overcome. it but it is a very important. that we continue to have this. conversation oh yeah. i love politicians too okay uh so mr. elon musk. said that uh lidar is a crutch. what are your thoughts.

i wouldn't characterize it exactly that. way uh i know i think lighter is. very important uh it is a key sensor. uh that you know we use just like other. modalities and as we discussed. our cars use cameras uh lidars and. radars. they are all very important they are. at the kind of the physical level they. are very different they have very. different you know physical. characteristics.

cameras are passive lighters and radars. are active you use different wavelengths. uh so that means they complement each. other uh. and very nicely and and together. combined they can be used to build a. much. safer and much more capable system so. you know to me it's more of a question. you know why the heck would you handicap. yourself and not use one. or more of those sensing modalities when.

they you know undoubtedly just make your. system. uh more capable and safer. now it you know. what might make sense for one product uh. or one business might not make sense for. another one. so if you're talking about driver assist. technologies you make certain design. decisions and you make certain. trade-offs. and you make different ones if you are. you know building a driver uh that deep. deploy in fully driverless. vehicles uh and you know and lighter.

specifically when this question comes up. i uh you know typically the criticisms. uh. that i hear or you know the. counterpoints. that cost and aesthetics. and like i i don't find either of those. honestly very compelling so on the cost. side. there's nothing fundamentally. prohibitive about you know the cost of. lighters you know radars used to be very. expensive uh before people start you. know.

uh before people need certain balances. and technology and you started to. to manufacture them uh massive scale and. deploy them in vehicles right. uh similarly with lighters and this is. where the lidars that we have on our. cars especially the fifth generation. uh you know we've been able to make. some pretty qualitative discontinuous. jumps in terms of the fundamental. technology that allow us to. manufacture those things at very. significant scale. and add a fraction of the cost of you. know both. our previous generation as well as a.

fraction of the cost of you know what. might be available. on the market you know off the shelf. right now and you know that improvement. will continue so i i think you know cost. is uh. not a real issue uh second one is uh you. know uh aesthetics. uh you know i don't think that's you. know a real issue either. uh um the beholder yeah. you can make lidar sexy again i think. you're exactly right i think it is sexy. like honestly i think foreign. you know i was actually somebody brought. this up to me um.

i mean all forms of lidar even. uh even like the ones that are like big. you can make. look i mean it can make look beautiful. like. there's no sense in which you can't. integrate it into design. like there's all kinds of awesome. designs i don't think. small and humble is beautiful it could. be. like you know brutalism or like it could. be. uh like harsh corners i mean like i said. like hot rods like i don't like i don't. necessarily like like oh man i'm gonna.

start so much controversy with this. i i don't like porsches. okay the porsche 911 like everyone says. the most beautiful. no it no it's like it's like a baby car. it doesn't make any sense. but everyone it's beauty's denied the. beholder you're already looking at me. like what's this kid talking about. you're happy to talk about you're. digging your own home the form and. function. and my take on the beauty of the. hardware that we put on our vehicles.

you know i will not comment on a porsche. monologue. okay all right so but aesthetics fine. but there's an underlying like. philosophical question. behind the kind of lighter question is. like how much. of the problem can be solved with. uh computer vision with machine learning. so i think without sort of. disagreements and so on. it's nice to put uh it on the spectrum.

because waymo is doing a lot of machine. learning as well. it's interesting to think how much of. driving if we look at five years. 10 years 50 years down the road would. can be. learned in almost more and more and more. end-to-end way if we look at what tesla. is doing. with the as a machine learning problem. they're doing a multi-task learning. thing where it's just they break up. driving into a bunch of learning tasks. and they have one single neural network. and they're just collecting huge amounts.

of data that's training that. i've recently hung out with george cotts. i don't know if you know george. uh i love him so much he's just an. entertaining human being we were off. mike talking about hunter s thompson. he's. he's the hunter that's thompson and. baton was driving okay so he. i didn't realize this with common ai but. they're like. really trying to do end to end they're. the machine. like looking at the machine learning. problem they're. really not doing multi-task learning but.

it's. uh it's it's computing the drivable area. as a machine learning task. and hoping that like down the line. this level two system this driver. assistance. will eventually lead to allowing you to. have a fully autonomous vehicle. okay there's an underlying deep. philosophical question there technical. question. of how much of driving can be learned. so lidar is an effective tool today. uh for actually deploying a successful.

service in phoenix. right that's safe that's reliable et. cetera et cetera but. uh the the question and i'm not saying. you can't do machine learning on lidar. but the the question is that like how. much of driving can be. learned eventually can we do fully. autonomous. that's learned yeah uh you know learning. is all over the place. and plays a key role in every part of. our system. i i as you said i would uh you know. decouple the sensing modalities.

from the you know ml and the software. parts of it. lighter radar cameras like it's all. machine learning. all of the object detection. classification of course like that's. what you know these. modern deep nuts and continents are very. good at you feed them raw data. massive amounts of raw data um and you. know. that's actually what our custom build. lighters and raiders are really good at. and radars they don't just give you. point estimates of you know objects in. space they give you raw. like physical observations and then you.

take all of that raw information you. know there's colors of the pixels. whether it's you know lighters returns. and some auxiliary information it's not. just distance. right and you know angle and distance is. much richer information that you get. from those returns. plus really rich information from the. radars you fuse it all together and you. feed it into those massive. ml models that then you know. lead to the best results in terms of you. know object uh deduction. classification you know state estimation. so there's a side interrupt but there is. a fusion. i mean that's something that people. didn't do for a very long time which is.

like at the. sensor fusion level i guess like early. on fusing the. information together whether so that the. the sensory. information that the vehicle receives. from the different modalities. or even from different cameras is. combined before it is fed into the. machine learning models. uh yes i think this is one of the trends. you're seeing more of that you mentioned. end to end there's different. interpretations of antenna there's kind. of the. purest interpretation now i'm gonna like. have one model. that goes from raw sensor data to like.

you know steering torque. and you know guest brakes that you know. that that's too much i don't think. that's the right way to do it. there's more you know smaller versions. of end to end. where you're you know kind of doing more. end-to-end learning or core training or. deep propagation. of kind of signals back and forth across. the different stages of your system. there's no really good ways it gets into. some fairly complex design choices where. on one hand you want modularity and the. compass. composite ability the composibility of. your system.

but on the other hand you don't want to. create interfaces that are too narrow. or too brittle to engineered where. you're giving up on the generality of. the solution. or you're unable to properly propagate. signal you know reach signal forward and. losses and you know back so you can you. know optimize the whole system jointly. uh so i would decouple and i guess what. you're seeing in terms of the fusion. of the sensing data from different. modalities as well as kind of fusion. at in the temporal level going more from. you know frame by frame. yeah where you know you would have one.

net that would do frame by frame. detection and camera and then you know. something that does frame by frame and. lighter and then radar and then you fuse. it you know in a weaker engineered way. later. like the field over the last you know. decade has been evolving in more kind of. joint fusion more end-to-end models that. are. solving some of these tasks you know. jointly and there's tremendous power in. that. and you know that that's that's that. that's the progression that kind of our. technology. our stack has been on as well now it's. your you know that so i would decouple. the kind of. sensing and how that information is used. from the role of ml in the entire stack.

and you know i guess it's uh i there's. trade-offs. uh and you know modularity and how do. you inject. inductive bias into your system right. this is. uh there's tremendous power in being. able to do that. so you know we have there's no. part of our system that is not heavily. that does not heavily you know leverage. uh. data-driven development or a. state-of-the-art ml. but there's mapping there's a simulator.

there's perception. you know object level you know. perception whether it's semantic. understanding prediction. decision making you know so forth and so. on um. it's and of course object detection and. classification like you're finding. pedestrians and cars and cyclists and. you know cones and signs and vegetation. and being very good at estimating kind. of detection classification and state. estimation. there's just stable stakes like like. that's step zero of this whole stack. you can be incredibly good at that. whether you use cameras or light as a.

radar but they're just you know that's. stable stakes that's just stub zero. beyond that you get into the really. interesting challenges of semantic. understanding of the perception level. you get into scene level reasoning you. get into very deep problems. uh that have to do with prediction and. joint production and interaction so. interaction between all of the actors in. the environment pedestrian cyclists. other cars and you get into decision. making. right so how do you build a lot of. systems so uh we. leverage ml very heavily in all of these. components i do believe that the best. results. you achieve by kind of using a hybrid.

approach. and having different types of ml. having different models with. different degrees of inductive bias that. you can have. and combining kind of model you know. free approaches with some you know model. based approaches and. some uh rule-based uh physics-based. uh systems so you know one example i can. give you is traffic lights. uh there's problem of the detection of. traffic light state and obviously that's.

a great problem for you know computer. vision confidence are. you know that's their bread and butter. right that's how you build that. but then the interpretation of you know. of a traffic light that you're going to. need to learn that right you you. read you don't need to build something. you know complex ml model that you know. infers with some you know precision and. recall that red means stop. like it was a it's a very clear. engineered signal with very clear. semantics. right so you want to induce that bias. like how you induce that bias and that.

whether you know it's a constraint. or a cost you know function in your. stack. but like it is important to be able to. inject that like clear semantic. signal into your stack and you know. that's what we do um. and but then the question of like and. that's when you. apply it to yourself when you're making. decisions whether you want to stop for a. red light. you know or not but if you think about. how other people. treat traffic lights we're back to the. ml version of that. because you know they're supposed to.

stop for a red light but that doesn't. mean they will so then you're back in. the like very. uh heavy uh ml domain where you're. picking up on like very subtle keys. about. you know that have to do with the. behavior of objects pedestrians cyclists. cars and the whole entire configuration. of the scene. that allow you to make accurate. predictions on whether they will in fact. stop. or run a red light so it sounds like a. ready. for waymo like machine learning is a. huge part of the stack. so it's a huge part of like uh not just.

so obviously the the first the level. zero or whatever you said which is like. just object detection of things that you. know with no that machine learning can. do but also starting to. to do prediction behavior and so on to. model the. what other or the other parties in the. scene entities in the scene are gonna do. so machine learning is more and more uh. playing a role in that. as well of course absolutely i think. we've been. and going back to the earliest days like.

you know darpa. even the grand challenge and team was. leveraging you know machine learning. i was like pre you know image nut and it. was very different type of ml. but uh and i think actually that was. before my time but the stanford team. on during the grand challenge had a very. interesting machine learned. system that would you know use lighter. and camera. when driving in the desert and it we had. built the model. uh where it would kind of extend the. range. of free space reasoning so we get a. clear signal from lighter.

and then it had a model that hey like. this stuff and camera kind of sort of. looks like this stuff and. lighter and i know this stuff and that. i've seen in lighter i'm very confident. there's free space so let me extend that. uh free space zone into the camera range. that would allow the vehicle to drive. faster. right and then we've been building on. top of that and kind of staying and. pushing the state of the art in a ml. in all kinds of different ml uh over the. years and in fact uh from the earlier. days i think you know 2010. is probably the year where google. uh maybe 2011 probably got got pretty.

heavily involved. in uh machine learning uh kind of deep. nuts. uh and at that time was probably the. only company that was very heavily. investing. in kind of state-of-the-art ml and. self-driving cars. right and they they they go ahead you. know hand in hand. and we've been on that journey ever. since we're doing uh pushing. a lot of these areas uh in terms of. research you know at waymo and we. collaborate. very heavily with the researchers in. alphabet. and like all kinds of mel yeah supervise. the male unsupervised male.

uh you know published some uh. interesting uh research papers in the. space uh. especially recently it's just super. super learning as well yeah so super. super active uh. of course there's you know kind of like. more uh mature stuff. like you know confidence for you know. object detection but there's some really. interesting really active uh. work that's happening in um kind of more. uh you know and bigger models and you. know models. that uh have more structure uh to them.

uh you know not just you know large. bitmaps and reasonable temporal. sequences. and some of the interesting. breakthroughs that you've you know we've. seen. in language models right you know. transformers you know. you know gpd 3 and friends uh there's. some really interesting applications of. some of the core breakthroughs to those. problems. of you know behavior prediction as well. as you know decision making and planning. right you. think about it kind of the the behavior. how you know the path the trajectories. the the how people drive.

and they have kind of a share a lot of. the fundamental structure. you know this problem there's you know. sequential. you know nature there's a lot of. structure uh in this representation. there is a strong locality kind of like. in sentences you know words that follow. each other. they're strongly connected but there's. also a kind of larger context that. doesn't have that locality and you also. see that in driving right what's. happening in the scene. as a whole has very strong implications. on. uh you know the kind of the next step in.

that sequence where. whether you're predicting what other. people are going to do whether you're. making your own decisions. or whether in the simulator you're. building generative models of. you know humans walking cyclists riding. another car is driving oh that's that's. all really fascinating like how. it's fascinating to think that uh. transformer models and all this. all the breakthroughs in language and. nlp that might be applicable to like. driving at the higher level. at the behavioral level that's kind of. fascinating um let me ask about pesky. little creatures called pedestrians and. cyclists.

they seem so humans are a problem if we. can get rid of them i would. um but unfortunately they're all sort of. a source of joy and love and beauty so. let's keep them around they're also our. customers oh for your perspective yes. yes. for sure there's some money very good. um but. uh i don't even know where i was going. oh yes pedestrians and cyclists. uh i you know. they're a fascinating injection into the. system of.

uh uncertainty of um. of like a game theoretic dance of what. to do and and also. they have perceptions of their own. and they can tweet about your product so. you don't want to run them over. from that perspective uh i mean i don't. know i'm joking a lot but that. i think in seriousness like you know. pedestrians are complicated um. uh computer vision problem a complicated. behavioral problem.

is there something interesting you could. say about what you've learned. from a machine learning perspective from. also an autonomous vehicle. and a product perspective about just. interacting with the humans in this. world. yeah just you know stayed on the record. we care deeply about the safety of. pedestrians. you know even the ones that don't have. twitter accounts um. thank you all right but you know not me. but yes i i'm glad i'm glad somebody. does okay uh but you know. in all seriousness safety of.

uh vulnerable road users pedestrians or. cyclists is one of our highest. priorities. we do a tremendous amount of testing. and validation and put a very. significant emphasis. on you know the capabilities of our. systems that have to do with safety. around those unprotected vulnerable road. users. um you know cars just you know discussed. earlier in phoenix we have completely. empty cars completely driverless cars. you know. driving in this very large area and you.

know some. people use them to you know go to school. so they'll drive through school zones. right. kids are kind of the very special class. of those vulnerable user road users. right you want to be. super super safe and super super. cautious around those so we take it very. very very seriously. um and you know what does it take uh to. uh be good at it uh you know. an incredible amount of uh. performance across your whole stack.

you know starts with hardware and again. you want to use all. sensing modalities available to you. imagine driving on a residential road at. night. and kind of making a turn and you don't. have you know headlights covering some. part of the space and like you know. a kid might run out and you know. lighters are amazing at that. they see just as well in complete. darkness as they do during the day right. so just again it gives you that extra. uh uh you know margin.

in terms of your capability and. performance and safety and quality. and in fact we oftentimes uh in these. kinds of situations we have our system. detect. something in some cases even earlier. than our trained operators. in the car might do especially in. conditions like you know. very dark nights um so starts with. sensing. then you know perception has to be. incredibly good and you have to be very. very good at. kind of detecting uh pedestrians. uh in all kinds of situations and all.

kinds of environments including. people in weird poses uh people kind of. running around. and you know being partially occluded um. so you know that that's stop number one. then you have to have in. very high accuracy and very low latency. in terms of your reactions. to you know what you know these uh. actors might do right and we've put a. tremendous amount of engineering and. tremendous amount of validation in to.

make sure our system performs. uh and you know oftentimes it does. require a very strong reaction. to do the same thing and we actually see. a lot of cases like that that's the long. tail of really. rare you know really uh kind of crazy. events. that contribute to the safety. around pedestrians like one one example. that comes to mind that we actually. happened uh in phoenix where we were uh. driving uh along and i think it was a 45. mile per hour road so in pretty high.

speed traffic. and there was a sidewalk next to it and. there was a cyclist. on the sidewalk and as. uh we were in the right lane and right. next to the site so it was a multi-lane. road. so as we got close to the cyclist on the. sidewalk. uh it was a woman and she tripped and. fell just you know fell right into the. path of our vehicle. right um and our you know cart. uh uh you know this was actually with a. test driver our test drivers uh. uh did exactly the right thing uh they.

kind of reacted and came to stop it. requires both very strong steering. and uh you know strong application of. the brake uh and then we simulated what. our system would have done in that. situation and it did. exactly the same thing it uh and that. that speaks to. all of those components of really good. uh state estimation and tracking. and like imagine you know a person on a. bike and they're falling over. and they're doing that right in front of. you right so you have to be real like. things are changing the appearance of. that whole. thing is changing right and the person. goes one way they're falling on the road. they're you know. being flat on the ground in front of you.

you know the the bike goes flying the. other. direction like the two objects that used. to be one they're now you know. uh are splitting apart and the car has. to like detect all of that uh like. milliseconds matter. and it doesn't it's not good enough to. just break you have to like steer. and break and there's traffic around you. so like it all has to come together. and it was really great uh to see in. this case and other cases like that. that we're actually seeing in the wild. that our system is you know performing. exactly the way uh that we would have. liked and is able to you know avoid. uh collisions like this such an exciting.

space for robotics. like in that split second to make. decisions of life and death. i don't know if the stakes are high in. the sense but it's also beautiful. that um um for somebody who loves. artificial intelligence. the possibility that an ai system might. be able to save a human life. that's kind of exciting as a as a. problem like to wake up. you get it's terrifying probably from. energy for an engineer to wake up. and to think about but it's also. exciting because it's like.

it's it's in your hands let me try to. ask a question that's often brought up. about autonomous vehicles. and it might be fun to see if you have. anything anything interesting to say. which is about the trolley problem. so uh a trolley problem is a interesting. philosophical construct. of uh that highlights and there's many. others like it. of the difficult ethical decisions that. uh we humans. have before us in this complicated world.

uh so. the specifically is the choice between. if you were forced to choose uh to kill. a group x of people versus a good why of. people like. one person if you didn't if you did. nothing you would kill one person but if. you would kill five people and if you. decide to swerve out of the way you. would only kill one person. do you do nothing or you choose to do. something you can construct all kinds of. sort of. ethical experiments of this kind that.

um i i think at least on a positive note. inspire you to think about like. introspect. what are the the physics. of our morality and there's usually not. good. answers there i think it people love it. because it's just an exciting. thing to think about i think people who. build autonomous vehicles usually roll. their eyes. because uh this is not this.

one as constructed this like literally. never comes up. in reality you never have to choose. between killing. one like one of two groups of people. but i wonder if you can speak to. is there some something interesting. to use an engineer of autonomous. vehicles that's within the trolley. problem. or maybe more generally are there. difficult. ethical decisions that you find that the. algorithm must make on the specific.

version of the trial problem which one. would you do. if you're driving the question itself is. a profound. question because we humans ourselves. cannot answer and that's the very. point uh i guess i would kill both. um yeah humans i think you're exactly. right and that you know humans are not. particularly good. i think they kind of phrased as a like. what would a computer do but like. humans you know are not very good and i. actually often times. i think that you know freezing and kind.

of not doing anything because. like you've taken a few extra. milliseconds to just process and then. you end up. like doing the worst of the possible. outcomes right so um i. i do think that as you've pointed out it. can be. a bit of a distraction and it can be a. bit of a kind of red herring i think. it's an interesting philosophy. discussion in the realm of uh philosophy. um right but in terms of what you know. how that affects the. actual engineering and deployment of. self-driving vehicles. i um it's not how you go about building.

a system right we have talked about how. you engineer a system. how you go about evaluating the. different components. and you know the safety of the entire. thing how do you kind of inject. the you know various model based safety. based arguments and you're like yes you. reason it parts the system. you know you reason about the. probability of a collision the severity. of that collision right. and that is incorporated and there's you. know you have to properly reason about. uncertainty that flows through the. system right so. you know those uh um you know.

factors definitely play a role in how. the cars don't behave but they have to. be more. of like the immersion behavior and what. you see like you're absolutely right. that these you know. clear uh theoretical problems that they. you know you you don't require that. in system and really kind of being back. to our previous discussion of like what. what you know what. what you know which one do you choose. well you know oftentimes. like you made a mistake earlier like you. shouldn't be in that situation. uh in the first place right and in. reality the system comes up.

if you build a very good safe and. capable driver. you have enough uh you know clues uh in. the environment that. you drive defensively so you don't put. yourself in that situation right. and again you know it has you know this. if you go back to that analogy of you. know precision and recall like okay you. can make a. very hard trade-off of the i1 but like. neither answer is really good. but what instead you focus on is kind of. moving the whole curve up. and then you focus on building the right. capability and the right defensive. driving so that you know you don't put. yourself in a situation like this.

i don't know if you have a good answer. for this but people love it when i ask. this question. about books um are there books. in um in your life that you've enjoyed. philosophical fiction. technical that had a big impact on you. as an engineer or as a human being. you know everything from science fiction. to a favorite textbook. is there three books that stand out that. you can think of. uh three books so i would uh you know. that impacted me. um i would say uh.

this one is you probably know it well. um but and not generally well known. i i think in the u.s or kind of. internationally the master. and margarita it's uh one of. actually my favorite uh books um. it is you know by a russian it's a novel. by. russian author uh mikhail bulgakov and. it's just it's it's a great book and. it's one of those books that you can. like. reread your entire life and it's very.

accessible you can read it as a kid. and like it's it you know it's that the. plot is interesting it's you know the. the devil. you know visiting the soviet union and. yeah but it. it like you read it reread it at. different stages of your life and. you yeah you enjoy it for different very. different reasons. and you keep finding like deeper and. deeper meaning uh and you know kind of. affected. you know hadn't definitely had an like. imprint on me. mostly from the probably kind of the. cultural stylistic. uh aspect like it makes you one of those.

books that you know is good and makes. you think but also. has like this really you know silly. quirky dark sense of you know humor hey. casper is the russian so that's more. than maybe perhaps many other books. on that like slight no just out of. curiosity one of the saddest things is. i've read that book. in english did you by chance read it in. english or in russian. uh in russian only in russian uh and i. actually that that is a question i had. uh uh kind of pose to myself every once. in a while like i wonder how well it.

translates. if it translates at all and there's the. language aspect of it and then there's. the cultural aspect so i. and actually i'm not sure if you know. either of those would so work well in. english. now i forget their names but so when the. covid lists a little bit i'm traveling. to paris. uh for for several reasons one it's just. i've never been to paris i want to go to. paris but. there's a the most famous translators. of uh destielski tolstoy of most of. russian literature. live there there's a couple they're. famous a man and a woman.

and i'm going to sort of have a series. of conversations with them and in. preparation for that i'm. starting to read dusty sk in russian so. i'm really embarrassed to say that i. read. this everything i've read russian. literature of like. serious depth has been in english even. though. i can also read i mean obviously in. russian but. for some reason it seemed. uh in the optimization of life it seemed. the improper decision to do to read in.

russian. like you know like i don't need to opt i. need to think in english not in russian. but now i'm changing my mind on that and. so the question of how well it. translates it's a really fundamental one. like it even with dostoyevsky. so from what i understand this death can. translate easier. uh others don't as much obviously the. poetry doesn't translate as well. i'm also the the music of a big fan of. vladimir wassotsky. he doesn't obviously translate well.

people have tried. but mastermind i don't know i don't know. about that one i just know it in english. you know it's fun. fun as hell in english so uh so but it's. a curious question and i want to study. it. rigorously from both the machine. learning aspect and also because i. want to do a couple of interviews in. russia. that i'm still unsure. of how to properly conduct an interview. across a language barrier it's a. fascinating question.

that ultimately communicates to an. american audience there's a few. russian people that i think are truly. special human beings. and i feel. like i sometimes encounter this with. some incredible scientists and maybe. you encounter this as well at some point. in your life that it feels like because. of the language barrier their ideas are. lost to history. it's a sad thing i think about like. chinese scientists or even authors that.

like. that we don't in english-speaking world. don't get to appreciate. some like the depth of the culture. because it's lost in translation. and i feel like i would love to show. that to the world. like i'm i'm just some idiot but because. i have this. like at least some semblance of skill in. speaking russian. i feel like and i know how to record. stuff on a video camera. i feel like i want to catch like gregory. pearlman who's a mathematician i'm not. sure if you're familiar with him.

yeah i want to talk to him like he's a. fascinating mind and to bring him to a. wider audience in english speaking. it'll be fascinating but that requires. to be rigorous about this question. of how well uh bulgakov translates. i mean i i know it's a it's a silly. concept but it's a fundamental one. because how do you translate and that's. that's the thing that uh google. translate is also facing. yeah uh as a as a more machine learning. problem but i. i wonder is a more bigger problem for ai.

how do we capture the magic that's there. in the language. i i think that's a really interesting. really. challenging problem i if you do read it. master and margarita. in uh english uh sorry in russian i'd be. curious. get your uh opinion and i think part of. it is language but part of it's just you. know centuries of culture. that the cultures are different so it's. hard to. connect that but uh okay so that was my. first one right you had to know tomorrow.

um the second one i would probably pick. the science fiction by the stragoski. brothers. uh you know it's up there with you know. isaac asimov and you know ray bradbury. uh and you know company uh the straguski. brothers kind of appealed more to me i. think more it made more of an impression. on me. uh growing up um can you i. apologize if i'm showing my complete. ignorance i'm so weak on sci-fi. which what what are they right oh um.

uh roadside picnic um. [Music]. uh hard to be a god uh. uh beetle in an ant hill. uh monday starts on saturday like it's. it's not just science fiction it's also. like has very interesting you know. interpersonal. and societal questions and some of the. language is just. completely hilarious. that's the one that's right oh. interesting monday starts on saturday so.

i need to read. okay oh boy you put that in the category. of science fiction. uh that one is i mean this was more of a. silly. you know humorous uh work i mean there. is kind of. profound too right science fiction right. is about you know this this research. institute and like. this it it has deep parallels to like. serious. research but the the setting of course. is that they're working on you know. magic. right and there's a lot of stuff so i i. i i that that's their style right. they go and you know other books are.

very different right. you know hard to be a god right it's. about kind of this higher society being. injected into this primitive world and. how they operate there. like some of the very deep ethical you. know questions there. right and like they've got this spectrum. some as you know more about kind of more. uh adventure style but like i i enjoy. all of their books there's. probably a couple actually one i think. that they consider their most important. work. i think it's the snail on an on a a hill. i don't know exactly how sure how it. translates i tried reading a couple of.

times i still don't get it. but everything else i fully enjoyed uh. and like for one of my birthdays as a. kid i got like their entire collection. like occupied a giant shelf in my room. and then like over the holidays i just. like. you know my parents couldn't drag me out. of the room and i read the whole thing. cover to cover. and it it uh i really enjoyed it. uh and that's it one more i thought for. the third one i you know maybe a little. bit darker. um uh but you know comes to mind is. orwell's 1984. uh. and i you know you asked what made an.

impression on me and books that people. should read that one i think falls in. the category of. both now you know definitely it's one of. those books that you read and. you just kind of you know put it down. and you stare in space for a while. uh yeah you know that that that kind of. work uh i i think there's. you know lessons there people uh should. not ignore. and you know nowadays with like. everything that's happening in the world. i. i can't help it but you know have my. mind jump to some. you know parallels uh with what orwell.

described and like there's this whole. you know concept of double think and. ignoring logic. and you know holding completely. contradictory opinions in your mind and. not have that not bother you and you. know sticking to the party line. yeah uh at all costs like you know. there's there's there's something. there if anything 2020 has taught me. and i'm a huge fan of animal farm which. is a kind of friendly. as a friend of 1984 by orwell. it's kind of another thought experiment.

of how our society. may go in directions that we wouldn't. like it to go. but if if anything that's been. [Music]. kind of heartbreaking to an optimist. about 2020. is that that society is kind of fragile. like we have this this is a special. little experiment we have going on. and not it's not unbreakable like. we should be careful to like preserve.

whatever special thing we have going on. i mean i think 1984 in these books brave. new world. they they're helpful in thinking like. stuff can go wrong in non-obvious ways. and it's like it's up to us to preserve. it. and it's like it's a responsibility it's. been weighing heavy on me because like. for some reason like uh. more than my mom follows me on twitter. and i feel like. i have i have like now somehow a.

responsibility to. um to this world and it. dawned on me that like me and millions. of others. are like the little ants that maintain. this little colony. right so we have a responsibility not to. be uh i don't know what the right. analogy is but. i'll put a flamethrower to the place we. want to. not do that and there's interesting. complicated ways of doing that as 1984. shows. it could be through bureaucracy it could. be through incompetence it could be.

through misinformation. it could be through division and. toxicity uh. i'm a huge believer in like that love. will be. the somehow the solution so. uh loving robots yeah. i i think you're exactly right. unfortunately i think it's uh less of a. flamethrower. type of next i think it's more of a in. many cases can be more of a slow boil. and that that's the danger let me ask uh. it's a fun thing to make a world-class.

roboticist engineer. and leader uncomfortable with a. ridiculous question about. life what is the meaning of life. at dmitry from a robotics and a human. perspective. you only have a couple minutes or one. minute to answer so. i don't know if that makes it more. difficult or easier actually yeah. you know they're very tempted to. uh quote uh one of the stories.

stories by uh uh isaac asimov actually. um actually titled appropriately titled. the last question uh. short story where you know the plot is. that you know humans build this super. computer you know this this this ai. intelligence and you know once. it's get power gets powerful enough they. pose this question to it you know. um how can the entropy in the universe. be reduced all right so your computer. replies. and as of yet insufficient information. to give a meaningful answer. right and then you know thousands of. years go by and they keep posing the.

same question the computer you know. it gets more and more powerful and keeps. giving the same answer yeah as of yet. insufficient information to give a. meaningful answer or something along. those lines. right and then you know keeps you know. happening and. happening you fast forward like millions. of years into the future and you know. billions of years and like at some point. it's just the only entity in the. universe. it's like absorbed all humanity and all. knowledge in the universe and it like. keeps posing the same question to itself. and you know. finally it gets to the point where it is. able to answer that question but of.

course at that point you know there's. you know the heat death of the universe. has occurred and that's the only entity. and there's nobody else to provide that. answer to so the only thing it can do is. to. you know answer it by demonstration so. it like you know recreates the. big bang right and resets the clock. right. but i i can try to give kind of a. a different version of the answer you. know maybe. uh not on the behalf of all humanity i. think that that might be a little. presumptuous for me to speak about the. meaning of life on. the behalf of all humans uh but at least.

you know personally. uh it changes right i think if you think. about kind of what. uh gives uh. you know you and your life meaning and. purpose and kind of what drives you. um it seems to. change over time right and the the the. lifespan. of you know your existence uh you know. when. just when you just enter this this world. right it's all about kind of new. experiences. and you get like new smells new sounds.

new emotions right. and like that's what's driving you right. you're experiencing. new amazing things right and that that's. magical right. that's pretty pretty pretty pretty. awesome right that gives you kind of. meaning. then you get a little bit older you. start more intentionally. uh learning about things right i guess. actually before you start intentionally. learning probably fun. fun is a thing that gives you kind of. meaning and purpose and purpose and the. thing you optimize for right. and like fun is good uh then you get you. know start learning and i guess that.

this this. joy of comprehension and discovery. is another thing that you know gives you. meaning and purpose and drives you right. then. you know you learn enough stuff and. it you want to give some of it back. right and so impact and contributions. back to you know technology or society. uh uh people uh you know local or more. globally yeah. is becomes a new thing that you know. drives a lot of kind of your behavior. and something that gives you purpose and.

that you derive you know positive. feedback from right. you know then you go and so on and so. forth you go through various stages of. life. if you have if you have kids. like that definitely changes your. perspective on things you know i have. three. that definitely flips some bits in your. head in terms of. you know what you care about and what. you optimize for and you know what. matters what doesn't matter right. so you know and so on and so forth right. and i i i. it seems to me that you know it's all of.

those things and as. kind of you go through life um. you know you want these to be additive. right. new experiences fun learning. impact like you want you want to you. know be accumulating other you know i. don't want to you know. stop having fun or experiencing new. things and i think it's important that. it just kind of becomes uh additive as. opposed to a replacement or subtraction. but you know views probably as far as i. got but you know ask me in a few years i. might have one or two more to add to the. list.

and before you know it time is up just. like it is for this conversation. uh but hopefully it was a fun ride it. was a huge honor to meet you as. you know i've been a fan of yours and a. fan of. google self-driving car and waymo for a. long time. i can't wait i mean it's one of the most. exciting if we look back in the 21st. century i truly believe it'll be one of. the most exciting things we. descendants of apes have created on this. earth so. i'm a huge fan and i can't wait to see. what you do.

next thanks so much for talking today. thanks thanks for having me and it's a. also a huge fan doesn't work honestly. and uh i really enjoyed it thank you. thanks for listening to this. conversation with dimitri dalgov. and thank you to our sponsors trial labs. a company. that helps businesses apply machine. learning to solve real world problems. blinkist an app i use for reading. through summaries of books. better help online therapy with a. licensed professional. and cash app the app i use to send money.

to friends. please check out these sponsors in the. description to get a discount. and to support this podcast if you. enjoyed this thing. subscribe on youtube review 5000 upper. podcast. follow on spotify support on patreon or. connect with me on twitter at lex. friedman. and now let me leave you with some words. from isaac. asimov science can amuse and fascinate. us all. but it is engineering that changes the. world. thank you for listening and hope to see. you next time.

you.