Wojciech Zaremba: OpenAI Codex, GPT-3, Robotics, and the Future of AI | Lex Fridman Podcast #215

the following is a conversation with. wojciech zaramba co-founder of openai. which is one of the top organizations in. the world doing artificial intelligence. research and development. wojciech is the head of language and. cogeneration teams building and doing. research on github copilot openai codex. and gpt. three and who knows. four five. six. n and. n plus one and he also previously led.

openai's robotic efforts. these are incredibly exciting projects. to me that deeply challenge and expand. our understanding of the structure and. nature of intelligence the 21st century. i think may very well be remembered for. a handful of revolutionary ai systems. and their implementations. gpt codex and applications of language. models and transformers in general. to the language and visual domains may.

very well be at the core of these ai. systems. to support this podcast please check out. our sponsors. they're listed in the description. this is a lex friedman podcast and here. is my conversation with wachek zaremba. you mentioned that sam altman asked. about the fermi paradox. and the people at open ai had really. sophisticated interesting answers so. that's when you knew this is the right.

team to be working with so let me ask. you about the fermi paradox about aliens. why have we not found overwhelming. evidence for aliens visiting earth. i don't have a conviction in the answer. but rather kind of probabilistic. perspective on what might be a let's say. possible answers it's also interesting. that the question itself even. can't touch on the you know your typical. question of what's the meaning of life. because like if you assume that like we. don't see aliens because they destroy. themselves that kind of upwards their.

focus on making sure that we won't. destroy ourselves yeah and at the moment. the. place where i am actually with my belief. and these things also change over the. time. is i think that we might be alone in the. universe which actually makes. life more or less a consciousness life. more kind of valuable and that means. that we should more appreciate it. have you always been alone so what's. your intuition about our galaxy our.

universe is it just. sprinkled with graveyards of intelligent. civilizations or are we truly is is life. intelligent life truly unique. at the moment my belief that it is. unique but i would say i could also. you know there was like some footage. released with ufo objects which makes me. actually doubt my own belief yes. yeah i can tell you one crazy answer. that i have heard yes. so. apparently when you look actually at the.

limits of computation. you can compute more. if the temperature of the universe would. drop down. so one of the things that. aliens might want to do if they are. truly optimizing to maximize amount of. compute which you know maybe can lead to. or let's say simulations or so it's. instead of wasting current entropy of. the universe because you know we by. living we are actually somewhat wasting. entropy. then you can wait for the universe to.

cool down such that you have more. computation that's kind of a funny. answer i'm not sure if i believe in it. but that would be one of the reasons why. you don't see aliens it's also possible. see some people say that maybe there is. not that much point in actually going to. other galaxies. if you can go inwards. so there is no limits of what could be. an experience. if we could you know connect machines to. our brains. while there are still some limits if we. want to explore universe yeah there.

could be a lot of. ways to go inwards too. once you figure out some aspect of. physics we haven't figured out yet maybe. you can travel to different dimensions i. mean. travel in three-dimensional space may. not be the most fun kind of travel there. may be like just a huge amount of. different ways to travel and it doesn't. require a spaceship going. slowly in 3d space space time it also. feels you know one of the problems is.

that speed of light is low and universe. is vast yeah and um. it seems that actually most likely if we. want to travel very far. then then we would instead of actually. sending spaceships with humans that wait. a lot we would. send something similar to what yuri. miller is working on these are like a. huge uh sail which is at first powered. power there is a shot of laser from an. earth and it can propel it to a quarter. of speed of light and uh sail itself.

contains a. few grams of equipment and that might be. the way to actually. transport matter through universe but. then when you think what would it mean. for humans it means that. we would need to actually put their 3d. printer and you know 3d print a human on. other planet i don't know play them. youtube or let's say or like a pre 3d. print like a huge human right away or. maybe a womb or so um yeah. with our current techniques of.

archaeology. if. a civilization was born and died. long long enough ago on earth we. wouldn't be able to tell and so that. makes me really sad. and so i think about earth in that same. way how can we leave some remnants if we. do destroy ourselves how can we leave. remnants for aliens in the future to. discover. like here's some nice stuff we've done. like wikipedia and youtube do we have it. like. in a satellite orbiting earth.

with a hard drive like how how do we say. how do we back up human civilization. uh for the good parts or. all of it is good parts. so that uh. it can be preserved longer than our. bodies can that's a. that's kind of a. it's a difficult question it also. requires the difficult acceptance of the. fact that we may die and if we die we. may. die suddenly as a civilization. so let's see i think it kind of depends.

on the cataclysm we have observed in. other parts of the universe that births. of gamma rays. these are. high energy. rays of light that actually can. apparently kill entire galaxy. so there might be actually nothing even. to. nothing to protect us from it i'm also. when i'm looking actually at the past. civilization so it's like aztecs or so. they disappear from the. surface of the earth and one can ask.

why is it the case. and. the way i'm thinking about it is. you know that definitely they had some. problem that they couldn't solve. and maybe there was a flat and all of a. sudden they couldn't drink there was no. potable water and they all died. and. i think that. so far. the best solution to such a problems is. i guess technology so i mean if they. would know that you can just boil water.

and then drink it after then that would. save their civilization and even now. when we look actually at the current. pandemic it seems that once again. actually science comes to rescue and. somehow science increases size of the. action space and i think that's a good. thing. yeah but nature. has a vastly larger action space but. still it might be a good thing for us to. keep on increasing action space. okay. looking at past civilizations yes.

but looking at the destruction of human. civilization. perhaps expanding the action space will. add. actions that are easily. acted upon easily executed and as a. result destroy. us. so let's see. i was pondering. why actually even. we have negative impact on the. globe because you know if you ask every. single individual they would like to.

have clean air. they would like healthy planet but. somehow it actually is not the case that. as a collective we are not going this. direction. i think that there exists very powerful. system to describe what we value that's. capitalism it assigns actually monetary. values to various activities at the. moment the problem in the current system. is that there are some things which we. value there is no cost assigned to it so. even though we value clean air or maybe. we also.

value. lack of destruction on the internet or. so at the moment. these quantities you know companies. corporations can pollute them uh for. free. so in some sense. i wish. or like and that's i guess purpose of. politics to. align the incentive systems and we are. kind of maybe even moving in this. direction the first issue is even to be. able to measure the things that we value.

then we can actually assign the monetary. value to them. yeah and that's so it's getting the data. and also. probably through technology enabling. people to vote. and to. move money around in a way that is. aligned with their values and that's. very much a technology question so like. having one president. and congress. and voting that happens every four years. or something like that. that's a very outdated idea there could.

be some technological improvements to. that kind of idea so. i'm thinking from time to time about. these topics but it also feels to me. that it's it's a little bit like a. it's hard for me to actually make. correct predictions what is the. appropriate thing to do i extremely. trust uh sam altman our ceo. on these topics he um okay i'm more on. the side of being i guess. naive hippie that. yeah. that's your life philosophy um.

well like i think self-doubt. and uh. i think hippie implies optimism those. those two things are pretty pretty good. way to operate. i mean still it is. hard for me to actually. understand how the politics works or. like uh how this like. exactly how the things would play out. and sam is a really excellent with it. what do you think is rarest in the. universe you said we might be alone.

what's hardest to build is another. engineering way to ask that. life. intelligence or consciousness so like. you said that we might be alone. which is the thing that's hardest to get. to. is it just the origin of life is it the. origin of intelligence is it the origin. of consciousness. so. um let me at first explain you my kind. of mental model what i think is needed. for life to appear.

um. so. i imagine that at some point there was. this primordial. zoop of. amino acids and maybe some proteins in. the ocean. and you know some proteins were turning. into some other proteins through. reaction. and you can almost think about this. uh cycle of what turns into what as. there is a graph essentially describing. which substance turns into some other. substance and essentially life means. that all the sudden in the graph has.

been created a cycle such that the same. thing keeps on happening over and over. again that's what is needed for life to. happen and in some sense you can think. almost that you have this gigantic graph. and it needs like a sufficient number of. edges for the cycle to appear. then um from perspective of intelligence. and consciousness. my current intuition is that they might. be. quite intertwined first of all it might. not be that it's like a binary thing. that you have intelligence or. consciousness it seems to be a.

more a. continuous component let's see if we. look for instance on the even networks. recognizing images and people are able. to show that the activations of these. networks correlate very strongly. with activations in visual cortex. of some monkeys the same seems to be. true about language models. also if you for instance.

look. if you train agent in a 3d world. at first you know it it it it barely. recognizes what is going on over the. time it kind of recognizes foreground. from a background over the time it kind. of knows where there is a foot. and it just follows it. over the time it actually starts having. a 3d perception so it is possible for. instance to look inside of the head of. an agent and ask what would it see if it. looks to the right and the crazy thing.

is you know initially when the agents. are very trained these predictions are. pretty bad over the time they they. become better and better you can still. see. that if you ask what happens when the. head is turned by 360 degrees for some. time they think that the different thing. appears and then at some stage they. understand actually that the same. thing's supposed to appear so they get. like a understanding of 3d structure. it's also you know very likely that they. have inside some. level of and of like a symbolic.

reasoning like they're particularly. symbols for other agents so when you. look at dota agents they collaborate. together and uh. and. now they they they have some. anticipation of uh if if they would win. battle they have some some expectations. with respect to other agents i might be. you know too much anthropomorphizing. um the the how the things look. look for me but then the fact that they.

have a symbol for other agents. and makes me believe that. at some stage as the uh you know as they. are optimizing for skills they would. have also symbol to describe. themselves this is like a very useful. symbol to have and this particularity i. would call it like a self-consciousness. or self-awareness. and still it might be different from the. consciousness so i guess the the way how. i'm understanding the word consciousness.

let's say the experience of drinking a. coffee or let's say experience of being. a butt. that's the meaning of the word. consciousness it doesn't mean to be. awake. yeah it feels. it might be also somewhat related to. memory and recurrent connections so um. it's kind of okay if you look at. anesthetic drugs they might be. uh like they essentially. they disturb. brain waste. such that.

[Music]. maybe memory is not not formed. so there's a lessening of consciousness. when you do that correct and so that's. one way to intuit what is consciousness. there's also kind of another. element here it could be that it's. you know this kind of self-awareness. module that you described. plus the actual subjective experience. is a storytelling module. that tells us a story about uh.

what we're experiencing. the. crazy thing so let's say i mean in. meditation they teach people. not to speak story inside of the head. and there is also some fraction of. population. who doesn't have actually narrator i. know people who don't have a right. narrator and you know they have to use. external people in order to. kind of. solve tasks that require internal. narrator.

so. it seems that it's possible to have the. experience without the talk. what are we talking about when we talk. about the internal narrator is that the. voice when you're like yeah i thought. that that that's what you are referring. to well i was referring more on the like. not an actual voice. i meant like. there's some kind of. like subjective experience. feels like it's.

it's fundamentally about storytelling to. ourselves. it feels like. like the feeling is a story. that is much. much simpler abstraction than the raw. sensory information. so it feels like it's a very high level. abstraction. that. is useful. for me to feel like. entity in this world. most. useful aspect of it is that.

because i'm conscious. i think there's an intricate connection. to me not one. wanting to die. so like. it's a useful hack to really. prioritize not dying. like those seem to be somehow connected. so i'm telling the story of like it's. richly feels like something to be me and. the fact that me exists in this world i. want to preserve me. and so that makes it a useful agent hack.

so i will just refer maybe to the first. part as you said about the kind of story. of describing who you are. i was. thinking about that even so you know. obviously i'm i'm i. like thinking about consciousness uh i. like thinking about the ai as well and. i'm trying to see analogies of these. things in ai what would it correspond to. so um. you know openly i trained a.

a. model called gpt. which. can generate a. pretty amusing text on arbitrary topic. and um. and one way to control gpd. is uh by putting into prefix at the. beginning of the text some information. what would be the story about. you can have even chat with uh. you know with gpt by saying that the. chat is with lex or elon musk or so.

and gpt would just. pretend to be you or elon musk or so. and. it almost feels that this uh. story that we give ourselves to describe. our life it's almost like a. things that you put into context of gpt. yeah the primary it's the and but the. the context we provide to gpt. is uh. is multimodal it's so gpt itself is. multimodal gpt itself uh hasn't learned.

actually from experience of single human. but from the experience of humanity it's. a chameleon you can turn it into. anything and in some sense by providing. context uh. it you know. behaves as the thing that you wanted it. to be and it's interesting that the. you know people have a stories of who. they are and as i said these stories. they help them to operate in the world. but it's also you know interesting.

i guess various people find it out. through meditation or so that. there might be some patterns that you. have learned. when you were a kid that actually are. not serving you anymore. and you also might be thinking that. that's who you are and that's actually. just the story. yeah so it's a useful hack but sometimes. it gets us into trouble it's a local. optima. you wrote that stephen hawking he. tweeted stephen hawking asked what. breathes fire into equations which meant.

what makes given mathematical equations. realize the physics of a universe. similarly. i wonder what breathes fire into. computation what makes given computation. conscious. okay so how do we engineer consciousness. how do you breathe fire. and magic into the machine. so. it seems clear to me that not every. computation is conscious i mean you can. let's say just keep on multiplying one.

matrix over and over again and my. gigantic matrix you can put a lot of. computation i don't think it would be. conscious so in some sense the question. is. what are the computations which could be. conscious. uh i mean so one assumption is. that it has to do purely with. computation that you can abstract away. matter and other possibilities that it's. very important was the realization of. computation that it has to do with some. uh uh force fields or so and they bring. consciousness at the moment my intuition.

is that it can be fully abstracted that. way so in case of computation you can. ask yourself what are the. mathematical objects or so that could. bring such a properties so for instance. if we think about the. models uh ai models then what they truly. try to do. or like models like gpt is uh. you know they try to predict a next word.

or so and this turns out to be. equivalent to. compressing. text. and because in some sense compression. means that. you learn the model of reality and you. have just to uh. remember where are your mistakes the. better you are in predicting the. and. and in some sense when we look at our. experience also when you look for. instance the car driving you know in. which direction it will go you are good. like a in prediction and um. you know it might be the case that the.

consciousness. is intertwined with compression it might. be also the case that self-consciousness. has to do with compressor trying to. compress itself so. um. okay i was just wondering what are the. objects in you know mathematics or. computer science which are mysterious. that could uh that that could have to do. with consciousness and then i thought um. you know you you see in uh mathematics. there is something called cadal theorem.

which means okay you have if you have. sufficiently complicated mathematical. system it is possible to point the. mathematical system back on itself in. computer sense there is uh something. called helping problem it's it's. somewhat similar construction so i. thought that you know if we believe that. that the that. under assumption that consciousness has. to do with uh with compression. uh. then you could imagine that the the as.

you keep on compressing things then at. some point it actually makes sense. for the compressor to compress itself. metacompression yeah consciousness is. metacompression. that's uh that's and i and an idea. and in some sense you know the creation. of it. thank you so uh. but do you think if we think of a. touring machine a universal touring. machine. can that achieve. consciousness. so is there some. thing beyond our traditional definition.

of computation that's required so it's a. specific computation and i said this. computation has to do with compression. and. the compression itself maybe other way. of putting it is like you are internally. creating the model of reality. in order like a it's like a you try. inside to simplify reality in order to. predict what's going to happen. and. that also feels somewhat similar to how. i think actually about my own conscious. experience so clearly i don't have. access to reality the only access to.

reality is through you know cable going. to my brain and my brain is creating a. simulation of reality and i have access. to the simulation of reality. are you by any chance uh aware of uh. the harder prize marcus hutter. he he made this prize. for compression. of wikipedia pages. and. there's a few qualities to it. one i think has to be perfect. compression which makes. i think that little quirk makes it much.

less um. applicable to the general task of. intelligence because it feels like. intelligence is always going to be messy. uh. like perfect compression is feels like. it's not the right goal but it's. nevertheless a very interesting goal so. for him intelligence equals compression. and so. the smaller you make the file. given a large wikipedia page. the more intelligent the system has to.

be yeah that makes sense so you can make. perfect compression if you store errors. and i think that actually what he meant. is you have algorithm plus errors and by. the way hooter hatter is a he was pa uh. phd advisor of shenleck who is the mind. uh. uh deep mind co-founder yeah yeah so. there's an interesting. and now he's a deep mind there's an. interesting uh network of people he's. one of the people that. i think. seriously took on the task of what would.

an agi system look like. i think for a longest time. the question of agi was not. taken. seriously or rather rigorously. and he did just that like mathematically. speaking what would the model look like. if you remove the constraints of it. having to be. having to have a. reasonable amount of memory reasonable.

amount of running time complexity uh. computation time what would it look like. and essentially it's it's a. half math half philosophical discussion. of uh how would like a reinforcement. learning type of framework look like for. an agi yeah so he developed a framework. even to describe what's optimal with. respect to reinforcement learning like. there is a theoretical framework which. is as you said. under assumption there is infinite. amount of memory and compute and there. was actually one person before his name.

is solomonov hutter extended. amount of work to reinforcement learning. but there exists a. theoretical algorithm which is optimal. algorithm to build intelligence and i. can actually explain you the algorithm. yes. let's go let's go so the task itself can. i just. pause. how absurd it is. for brain in a skull trying to explain. the algorithm for intelligence just go.

ahead it is pretty crazy it is pretty. crazy that you know the brain itself is. actually so small and it can ponder. how to design algorithms that optimally. solve the problem of intelligence okay. all right so what's the algorithm so. let's see so first of all the task. itself is. described as. you have infinite sequence of zeros and. ones. okay you read n bits and you are about. to predict n plus one bit. so that's the task and you could imagine.

that every task could be casted as such. a task so if for instance you have. images and labels you can just turn. every image into sequence of zeros and. ones then label you concatenate labels. and you and that that's actually the the. and you could you could start by having. training data first and then afterwards. you have test data. so theoretically any problem could be. casted as a problem of predicting zeros. and ones on this infinite type so um.

so let's say you read already n bits and. you want to predict n plus one bit. and i will ask you to write. every possible program that generates. these end bits okay so. and you can have you you choose. programming language it can be in python. or c. and the difference between programming. languages. might be there is a difference by. constant. asymptotically your predictions will be. equivalent.

so you you read and beats you enumerate. all the programs that produce these and. end bits in their output. and then in order to predict n plus one. bit you actually weight. the programs according to their length. and there is like some specific formula. how you weight them and then the n plus. one bit prediction is the prediction uh. from each of this program according to. that weight. like statistically you statistically.

pick so the smaller the program the more. likely you you are to pick the its. output. so uh that's that algorithm is grounded. in the hope. or the intuition that the simple answer. is the right one it's a formalization of. it yeah um it also. means like if you would ask the question. after. how many years. would you know. sun explode. you can say. it's more likely the answer is.

to some power because it's a shorter. program. yeah. and then other. well i don't have a good intuition about. how different the space of short. programs are from the space of large. programs. like. what is the universe where short. programs. uh like run things. uh so as i said the things have to agree. with end beats so even if you have. you you need to start okay if if you. have very short program and they're like.

uh still some as if it's not perfect. with prediction of n bits you have to. start errors what are the errors and. that gives you the full program that. agrees on end beats. oh so you don't agree perfectly with the. end bits and you store. that's like a longer a longer program. slightly longer program. because it contains these extra bits of. errors that's fascinating what's what's. your intuition. about. the the programs. that are able to do cool stuff like. intelligence and consciousness are they.

uh. perfectly like is is it uh. is there if then statements in them so. like is there a lot of exceptions that. they're storing so um you could imagine. if there would be tremendous amount of. if statements yeah then they wouldn't be. that short in case of neural networks. you could imagine that. what happens is uh. they. when you start with an uninitialized. neural network uh it stores internally.

many possibilities how the. how the problem can be solved and sgd is. kind of magnifying some some. some. paths which are slightly. similar to the correct answer so it's. kind of magnifying correct programs and. in some sense hdd is a search algorithm. in the program space and the program. space is represented by uh you know kind. of the wiring inside of the neural. network and there's like an insane. number of ways how that features can be.

computed. let me ask you the high level basic. question that's not so basic. what is deep learning. is there a way you'd like to think of it. that is different than like a generic. textbook definition. the thing that i hinted just a second. ago is maybe the uh closest to how i'm. thinking these days about um deep. learning so. now the statement is. uh neural networks can represent some.

programs. uh it seems that various modules that we. are actually adding up to are like a you. know we we want networks to be deep. because we we want multiple steps of the. computation. and. and deep learning provides the way to. represent space of programs which is. searchable and it's searchable with. stochastic gradient descent so we have. an algorithm to search over a humongous. number of programs. and gradient descent kind of bubbles up.

the things that are tend to give correct. answers so. a neural network. with a with fixed weights that's. optimized do you think of that as a. single program um so there is a. work by christopher olach where he. so he works on interpretability of. neural networks and he was able to. uh. to identify inside of the neural network. for instance a detector of a wheel for a.

car or the detector of a mask for a car. and then he was able to separate them. out and assemble them uh together using. a simple program uh for the detector for. a car detector that's like uh if you. think of traditionally defined programs. that's like a function within a program. that this particular neural network was. able to find and you can tear that out. just like you can copy and paste from. stack overflow. that. so uh any program is a composition of.

smaller programs. yeah i mean the nice thing about the. neural networks is that it allows the. things to be more fuzzy than in case of. programs. in case of programs you have this like a. branching this way or that way and the. neural networks they they have an easier. way to. to be somewhere in between or to share. things. what to use the most beautiful or. surprising idea in deep learning. in the utilization of these neural. networks which by the way for people who.

are not familiar. neural networks is a bunch of uh. what would you say it's inspired by the. human brain there's neurons there's. connection between those neurons there's. inputs and there's outputs and there's. millions or billions of those neurons. and. the learning. happens. uh by adjusting the weights on the edges. that connect these neurons thank you for. giving definition that. i supposed to do it but i guess you have. enough empathy to listeners to actually.

know that that might be useful no that's. like. so i'm asking plato of like what is the. meaning of life he's not going to answer. you're being philosophical and deep and. quite profound talking about the space. of programs which is just very. interesting but also for people who are. just not familiar with the hell we're. talking about when we talk about deep. learning anyway sorry what is the most. beautiful. or surprising idea to you in in um in. all the time you've worked at deep. learning and you worked on a lot of.

fascinating projects. applications of neural networks. it doesn't have to be big and profound. it can be a cool trick yeah i mean i'm. thinking about the trick but like it's. still amusing to me that it works at all. yeah that let's say that the extremely. simple algorithm stochastic gradient. descent which is something that i would. be able you know to derive on the piece. of paper to high school student uh when. put at the. ins at the scale of you know thousands. of machines actually.

uh can create. the. behaviors we which we called kind of. human like behaviors. so in general. any applications to cast a gradient. descent to neural networks is. is amazing to you so that or is there a. particular application. in natural language. reinforcement learning. uh. and also would you attribute. that success too is it just scale.

what profound insight can we take from. the fact that. the thing works for. gigantic. uh sets of variables. i mean the interesting thing is these. algorithms they were. invented uh decades ago. and. people actually. gave up on the idea yeah and um. you know back then they thought that we. need profoundly different algorithms and. they spent a lot of cycles on very. different algorithms and i believe that.

you know we have seen that various. various innovations that say like. transformer or or dropout or so they can. uh you know pass the help but it's also. remarkable to me that this algorithm. from 60s or so. or i mean you can even say that the. gradient descent was invented by leibniz. in i guess 18th century or so that. actually. is the. core of learning. in the past people are.

it's almost like a out of the maybe an. ego people are saying that it cannot be. the case that such a simple algorithm is. there you know. uh. could solve complicated problems. so they were in search for the. other algorithms and as i'm saying like. i believe that actually we are in the. game where there is there are actually. frankly three levels there is compute. there are algorithms and there is data. and if we want to build intelligent. systems we have to.

pull all three levers. and they are actually multiplicative. and it's also interesting so you ask is. it only compute. people internally they did the studies. to determine how much gains they were. coming from different levels and so far. we have seen that more gains came from. compute than algorithms but also we are. in the world that in case of compute. there is a kind of you know exponential. increase in funding and at some point. it's impossible to. invest more it's impossible to you know.

invest 10 trillion dollars. because we are speaking about that. let's say all taxes in u.s. uh but you're talking about money there. could be innovation. in the compute that's that's true as. well. so i mean they're like a few pieces so. one piece is human brain is an. incredible super computer. [Music]. and they're like a. it. it has. 100 trillion. parameters or like a if you try to count.

various quantities in the brain there. are like a neurons synapses that small. number of neurons there is a lot of. synapses yeah it's unclear even how to. map. synapses. to. two parameters of neural networks but. it's clear that there are many more yeah. so it might be the case that our. networks are still somewhat small. it also might be the case that they are. more efficient than brain or less. efficient by some by some huge factor.

i also believe that there will be like a. you know at the moment we are at the. stage that the these neural networks. they require 1000x or like a huge factor. of more data than humans do and it will. be a matter of. there will be algorithms that. vastly decrease sample complexity i. believe so but the place where we are. heading today is dark domains which. contains million x. more. data and even though computers might be.

1 000 times slower than humans in. learning that's not the problem okay for. instance. i believe that. it should be possible to create super. human therapies. uh by uh. and and then they're like even simple. steps of of doing what of of doing it. and you know that the core reason is. there is just machine will be able to. read way more. transcripts of therapies and then it. should be able to speak simultaneously.

with many more people and it should be. possible to optimize it uh all in. parallel. and well there's now you're touching on. something i deeply care about and think. is way harder than we imagined. um. what's the goal of a therapist what's it. called therapies. so okay so one goal now this is. terrifying to me. but there's a lot of people that. contemplate suicide suffer from. depression. and they could significantly be helped.

with therapy. and the idea that an ai algorithm might. be in charge of that. it's like a life and death task. it's uh. the stakes are high. so one. goal for a therapist whether human or ai. is to prevent suicide ideation to. prevent suicide how do you achieve that. so. let's see. so. to be clear i don't think that the.

current models are good enough for such. a task because it requires insane amount. of understanding and patty and the. models are far from this place but it's. but do you think that understanding. empathy that signal is in the data um i. think there is some signal in the data. yes i mean there are plenty of. transcripts of conversations. and it is possible to. it is possible from it to understand. personalities it is possible from it to. understand uh if conversation is.

a friendly. uh amicable uh. antagonistic it is i believe that the. you know given the fact that the models. that we train now. they can. they can have. they are chameleons that they can have. any personality they might turn out to. be better in understanding. uh personality of other people than. anyone else and they feel pathetic to be. empathetic yeah. interesting uh but i wonder if there's.

some level. of. multiple modalities required. to be able to. be empathetic of the human experience. whether language is not enough to. understand death to understand fear to. understand. uh childhood trauma. to understand uh wit and humor required. when you're dancing with the person who. might be depressed or suffering. both humor and hope and love and all.

those kinds of things. so there's another underlying question. which is self-supervised versus. supervised. so can you get. that from the data by just reading. a huge number of transcripts i actually. so i think that reading huge number of. transcripts is a step one it's like the. same way as you cannot learn to dance if. just from youtube by watching it you. have to actually try it out yourself. yeah and so i think that here that's a.

similar situation i also wouldn't deploy. the system in the high-stakes situations. right away but kind of see gradually. where. it goes and. obviously initially. it would have to go hand with a hand in. hand with humans but. at the moment we are in the situation. that actually. there is many more people who actually. would like to have a therapy or. or speak with with someone then there. are therapies out there okay you know i. was. so so.

fundamentally i was thinking what are. the things that. can vastly increase people well-being. therapy is one of them i think. meditation is other one i guess maybe. human connection is a third one and i. guess. pharmacologically it's also possible. maybe direct brain stimulation or. something like that but these are pretty. much options out there then let's say. the way i'm thinking about the agi. endeavor is by default that's an. endeavor to. increase amount of wealth and i believe.

that we can vastly increase amount of. wealth. for everyone and simultaneously so i. mean they're like two endeavors that. make sense to me one is like essentially. increase amount of wealth and second one. is uh increase overall human well-being. and those are coupled together and they. they can okay i would say these are. different topics one can help another. and uh you know therapist is a funny. word because i see friendship and love. as therapy i mean so therapist broadly.

defined as just friendship as a friend. so like therapist is has a very kind of. clinical sense to it but. what is human connection. you're like uh. not to get all camus and dostoyevsky on. you but you know life is suffering and. we draw. we. seek connection with other humans as we. desperately try to make sense of this. world. in the deep overwhelming loneliness that.

we feel. inside. so i think connection has to do with. understanding. and i think that almost like a lack of. understanding causes suffering if you. speak with someone and you. do you feel ignored that actually causes. pain if you are feeling deeply. understood that actually. they they might not even tell you what. to do in life but like a pure. understanding or just being heard. understanding is a kind of. it's a lot you know just being heard.

feel like you're being heard. like somehow. that's uh alleviation temporarily of the. loneliness. that if somebody. knows you're here. with their body language with the way. they are with the way they look at you. with the way they talk. you feel less alone for a brief moment. yeah very very much agree so i thought. in the past about uh somewhat similar. question to yours which is what is love.

uh rather what is connection yes and um. and obviously i think about these things. from ai perspective what would it mean. um. so i said that the you know intelligence. has to do with some compression which is. more or less like i can say almost. understanding of what is going around it. seems to me that uh other aspect is. there seem to be reward functions and. you can have a you know reward for. uh food for maybe human connection for.

uh let's say warmth. uh. sex and so on. and um. and it turns out that the various people. might be optimizing slightly different. reward functions they essentially might. care about different things. and um. in case of. love at least the love between two. people you can say that the um you know. boundary between people dissolves to. such extent that. they end up optimizing each other reward. functions.

yeah. oh that's interesting um. the success of each other yeah in some. sense i would say love means. uh. helping others to optimize their uh. reward functions not your reward. functions not the things that you think. are important but the things that the. person cares about you try to help them. to optimize it so love is uh. if you think of two reward functions you. just it's a condition yeah you combine.

them together yeah pretty much maybe. like with a weight and it depends like. the dynamic of the relationship yeah i. mean you could imagine that if you are. fully uh optimizing someone's reward. function without yours then yeah then. maybe are creating code dependency or. something like that yeah. i'm not sure what's the appropriate. weight but the interesting thing is i. even. i even think that the. individual person. we ourselves we are actually. less of a. unified insight so for instance if you.

look at the donut on the one level you. might think oh this like it looks tasty. i would like to eat it on another level. you might tell yourself i shouldn't be. doing it because. i want to gain muscles so and you know. you might do it regardless kind of. against yourself so it seems that even. within ourselves they're almost like a. kind of intertwined personas. and. i believe that the self-love. means that. the love between all these persons which. also means being able to.

love love. yourself when we are angry or stressed. or so combining all those reward. functions of the different selves you. have yeah and accepting that they are. there okay you know often people they. have a negative self-talk or they say i. don't like when i'm angry and like i try. to imagine. try to imagine if there would be. like a. small baby alex like a five years old. who's angry angry and then you're like. you shouldn't be angry like stop being.

angry yeah but like instead actually you. want the legs to come over give him a. hug and he's like i say it's fine okay. you can't be angry as long as you want. yeah then he would stop. or. or maybe not or maybe not but you cannot. expect it even yeah. but still that doesn't explain the why. of love like why is love part of the. human condition why is it useful. to combine the reward functions. it seems like. that doesn't i mean.

i don't think reinforcement learning. frameworks can give us answers to why. even even the hudder. framework has an objective function. that's static so we came to existence as. a consequence of evolutionary process. and in some sense the purpose of. evolution is survival and then the. this. complicated optimization objective. baked into us let's say compression. which might help us. operate in the real world and it bake. into us various reward functions yeah.

and then to be clear at the moment we. are operating in the regime which is. somewhat out of distribution where the. event evolution optimized us it's almost. like love is a consequence of. cooperation that we've discovered is. useful correct in some way it's even the. case if you i just love the idea that. love is like the out of distribution. or it's not out of distribution it's. like as you that it evolved for. cooperation. yes and i believe that the cop like a in. some sense cooperation ends up helping. each of us individually so it makes.

sense evolutionary and there is a in. some sense and you know love means there. is this dissolution of boundaries that. you have a shared reward function and we. evolve to actually identify ourselves. with larger groups so we we can identify. ourselves you know with a family we can. identify ourselves with a country to. such an extent that people are willing. to give away their life for country. [Music]. so there is we are wired actually even. uh. for love and at the moment i guess.

the. maybe. it would be somewhat more beneficial if. you will if we would identify ourselves. with all the humanity as a whole so so. you can clearly see when people travel. around the world when they run into. person from the same country they say oh. which ctr and all this like all of a. sudden they find all these similarities. they they they find some they befriend. those folks earlier than others so there. is like a sense some sense of the. belonging and i would say i think it.

would be overall good thing to the word. for people. to. move towards. i think it's even called open. individualism and move toward the. mindset of a larger and larger groups so. the challenge there. that's a beautiful vision and i share it. to expand that circle of empathy that. circle of love towards the entirety of. humanity but then you start to ask well. where do you draw the line. because why not expand it to other. conscious beings and then at the finally.

for our discussion. something i think about. is why not expand it to ai systems. like we we start respecting each other. when the other the person the entity on. the other side. has the capacity to suffer because then. we develop a capacity to sort of. empathize. and so. i could see ai systems that are. interacting with humans. more and more having conscious like.

displays. so like they display consciousness. through language and through other means. and so then the question is like well is. that consciousness. because they're acting conscious. and so. you know the reason we don't like. torturing animals. is because. they look like they're suffering when. they're tortured. and if ai looks like it's suffering. when it's tortured. how is that not.

requiring of the same kind of empathy. from us and respect and rights. that animals do and other humans do i. think it requires empathy as well i mean. i would like. i guess us or humanity or so make a. progress in. understanding what consciousness is. because i don't want just to be speaking. about that the philosophy but rather. actually make a scientific uh to have a. like a you know there was a time that. people thought that. there is a force.

of life. and. the. things that have this force they are. alive. and. i think that there is actually a path to. understand exactly what consciousness is. and. um in some sense it might require. essentially putting probes inside of a. human brain. what neuralink. does so the goal there i mean there's. several things with consciousness that. make it a real discipline which is one. is rigorous measurement of consciousness.

and then the other is the engineering of. consciousness which may or may not be. related i mean you could also run into. trouble like for example in the united. states. for the department d.o.t department of. transportation and a lot of different. places put a value on human life. i think dot's. uh values nine million dollars per. person. sort of in that same way you can get. into trouble. if you put a number on how conscious a. being is.

because then you can start making policy. if a cow. is uh 0.1. or like um. 10 as conscious as a human then you can. start making calculations and might get. you into trouble but then again that. might be a very good way to do it. i would like uh. to move to that place that actually we. have scientific understanding what. consciousness is yeah and then we'll be. able to actually assign value and i. believe that there is even the path for.

the experimentation in it so uh you know. we said that you know you could put the. probes inside of the brain there is. actually few other things that you could. do with devices like neuralink so you. could imagine that the way even to. measure if ai system is conscious. is by literally just plugging into the. brain. and i mean that that seems that's kind. of easy but the plugging into the brain. and asking person if they feel that. their consciousness expanded.

this direction of course has some issues. you can say you know if someone takes a. psychedelic drug they might feel that. their consciousness expanded even though. that drug itself is not conscious. right so like you can't fully trust the. self-report of a person saying their. their consciousness is expanded or not. let me ask you a little bit about. psychedelics because uh there's been a. lot of excellent research on uh. different psychedelics psilocybin mdma. yeah even dmt.

drugs in general marijuana too. uh what do you think psychedelics do to. the human mind it seems they take. the human mind to some interesting. places. is that just a little uh hack. a visual hack. or is there some profound expansion of. the mind. so let's see i i don't believe in magic. i believe in that i believe in. in science in. in causality. still let's say and then as i said like.

i think that the brain. that the our subjective experience of. reality is uh. we live in the simulation run by our. brain and the simulation that our brain. runs. they can be very pleasant or very. hellish. drugs they are changing some hyper. parameters of the simulation it is. possible thanks to change of these hyper. parameters to actually look back on your. experience and even see that the given.

things that we took for granted they are. changeable. so they allow to have a. amazing perspective there is also. for instance the fact that after dmt. people can see the. full movie inside of their head. gives me further belief. that the brain can generate that full. movie that the brain is actually. learning the model of reality to such. extent that it tries to predict what's. going to happen next yeah very high.

resolution so it can replay realities. actually extremely high resolution. and it's also kind of interesting to me. that somehow there seems to be some. similarity between. these uh drugs and meditation itself and. i actually started even these days to. think about meditation as a psychedelic. and do you practice meditation. i i practice meditation i mean i once. few times on the. retreats and it feels after like after.

second or third day of meditation. there is a there is almost like a sense. of you know tripping. what does the meditation retreat entail. so. i mean you you wake up early in the. morning and you meditate for extended. period of time. and alone. yeah so it's optimized even though there. are other people it's optimized for. isolation so you don't speak with anyone.

you don't actually look into other. people's eyes. and. you know you sit on the chair and. say the passage meditation tells you uh. to focus on the breath so you try to put. all the all attention into breathing and. breathing in and breathing out. and the. crazy thing is that as you focus. attention like that. after some time. their stamps starts coming back like.

some. memories that you completely forgotten. it almost feels like um that you have a. mailbox and then you. you know you are just like a archiving. email one by one. and at some point at some point there is. like a. amazing feeling. of getting to mailbox zero. zero emails and uh it's very pleasant. it's it's kind of it's it's. it's. crazy to me. that.

that once. you resolve these. inner stories or like inner traumas. then once there is nothing. uh left. the default state of human mind is. extremely peaceful and happy extreme. like some sense it it feels that. it feels. at least to me in the way how when i was. a. child that i can look at any object and.

it's very beautiful i have a lot of. curiosity about the simple things and. that's where usually meditation takes me. are you. what are you experiencing are you just. taking in simple sensory. information and they're just enjoying. the rawness of that sensory information. so there's no. there's no memories all that kind of. stuff you're just enjoying. being. yeah pretty much i mean still there is a. there it's it's thoughts are slowing.

down sometimes they pop up but it's also. somehow the extended meditation takes. you to the space that they are. way more friendly you know way more. positive um. there is also this uh this thing that. we've actually. it almost feels that the. it almost feels that the we are. constantly getting a little bit of a. reward function and we are just. spreading this reward function on.

various activities but if you stay still. for extended period of time it kind of. accumulates accumulates accumulates. and. there is a there is a sense there is a. sense that at some point it passes some. threshold and it feels as. drop is falling into kind of ocean of. love and bliss and that's like a. this is like a very pleasant and as i'm. saying okay. that corresponds to the subjective. experience. some people. uh i guess in spiritual community they.

describe it that that's the reality and. i would say i believe that they're like. all sorts of subjective experience that. one can have and. i believe that for instance meditation. might take you to the subjective. experiences which are very pleasant. collaborative and i would like a word to. move toward a more collaborative uh. place. yeah i would say that's very pleasant. that i enjoy doing stuff like that i i. i wonder how that maps to your uh.

mathematical model of love with the. the reward function combining a bunch of. things. it seems like our life. then is we're just we have this reward. function and we're accumulating a bunch. of stuff in it. with weights. it's like um. like multi-objective. and. what meditation is is you just remove. them remove them until the weight on one.

or just a few is is very high and that's. where the pleasure comes from yeah so. something similar how i'm thinking about. this so i told you that there is like a. there is a story of who you are. and i think almost about it as a you. know text prepended to gpt. yeah and. some people refer to it as ego okay it's. like a story. who who you are okay so ego is the. prompt for gpt three gpg yes yes and.

that's description of you and then with. meditation you can get to the point that. actually you experience things without. the prompt. and you experience things like as they. are you are not biased over the. description how they supposed to be. uh. that's very pleasant and then with. respect to the reward function uh it's. possible to. get to the point that the there is. dissolution of self. and therefore you can say that they are. you you're having a you're or like your. brain attempts to simulate the reward.

function of everyone else or like. everything that's there is this like a. love which feels like a oneness with. everything. and that's also you know very beautiful. very pleasant at some point. you might have a lot of altruistic. thoughts during that moment and then. the self uh always comes back how would. you recommend. if somebody is interested in meditation. like a big thing to take on as a project. would you recommend a meditation retreat. how many days what kind of thing would. you recommend i think that actually.

retreat is the way to go and it almost. feels that. as i said like a meditation is a. psychedelic but. when you take it in the small dose you. might barely feel it once you get the. high dose actually you're gonna feel it. um. so even cold turkey if you haven't. really seriously meditated for a. prolonged period of time just go to a. retreat yeah how many days how many days. start the weekend one weekend so like. two three days. and it's like it's interesting that.

first or second day it's hard and at. some point it becomes easy. there's a lot of seconds in a day how. hard is the meditation retreat just. sitting there in a chair. so the thing is actually. it literally just depends on your uh. on death your own framing like if you. are in the mindset that you are waiting. for it to be over or you are waiting for. nirvana to happen it will be very. unpleasant yeah and in some sense even.

the. difficulty it's not even in. the lack of being able to speak with. others like. you are sitting there your. legs will hurt from sitting. in terms of like the practical things do. you experience kind of discomfort like. physical discomfort of just sitting like. your your butt being numb your. legs being sore all that kind of stuff. yes you experience it and then the. they teach you to observe it. rather and it's like a the crazy thing.

is. you at first might have a feeling toward. trying to escape it yeah and that. becomes very apparent that that's. extremely unpleasant and then you just. just observe it and. at some point it it just becomes uh it. just is. it's like a i remember with ilya told me. some time ago that uh you know he takes. a cold shower and his mindset of taking. a court cold shower was to. embrace suffering yeah excellent i do. the same there's the art style yes my.

style. i like this. so my style is actually i also sometimes. take cold showers it is purely observing. how the water goes through my body like. a purely being present not trying to. escape from there yeah and i would say. then it actually becomes pleasant. it's not like ah well that that's. interesting um. i i'm also that mean that's that's the. way to deal with anything really.

difficult especially in the physical. space is. to observe it. to say it's pleasant. it's a i would use a different word. your uh. you're accepting of the full beauty of. reality i would say because say pleasant. but yeah i mean in some sense it is. pleasant that's the only way to deal. with a cold shower. is to to become an observer and to find. joy in it.

same with like really difficult physical. uh exercise or like running for a really. long time endurance events. just anytime you're exhausted any kind. of pain i think the only way to survive. it is not to resist it just to observe. it. you mentioned ilya elias discover. he's very he's our chief scientist but. also he's very close friend of mine he. co-founded open air with you i've spoken. with him a few times he's brilliant i. really enjoy talking to him.

his mind just like yours works in. fascinating ways. now both of you are not able to define. deep learning simply. uh what's it like having him. as somebody you have technical. discussions with. on in space machine learning. deep learning ai but also life. what's it like when these two uh agents. get into a self-play situation in in a.

room what's it like collaborating with. him. so i believe that we have. extreme uh respect to each other so. um. i mean. i love ilia's insight both like uh. i guess about consciousness uh life ai. but uh in terms of the it's interesting. to me because. you're. a brilliant. uh. thinker in the space of machine learning. like intuition like digging deep.

in what works. what doesn't why it works why it doesn't. and so is ilia i'm wondering if there's. interesting. deep discussions you've had with him in. the past or disagreements that were very. productive so i can say. i also understood over the time where. are. my strengths so obviously we have plenty. of ai discussions and. um. and you know i myself have plenty of. ideas but like i consider ilya.

one of the most prolific ai scientists. in the entire world. and. i think that. um i realized that maybe my super skill. is. being able to bring people to. collaborate together that i have some. level of empathy that is unique in ai. world and that might come you know from. either meditation psychedelics or let's. say i read just hundreds of books on. this topic so and i also went through a. journey of you know i develop all sorts. of algorithms so i think that.

maybe i can. that's my. super human skill uh. ilia is. one of the best ai scientists but then. i'm pretty good in assembling teams and. i'm also not holding two people like i'm. growing people and then people become. managers that open yeah there's room any. of them like a research manager. so you you find. you find places where you're excellent. and and he finds like his his deep.

scientific insights is where he is and. you find ways you can. the puzzle pieces fit together correct. okay you know ultimately for instance. let's say ilia he doesn't manage people. uh that's not. what he likes or so um. i i like i like hanging out with people. by default i'm an extrovert and i care. about people oh interesting okay. okay cool so that that fits perfectly. together but i i mean uh i also just. like your intuition about various. problems in machine learning.

he's definitely one i really enjoy. i remember talking to him. about something i was struggling with. which is. coming up with a good model for. pedestrians. for human beings across the street in. the context of autonomous vehicles. and he immediately started to like. formulate a framework within which you. can evolve a model for pedestrians like. through self-play all that kind of. mechanisms.

the depth of thought on a particular. problem especially problems he doesn't. know anything about. is fascinating to watch. it makes you realize like um. yeah the the limits of the. that the human intellect might be. limitless. or it's just impressive to see a descent. on the vape come up with clever ideas. yeah i mean so even in the space of deep. learning when you look at various people. there are people you know who. invented.

some breakthroughs once but there are. very few people who did it multiple. times and you can think if someone. invented it once. that might be just a shared luck. and if someone invented it multiple. times you know if a probability of. inventing it once is one over a million. then probability of inventing it twice. or three times would be one over a. million square. or to the power of three. which which would be just impossible so. it literally means that it's it's given. that uh it's not the luck yeah and ilea.

is one of these few people who um. who have uh a lot of these inventions in. his arsenal it also feels that the. now for instance if you think about. folks like gauss or euler. and you know. at first they read a lot of books. and then they did thinking and then they. figure out math. and that's how it feels with ilya yeah. you know at first he read stuff and then. like he spent his thinking cycles.

and. that's a really good way to put it. when i talk to him. [Music]. i. i see thinking. he's actually thinking. like he makes me realize that there's. like deep thinking that the human mind. can do like most of us are not thinking. deeply. like you really have to put a lot of. effort to think deeply like i have to. really put myself in a place where i. think deeply about a problem it takes a.

lot of effort it's like a it's like an. airplane taking off or something you. have to achieve deep focus he he's just. uh. he's what is it. his brain is like a vertical takeoff. in terms of airplane analogy so it's. interesting but. it i mean cal newport talks about this. as ideas of deep work. it's you know most of us don't work much. at all in terms of like. like deeply think about particular. problems whether it's math engineering.

all that kind of stuff. you want to go to that place often and. that's real hard work and some of us are. better than others at that so i think. that the big piece has to do with. actually even engineering your. environment such that it's conducive to. that yeah so um. see both ilia and i uh on the frequent. basis we kind of disconnect ourselves. from the world in order to be able to do. extensive amount of thinking yes so ilia. usually.

he just. leaves ipad. at hand he loves his ipad. and. for me i'm even. sometimes you know just going for a few. days to different location to airbnb i'm. turning off my phone. and there is no access to me yeah. and. that's extremely important for me to be. able to actually just formulate new. thoughts to do deep work rather than to. be reactive and the the older i am the. more of these like random tasks are at.

hand. before i go on to that uh thread let me. return. to our friend gpt. let me ask you another ridiculously big. question. can you give an overview of what gpt 3. is. or like you say in your twitter bio gpt. n plus one. how it works. and why it works so um gpt 3 is a. humongous neural network and let's. assume that we know what is neural.

network okay by the definition. and it is trained on the entire internet. and just to predict. next word so let's say it sees part of. the uh article and it the only task that. it has at hand it is to say what would. be the next word uh what would be the. next word. and it becomes uh. really exceptional at the task of. figuring out what's the next word so you. might ask. why would this be an important task why.

would it be important to predict what's. the next word. and it turns out that a lot of problems. uh can be formulated. uh. as a text completion problem so gpt is. purely uh learning to complete the text. and you could imagine for instance if. you are asking a question who is a. president of united states. then gpt can give you an answer to it. it turns out that many more things can. be formulated this way you can format.

text. in the way that you have sentence in. english. you make it even look like a some. content of a website uh elsewhere which. would be teaching people how to. translate things between languages so it. would be en colon. text in english fr colon and then you uh. and then you ask people and then you ask. model to to continue and it turns out. that the such a model is predicting. translation from english to french the. crazy thing is that.

this model. can be used for way more sophisticated. tasks so you can format text such that. it looks like a conversation between two. people and that might be a conversation. between you and elon musk and because. the model read all the texts about elon. musk. it will be able to predict elon musk. words as it would be elon musk it will. speak about colonization of. mars. about sustainable future and so on and. it's also possible to.

to even give arbitrary personality to. the model you can say here is a. conversation with a friendly ai bot. and the model uh will complete the text. as a friendly ai bot so i mean. how do i express how. amazing this is so. just to clarify. a conversation generating a conversation. between me and elon musk. it wouldn't just generate good. examples of what elon would say.

it would get the syntax all correct so. like interview style you would say like. elon colon and lex con like it it's not. just like uh. inklings of. semantic. correctness. it's like the whole thing grammatical. syntactic. semantic. it's just really really impressive. uh generalization. yeah i mean i also want to you know.

provide some caveats so it can generate. few paragraphs of coherent text but as. you go to uh longer pieces it actually. goes off the rails okay if you would uh. try to write a book it won't work out uh. this way what way does it go off the. rails by the way is there interesting. ways in which it goes off the rails like. what falls apart first so the model is. trained on the all the existing data. that is out there which means that it is.

not trained on its own mistakes so for. instance if it would make a mistake then. uh i kept so to give give you an example. so let's say i have a conversation with. a. model pretending that is elon musk. and then i start putting some i'm start. actually making up things which are not. factual. um i would say like twitter. but i gotcha sorry yeah um okay. i don't know i would say that elon is my.

wife. and the model will just. keep on carrying it on and as if it's. true. yes and in some sense if you would have. a normal conversation with elon he would. be what the fuck. yeah there would be some feedback. between so the the model is trained on. things that humans have written but. through the generation process there's. no human in the loop feedback correct. that's fascinating makes sense so it's. magnified it's like the errors get. magnified and magnified right and it's a.

it's also interesting. i mean first of all humans have the same. problem it's just that we. uh we make. fewer errors and magnify the errors. slower i think that actually what. happens with humans is if you have a. wrong belief about the world as a kid. then very quickly you will learn that. it's not correct because you are. grounded in reality and you are learning. from your new experience yes. but do you think the model can correct. itself too.

it through the power of the. representation. and so the absence. of. elon musk being your wife. information on the internet want to. correct itself. there won't be examples like that so the. errors would be subtle at first. saddle at first and in some sense. you can also say that the data that is. not out there is the data which would. represent how the human learns. that's an a and and maybe model would be.

trained on such a data then it would be. better off how intelligent is gpt 3 do. you think like when you think about the. nature of intelligence. it seems exceptionally. impressive. but then if you think about the big agi. problem is this footsteps along the way. to agi. so. let's see seems that intelligence itself. is there are multiple axis of it and. i would expect that the.

the systems that we are building they. may end up being super human on some. axis. and sub human on some other axis it. would be surprising to me on all axis. simultaneously they would become. superhuman. of course people ask this question is. gpt a spaceship that. that would take us to moon or are we. putting a building a ladder to heaven. that we are just building bigger and. bigger ladder and we don't know in some. sense. uh which one of these two which one is.

better. i'm trying to i like stairway to heaven. that's a good song so i'm not exactly. sure which one is better but you're. saying like the the spaceship to the. moon is actually effective. correct so people who criticize gpt yeah. they say jarga is just. building a. taller a ladder. and it will never reach the moon. and. at the moment i would say the way i'm. thinking is this like a scientific. question. and i'm also in heart i'm a builder.

creator and like i'm thinking let's try. out let's see how far it goes and so far. we see constantly that there is a. progress yeah. so what do you think. gpt4. gpt5 gpt n plus one. will uh. there'll be a phase shift like a. transition to a to a place where. we'll be truly surprised then again like. gpt3 is already very like truly.

surprising the people that criticize. gpg3 as it's there as a what is it. ladder to heaven. i think too quickly get accustomed to. how impressive it is that the prediction. of the next word can achieve such. depth of semantics accuracy of syntax. grammar and semantics. um do you do you think. gpt four and five and six will continue. to surprise us. i mean definitely there will be more. impressive models there is a question of.

course if there will be a phase shift. and. the also even the way i'm thinking about. the about these models is that. when we build these models. you know we see some level of the. capabilities but we don't even fully. understand everything that the model can. do and actually one of the best things. to do is to. allow other people to probe the model to. even see what is possible. hence the.

using gpg as an api. and opening it up to the world yeah i. mean so when i'm thinking from. perspective of. there like a obviously various people. are that have concerns about agi. including myself. and then when i'm thinking from. perspective what's the strategy even to. deploy these things to the world. the. the one strategy that i have seen many. times working is the iterative. deployment that you deploy. um slightly better versions and you.

allow other people to criticize you so. you actually are tried out you see where. are their fundamental issues and it's. almost you don't want to be in that. situation that you are holding into. powerful system and there's like a huge. overhang then you deploy it and it might. have a random chaotic impact on the. world so you actually want to be in the. situation that they are gradually. deploying systems. i asked this question of ilio let me ask. you. you this question.

i've been reading a lot. about stalin and power. if you're in possession of a system. that's. like agi that's exceptionally powerful. do you think your character integrity. might become corrupted. like famously power corrupts and. absolute power corrupts absolutely. so i believe that. you want at some point to.

work toward distributing the power. i think that. you want to be in the situation. that actually agi is not controlled by a. small number of people. but. essentially. by a larger collective so the thing is. that requires a george washington style. move. in the ascent to power there's always a. moment when somebody gets a lot of power. and they have to have the integrity. and uh the moral compass to give away.

that power. that humans have been. good and bad throughout history at this. particular step and i wonder. i wonder we like blind ourselves in uh. for example. between nations a race. uh towards uh. yeah ai race between nations we might. blind ourselves and justify to ourselves. the development of ai without. distributing the power. because we want to defend ourselves. against china against russia that kind.

of that kind of logic. and. i wonder. how we um. how we design governance mechanisms that. um prevent us from. becoming power hungry and in the process. destroying ourselves. so let's see i have been thinking about. this topic quite a bit but i also want. to admit that. uh once again i actually want to rely. way more on sam outman on it hero than a. heroed an excellent block.

on how even to distribute wealth. and his proper he proposed in his block. to tax. equity of the companies rather than. profit and to distribute it and this is. this is an example of. uh washington move. i guess i personally have insane trust. in some. he already spent plenty of money running. a. universal basic income.

project. that like gives me i guess. maybe some level of trust to him but i. also. i guess. love him as a friend yeah. i wonder because we're sort of summoning. a new set of technologies. i wonder if we'll be. cognizant like you're describing the. process of open ai but it could also be. at other places like in the us. government right. both china and the us are now.

full steam ahead on autonomous weapons. systems development. and that's really worrying to me because. in the framework of something being. a national security danger or military. danger you can do a lot of pretty dark. things. that blind our moral compass. and i think ai will be one of those. things. in some sense the the mission. and the work you're doing at openai.

is like the counterbalance to that so. you want to have more open ai and less. autonomous weapon systems i i i like. these statements like to be clear like. this interesting and i'm thinking about. it myself but uh. this is a place that i i okay. i put my trust actually. in some hence because it's extremely. hard for me to reason about it yeah i. mean one important statement to make is. um. it's good to think about this yeah no. question about right no question even.

like. low-level quote-unquote engineer. like there's such a. i remember i i programmed a car uh our. rc car. they went really fast like 30 40 miles. an hour. and i remember i was like sleep deprived. so i programmed it. pretty crappily and it like uh. the the code froze so it's doing some. basic computer vision and it's going. around on track but it's going full.

speed. and uh there's a bug in the code that uh. the car just. went it didn't turn it went straight. full speed and smashed into the wall i. remember thinking. the seriousness with which you need to. approach the design of artificial. intelligence systems and the programming. of artificial intelligence systems. is high because the consequences are. high like that little car smashing it to. the wall.

for some reason i immediately thought of. like an algorithm that controls nuclear. weapons. having the same kind of bug and so like. the lowest level engineer and the ceo of. a company all need to have the. seriousness. in approaching this problem and thinking. about the worst case consequences so i. think that is true i mean the. what i also recognize in myself and. others even asking this question is that. it evokes a lot of fear. and fear itself ends up being actually.

quite debilitating. the place where i arrived at the moment. might sound cheesy or so but it's almost. to. build things out of love rather than. fear yeah. i can focus on how. i can you know maximize the value how. the systems that i'm building might be. uh. useful. i'm not saying that the fear doesn't. exist out there and like it totally.

makes sense to minimize it. but i don't want to be working because. uh i'm scared i want to be working out. of passion out of curiosity out of the. you know looking forward for the. positive future. with uh. the definition of love arising from a. rigorous practice of empathy so not just. like your own conception of what is good. for the world but uh always listening to. others. correct like at the love where i'm. considering reward functions of others.

others. to infil limit to infinity is like a sum. like one to n where n is uh seven. billion or whatever it is not not. projecting my reward functions on others. yeah exactly. okay. can we just take a step back to. something else super cool which is uh. opening up codex. can you give an overview of what. open-air codecs and github co-pilot is. how it works. and why the hell it works so. well so with gpd3 we noticed that the.

system. um you know that system training all the. language out there started having some. rudimentary coding capabilities so we're. able to ask it you know to. implement addition function between two. numbers and indeed it can write python. or javascript code for that and then we. thought um we might as well just go full. steam ahead and try to create a system. that is actually good. at what we are doing every day ourselves. which is programming.

we optimize models. for proficiency in coding we actually. even created models that both have a. comprehension of language and code. and codex is api for these models so. it's first pre-trained on language. and then. i don't know if you can say fine-tuned. because there's a lot of code. but it's language and code it's language. and code. it's also optimized for various things.

like let's say low latency and so on. codex is the api that's similar to gpd3. we expect that there will be. proliferation of the potential products. that can use coding capabilities and i. can. i can speak about it in a second. compiled is the first product. and developed by github so as we're. building uh models we wanted to make. sure that these models are useful. and we work together with github on. building the first product co-pilot is. actually as you code it suggests you.

code completions and we have seen in the. past they're like a various tools that. can suggest how to like a few characters. of the code or the line of code the the. thing about copilot is it can generate. 10 lines of code you. it's often the way how it works is you. often write in the comment what you want. to happen because. people in comments they describe what. happens next so. um these days when i code instead of. going to google to search.

for the appropriate code to solve my. problem i say oh for this array could. you smooth it and then you know it. imports some appropriate libraries and. say it uses numpy convolution or so i. that i was not even aware that exists. and it does the appropriate thing. um so you you write a comment maybe the. header of a function and it completes. the function. of course you don't know what is the. space of all the possible. small programs it can generate. what are the failure cases how many edge.

cases how many subtle. errors there are how many big errors. there are it's hard to know but the fact. that it works at all on in a large. number of cases is incredible it's like. a. it's a kind of search engine. into code that's been written on the. internet. correct so for instance. when you search things online then. usually you get to the. some particular. case like if you go to stack overflow. people describe that one particular.

situation uh and then they seek for a. solution but in case of uh co-pilot it's. aware of your entire context and in. contexts oh these are the libraries that. they are using that's the set of the. variables that is initialized and on the. spot it can actually tell you what to do. so the interesting thing is. and we think that the copilot is one. possible product using codex but there. is a place for many more so. internally we tried out you know to. create other fun products so it turns.

out that a lot of tools out there. let's say google calendar or microsoft. word or so. they all have uh internal api to build. plugins around them. so there is a way in the sophisticated. way to control calendar or microsoft. word today if you want. if you want more complicated behaviors. from these programs you have to add a. new button for every behavior. but it is possible to use codex and. tell for instance to calendar.

could you schedule an appointment with. blacks. next week after 2 pm and either writes. corresponding piece of code. and that's the thing that actually you. want so interesting so. what you figure out is there's a lot of. programs with which you can interact. through code. and so there you can generate that code. from natural language. that's fascinating and that's somewhat. like also closest to. uh what was the promise of siri or alexa. yeah so previously all these behaviors.

they were had. hard coded yeah and it seems that codex. on the fly can pick up the api of let's. say given software yeah and then it can. turn the language into use of this api. without hard coding you can find it can. translate to machine language correct it. to uh so for example this would be. really exciting for me like for um adobe. products like photoshop. uh which is the i think actionscript i. think there's a scripting language that. communicates with them same with.

premiere. and you could imagine that that allows. event to. do coding by voice on your phone. so for instance in the past okay as of. today i'm not editing word documents on. my phone because it's just the keyboard. is too small but if i would be able to. tell. to my phone you know uh make the header. large and then move the paragraphs. around and it does actually what i want. so i can tell you one more cool thing or. even how i'm thinking about codex.

so if you look actually at the evolution. of. of computers. we started with very primitive. interfaces which is a punch card and. punch card essentially. you make a holes in the. in the plastic card to indicate zeros. and ones. and. during that time there was a small. number of specialists who were able to. use computers and by the way people even. suspected that there is no need for many. more people to use computers. but then we moved from punch cards to.

at first assembly then c. and these programming languages they. were slightly higher level they allowed. many more people to code and they also. led to more of a proliferation of. technology and. you know further on there was a jump to. say from c plus plus to java and python. and every time it has happened. more people are able to code and we. build more technology and it's even you. know.

hard to imagine now if someone will tell. you that you should write code in. assembly instead of let's say python or. or. or java or javascript and codex is yet. another step toward kind of bringing. computers closer to humans such that you. communicate with a computer. with your own language. rather than with a specialized language. and. i think that it will lead to. an increase of number of people who can. code. yeah and then and the kind of. technologies that those people will.

create is. like it's innumerable it could you know. it could be a huge number of. technologies we're not predicting at all. because that's less and less requirement. of uh. having a technical mind. a programming mind you're not opening it. to the world of. um. other kinds of minds creative minds. artistic minds all that kind of stuff i. would like for instance biologists who. work on dna to be able to program and. not to need to spend a lot of time uh. learning it and i i believe that's a.

good thing to the word and i would. actually add out that so at the moment. i'm a managing codex team and also. language team and i believe that there. is like a plenty of brilliant people out. there. and they should apply. oh okay yeah awesome so what's the. language in the codexes so those are. kind of. they're overlapping teams so it's like. gpt the raw language and then the codex. is like applied to programming. correct and they are quite intertwined. there are many more teams involved.

making these uh. models. extremely efficient and deployable for. instance there are people who are. working to you know. make our data centers uh amazing or. there are people who work on pro putting. these models into production. or uh. or even pushing it at the very limit of. the scale. so all aspects from from the. infrastructure to the actual machine. learning so i'm just saying that. multiple teams while the. and the team working on codex and.

language uh i guess i'm i'm directly. managing them i would like i would love. to hire yeah if you're interested in. machine learning. this is probably one of the most. exciting uh problems and like systems to. be working on because it's actually it's. it's pretty cool like what what uh the. program synthesis like generating of. programs is very interesting very. interesting problem that has echoes of. reasoning and intelligence in it. it and i think there's a lot of. fundamental questions that you might be.

able to sneak. sneak up to by generating programs yeah. the one more exciting thing about the. programs is that so i said that the. um you know the in case of language that. one of the troubles is even evaluating. language so when the things are made up. you you need somehow. either a human. to say that this doesn't make sense or. so in case of program there is one extra. level that we can actually execute. programs and see what they evaluate to.

so that process might be somewhat. more automated in in order to improve. the uh qualities of generations and. that's not saying so like the wow that's. really interesting so for the language. that you know the simulation to actually. execute it as a human mind yeah for. programs there is a there is a computer. on which you can evaluate it. wow. that's a. brilliant little. insight that the thing compiles and runs.

that's first. and second you can evaluate on a like do. automated unit testing. and in some sense. it seems to me that we will be able to. make a tremendous progress you know. we are in the paradigm that there is. way more data and there is like a. transcription of millions of uh of uh. software engineers yeah. yeah. so. i mean you just me because i was going. to ask you about reliability the thing.

about programs is you don't know if. they're going to. like a program that's controlling a. nuclear power plant has to be very. reliable so i i wouldn't start with. controlling nuclear power plant can i be. one day but that that's not actually. that's not on the current roadmap that's. not that's step one and you know it's. the russian thing you just want to go to. the most powerful destructive thing. right away. run by javascript but i got you so it's. a lower impact but nevertheless what.

you're making me realize. it is possible to achieve some levels of. reliability by doing testing. and i thought you could imagine that. them you know maybe there are ways for a. model to. write even code for testing itself and. so on. and there exists a ways to create the. feedback loops that the model could keep. on improving. by writing programs that generate tests. for the instance for instance. and that's how we get consciousness. because it's meta compression that's.

what you're going to write that's the. comment that's the prompt that generates. consciousness. compressor of compressors you just write. that. do you think the code that generates. consciousness would be simple. so. let's see i mean ultimately the core. idea behind will be simple but there. will be also decent amount of. engineering. involved like in some sense. it seems that you know spreading these. models on many machines.

and it's not that trivial yeah and. we find all sorts of innovations that. make our models more efficient. i believe that. first models. that i guess are conscious are like a. truly intelligent they will have all. sorts of. tricks. but then again there's uh. which is certain argument that maybe. the tricks are temporary thing yeah they. might be temporary things and in some. sense it's also even important. to um.

to. know that even the cost of a trick so. sometimes people are eager to put the. trick. while forgetting that there is a cost of. maintenance. or like a long-term cost long-term cost. or maintenance or maybe even. flexibility of code to actually. implement new ideas so even if you have. something that gives you 2x but it. requires you know 1000 lines of code i'm. not sure if it's actually worth it so in. some sense you know if it's five lines. of code and 2x i would take it.

and and we we we see many of this but. also you know that requires some level. of. i guess lack of attachment to code that. we are willing to remove it yeah. so you led the open ai robotics team can. you give an overview of of the cool. things you're able to accomplish what. are you most proud of. so when we started robotics we knew that. actually reinforcement learning works. and it is possible to. solve very complicated problems.

like for instance alphago is an evidence. that it is possible to to build. superhuman and gold players dota 2 is a. an evidence that is possible to. build superhuman uh. agents playing dota so i asked myself a. question you know what about robots out. there could we train machines to solve. arbitrary tasks in the physical world. our approach was i guess let's pick a. complicated problem that.

if we would solve it that means that we. made some uh significant progress in the. domain and then we went after the. problem. so um we noticed that actually the. robots out there they are kind of at the. moment optimized per task so you can. have a robot that it's like if you have. a robot opening a battle it's very. likely that the end factor is a battle. opener. and. and in some sense that's a hack to be. able to solve a task which makes any. task easier and um ask myself so what.

would be a robot that can actually solve. many tasks yeah and we conclude that. that. like a human hands have such a quality. that indeed they are you know you have. five kind of tiny arms attached. individually they can manipulate. pretty broad spectrum of objects so we. went after a single hand like a trying. to solve rubik's cube single-handed we. picked this task because we thought that. there is no way to.

harcode it and it's also we picked the. robot on which it would be hard to. hardcode it and. we went after the solution such that. it could generalize to other problems. and just to clarify it's. one robotic hand solving the rubik's. cube the hard part isn't the solution to. the rubik's cube is the manipulation of. the uh of like having it not fall out of. the hand having it. use the uh. five baby arms. to uh what is it like rotate different.

parts of the rubik's cube to achieve the. solution correct yeah so what uh what. was the hardest part about that. what was the approach taken there what. are you most proud of obviously we have. like a strong belief in reinforcement. learning. and uh. you know one path it is to do. reinforcement learning the real world. other path is to. the simulation in some sense the. tricky part about the real world is at. the moment our models they require a lot. of data there is essentially no data.

and i did we decided to go through the. path of the simulation and in simulation. you can have infinite amount of data the. tricky part is the fidelity of the. simulation and also can you in. simulation represent everything that you. represent otherwise in the real world. and you know it turned out that uh. that you know because there is lack of. fidelity it is possible to that what we. what we. arrived at is training a model that.

doesn't solve one simulation but it. actually solves the. entire range of simulations which uh. vary uh in terms of like uh what's the. exactly the friction of that cube or the. weight or so. and the. single ai that can solve all of them. ends up working well with the reality. how do you generate the different. simulations so. you know there's plenty of parameters. out there we just pick them randomly and. and in simulation model just goes for.

thousands of years and keeps on solving. rubik's cube in each of them and the. thing is the neural network that we used. it has a memory. and as it presses for instance the side. of the of the cube it can sense oh. that's actually this side was. uh difficult to press i should press it. stronger and throughout this process. kind of. learns even how to. how to solve this particular instance of. the rubik's cube back even mass it's.

kind of like a. you know sometimes when you. go to a gym and after. after bench press you. try to lift the. and you kind of forgot uh and and your. hand goes like yeah right away because. kind of you got this to maybe different. weight yeah and it takes a second to. adjust yeah. and this kind of of a memory that model. gained through the process of. interacting with the cube in the. simulation. i appreciate you speaking to the.

audience with the bench press all the. bros in the audience. probably working out right now there's. probably somebody listening to this. actually doing bench press. so maybe. uh put the bar down and pick up the. water bottle and you'll know exactly. what uh what check is talking about okay. so what uh. what was the hardest part of getting the. whole thing to work so the hardest part. is. at the moment when it comes to a. physical world.

when it comes to robots. they require maintenance it's hard to. replicate a million times it's. it's also it's hard to replay things. exactly. i remember this situation that. one guy. at our company he had like a model that. performs way better than other models in. solving rubik's cube and. you know we kind of didn't know what's. going on. why it's that. and.

it turned out. that you know he was running it from his. laptop that had better cpu. or. uh or better or maybe local gpu as well. and uh because of that there was less of. a latency and the model was the same. and that actually. made solving rubik's cube more reliable. so in some sense there might be some. saddlebacks like that when it comes to. running things in the real world. even hinting on that. you could imagine that the initial. models you would like to have models.

which are insanely huge neural networks. and you would like to give them even. more time for thinking. and when you have these real-time. systems. then. you might be constrained actually by the. amount of latency. and. ultimately i would like to build the. system that it is. worth for you to wait five minutes. because it gives you the answer. that you are willing to wait for five. minutes so latency is a very unpleasant. constraint underwish to operate correct.

and also there is actually one more. thing which is tricky about robots. there is actually. no not much data so the data that i'm. speaking about would be a data of. first person experience from the robot. and like a gigabytes of data like that. if we would have gigabytes of data like. that of robot solving various problems. it would be very easy to make a progress. on robotics and you can see that in case. of text or code there is a lot of data.

like a first person perspective data on. the writing code. yeah so you had this. you mentioned this really interesting. idea that. if you were to build like a successful. robotics company so open as mission is. much bigger than robotics this is one of. the. one of the things you've worked on. but if it was a robotics company they. you wouldn't so quickly dismiss. supervised learning i correct that you. would build a robot. that. was perhaps one like.

um an empty shell like dumb and they. would operate under tele operation. so you would invest. that's just one way to do it invest in. human super like direct human control of. the robots as it's learning and over. time add more and more automation. that's correct so let's say that's how i. would build a robotics company today. if i would be building a robotics. company which is you know spent 10. million dollars or so. recording human trajectories controlling.

a robot after you find. a thing that the robot should be doing. that there's a market fit for like that. you can make a lot of money with that. product correct correct yeah. so. i would record data and then i would. essentially train supervised learning. model on it. that might be the path today. long term i think that actually what is. needed is to train powerful models over. video. so. um you have seen maybe a models that can.

generate images like dali. and people are looking into models. generating videos they're like various. algorithmic questions even how to do it. and it's unclear if there is enough. compute for this purpose. but. i i suspect that the models that which. would have a. level of understanding of video same as. gpt has the level of understanding of. text. could be used. to train robots to solve tasks they. would have a lot of common sense.

if one day. i'm pretty sure one day. there will be a robotics company. by robotics company i mean the primary. source of income is is from robots. that is worth over. 1 trillion dollars. what do you think that company will do i. think self-driving cars no. it's interesting because my mind went to. personal robotics robots in the home. it seems like there's much more market. opportunity there.

i think it's very difficult to achieve. i mean this this. this might speak to something important. which is i understand self-driving much. better than understand robotics in the. home so i understand how difficult it is. to actually solve self-driving. to uh to a level not just the actual. computer vision and the control problem. and just the basic problem self-driving. but. creating a product. that would undeniably. be um.

that will cost less money like it will. save you a lot of money like orders the. magnitude less money that could replace. uber drivers for example so car sharing. that's autonomous that creates. a similar or better experience in terms. of how quickly you get from a to b or. just whatever the the pleasantness of. the experience. the efficiency of the experience the. value of the experience and at the same. time the car itself costs cheaper. i think that's very difficult to achieve. i think there's a lot more.

um low hanging fruit in the home. that that could be i also want to give. you perspective on. like how challenging it would be at home. or like it maybe kind of depends on the. exact problem that you'd be solving okay. if we are speaking about these robotic. arms. and hence. these things they cost tens of thousands. of dollars or maybe 100k. and. you know maybe obviously maybe there.

would be economy of scale these things. would be cheaper. but actually for any household to buy. the price would have to go down to maybe. thousand bucks. yeah i personally think. that uh. so self-driving car it provides a clear. service i don't think robots in the home. they'll be a trillion dollar company. will just be all about service. meaning it will not necessarily be about. like a robotic arm that. helps you i don't know open a bottle.

or wash the dishes or. any of that kind of stuff it has to be. able to take care of that whole the. therapist thing you mentioned. i i think that's um of course there's a. line between what is a robot and what is. not. like doesn't really need a body but you. know some. uh ai system with some embodiment i. think. so the tricky part when you think. actually what's the difficult part is. um. when the robot has. like when there is a diversity of the.

environment with which the robot has to. interact that becomes hard so you know. on one spectrum you have. industrial robots as they are doing over. and over the same thing it is possible. to some extent to prescribe the. movements and with very small amount of. intelligence the the movement can be. repeated millions of times um the it. there are also you know various pieces. of industrial robots where it becomes. harder and harder like for instance in. case of tesla it might be a matter of. putting a a rack inside of a car.

and you know because the rack kind of. moves around it's it's not that easy. it's not exactly the same every time it. ends up being the case that you need. actually humans to do it. and while you know welding cars together. it's a very repetitive process. and then in case of self-driving itself. the difficulty has to do with the. diversity of the environment but still. the car itself and the problem that you. are solving is.

you try to avoid even interacting with. things you are not touching anything. around because touching itself is hard. and then if you would have in the home. uh robot that you know has to touch. things and like if these things they. change the shape if there is a huge. variety of things to be touched then. that's difficult if you are speaking. about the robot which there is you know. head that is smiling in some way with. cameras that it doesn't you know touch. things that's relatively simple. okay so.

to both agree and to push back. so you're referring to touch like. soft robotics like the actual touch. but. i would argue that you could formulate. just basic interaction. between um like non-contact interaction. is also a kind of touch and that might. be very difficult to solve that's the. basic this not disagreement but that's. the basic open question to me. with self-driving cars and disagreement. with elon which is how much interaction.

is required to solve self-driving cars. how much touch is required you said that. in your intuition touch is not required. and my intuition to create a product. that's compelling to use you're going to. have to uh. interact with pedestrians not just avoid. pedestrians but interact with them. when we drive around in major cities. we're constantly threatening everybody's. life with our movements. and that's how they respect us there's a.

game theoretically going on with. pedestrians. and. i am afraid you can't just. formulate. autonomous driving as a collision. avoidance problem so i i think it goes. beyond like a collision avoidance is the. first order approximation. but then at least in case of tesla they. are gathering data from people driving. their cars. and i believe that's an example of. supervised learning data that they can. train their models uh on and they are. doing it.

which you know can give the model this. like. another level of. of a behavior that is needed to actually. interact with the real world yeah it's. interesting how much data. is required to achieve that. um. what do you think of the whole tesla. autopilot approach the computer vision. based approach with multiple cameras and. there's a data engine it's a multi-task. multi-headed neural network and it's. this fascinating process of uh similar.

to what you're talking about. with the the robotics approach. uh which is you know you deploy neural. network and then there's humans that use. it. and then it runs into trouble in a bunch. of places and that stuff is sent back so. like. the deployment discovers a bunch of edge. cases and those edge cases are sent back. for supervised annotation thereby. improving the neural network and that's. deployed again. it goes over and over until the the. network becomes really good at the task. of driving becomes safer and safer what.

do you think of that kind of approach to. robotics i believe that's the way to go. so in some sense even when i was. speaking about you know collecting. trajectories from humans that's like a. first step and then you deploy the. system and then you have humans revising. the. all the issues and in some sense. like this approach converges to system. that doesn't make mistakes because for. the cases where there are mistakes you. got their data how to fix them and the. system will keep on improving so there's. a very.

to me difficult question of how hard. that you know how long that converging. takes how hard it is. uh the other aspect of autonomous. vehicles this probably applies to. certain robotics applications. is society right they put. as as the quality of the system. converges. so one there's a human factors. perspective of psychology of humans. being able to supervise those uh even. with teleoperation those robots and the. other society willing to accept robots.

currently society is much harsher on. self-driving cars than it is on human. driven cars in terms of the expectation. of safety so the bar is set much higher. than for humans and we're so if there's. a death in an autonomous vehicle that's. seen as much more. much more dramatic than a death in a. human driven vehicle. part of the success of deployment of. robots is figuring out how to make. robots part of society both on the just.

the human side on the media journalist. side and also on the policy government. side and that seems to be uh. maybe you can put that into the. objective function to optimize. but that is that is definitely um. a tricky one and i wonder if that is. actually the trickiest part for. self-driving cars or any system that's. safety critical. it's not the algorithm it's the society. accepting it. yeah i i would say. i believe that.

the part of the process of deployment is. actually showing people that the given. things can be trusted yeah and you know. trust is also like a glass that is. actually really easy to crack it yeah. and damage it and. i think that's actually. very common with uh. with innovation. that there is some resistance toward it. yeah. and. it's just the natural progression so in. some sense people will have to keep on.

proving that indeed these systems are. worth being used and i would say. i also found out that. often the best way to convince people. is by letting them experience it yeah. absolutely that's the case for tesla. autopilot for example. that's the case with uh yeah with. basically robots in general it's it's. kind of funny to hear people talk about. robots like. there's a lot of fear. even like legged robots but when they. actually interact with them.

there's joy. i love interacting with them and the. same with the car. with the robot. if it starts being useful. i think people immediately understand. and if the product is designed well they. fall in love you're right. it's actually even similar when i'm. thinking about co-pilot the github. co-pilot there was a spectrum of. responses that people had and uh. ultimately. uh. the important piece was to let people. try it out and then many people just.

loved it especially like. programmers yeah programmers but like. some of them you know they came with a. fear. yeah but then you try it out and you. think actually that's cool okay and you. know you can try to resist the same way. as you know you could resist moving from. punch cards to let's say. c plus or so. and. it's a little bit futile. so we talked about generation program. generation of language. even.

self-supervised learning in the visual. space for robotics and then. reinforcement learning what do you and. like this whole beautiful. spectrum of ai. do you think is a good. benchmark a good test to strive for. to achieve intelligence that's a strong. test of intelligence you know it started. with alan turing and the touring test. maybe you think natural language. conversation is a good test. so you know it would be nice if for. instance machine would be able to solve.

riemann hypothesis in math. that would be i think that would be very. impressive so theorem proving. is that to you proving theorems is a. good. oh oh like one thing that the machine. did you would say damn. exactly. okay. that would be quite. quite impressive i mean the the tricky. part about the benchmarks is. um you know as we are getting closer. with them we have to invent new. benchmarks there is actually no ultimate.

benchmark out there yeah see my thought. with the riemann hypothesis would be. the moment the machine proves it would. say okay well then the problem was easy. that's what happens and i mean in some. sense um that's actually what happens. over the years in ai that like uh. we get used to things very quickly you. know something i talked to rodney brooks. i don't know if you know that is. he called alpha zero homework problem. because he was saying like there's. nothing special about it it's not a big.

leap and i i didn't. well he's coming from one of the aspects. that we referred to as he was part of uh. the founding of irobot which deployed. now tens of millions of robot in the. home so. if you see robots. that are actually in the homes of people. as the legitimate. instantiation of artificial intelligence. then yes maybe an ai that plays a silly. game like going chess is not a real. accomplishment but to me it's it's a. fundamental leap but i think we as.

humans then say okay well then. that uh that game of chess or go wasn't. that difficult compared to the thing. that's currently unsolved so my. intuition is that. from perspective of the evolution of. you know these ai systems we'll at first. see the tremendous progress in digital. space and the you know the main thing. about digital space is also that you can. everything is that there is a lot of. recorded data plus you can very rapidly. deploy things to billions of people. while in case of uh physical space the.

deployment part takes multiple years you. have to manufacture things and. you know delivering it to actual people. it's very hard. so i'm expecting that the first and the. prices in digital space. of goods they would go you know down to. the let's say marginal costs are to zero. and also the question is how much of our. life will be in digital because it seems. like we're heading towards more and more.

of our lives being in the digital space. so like. innovation in the physical space might. become less and less significant like. why do you need to drive anywhere. if most of your life is spent in virtual. reality i still would like you know to. at least at the moment my impression is. that i would like to have a physical. contact with other people and that's. very important to me and we don't have a. way to replicate it in the computer it. might be the case that over the time it. will change. like in 10 years from now why not have.

like an arbitrary infinite number of. people you can interact with some of. them are real some are not. with uh arbitrary characteristics that. you can define based on your own. preferences i think that's maybe where. we are heading and maybe i'm resisting. the future yeah. i'm telling you. i if i got to choose. if i could live in elder scrolls skyrim. versus the real world i'm not so sure i. would stay with the real world.

yeah i mean the question is so will vr. be sufficient to get us there or do do. you need to you know plug electrodes in. the brain. and it would be nice if these electrodes. wouldn't be invasive yeah. or at least like provably. non-destructive. but in in the digital space do you do. you think we'll be able to solve the. touring test the spirit of the touring. test which is do you think we'll be able. to. achieve compelling natural language.

conversation between people like have. friends that are ai systems on the. internet. i thought i think it's doable do you. think the current approach of gbt. will take us there so there is you know. the the part of at first learning all. the content out there and i i think that. steel system should keep on learning as. it speaks with you yeah. and i think that should work the. question is how exactly to do it and you. know obviously. we have people at open air asking these. questions.

and kind of at first pre-training on all. existing content is like a backbone and. it's a decent backbone. do you think ai. needs a body. connecting to our robotics question. to uh truly connect with humans or can. can most of the connection be in the. digital space. so let's see. we know that there are people who met. each other online and they felt in love. yeah. so it seems that it's conceivable to.

establish connection which is purely. through internet. and of course it might be more. compelling than more modalities you add. so it would be like you're proposing. like a tinder but for ai. are you like swipe right and left and. half the systems are ai and the other is. uh. humans and you don't know which is which. that would be ours that would be our. formulation of touring test the the. moment ai is able to achieve more swipe.

right or left whatever. the the moment is able to be more. attractive than other humans. it passes the torrent test then you. would pass the turing test in. attractiveness that's right well no like. attractiveness just declare conversation. not just visual right it's also. attractiveness. with wit and humor and uh whatever. whatever makes conversations pleasant. for humans. okay all right um. so so you're saying uh it's possible to.

achieve in the digital space in some. sense i would almost ask that question. why wouldn't that be possible. right. well. i have this argument with my dad all the. time he thinks that touch and smell are. really important. so they can be very important and i'm. saying the initial systems they won't. have it. still i wouldn't like their people being. born. without these senses. and. you know i believe that they can still. fall in love and have meaningful life.

yeah i wonder if it's uh possible to go. close to all the way by just training on. transcripts of conversations. like i wonder how far that takes us so i. i think that actually still you want. images like i would like so i don't have. kids but like i could imagine the. having ai tutor it has to see you know. kids. drawing some pictures on their paper. and and also facial expressions and all. that kind of stuff we use uh dogs and.

humans use their eyes and. uh to communicate with each other i. think this that's that's a really. powerful mechanism of communication body. language too. that uh words are much uh lower. bandwidth and for body language we still. you know we kind of have a system that. displays an image. of its or facial expression on the. computer it doesn't have to move you. know mechanical pieces or so so i think. that uh you know there is like kind of a. progression you can imagine that text. might be the simplest to tackle.

but. this is not a complete. human experience at all you expand it to. let's say. images both for input and output and. what you describe is actually the final. i guess frontier what makes us human the. fact that we can touch each other or. smell or so and it's the hardest from. perspective of data and deployment. and. i okay i believe that these things might. happen gradually. are you excited by that possibility this.

particular application of. human to ai. friendship and interaction so let's see. like would you uh do you look forward to. a world you you said you're living with. a few folks and you're very close. friends with them. do you look forward to a day where one. or two of those friends are ai systems. so if the system would be truly wishing. me well. rather than being in the situation that. it optimizes for my time to interact. with the system. the line.

between those is it's a gray. it's a gray area i i think that's the. distinction between. love and possession and these things. they might be often correlated for. humans but it's it like like a you you. might find that there like some friends. with whom you haven't spoken for months. yeah and then you know you pick up the. phone it's as the time hasn't passed. they are not holding to you. and i will i wouldn't like to have ai. system that you know it's.

it's. trying to convince me to spend time with. it i would like the system to optimize. for what i care about and help me. in achieving my own goals. but there's some i mean i don't know. there's some manipulation there's some. possessiveness there's some insecurities. this fragility all those things are. necessary. to form a close friendship over time to. go through some dark shit together some. bliss and happiness together i feel like.

there's a lot of greedy self-centered. behavior within that process. my intuition but i might be wrong is. that. human computer interaction doesn't have. to go through uh. computer being greedy possessive and so. on it is possible to train systems maybe. that they actually. you know they are i guess prompted or. fine-tuned or so. to truly optimize for what you care. about and you could imagine that you.

know. that the way how the process would look. like is at some point. we as a human as we look at the. transcript of the conversation or like. an entire interaction and we say. actually here there was more loving way. to go about it and we supervise system. toward being more loving. or maybe we train the system such that. it has a reward function toward being. more loving yeah or maybe the. possibility of the system being an. asshole.

and manipulative and possessive every. once in a while is a feature not a bug. because some of the. happiness that we experience when two. souls meet each other when two humans. meet each other is a kind of break from. the assholes in the world. and so you need assholes and ai as well. because like. it'll be like a breath of fresh air to. discover an ai that. the three previous ais you had are too.

friendly. or no no or or cruel or whatever it's. like some kind of mix. and then this one is just right but you. need to experience the full spectrum. like i think you need to be able to. engineer assholes so let's see. because there's some level to us of. being appreciate to appreciate the human. experience. we need the dark. and the light so that kind of reminds me. um i met a while ago at the meditation.

retreat uh. one woman and. um you know beautiful beautiful woman. and she had a she had a crutch okay she. had the trouble uh walking on one deck i. asked her what has happened. and. she said that five years ago she was in. maui hawaii. and she was eating a salad and some. snail fell into the salad and apparently. there are neurotoxic.

snails over there and she got into coma. for a year okay oh wow. and. apparently there is you know high chance. of even just dying but she was in the. coma at some point. she regained partially consciousness she. was able to hear people in the room. people behave as she wouldn't be there. you know at some point she started being. able to speak but she was mumbling like. a barely able to to express herself and. at some point she got into wheelchair.

then at some point she actually noticed. that she can move her uh. a toe. and then she knew that she will be able. to walk. and then you know that's where she was. five years after and she said that since. then she appreciates the fact that she. can move her toe. and i was thinking. do i need to go through such experience. to appreciate that i have i can move my. toe well that's really good story a. really deep example yeah. and in some sense it might be the case.

that we don't see. light if we haven't went through the. darkness but i wouldn't say that we. should. we shouldn't assume that that's the case. which. may we maybe will do engineer shortcuts. yeah ilia had. this you know belief that maybe one has. to go for a week or six months. to some challenging camp yeah. to just experience you know a lot of. difficulties and then comes back and.

actually. everything is bright everything is. beautiful i'm with iliana it must be a. russian thing where are you from. originally i'm i'm polish polish. okay. i'm tempted to say that explains a lot. but uh yeah there's something about the. russian the necessity of suffering i. believe i believe suffering or rather. struggle is necessary i believe that. struggle is necessary i mean in some. sense. you. even look at the story of any superhero. in that movie it's not that it was like.

everything like it goes easy easy i like. how that's your ground truth. it's the story of superheroes okay. uh you mentioned that you used to do. research at night and go to bed at like. 6 a.m or 7 a.m. i still do that. often. um. what uh sleep schedules have you tried. to make for a productive and happy life. like is there. um is there some interesting wild. sleeping patterns that you engaged that. you found that works really well for you.

i tried at some point. decreasing number of hours of sleep like. gradually. like a half an hour every few days less. you know i was hoping to just save time. that clearly didn't work for me like at. some point there's like a phase shift. and. i felt tired all the time. uh. you know there was a time that i used to. work during the nights the nice thing. about the nights is that no one disturbs. you. and. even i remember. when i was.

meeting for the first time with greg. brookman his cto and chairman of openai. our meeting was scheduled to 5 pm. and i overstepped for the meeting. over slept for the meeting yeah 5 p.m. yeah now you sound like me that's. hilarious okay yeah and uh at the moment. in some sense uh. my sleeping schedule also has to do with. the fact that i'm. interacting with people. i sleep without an alarm. so.

so yeah the the team thing you mentioned. extrovert thing because. most humans operate during a certain set. of hours. you're forced to then operate at the. same set of hours. but. i'm not quite there yet. i found a lot of joy just like you said. working through the night. because it's quiet. because the world doesn't disturb you. and there's some aspect. counter to everything you're saying. there's some joyful aspect to sleeping.

through the mess of the day. because uh people are having meetings. and sending emails and there's drama. meetings i can sleep through all the. meetings you know i have meetings every. day and they prevent me from having. sufficient amount of time for. focus work. and. then. i modified my calendar and i said that. i'm out of office wednesday thursday and. friday every day and i'm having meetings. only monday and tuesday. and that vastly.

positively influenced my mood that i. have literally like had three days for. fully focused work yeah so there's. better solutions to this problem than. staying awake all night okay. you've been part of development of some. of the greatest ideas in artificial. intelligence what would you say is your. process for developing good novel ideas. you have to be aware that. clearly there are many other brilliant. people around. so. you have to ask yourself a question. why the given idea.

let's say wasn't uh tried by someone. else. and in some sense it has to. do with. you know kind of simple it might sound. simple but like i'm thinking outside of. the box and what do i mean here. so for instance for a while. people in academia they assumed. that. you have a fixed data set. and then you optimize the algorithms. in order to get the best performance.

and. that was so in great assumption. that no one thought about. training models on anti-internet. or like that that maybe some people. thought about it but if it felt too too. many as unfair. and in some sense that's almost like a. it's not my idea or so but that's an. example of breaking a typical assumption. so you want to be in the paradigm that. you are breaking a typical assumption.

in the context of the ai community. getting to pick your dataset as cheating. correct and in some sense so that was a. that was assumption that many people had. out there. and then if you free yourself from. assumptions. then. they are. likely to achieve something that others. cannot do and in some sense if you are. trying to do exactly the same things as. others. it's very likely that you're gonna have. the same results yeah i.

but there's also that kind of tension. which is uh. asking yourself the question why. haven't others done this. because um. i mean i get a lot of good ideas. but i think probably most of them suck. when they meet reality. so so actually i think the other big. piece. is. uh getting into habit of generating. ideas training your brain toward. generating ideas and not even.

suspending judgment of the ideas. so in some sense i noticed myself that. even if i'm in the process of generating. ideas if i tell myself oh that was a bad. idea. then that actually interrupts the. process and i cannot generate more ideas. because i'm actually focused on the. negative part why it won't work yes. but. i created also environment in the way. that it's very easy for me to to store. new ideas so for instance next to my bed.

i have a. voice recorder. and it happens to me often like i wake. up in that during the night and i have. some idea in the past i was. writing them down on my phone but that. means you know turning off this turning. on the screen and that wakes me up or. like pulling a paper which requires you. know turning on the. light these days i just start recording. it. what do you think i don't know if you. know who jim keller is i know team color. he's a big proponent of thinking hard on.

a problem right before sleep so that he. can sleep through it and solve it in a. sleep. or like come up with radical stuff in. his sleep he was trying to get me to do. this so. it happened from. my experience perspective it happened to. me many times during the high school. days when i was doing mathematics. that i had the solution to my problem as. i woke up. at the moment regarding thinking hard.

about the given problem is. i'm trying to actually devote. substantial amount of time to think. about important problems not just before. the sleep. like i'm organizing amount of the huge. chunks of time such that i'm not. constantly working on the urgent. problems but i actually have time to. think about the important one so you do. it naturally but his idea is that you. kind of. prime your brain. to make sure that that's the focus you. know oftentimes people have other. worries in their life that's not. fundamentally deep problems like i don't.

know uh just stupid drama in your life. and even at work all that kind of stuff. he wants to kind of. pick the most important problem. that you're thinking about and go to bed. on that i think that's why i mean the. other thing that comes to my mind is. also i feel the most fresh in the. morning. so during the morning i try to work on. the most important things rather than. i'm just being pulled by urgent things. or checking email or so. what do you do with the cause i've been.

doing the voice recorder thing too but i. end up recording so many messages it's. hard to organize. i have the same problem now i have heard. that. google pixel is really good in. transcribing text and i might get a. google pixel just for the sake of. transcribing text yeah people listening. to this if you have a good voice. recorder suggestion that transcribed. please let me know. i it's some of it is uh this has to do. with uh. uh open ai codex too like some of it is. simply like the friction.

i need uh. apps that remove that friction between. voice. and the organization of the resulting. transcripts and all that kind of stuff. um but yes you're right absolutely like. during uh for me it's walking sleep too. but walking and running. especially running. get a lot of thoughts during running and. there's there's no. good mechanism for recording thoughts so. one more thing that i do i have a. separate phone.

uh which i. which has no apps. and maybe it has like a. audible or let's say kindle no one has. this phone number this kind of my. meditation phone yeah and. i try to. expand the amount of time that that's. the phone that i'm having i it has also. google maps if i need to go somewhere. and i also use this phone to write down. ideas. ah that's really good idea. that's a really good idea often actually. what i end up doing is even sending a.

message from that phone to the other. phone so that's actually my way of. recording messages or i just put them. into notes i love it. what advice would you give to a young. person high school. college. about how to be successful you've done a. lot of incredible things in the past. decade. so maybe maybe of some. something there might be something there. might be something. i mean. might sound like a. simplistic or so but i would say.

literally just. follow your passion double down on it. and if you don't know what's your. passion just figure out what could be a. what could be a passion so this that. might be an exploration. when i was in elementary school was math. and chemistry. and i remember. for some time i gave up on math because. my school teacher she told me that i'm. dumb. and i i i guess maybe an advice would be. just ignore people if they tell you that.

you're dumb. you mentioned something offline about. chemistry and explosives. um. what was that about so let's see. so a story goes like that i can. i got into chemistry maybe i was like a. second grade of my elementary school. third grade. uh i started going to chemistry classes. uh. i i really love building stuff.

and. i did all the experiments that they. described in the book okay you know how. to create oxygen with vinegar and. and baking soda so okay. so i did all the experiments. and at some point i was you know so. what's next what can i do. and uh. explosives they also it's like a you. have a clear reward signal you know if. the thing worked or not. so i remember. at first i got i got interested in.

producing hydrogen that was kind of. funny experiment from school you can. just burn it and then i moved to. uh nitroglycerin so that's also. relatively easy to synthesize. i started producing essentially. dynamite and detonating with it with my. friend i remember there was a you know. there was at first like maybe. two attempts that i went with a friend. to detonate what we built. and it didn't work out and like a third. time he was like ah it won't work like.

uh let's don't waste time. and um now we were. i was carrying this uh. this. you know that tube with dynamite i don't. know pound or so. dynamite in my backpack or like riding. on the bike to the edges of the city. [Laughter]. yeah and. attempt number three. this would be to number three attempt. number three. and uh. now we we dig a hole to.

uh put it inside it actually had the uh. you know electrical detonator. we we draw a cable behind the tree. i even i never had i haven't ever seen. like a explosion before so i thought. that there will be a lot of sound. and but you know we're like laying down. and i'm holding the cable and the. battery at some point you know it kind. of like a three to one. and uh. i just connected it and it felt like at. the ground shake it was like a more like.

a. sound and then. the soil started kind of lifting up and. started falling on us yeah. wow and then uh. now the friends said let's let's make. sure next time we have helmets. but also you know. i'm happy that nothing happened to me it. could have been the case that i i lost. the limb or so yeah but that's childhood. of an engineering mind. with a strong reward signal.

of an explosion. i love it. my there's some aspect of uh chemists. the the the chemist i know like my dad. with plasma chemistry plasma physics he. was very much into explosives too it's a. worrying quality of. people that work in chemistry that they. love i think it is that exactly is the. the strong signal that the thing worked. there is no doubt there's no doubt. there's some magic it's almost like a. reminder that physics works that.

chemistry works it's cool it's almost. like a little glimpse at nature that you. yourself engineer i that's why i really. like artificial intelligence especially. robotics is you create. a little piece of nature. and in some sense even for me with. explosives the motivation was creation. rather than distraction yes exactly. in terms of advice i forgot to ask. about just machine learning and deep. learning for people who are specifically. interested in.

machine learning how would you recommend. they get into the field. so um i would say implement everything. and also there is plenty of courses. so like from scratch um so on different. levels of abstraction in some sense but. i would say brain or implement something. from scratch or implement something from. a paper or implement something you know. from podcasts that you have heard about. i would say that's a powerful way to. understand things so it's often the case. that you read the description and you. think you understand but you truly.

understand once you build it. then you actually know what really meant. that in the description. is there particular topics that you find. people just. fall in love with so i i've seen. i tend to uh really enjoy reinforcement. learning. because it's it's much more. it's much easier to get to a point where. you feel like you created something. special. like like fun games kind of things that. are rewarding it's rewarding yeah.

uh as opposed to like. uh reimplementing from scratch. more like supervised learning kind of. things it's it's yeah so you know if if. someone would optimize for things to be. rewarding then it feels that the things. that are somewhat generative they have. such a property so yes you have for. instance yes adversarial networks or you. have just even generated language models. and you could you can even see. um internally we have seen this thing.

with our releases so we have a we. released recently two models there is. one model called dali that generates. images and there is other model called. clip that actually uh. you provide. various possibilities what could be the. answer to what is on the picture and it. can tell you which one is the most. likely okay. and in some sense in case of the. first one dali. it is very easy for you to understand. that actually there is magic going on uh.

and in the in case of the second one. even though it is insanely powerful and. you know people from a vision community. they as they started probing it inside. they actually understood. how far it goes it's difficult for. person at first to see. how well it works. and that's the same as you said that in. case of supervised learning models you. might not kind of see. or it's not that easy for you to. understand the the strength.

even though you don't believe in magic. to see the magic let's say that magic. it's a generative that's really. brilliant so anything that's generative. because then you are. at the core of the creation you get to. experience creation. without much effort unless you have to. do it from scratch but and it feels that. you know humans are wired there is some. level of reward for creating stuff yeah. like of course different people have a. different weight on this reward yeah.

in the big objective function in the big. objective. of a person of a person uh you wrote. that beautiful. is what you intensely pay attention to. even a cockroach is beautiful if you. look very closely. can you expand on this what is. beauty. so. what i'm i wrote here actually. corresponds to my subjective experience. that i had. through extended periods of meditation. it's it's pretty crazy that at some.

point the meditation gets you to the. place that you have really. increased uh focus increase attention. and then you look at the very simple. objects that were all the time around. you can look at the table or on the pen. or at that nature. and. you notice more and more details. and it becomes very pleasant to look at. it. and it once again it kind of reminds me. my childhood.

uh like i just pure joy. of being. it's also i have seen even the reverse. effect that. by default regardless of what we possess. we very quickly get used to it. and you know you can have a very. beautiful house. and. if you don't put sufficient effort. you're just gonna get used to it. and it doesn't bring any more joy. regardless of what you have yeah. well i actually.

i find that material possessions. get in the way of that experience of. pure joy. so i've always i've been very fortunate. to just find joy in simple things just. just like you're saying. just like i don't know objects in my. life just stupid objects like this cup. like thing you know just objects sounds. okay i'm not being eloquent but. literally objects in the world.

they're just full of joy because it's. like. i can't believe. one i can't believe that i'm. fortunate enough to be alive to. experience these objects and then two i. can't believe. humans are clever enough to have built. these objects. the the hierarchy of pleasure that that. uh provides is infinite i mean even if. you look at the cup of water so you know. you see first like a level of like a. reflection of light but then you think. you know man there's like a trillions. upon of trillions of particles bouncing.

uh against each other there is also uh. the tension on the surface that you know. if the back back could like a stand on. it and move around and you think it also. has this like a magical property that as. you decrease temperature. it actually expands in volume which. allows for the. you know legs to freeze on the on the. surface and then at the bottom to have. actually uh not freeze which allows for. life like a. crazy yeah you look.

in detail at some object and you think. actually you know this table that was. just the figment of someone's. imagination at some point and then there. was like thousands of people involved to. actually manufacture it yeah and put it. here and by default no one cares. [Laughter]. and then you can start thinking about. evolution how it all started from single. cell organisms. that led to this table and and okay. these thoughts they give me life. appreciation yeah and. even lack of those just the pure raw. signal also gives their life.

appreciation see the thing is. and then that's coupled. for me with the sadness that the whole. ride ends. and perhaps is deeply coupled in that. the fact that this experience this. moment ends. gives it. gives it an intensity that i'm not sure. i would otherwise have. so in that same way i try to meditate on. my own death often. do you think about your mortality. are you afraid of death.

so. fear of death is like one of the most. fundamental fears that each of us has we. might be not even aware of it it. requires to look inside to even. recognize that it's. out there and there is still let's say. this property. of uh nature that if things would last. forever then they would be also boring. to us. the fact that the things change in some. way. gives any meaning to them.

i also you know found out that. it seems to be. very healing to people to. have. this short experiences. uh like i guess psychedelic experiences. in which they experience. death of self. in which they let go of this fear and. then maybe can even increase the. appreciation of the moment and it seems.

that many people they uh. they. they can easily comprehend fine the fact. that their money is finite while they. don't see that time is finite. i have this like a discussion with ilya. from time to time he's saying you know. man like uh. the lack will pass very fast at some. point i will be 40 50 60 70 and then. it's over. this is true which also makes me believe.

that you know that every single moment. it is so unique. that. should be appreciated and this also. makes me think that i should be acting. on my life. because otherwise it will pass. i also like this framework of thinking. from jeff bezos on regret minimization. that like i would like. if i will be at that death bed to look. back on my life. and.

and not regret that i haven't. done something it's usually you might. regret that you haven't. tried i'm fine with failing. i haven't tried. uh what's the nature eternal occurrence. tried to live a life that if you had to. live it infinitely many times. that would be the. you'll be okay with. that kind of life. so try to live it optimally. i can say that. it's almost like i'm.

unbelievable to me. where i am in my life i'm extremely. grateful for actually people. whom i met i would say i think that i'm. decently smart and so on. uh. but i think that. actually to great extent where i am has. to do with that people who i met. would you be okay. if after this conversation you died. so. if i'm dead then it kind of.

i don't have a choice anymore. so in some sense there's like plenty of. things that i would like to try out in. my life. [Music]. i feel that you know i'm gradually going. one by one and i'm just doing them. i think that the list will be always. infinite yeah. so might as well go today. yeah i mean to be clear i'm not looking. forward to die. i would say if there is no choice i. would accept it. but like uh in some sense i'm.

if there would be a choice if there. would be possibility to live i would. fight for a living. i find um. it's more honest than real to think. about you know dying today at the end of. the day. that seems to me to at least to my brain. more honest slap in the face. as opposed to i still have 10 years. like today. then then i'm much more about. appreciating the cup and the table and. so on and less about like silly worldly.

accomplishments and all those kinds of. things. we we have in the company a person who. say at some point found out that they. have cancer and that also gives you know. huge perspective with respect to what. matters now yeah and you know often. people in situations like that they. conclude that actually what matters is. human connection. and. love and uh that's people conclude also. if you have kids because kids is family. you uh i think tweeted.

we don't assign the minus infinity. reward to our death. such a reward would prevent us from. taking any risk we wouldn't be able to. cross the road in fear of being hit by a. car so in the objective function you. mentioned fear of death might be. fundamental to the human condition. so. as i said let's assume that they're like. a reward functions in our brain. and. and the interesting thing is. even realization how different reward.

functions can play with your behavior. as a matter of fact i wouldn't say that. you should assign infinite. negative reward to anything because that. messes up the math. the math doesn't work out it doesn't. work out and as you said even you know. uh government or some insurance. companies you said they assign 99. million dollars to human life yeah and. i'm just saying it with respect to. that might be a harsh statement to.

ourselves but in some sense that there. is a finite value of our own life. i'm trying to put it from perspective of. being. less. of being more egoless. and realizing fragility of my own life. and. in some sense. the. fear of death. might prevent you from acting. because anything can cause death. yeah and i'm sure actually if you were.

to put death in the objective function. there's probably so many aspects to. death and fear of death and. realization of death and mortality. there's just whole components of. finiteness. of not just your life but every. experience and so on you're gonna have. to formalize mathematically and also you. know that might lead to. um. you spending a lot of compute cycles. on this like a.

and deliberating this terrible future. instead of experiencing now. and that in some sense is also kind of. unpleasant simulation to run in your. head yeah. do you think there's an objective. function that describes the entirety of. uh. human life. so you know usually the way you ask that. is what is the meaning of life. is there um. a universal objective functions that. captures the why of life so yeah i mean.

i suspected that they will ask this. question but it's also a question that i. asked myself many many times. see i can tell you a framework that i. have these days to think about these. questions so i think that fundamentally. meaning of life has to do. with some of our reward functions that. we have in brain and they might have to. do with. let's say for instance curiosity. or. human connection which might mean. understanding others.

it's also possible for a person to. slightly modify their reward function. usually they mostly stay fixed but it's. possible to modify reward function and. you can pretty much choose so in some. sense reward functions optimizing reward. functions they will give you life. satisfaction. is there some randomness in the function. i think when you are born there is some. randomness like you can see that some. people for instance they. they care more about building stuff some. people care more about caring for others.

some people. that there are all sorts of uh default. reward functions and then in some sense. you can ask yourself what's this like. what is the satisfying way for you to go. after this reward function and you just. go after this reward function and you. know some people also ask are these. reward functions real. i almost think about it. as. let's say. if you would have to discover. mathematics. in mathematics you are likely to run. into various objects like a complex.

numbers. or differentiation some other objects. and these are very natural objects that. arise and similarly the reward functions. that we are having in our brain they are. somewhat very natural that you know. there is a reward function for. for understanding. like a comprehension yeah uh. curiosity and so on so in some sense. they are in the same way natural as. their natural objects in mathematics. interesting so you know there's the uh.

the old. sort of debate is mathematics invented. or discovered you're saying reward. functions are discovered so nature so. nature's provided some you can still. let's say expanded throughout the life. some of the reward functions they might. be futile like for instance there might. be a reward function maximize amount of. wealth. yeah and this is more like a a learning. reward function. and but we know also that some reward. functions if you optimize them you won't. be. quite satisfied.

well i don't know which part of your. reward function resulted in you coming. today but i am deeply appreciative that. you did spend your valuable time with me. watching is really fun talking to you. you're you're brilliant you're a good. human being and it's an honor to meet. you and an honor to talk to you thanks. for talking today brother. thank you alex a lot i appreciated your. questions here. i had a lot of time being here. thanks for listening to this. conversation with welch and ramba to.

support this podcast please check out. our sponsors in the description. and now let me leave you some words from. arthur c clarke who is the author of. 2001 a space odyssey. it may be that our role on this planet. is not to worship god but to create him. thank you for listening and hope to see. you next time.

you.