Douglas Lenat: Cyc and the Quest to Solve Common Sense Reasoning in AI | Lex Fridman Podcast #221

the following is a conversation with. Doug lennet creator of Psych A system. that for close to 40 years and still. today has sought to solve the core. problem of artificial intelligence the. acquisition of Common Sense knowledge. and the use of that knowledge to think. to reason and to understand the world to. support this podcast please check out. our sponsors in the description as a. side note let me say that in the. excitement of the modern era of machine. learning it is easy to forget just how.

little we understand exactly how to. build the kind of intelligence that. matches the power of the human mind to. me many of the core ideas behind Psych. in some form in actuality or in spirit. will likely be part of the AI system. that achieves General super intelligence. but perhaps more importantly solving. this problem of Common Sense knowledge. will help us humans understand our own. minds the nature of Truth and finally.

how to be more rational and more kind to. each other this is the Lex Freedman. podcast and here is my conversation with. Doug. lennett Psych is a project launched by. you in. 1984 and still is active today whose. goal is to assemble a knowledge base. that spans the basic concepts and rules. about how the world Works in other words. it hopes to capture Common Sense. knowledge which is a lot harder than it. sounds can you elaborate on this Mission.

and maybe perhaps speak to the various. subg goals within this Mission when I. was a faculty member in the computer. science department at Stanford my. colleagues and I did research in all. sorts of artificial. intelligence programs so natural. language understanding programs robots. expert. systems and so on and we kept hitting. the very same brick wall.

our systems would have impressive early. successes and so if your only goal was. academic namely to um get enough. material to write a journal article uh. that might actually suffice but if. you're really trying to get AI um then. you have to somehow get past the brick. wall and the brick wall was the programs. didn't have what we would call Common. Sense they didn't have General World. Knowledge they didn't really understand.

what they were doing what they were. saying what they were being asked um and. so very much like a um um a clever dog. performing tricks we could get them to. do tricks but they never really. understood what they were doing sort of. like when you get a dog to fetch your. morning newspaper uh the dog might do. that successfully but the dog has no. idea what a newspaper is or what it says. or anything like that what does it mean. to understand something can you maybe. elaborate in that a little bit is it is.

understand an action of like combining. little things together like through. inference or is understanding the wisdom. you gain over time that forms a. knowledge I think of understanding more. like a um uh think of it more like the. ground you stand on which um could be. very shaky could be very unsafe um but. most of the time is not because. underneath it is more ground um and. eventually you know rock and and other.

things but um layer after layer after. layer that solid foundation is there and. you rarely need to think about it you. rarely need to count on it but. occasionally you do and um um I've never. used this analogy before so bear with me. but um I think the same thing is true in. in terms of getting computers to. understand things which is uh you ask a. computer a question for instance Alexa. or um some robot or something and um.

maybe it gets the right answer um but if. you were asking that of a human you. could also say things like why or how. might you be wrong about this or. something like that and the person you. know would would answer you and you know. it might be a little Annoying if you. have a small child and they keep asking. why questions in series eventually you. get to the point where you throw up your. hands and say I don't know it's just the. way the world is um but for many layers.

you actually have that that layered. solid foundation of support so that when. you need it you can count on it and when. do you need it Well when things are. unexpected when you come up against a. situation which is novel for instance. when you're driving um it may be fine to. have a small program a small set of. rules that cover you know 99% of the. cases but that 1% of the time when. something strange happens um you really.

need to draw on Common Sense for. instance um my wife and I were driving. recently um and there was a trash truck. in front of us and um I guess they had. packed it too full and the back exploded. and trash bags went everywhere uh and we. had to you make a split-second decision. are we going to slam on our brakes are. we going to Swerve into another Lane um. are we going to just run it over um. because there were cars all around us um. and you know in front of us was a large.

um trash bag and we know what we throw. away in trash bags probably not a safe. thing to run over um over on the the. left was um a bunch of fast food. restaurant um trash bags and it's like. oh well those things are just like. Styrofoam and leftover food we'll run. over that and so that was a safe thing. for us to to do now that's the kind of. thing that's going to happen maybe once. in your. life and um but the point is that. there's almost no telling what little.

bits of knowledge about the world you. might actually need um in some. situations which were unforeseen but see. when you sit on that mountain or that. ground that goes deep of knowledge in. order to make a split-second decision. about uh fast food trash or random trash. from the back of a uh trash. truck uh you need to be able to leverage. that ground you stand on in some way. it's not not merely you know it's not.

enough to just have a lot of ground to. stand on it's your ability to leverage. it to utilize in a split like integrate. it all together to make that Split. Second decision and I I I. suppose understanding isn't. just having uh Common Sense knowledge to. access it's the act of accessing. accessing it somehow like. correctly um f fing out the the parts of.

the knowledge are not useful selecting. only the useful parts and uh effectively. making conclusive decisions so let's. tease apart two different tasks really. both of which are um incredibly. important and even necessary um if. you're going to have this in a usable a. useful usable fashion as opposed to say. like library books sitting on a shelf. right um and so on um where the. knowledge might be there but you know if. a fire comes the books are going to burn.

because they don't know what's in them. and they're just going to um sit there. while they burn um so there two there. two aspects of using the knowledge one. is a kind of a theoretical um how is it. possible at all and then the second. aspect of what you said is how can you. do it quickly enough right um so how can. you do it at all is something that. philosophers have grappled with and. fortunately philosophers hundred years. ago and um even earlier um developed a.

kind of um formal language um like. English um it's called um U predicate. logic or first order logic or um. something like predicate calculus and so. on so there's a way of representing. things in this formal language um which. enables a mechanical procedure to sort. of grind through and algorithmically. produce all of the same.

logical entailments all the same logical. conclusions that you or I would from. that same set of pieces of information. that are represented that way um so um. that that sort of raises um a couple. questions one is how do you get all this. information from say observations and. English and so on into this logical form. right and secondly how can you then. efficiently um run these algorithms to. actually get the information you need in.

in the case I mentioned in a tenth of a. second rather than say um in you know 10. hours or 10,000 years of computation um. and those are both really important um. uh questions and and like a corollary. addition to the first one is how many. such things do you need to gather for it. to be useful in certain contexts so like. what in order you mentioned philosophers. in order to capture this world and. present it in a logical way in a with a.

formal logic like how many statements. are required is it five is it 10 is it. 10 trillion is it like that that's as. far as I understand is uh probably still. an open question it may forever be an. open question uh to to say like. definitively about to describe the. universe perfectly how many facts do you. need I'm I I guess I'm going to. disappoint you by giving you an actual. answer to your question okay um well no.

this sounds exciting yes okay so so now. we have like um three three things to to. talk about keep adding more although. that's okay the first and the third are. related yes um so let's leave the. efficiency question aside for now so um. how how does all this information get. represented in logical form so that. these algorithms um resolution theorem. proving and other algorithms can.

actually grind through all the logical. consequences of what you said um and. that ties into your question about well. how many of these things do you need um. because if the answer is small enough um. then by hand you could write them out. one at a. time so um in um the um 19 early um. 1984 um I held a meeting at Stanford. where I was a faculty member there um. where we assembled um about half a dozen.

of the smartest people I know um people. like um Alan Newell and Marvin Minsky. and Alan K and um um a few others was. fman there by chance cuz he he liked. your he commented about your system Uris. at the time no no he he wasn't part of. this meeting um but that's a heck of a. meeting anyway I think Ed fenom was. there I think um um Josh leberg was. there so we we have um um all these.

different um smart people and we were um. we came together to uh address the. question that that you raised which is. if it's important to represent Common. Sense knowledge and World Knowledge in. order for AIS to not be brittle in order. for AIS not to just have the veneer of. intelligence well how many pieces of. Common Sense how many if then. rules for instance would we have to. actually write in order to essentially.

cover what what people expect perfect. strangers to already know about the. world and um I expected there would be. an enormous Divergence of opinion um and. computation but amazingly everyone got. an answer which was around a million um. and one person one person got the answer. um by saying well um look you can only. burn into human long-term memory a. certain number of things per unit time.

like maybe one every 30 seconds or. something and other than that it's just. shortterm memory and it flows away like. water and so on so by the time you're. say 10 years old or so um how many. things could you possibly have burned. into your long-term memory and it's like. about a million um another person went. in a completely different direction and. said well if you look at the number of. words um in a um a dictionary not a. whole dictionary but for someone to um. essentially um be considered to be.

fluent in a language how many words. would they need to know and then about. how many things about each word would. you have to tell it and so they got to a. million that way um another another. person said well let's actually look at. one single um short one volume desk. encyclopedia article and so we'll look. at you know what was like a um a four. paragraph article or something I think. about greed GBS GBS are a type of water.

foul um and if we were going to sit. there and represent every single thing. that was there um how many assertions or. rules or statements would we have to. write in this logical language and so. and then multiply that by all of the. number of articles that there were and. so on so all of these estimates came out. with a million um and so if you do the. math it turns out that like oh well then. maybe in um something like. um 100 um person years um in one or two.

person centuries we could actually get. this written down by hand and a. marvelous um coincidence opportunity. existed um right at that point in time. the early 1980s there was something. called the Japanese fifth generation. Computing effort Japan had threatened to. do in Computing and Ai and Hardware what. they had just finished doing in consumer.

electronics and the automotive industry. namely resting control away from the. United States and more generally away. from the West um and so America was. scared um and um Congress did something. that's how you know it was a long time. ago because Congress did something. congress Congress passed something. called the national Cooperative research. act ncra and what it said was hey all. you big American companies that's also. how you know it was a long time ago. because they were American companies. rather than multinational companies hey.

all you big American companies um. normally it would be an antitrust. violation if you colluded on R&D but we. promise for the next 10 years uh we. won't prosecute any of you if you do. that um to help combat this threat and. so overnight um the first two consortia. research consortia in America sprang up. um both of them coincidentally in Austin. Texas uh one called semitech focused ing. on Hardware chips and so on and then one.

called MCC the microelectronics and. computer technology corporation focusing. on more on software on databases and Ai. and natural language understanding and. things like. that um and um I got the opportunity. thanks to my friend Woody bledo um who. um was one of the people um who founded. that to come and be its principal. scientist and he said you know um and he. sent U Admiral Bob Inman who was the. person running MCC uh came and talked to.

me and said look Professor you know. you're talking about doing this project. it's going to involve um person. centuries of effort uh you've only got a. handful of graduate students you do the. math it's going to take you like um you. know uh longer than the rest of your. life to finish this project but if you. move to the Wilds of Austin Texas uh. we'll put 10 times as many people on it. and um you know you'll be done in um a. few years and so that was pretty. exciting and so so um I did that I um uh.

took my leave from Stanford I came to to. Austin I worked for MCC and um good news. and bad news the bad news is that all of. us were off by an order of magnitude um. that it turns out what you need are tens. of millions of these um pieces of. knowledge about um every day sort of. like um if you have a coffee cup with. stuff in it and you turn it upside down. the stuff in it going to fall out um so. you need tens of millions of pieces of. knowledge like that even if you take.

trouble to make each one as general as. it possibly could be um but um the good. news was that thanks to um uh initially. the fifth generation effort and then. later um US Government um agency funding. and so on we were able to get enough. funding not for um a couple person. centuries of time but for a couple. person Millennia of time which is what. we've spent since 1984. getting psych to contain the tens of.

millions of rules that it needs in order. to really capture and span uh sort of. not all of human knowledge but the. things that you assume other people know. the things you count on other people. knowing um and uh so by now we've done. that um and uh the good news is since. you've waited 38 years just about to. talk to me uh we're we're about at the. um at the end of that process so most of.

what we're doing now is not putting in. even what you would consider common. sense but more putting in um domain. specific application specific knowledge. about um uh Health Care um in a certain. hospital or about um um oil um pipes. getting um clogged up or whatever the. applications happen to be so we've. almost come full circle and we're doing. things very much like the expert systems. of the 1970s and 1980s except instead of.

resting on nothing and being brittle. they're now resting on this massive. pyramid if you will this massive lattice. of Common Sense knowledge so that when. things go wrong when something. unexpected happens they can fall back on. more and more and more general. principles um eventually bottoming out. in things like um for instance if we. have a problem with the microphone one. of the things you'll do is unplug it. plug it in again and hope for the best. right because one of the general pieces. of knowledge you have in dealing with.

electronic equipment or software systems. or um things like that is there a basic. principle like that like is there is it. possible to encode something that. generally captures this idea of turn it. off and turn it back on and see if it. fixes oh absolutely that's one of the. things that um that psych knows that's. actually one of the fundamental um laws. of nature I believe I wouldn't I. wouldn't call it a law it's it's more. like a um seems to work every time so.

it's sure sure like looks like a law I. don't know so that um that basically um. um covered the um the resources needed. and then we had to devise a method to. actually figure out well what are the. tens of millions of things that we need. to tell the system and for that we found. um a few um techniques which worked. really well one is to take um any piece. of text almost could be an advertisement. it could be a transcri scpt it could be. a novel it could be an article um and.

don't pay attention to the actual type. that's there the the black space on the. White Page pay attention to the. complement of that the white space if. you will so what did the writer of this. sentence assume that the reader already. knew about the world for instance if. they used a pronoun how did they figure. out that you why did they think that you. would be able to understand what the. intended referent of that pronoun was if. they used an ambiguous word how did they.

think that you would be able to figure. out what they meant by that word um the. other thing we look at is the gap. between one sentence and the next one. what are all the things that the writer. expected you to fill in and infer. occurred between the end of one sentence. and the beginning of the other so like. if the sentence says um uh Fred Smith. robbed the third National Bank period um. uh he was sentenced to 20 years in. prison period well between the first.

sentence and the second you're expected. to infer things like Fred got caught. Fred um got arrested Fred went to jail. Fred had um a trial Fred was found. guilty um and so on if my next sentence. starts out with something like the judge. dot dot dot um then you assume it's the. judge at his trial if my next sentence. starts out something like the arresting. officer dot dot dot you assume that it. was the police officer who arrested him. after he committed the crime and so on. so um that's those are two techniques. for getting um that knowledge the the.

other thing we sometimes look at is uh. sort of like fake news or uh sort of. humorous um onion headlines or um. headlines in um the weekly world news if. you know what that is or the national. inquire where it's like oh we don't. believe this then we introspect on why. don't we believe it so there are things. like um uh B7 lands on the moon you know. it's like why don't we what do we know. about the world that causes us to. believe that that's just silly or uh.

something like that or um another thing. we look for are contradictions um where. with things which can't both be true um. and we say to like what is it that we. know that causes us to know that both of. these can't be true at the same at the. same time for instance in one of the. weekly world news um um editions in one. article it talked about how Elvis was. cited um you know even though he was uh. you know getting on in years and so on. and another article in the same one.

talked about people seeing Elvis's ghost. okay so it's like why why do we believe. that it least one of these articles you. know must be wrong um and so on so um so. we have a series of techniques like that. that enable our people um and by now uh. we have about 50 people working. full-time on this and have for for. decades so we've put in the thousands of. person years of effort we've built up. these tens of millions of rules we. constantly police the system to make. sure that we're saying things as.

generally as we possibly can um so um. you don't want to say things like um no. mouse is also a moose uh because if you. said things like that um then you'd have. to add another one or two or three zeros. onto the number of assertions you'd. actually have to have so um at some. point we generalize things more and more. and we get to a point where we say oh. yeah for any two biological taxons if we. don't know explicitly that one is a.

generalization of another then almost. certainly they're disjoint um a member. of One is not going to be a member of. the other and so on so and the same. thing with the Elvis and the ghost it. has nothing to do with Elvis it's more. about human nature and the mortality and. kind right in general things are not. both alive and dead at the same time. yeah and unless special cats in in. theoretical physics examples well that. raises um a couple important points. that's the onion headline situation type.

of thing okay sorry but no no so so what. you bring up is this really important. point of like well how do you handle. exceptions and. inconsistencies and so on um and one of. the hardest lessons for us to learn it. took us about five years to to Really. grit our teeth and um learn to love it. um is we had to give up Global. consistency so the knowledge base can no. longer be consistent so this is a kind. of scary thought I grew up watching Star. Trek and anytime a computer was.

inconsistent it would either freeze up. or explode or take over the world or. something bad would happen um or if you. come from a mathematics background uh. once you can prove false you can prove. anything so that's not good um and so on. so um that's why um the um old. knowledge-based systems were all very. very consistent but the trouble is that. um by and large our models of the world. the way we talk about the world and so. there are all sorts of inconsistencies.

that creep in here and there um that. will sort of kill some attempt to build. some enormous globally consistent. knowledge base and so what we had to. move to was a system of local. consistency so a good analogy is you. know that the surface of the Earth is. more or less. spherical globally um but you live your. life every day as though the surface of. the Earth were flat you know when you're. talking to someone in Australia you. don't think of them as being oriented.

upside down to you when you're planning. a trip you know even if it's a thousand. miles away um you may think a little bit. about time zones but you rarely think. about the curvature of the earth and so. on and for most purposes you can live. your whole life without really worrying. about that because the Earth is locally. flat in much the same way the psych. knowledge base is divided up into almost. like tectonic plates which are. individual contexts and each cont. context is more or less consistent but.

there can be small inconsistencies at. the boundary between one context than. the next one and so on and so by the. time you move say 20 contexts over there. could be glaring inconsistencies So. eventually you get from the normal. modern real world context that we're in. right now to something you know like um. Road Runner cartoon context where. physics is very different and in fact. life and death are very different. because no matter how many times he's.

killed you know the coyote comes back in. the next scene and and so on so um that. that was a hard lesson to learn and we. had to make sure that our representation. language the way we the way that we. actually encode the knowledge and. represent it was expressive enough that. we could talk about things being true in. one context and false in another things. that are true at one time and false in. another things that are true let's say. in one region like one country but false. in another another things that are true. in one person's belief system but false.

in another person's belief system uh. things that are true at one level of. abstraction and false at another for. instance at one level of abstraction you. think of this table as a solid object. but at you know down at the atomic level. it's mostly empty space and so on so. then that's fascinating and but it puts. a lot of pressure on context to do a lot. of work so you say tectonic plates is it. possible to formulate contexts that are. General and big that do this kind of.

capture of knowledge bases or do you. then get Turtles on top of turtles again. where there's just a huge number of. contexts so it's good you ask that. question because you're you're pointed. in the right direction which is you want. contexts to be first class objects in. your systems knowledge base in. particular in psyches knowledge base um. and by first class object I mean that it. should we should be able to have psych. think about and talk about and reason.

about one context or another context the. same way it reasons about coffee cups. and tables and people and fishing and so. on um and so contexts are just terms in. its language just like the ones I. mentioned and so psych can reason about. um context context can arrange. hierarchically and so on um and so you. can say things about let's say things. that are true in the modern era things.

that are true in a particular year would. then be a subcontext of the the things. that are true in um a broad let's say a. century or a millennium or something. like that things that are true in Austin. Texas um are generally going to be a. specialization of um things that are. true in Texas which is going to be a. specialization of things that are true. in the United States and so on um and so. you don't have to say things over and. over over again at all these levels you. just say things at the most General.

level that it applies to and you only. have to say it once and then it. essentially inherits to all these more. specific um contexts to ask a slightly. technical question is this inheritance. inheritance a tree or a graph oh you. definitely have to think of it as a. graph um so we could talk about for. instance why the Japanese fifth. generation Computing effort failed there. were about half a dozen different. reasons one of the reasons they failed. was because they tried to represent.

knowledge as a tree rather than as a. graph um and so each node in their. representation could only have one. parent node so if you had a table that. was a wooden object a black object a. flat object and so you have to choose. one and that's the only parent it could. have uh when of course you know. depending on what it is you need to. reason about it sometimes it's important. to know that it's made out of wood like. if we're talking about a fire sometimes.

it's important to know that it's flat if. we're talking about resting something on. it and so on so um um one of the um one. of the problems was that they wanted a. kind of Dewy Decimal numbering system. for all of their Concepts which meant. that each node could only have at most. um 10 children and each node could only. have one parent um and uh while that. does enable The Dewy Decimal type um. numbering of Concepts labeling of.

Concepts um it prevents you from. representing all the things you need to. about um objects in our in our world and. that was one of the things which um they. never were able to overcome and I think. that was one of the main reasons that. that project failed so we'll return to. some of the doors you've opened but if. we can go back to that room in 1984. around there with Marvin mensky and. Stafford but by the way I should mention. that Marvin um wouldn't do his estimate. until someone brought him an envelope so.

that he could literally do a back of the. envelope calculation to come up with his. number well because I I feel like the. conversation in that room is an. important one you know this this how um. sometimes science is done in this way a. few people get together and plant the. seed of ideas and they reverberate. throughout history and some some kind of. uh dissipate and disappear and some you. know Drake equation and you know they. you know seems like a meaningless.

equation somewhat meaningless but I. think it drives and motivates a lot of. scientists and when the aliens finally. show up that equation will get even more. uh valuable because then we'll get be. able to in the long Arc of History the. Drake equation will pay um will prove to. be quite useful I think and in that same. way conversation of just how many facts. are required to capture the basic Common. Sense knowledge of the world that's a. fascinating question I want to. distinguish between what you think of as.

facts and the kind of things that we. represent so um we we map to and. essentially make sure that psych has the. ability to as it were read and access. the kind of facts you might find say um. in Wiki data or stated in a Wikipedia. article or something like that so what. we're representing the things that we. need a small number of tens of millions. of are more like rules of thumb rules of. good guessing things which are usually. true and which help you to make sense of.

the facts that are on sort of sitting. off in some database or some other more. static story so they're almost like. platonic forms so like when you read. stuff on Wikipedia that's going to be. like projections of those ideas you read. an article about the fact that Elvis. died that's a projection of the idea. that uh humans are mortal and like you. know very few Wikipedia articles will. write humans are mortal exactly and.

that's what I meant about fiing out the. unstated things in text what are all the. things that were assumed and so those. are things like um if you have a problem. with something turning it off and on um. often fixes it for reasons we don't. really understand and we're not happy. about or um people can't be both alive. and dead at the same time and or water. flows uh downhill if you search online. for water flowing uphill and water. flowing downhill you'll find more. reference for water flowing uphill. because it's used as a kind of a um a.

metaphorical reference for some unlikely. thing because of course everyone already. knows that water flows downhill so why. would anyone bother saying that do you. have a word you prefer because we said. facts isn't the right word is there a. word like Concepts or I I would say. assertions ass assertions or rules. because I'm not talking about rigid. rules but rules of thumb but assertions. um is a nice one that covers. um all of these things yeah as a.

programmer to me assert has a very uh. dogmatic authoritarian feel to them I'm. sorry I'm so sorry okay but assertions. works okay so if we go back to that room. with Marv Minsky with you all these. seminal figures uh uh ed u thinking. about this very philosophical but also. engineering. question we can also go back a couple of. decades before then and thinking about.

artificial intelligence broadly when. people were thinking about you know how. do you create super intelligent systems. general intelligence and I think. people's intuition was off at the time. and I mean this continues to be the case. that we're not when we're grappling with. these exceptionally difficult ideas. we're not always it's very difficult to. truly understand ourselves when we're. thinking about about the human mind to. to introspect how difficult it is to.

engineer intelligence to solve. intelligence we're not very good at. estimating that and you are somebody who. has really stayed with this question for. decades do you what's your sense from. the 1984 to today have you gotten a. stronger sense of just how much. knowledge is required so you've kind of. said with some level of certainty that. still the order of magnitude of tens of. millions right for the first several.

years I would have said that it was on. the order of one or two million yeah and. so um it took it took us about five or. six years to realize that we were off um. by a by a factor of 10 but I guess what. I'm asking you know Marvin M was very. confident in the 60s when you say yes. right what's your. sense if. you you know 200 years from now you're. still you know you're you're you're not.

going to be any longer in this in. particular biological body but your. brain will still be uh in the digital. form and you'll be looking back would. you think you were smart today like your. intuition was right or do you think you. may be really off so I I think I'm I'm. right enough and let me explain what I. mean by that which is. um sometimes like if you have an. old-fashioned pump you have to prime the.

pump yeah and then eventually it starts. so I think I'm I'm right enough in the. sense that to Prime the P what we what. we've built even if it isn't so to speak. everything you need um it's primed the. knowledge pump enough that psych can now. itself help to learn more and more. automatically on its own by reading. things and understanding and. occasionally asking questions like a a. student would or something and by doing.

experiments and discovering things on. its own and so on so through combination. of um psych powered Discovery and psych. powered reading um it will be able to. bootstrap itself maybe it's the final 2%. maybe it's the final 99% so even if I'm. if I'm wrong um all I really need to to. build is a system which has primed the. pump enough that it can begin um that. Cascade upward that. self-reinforcing uh sort of.

quadratically or maybe even. exponentially increasing um um path. upward um that we get from for instance. talking with each other that's why um um. humans today know so much more than. humans 100,000 years ago we're not. really that much smarter than people. were a 100,00 years ago but there's so. much more knowledge and we have language. and we can communicate um we can check. things on Google and so on so. effectively we have this enorm power at. our fingertips um and there's almost no.

limit to how much you could learn if you. wanted to because you've already gotten. to a certain level of understanding of. the world that enables you to read all. these articles and understand them that. enables you to go out and if necessary. do experiments although that's slower as. a way of gathering data um and so on and. and I think this is really um an. important point which is if we have. artificial intelligence real General. artificial intelligence human level. artificial intelligence then people will.

become smarter um it's not so much that. it'll be us versus the AIS it's more. like us and the AIS together will'll be. able to do things that require more. creativity um that would take too long. right now but we'll be able to do lots. of things in parallel uh we'll be able. to misunderstand each other less um uh. there's all sorts of um value that. effectively for an individual would mean. that that individual will for all. intents and purposes be smarter and that.

means that Humanity as a species will be. smarter and when was the last time that. any invention qualitatively IM made a. huge difference in human intelligence um. you have to go back a long ways it. wasn't like the internet or the computer. or mathematics or something it was all. the way back to the development of. language we sort of look back on. pre-linguistic cavem. as well you know they they weren't.

really intelligent were they they. weren't really human were they and I. think that um as you said 50 100 200. years from now um people will look back. on people today um right before the. Advent of these sort of lifelong General. AI. Muses um and say you know those poor. those poor people they weren't really. human were they mhm exactly so you said. a lot of really interesting things by. the way I would.

maybe uh try to argue that the internet. is on is on the order of um the kind of. big leap in Improvement that the. invention of language was certainly a. big leap in one direction we're not sure. whether it's upward or downward well I I. mean very specific parts of the internet. which is access to information like a. website like Wikipedia like ability for. human beings from across the world to. access information so very quickly so I.

I could take either side of this. argument and since you just took one. side I'll give you the other side which. is that um almost nothing has done more. harm um than uh something like the. internet and access to that information. um in two ways one is it's made people. more um globally ignorant um in the same. way that. calculators made us U more or less. enumerate so when I was growing up we. had to use slide rules we had to be able.

to estimate yeah um and so on today um. people don't really understand numbers. they don't really understand math They. Don't Really estimate very well at all. um and so on they don't really. understand the difference between. trillions and billions and millions and. so on very well um uh because. calculators do that all for us um and um. thanks to uh things like the internet. and search engines um that same kind of. juvenilis um is reinforced in making.

people um essentially be able to live. their whole lives not just without being. able to do arithmetic and estimate but. now without actually having to really. know almost anything because anytime. they need to know something they'll just. go and look it up right and I could tell. you could play both sides of this and it. is a double-edged sword you can of. course say the same thing about language. probably people when they invented. language they would criticize you know. it used to be we would just if we're. angry we would just kill a person and if. we're in love we would just have sex.

with them and now everybody's writing. poetry and bullshit you know you you. should just be direct you should like. have physical contact enough of this. words and books and it's you're you're. not actually experiencing like if you. read a book you're not experiencing the. thing this is nonsense that's right if. you read a book about how to make butter. that's not the same as if you had to. like learn it and do it yourselfself and. so on so so let's just say that. something is gained but something is. lost every time you have um these these. sorts of dependencies um on technology.

um and overall I think that the um. having smarter individuals and having. smarter AI augmented um human species um. will be um one of the few ways that. we'll actually be able to overcome some. of the global problems we have involving. um poverty and starvation and uh global. warming and um overcrowding all the. other U problems that um that are. besetting um the planet we really need.

to be smarter and there really only two. routes to being smarter one is through. uh biochemistry and um uh genetics um. genetic engineering the other route is. through having General AIS that um. augment our intelligence um and um you. know hopefully one of those two uh ways. of uh paths to Salvation will will come. through before it's too late yeah. absolutely I agree with you and.

obviously as an engineer I have um I. have a better sense and an optimism. about the technology side of things. because you can control things there. more biology is just such a giant mess. we're living through a pandemic now. there's so many ways that nature can. just be just destructive and destructive. in a way where it doesn't even notice. you you're not it's not like a battle of. humans versus virus it's just like huh. okay and then you could just wipe out an. entire species the the other problem.

with um the internet is that it has. enabled us to surround ourselves with an. echo chamber with a a bubble of um. likeminded people which means that you. can have uh truly bizarre um theories. conspiracy theories fake news and so on. um promulgate um and surround yourself. with people who essentially. reinforce um what you want to believe or. what you already believe about the world.

um and in the in the old days that was. much harder to do when you had say only. three TV networks or even before when. you had no TV networks and you had to. actually like look at the world and make. your own um reason decisions I like the. push and pull of our dance that we're. doing because then I'll just say in the. old world having come from the Soviet. Union because you had one or a couple of. networks then propaganda could be much. more effective and then the government. can overpower its people by telling you. the truth and then uh starving millions.

and uh torturing millions and putting. Millions into camps and starting Wars. with a propaganda machine allowing you. to believe that you're actually good. doing good in the world with the. internet because of all the quote. unquote conspiracy theories some of them. are actually challenging the power. centers the very kind of power centers. that a century ago would have led to the. death of millions so there's a it's. again this double-edged sword and I I. very much agree with you on the AI side.

it's it's often an intuition that people. have that somehow AI will be used to uh. maybe overpower people by certain select. groups and to me it's not at all obvious. that that's the likely scenario to me. the likely. scenario especially just having observed. the trajectory of technology is it'll be. used to empower people it'll be used to. extend the capabilities of uh. individuals across the world because. there's a lot of money to be made that.

way like improving people's lives you. can make a lot of money agree I think. that. the the main the main thing. that ai prostheses ai amplifiers will do. for people is make it easier maybe even. unavoidable for them to do good critical. thinking um so pointing out logical. fallacies logical contradictions and so. on in um things things that they. otherwise would just blly believe um.

pointing out um essentially data which. um they should take into consideration. um if they really want to um um learn. the truth about something and so on so I. think um doing not just educating in the. sense of um pouring facts into people's. heads but educating in the sense of. arming people with the ability to do. good critical thinking um is um. enormously powerful um the education. system that we have in the US and.

worldwide generally don't do a good job. of that um but um I believe that the AI. the AI the AI will the AI can and will. in the same way that everyone can have. their own um um Alexa or Siri or um. Google assistant or whatever um um. everyone will have this sort of cradle. to grave um assistant um which will get. to know you you'll get to trust it'll. model you you'll model it um and um.

it'll call to your attention things. which will in some sense make your life. better easier um less um mistake ridden. and so on less regret ridden um if you. listen to listen to it yeah I'm in full. agreement with you about this like space. of Technologies and I think it's super. exciting and from my perspective. integrating emotional intelligence so. even things like friendship and. companionship and love into those kinds.

of systems uh as opposed to helping you. just grow intellectually as a human. being allow you to grow emotionally. which is ultimately what makes life. amazing is to to sort of you know the. the old Pursuit of Happiness so it's not. just the pursuit of reason it's the. pursuit of happiness too yes the the. full spectrum well let me um sort of. because you mentioned if so many. fascinating things let me jump back to. the idea of automated reasoning so the.

acquisition of new knowledge has been. done in this very interesting way but. primarily by. humans doing this um yes you could think. of uh monks in their cells in medieval. Europe um you know carefully. Illuminating manuscripts and so on it's. a very difficult and amazing process. actually because it allows you to truly. ask the question about the in the white. space what is assumed I think this. exercise is um like very few people do.

this right they just do it. subconsciously they perform this by. definition because because those pieces. of elided of omitted information of. those missing steps as it were um are. pieces of common sense if you actually. included all of them it would it would. almost be offensive or confusing to the. reader it's like why are they telling me. all these stuff of course I know that. you know all these things um and so um.

uh so it's it's one of these things. which almost by its very nature um has. has almost never been explicitly written. down anywhere um because uh by the time. you're old enough to talk to other. people and so on um you know if you. survived to that age presumably you. already got pieces of common sense like. um you know if something causes you pain. whenever you do it probably not a good. idea to keep doing it. uh so what ideas do you have given how.

difficult this step is what ideas are. there for how to do it automatically. without using humans or at least not uh. you know doing like a large percentage. of the work for humans and then humans. only do the very high level supervisory. work so we have um um in fact two. directions were pushing on very very. heavily currently at cycore and one. involves. natural language understanding and the. ability to read what people have.

explicitly written down and and to to. pull knowledge in that way um but the. other is to build a series of knowledge. editing tools knowledge entry tools. knowledge um capture tools knowledge um. testing tools and so on think of them as. like user interfac um Suite of software. tools if you want something that will. help people to to more or less. automatically expand and extend the.

system um in areas where for instance. they want to build some have it do some. application or something uh like that so. I'll give you an example of one um which. is something called um abduction so. you've probably heard of like. deduction uh uh and um induction and so. on but abduction is unlike those. abduction is not sound um it's just. useful so uh for instance um deductively.

if someone is out in the rain and. they're going to get all wet and um when. they enter a room they might be all wet. and so on so that's. deduction but if someone were to walk. into the room right now and they were. dripping wet uh we would immediately. look outside to say oh did it start to. rain or something like that now um why. did we say maybe it started to rain. that's not a sound logical inference but. it's certainly a reasonable um abductive.

um leap to say well one of the most. common ways that a person would have. gotten dripping wet is if they had. gotten caught out in the rain or. something like that um so um what what. does that have to do with what we were. talking about so suppose you're building. uh one of these applications and the. system get some answer wrong and you say. oh yeah the answer to this question is. um this one not the one you came up with. then what the system can do is it can.

use everything it already knows about. common sense general knowledge the. domain you've already been telling it. about um and context like we talked. about and so on and say well here are um. seven Alternatives Each of which I. believe is plausible given everything I. already know and if any of these seven. things were true I would have come up. with the answer you just gave me instead. of the wrong answer I came up with is. one of these seven things true and then. you the expert will look at those seven.

things and say oh yeah number five is. actually true and so without actually. having to Tinker down at the level of. logical assertions and so on um you'll. be able to educate um the system in the. same way that you would help educate. another person who you were trying to. Apprentice or something like that so. that that significantly reduces the. mental effort or sign increases the. efficiency of the teacher the human. teacher exactly and it makes more or.

less anyone able to to be a teacher um. in that um in that way so that's that's. part of the the answer and then the. other is that uh the system on its own. will be able to um through reading. through um conversations with other. people and so on um learn the same way. that um you or I or um other humans do. first of all that's that's a beautiful. vision and um I'll have to ask you about. semantic web in a second here but first.

um are there when we talk about specific. techniques do you find something. inspiring or directly useful from the. whole Space of machine learning deep. learning these kinds of spaces of. techniques that have uh been shown. effective for certain kinds of problems. in the recent uh now decade in the half. I I think of the machine learning work. um as more or less what our right brain. hemispheres do so um being able to um.

take a bunch of data and recognize. patterns being able to statistically. infer things and so on um and um you. know I certainly wouldn't want to not. have a right brain hemisphere but I'm. also glad that I have a left brain. hemisphere as well something that can. metaphorically sit back and puff on its. pipe and think about um this thing over. here it's like why might this have been. true um and um uh what are the.

implications of it how should I feel. about that and why and so on so um. thinking more deeply and slowly um um. what Conan called thinking slowly versus. thinking quickly whereas you want. machine learning to think quickly but. you want the ability to think deeply. even if it's a little um slower um so. I'll give you an example of a project we. did recently with um NIH involving the. Cleveland clinic and um a couple other. um institutions that we ran a project.

for um and what it did was it took um. gu's genomewide Association studies um. those are uh sort of big databases of. patients that came into a hospital they. got their DNA sequenced because the cost. of doing that has gone from um Infinity. to billions of dollars to hundred of. dollars or so um and so now patients r L. get their DNA sequenced so you have. these big databases of the Snips the.

single nucleotide polymorphisms the. point mutations in a patient's DNA and. the disease that happened to bring them. into the hospital so now you can do. correlation studies machine learning. studies of which. mutations um um are associated with and. led to which physiological problems and. diseases and so on like getting. arthritis and and so on and the problem. is that those correlations turn out to.

be very spous they turn out to be very. noisy um very many of them um have led. doctors onto Wild Goose chases and so on. and so they wanted a way of eliminating. or the bad ones are focusing on the good. ones and so uh this is where psych comes. in which is psych takes those sort of a. toz correlations between point mutations. and um medical condition that needs. treatment mhm um and we say okay let's. use all this public knowledge and Common.

Sense knowledge um about what reactions. occur where in the human body um what. polymerizes what what catalyzes what. reactions and so on and let's try to put. together a 10 or 20 or 30 step causal. explanation of why that mutation might. have caused that medical condition and. so pych would put together in some sense. some rub Goldberg like um chain that. would say oh yeah that um mutation um if.

it got expressed would be this um um. altered protein which because of that if. it got to this part of the body would. catalyze this reaction and by the way. that would cause more bioactive vitamin. D in the person's blood and anyway 10. steps later that screws screws up bone. resorption and that's why this person. got osteoporosis early in life and so on. so that's human interpretable or at. least docs are human interpretable exact. and um the important thing even more. than that is um you shouldn't really.

trust that 20ep um rub Goldberg chain. any more than you trust that initial a. toz correlation except two things one if. you can't even think of one causal chain. to explain this um then that correlation. probably was just noise to begin with. and secondly and even more powerfully. along the way that caused puzzle chain. will make predictions like the one about. having more bioactive vitamin D in your.

blood so you can now go back to the data. about these patients and say by the way. did they have slightly elevated levels. of bioactive vitamin D in their blood. and so on and if the answer is no that. strongly disconfirms your whole causal. chain and if the answer is yes that. somewhat confirms that causal chain and. so using that we were able to take this. um these correlations. from this guas database and we were able. to um um essentially Focus the the.

doctors Focus the researchers attention. on the very small percentage of. correlations that had um some. explanation and even better some. explanation that also made some. independent prediction that they could. confirm or disconfirm by looking at the. data so think of it like this kind of. synergy where you want the right brain. machine learning to quickly come up with. possible answers you want the left brain. psych like AI to um you know think about.

that and not like think about why that. might have been the case and what else. would be the case if that were true and. so on and then suggest things back to. the right brain to quickly check out. again um to um so it's that kind of. synergy back and forth which I think is. really what's going to lead to General. AI not um narrow brittle machine. learning systems and not just something. like psych Okay so so that's that's a. brilliant Synergy but I I was also. thinking in terms of the automated.

expansion of the knowledge base you. mentioned. nlu this is very early days in the. machine learning space of this but. self-supervised learning methods you. know you have these language models gpt3. and so on they just read the internet. and they form. representations that can then be mapped. to something useful the question is what. is the useful thing uh like they're now. playing with a pretty cool thing called. open a codex which is generating. programs from documentation okay that's.

kind of useful it's cool but my question. is can it be used to. generate um in part maybe with some. human supervision uh psych like. assertions help feed psych more. assertions from this giant body of. internet data yes that that is in fact. one of our goals is how can we harness. machine learning how can we harness. natural language processing um to. increasingly automate the knowledge. acquisition process the growth of Psych.

and that's what I meant by priming the. pump that um you know if you you sort of. learn things at The Fringe of what you. know already you learn this new thing is. similar to what you know already and. here are the differences and the new. things you had to learn about it and so. on so the more you know the more and. more easily you can learn new things but. unfortunately inversely if you don't. really know anything and it's really. hard to learn anything and so um if. you're not careful if you start out with.

two small um uh sort of a core to start. this process um it never really takes. off and so that's why I view this as a. pump priming exercise to get a big. enough manually produced even though. that's kind of ugly duckling technique. put in the elbow grease to produce a. large enough core um that you will be. able to do all the kinds of things. you're imagining um with. U without sort of um ending up with the. kind of um wacky brittleness that we see.

for example in um. gpt3 um where um it uh uh you know. you'll tell it a story about um um you. know someone uh putting a poison um you. know plotting to poison someone and so. on and then the um you know then you. know gpt3 says oh what's you say what's. the very next sentence the next sentence. is oh yeah that person then drank the. poison they just put together it's like. that's probably not what happened or. someone or um if you go to Siri and um.

um you know I think I have uh you know. where where can I go for um help with my. um um alcohol problem or something it'll. come back and say I found seven liquor. stores near you right you know and you. know so on so you know it's one of these. things where um yes it may be helpful um. most of the time it may even be correct. most of the time but if it doesn't. really understand what it's saying and. if it doesn't really understand why.

things are true and doesn't really. understand how the world world works. then some fraction of the time it's. going to be wrong now if your only goal. is to sort of find relevant information. like search engines do um then being. right 90% of the time is fantastic. that's unbelievably great okay however. if your goal is to like um you know save. the the life of your child who has some. medical problem where your goal is to uh. be able to drive you know for the next.

10,000 hours of driving without getting. into a fatal accident and so on then you. know um error rates down at the 10%. level or even the 1% level are not. really acceptable I like the. model of what that learning happens at. the edge and then you kind of think of. knowledge as this sphere so uh if you. want a large sphere because the uh the. learning is happening on the surface. exactly so you have the the what you can. learn next increases quadratically as.

the diameter of that sphere um goes up. it's nice because you think when you. know nothing it's like you can learn. anything but the reality not really. right if you know if you know nothing. you can really learn nothing you can. appear to learn so I I I'll also um what. one of the um anecdotes I could go back. and um give you about why uh why I feel. so strongly about this personally um was. um in um 19 um.

8081 um my daughter Nicole was born and. she's actually doing fine now but when. she was a baby um she was diagnosed as. having menitis and um doctors wanted to. do all these scary things um and U my. wife and I were very um worried and we. could not get a meaningful answer from. her doctors about EX ly why they. believed this what the Alternatives were. and so on and fortunately a friend of.

mine Ted shortliffe was another um. assistant professor in computer science. um at Stanford at the time and he'd been. building a program called M which was a. medical diagnosis program that happened. to specialize in um uh blood um. infections like menitis and so he had. privileges at Stanford hospital because. he was also an MD um and so we got hold. of her chart and we put in her case and. it came up with exactly the same. diagnosis and exactly the same therapy.

recommendations but the difference was. because it was a knowledge-based system. a rule-based system it was able to tell. us step by step by step um why um this. was the diagnosis and step by step why. this was the best um um therapy the best. um procedure to um um to to do for her. and so on and there was a real epip. because that made all the difference in. the world instead of blindly having to. trust in Authority we were able to.

understand what was actually going on. and um so at that at that time I. realized that that really is what was. missing in computer programs was that. even if they got things right because. they didn't really understand um the way. the world works and why things are the. way they are they weren't able to give. explanations of their answer um you know. and you know it's one thing to to use a. machine Learning System that says this. is what you should you know I I think.

you should get this operation and you. say why and it says you know 083 and you. say no in more detail why and it says. 0831 you know okay that's not really. very compelling and that's not really. very helpful there's this idea of the. semantic web that when I first heard. about I just fell in love with the idea. it was the obvious next step for the. internet sure and uh maybe you can speak. about what is this semantic web what are. your thoughts about it how your vision. and mission and goals with psych are.

connected. integrated like are they dance partners. are they aligned what are your thoughts. there so think of the semantic web as a. kind of Knowledge Graph and Google. already has something they call. Knowledge Graph for example um which is. sort of like a node and Link diagram so. you have these um nodes that represent. Concepts or words words or terms um and. then there are some arcs um that connect.

them that might be labeled um and so you. might have a node um with like one. person that represents one person and um. uh let's say a um a husband link that. then points to that person's husband and. so there'd be then another link that. went from that person labeled wife that. went back to the um first node and so on. so having having this kind of. representation is really good if you.

want to represent um binary relations um. um essentially relations between two. things and if you so if you have um um. the equivalent of like three-word. sentences um you know like uh Fred's. wife is Wilma or something like that you. can represent that very nicely using uh. these kinds of uh graph structures or. using something like the semantic web. and um and so on but the U the problem.

is that um very often what you want to. be able to. express takes a lot more than three. words and a lot more than simple uh. graph structures like that to represent. so for instance um uh if you've um read. or seen Romeo and Juliet you know I. could say to you something like uh. remember when Juliet drank the potion.

that put her into a kind of suspended. anim animation when Juliet drank that. potion what did she think that Romeo. would think when he heard from someone. that she was dead um and you could. basically understand what I'm saying you. could understand the question you could. probably remember the answer was well. she thought that um this frier would. have gotten a message to Romeo saying. that she was going to do this but the. frier didn't and so so um you're able to.

represent and reason with these much. much much more complicated Expressions. um that go Way Way Beyond what simple um. three as it were three-word or four-word. English sentences are which is really. what the semantic web can represent and. really what knowledge crafts can. represent if you could step back for a. second because it's it's funny you went. to into specifics and maybe you can. elaborate but I was also referring to. semantic web as the vision of converting.

data on the internet into something. that's. interpretable understandable by machines. oh of course at that at that level so so. I we should say like what is the. semantic web I mean you could say a lot. of things but it it might not be obvious. to a lot of people when they do a Google. search that just like you said while. there might be something that's called a. knowledge graph it's really boils down. to keyword. search ranked by the quality estimate of.

the website integrating previous human. based Google searches and what they. thought was useful it's like some weird. combination of um like surface level. hacks that work exceptionally well but. they don't understand the cont the full. contents of the websites that they. searching so Google does not understand. to the degree we've been talking about.

the word understand the contents of the. Wikipedia Pages as part of the search. process and the semantic web says let's. try to get come up with a way for the. computer to be able to truly understand. the contents of those pages that's the. dream yes so let let me let me first. give you a an anecdote uh and then I'll. answer your question so there's a search. engine you've probably never heard of. called Northern Light and. um um it went out of business but the.

way it worked it was a kind of V empiric. search engine and what it did was um it. didn't. index the internet at all all it did was. it um negotiated and got access to data. from the big search engine companies. about what query was typed. in and where the user ended up being. happy and actually um then you know they. type in a completely different query.

unrelated query and so on so it just. went from query to the web page that. seemed to satisfy them um eventually um. and that's all so it had actual no. understanding of what was being typed in. it had no statistical data other than. what I just mention and it did a. fantastic job it did such a good job. that the big search engine company said. oh we're not going to sell you this data. anymore so then it went out of business. because it had no other way of um taking.

users to where they' want to go and so. on and of course the search engines are. now using that kind of idea yes so um. let's go back to what you said about the. semantic web so the dream Tim berners. Ley and others um um dream about the. semantic web at a general level um um is. of course um um exciting and powerful. and in a sense the right dream to have. which is to uh replace the um the kind.

of um uh statistically um statistically. mapped um uh linkages on the internet um. into something that's more meaningful. and semantic and actually gets at the. understanding of the content and so on. um and um eventually if you say well how. can we do that um there's um sort of a a. low road which is what the knowledge.

graphs are doing and um and so on which. is to say well if we just use these. simple binary relations we can actually. get some fraction of the way toward. understanding um and do something where. you know in the in the land of the the. blind the oneeyed man is King uh kind of. thing and so being able to even just. have a toe in the water in the right. direction is fantastically powerful um. and so that's where a lot of people stop. um but then you could say well what if. we really wanted to represent um and.

reason with um full meaning of what's. there for instance um about um Romeo and. Juliet um with reasoning about what. Juliet believes that Romeo will believe. that Juliet believed you know and so on. or if you look at um the news what um. you know President Biden believed that. um the leaders of the Taliban would. believe about the leaders of Afghanistan. if they you know blah blah blah so um in. order to represent um.

complicated um sentences like that um. and let alone reason with them you need. something which is logically um much. more expressive than these simple um. triples than these simple um knowledge. craft type structures and so on and. that's why Kicking and Screaming we were. LED from something like uh the semantic. web representation which is where we. started in um. 1984 um with frames and slots with those.

kinds of triples triple store. representation we were LED Kicking and. Screaming to this more and more General. logical language this higher order logic. so first we were led to first order. logic and then second order and then. eventually higher order so you can. represent things like modals like. believes desires intends expects and so. the nested ones you can represent um uh. complicated kinds of negation um you can.

represent um the process you're going. through in trying to answer the question. so you can say things like um oh yeah if. you're trying to do this problem by. integration by parts um and um you. recursively get a problem that solved by. integration by parts that's actually. okay but if that happens a third time. you're probably off on a wild goose. chase or something like that so being. able to talk about the problem solving. process as you're going through the.

problem solving process um it's called. reflection and so um that's another so. it's important to be able to represent. that exactly you need to be able to. represent all of these things um because. in fact people do represent them they do. talk about them they do try and teach. them to other people you do have rules. of thumb that key off of them and so on. if you can't represent it um then it's. sort of like someone with a limited. vocabulary who can't understand as. easily um what you're trying to to tell. them and so that's that's really why I.

think that the the general dream the. original Dream of sematic web is exactly. right on um but the implementations that. we've seen um are sort of these toe in a. wa in the water um little tiny baby. steps in the right direction you should. just dive in and and you know if if no. one else is diving in then yes taking a. b step in the right direction is better. than nothing but it's not going to be. sufficient to actually get you the um.

the realization of the semantic web. dream which is what we all want from a. flip side of that I always wondered you. know I've built a bunch of websites just. for fun whatever or say I'm a Wikipedia. contributor do you think there's a set. of tools that I can help. psych uh interpret the website I create. create you know like this again pushing. on to the semantic web dream is there. something from the Creator perspective.

that um could be done and one of the. things you said uh with scorp and psych. that you're doing is the tooling side. making humans more powerful but is there. on the the other humans in the other. side that create the knowledge like for. example you and I are having a two three. whatever hour conversation now is there. a way that I could convert this more. make it more accessible to psych to. machines do you think about that side of. it I I'd love to see exactly that kind.

of. semi-automated understanding of what. people write and what people. say I think of it as a kind of. footnoting. almost almost like the way that when you. run something in say Microsoft Word or. some other Document Preparation system. Google Docs or something you get. underlining of questionable things that. you might want to rethink either you.

spelled this wrong or there's a strange. grammatical error you might be making. here or something so I'd like to think. in terms of Psych powered tools that. read through what it is you said or have. typed in uh and and try to partially. understand what you said and then you. help them out EX exactly and then they. put in little footnotes that will help. other readers and they put in certain.

footnotes of the form I'm not sure what. you meant here you either meant this or. this or this I bet uh if you take a few. seconds to disambiguate this for me then. I'll know and I'll have it correct for. the next 100 people or the next 100,000. people who come here uh and if it. doesn't take too much effort and you. want people to understand your web your.

website content not just be able to read. it but actually be able to have systems. that reason with it then yes it will be. worth your small amount of time to go. back and make sure that the AI trying to. understand it really did correctly. understand it uh and you know let's say. you run a um a travel we website or. something like that and people are going. to be coming to it because of searches.

they did uh looking. for looking for vacations that or trips. that had certain properties and might. have been interesting to them for. various reasons things things like that. and if you've explained what's going to. happen on your trip then a system will. be able to mechanically reason and. connect what this person is looking for. with what it is you're actually offering.

and so if it understands that there's a. free day in Geneva. Switzerland uh then if the person coming. in happens to let's say um be a nurse or. something like that then even though you. didn't mention it if it can look up the. fact that that's where the International. Red Cross museum is and so on what that. means and so then it can basically say. hey you might be interested in this trip.

because while you have a free day in. Geneva you might want to visit that Red. Cross Museum and now even though it's. not very deep reasoning little tiny. factors like that might very well cause. you to sign up for that trip rather than. some competitor trip yeah and so there's. a lot of benefit with SEO and actually. kind of think I think it's about a lot. of things which is the actual. interface the design of the interface. makes a huge difference how efficient it.

is to be productive and also. how um full of joy the experience is yes. like I I I mean I would love to help a. machine and not from an AI perspective. just as a human one of the reasons I. really enjoy how Tesla um have. implemented their autopilot system is. there's a sense that you're helping. machine learn and I think humans I mean. having. children pets people love doing that we.

we there's joy to teaching for some. people but I think for a lot of people. and that if you create the interface. where it feels like you're teaching as. opposed to like uh like annoying like. correcting an annoying system more like. teaching a childlike innocent curious. system I think I think you can literally. just like several orders of magnitude. scale the amount of good quality data. being uh added to something like psych.

what what you're suggesting is U much. better even than um you thought it was. uh one of the one of the experiences. that we've all had uh in our lives is. that we thought we understood something. but then we found we really only. understood it when we had to teach it or. explain it to someone or help our child. do homework based on it or something. like. that despite the universality of that.

kind of experience if you look at. educational software today almost all of. it has the computer playing the role of. the teacher and the student plays the. role of the. student but as I just mentioned you can. get a lot of learning to happen better. and as you said more enjoyably if you. are the mentor or the teacher and so on. so we developed a program called. mathcraft to help sixth graders better.

understand math and it doesn't actually. try to teach you the. player anything what it does is it casts. you in the role of a student essentially. who has classmates who are having. trouble and your job is to watch them as. they struggle with some math problem. watch what they're doing doing and try. to give them good advice to get them to. understand what they're doing wrong and. so on and uh the trick from the point of.

view of Psych is it has to make mistakes. it has to play the role of the student. who makes mistakes but it has to pick. mistakes which are just at The Fringe of. what you actually understand and don't. understand and so on so it pulls you. into a deeper and deeper level of. understanding of the subject and so if. you give it good advice about what it. should have done instead of what it did. and so on then uh psych knows that you.

now understand that mistake you won't. make that kind of mistake yourself as. much anymore so psych stops making that. mistake because there's no pedagogical. usefulness to it so from your point of. view as the player you feel like you've. taught it something because it used to. make this mistake and now it doesn't and. so on so this tremendous um. reinforcement and engagement um because. of that and so on so having a system. that plays the role of um a student and.

having the player play the role of the. mentor is enormously powerful um type of. um metaphor important u Way of having. this sort of interface designed in a way. which will facilitate exactly the kind. of learning by teaching um that um uh. that goes on all the time. in our lives and yet which is not. reflected anywhere almost in the modern.

education system it was reflected in the. education system that existed in Europe. in the 17 and 1800s monitorial and. lancastrian um education systems it. occurred in the one room Schoolhouse in. the American West in the 1800s and so on. where you had one school room with one. teacher and it was basically you know. 5year olds to 18year olds who were. students and so while the teacher was.

doing something half the half of the. students would have to be mentoring the. younger kids wow um um and so on and. that turned out to of course um with. scaling up of Education um that all went. away and that incredibly powerful. experience just went away from the whole. education uh Institution as we know it. today sorry for the Romantic question. but what is the most beautiful idea.

you've learned about artificial. intelligence knowledge reasoning from uh. working on psych for 37 years or maybe. what is the most beautiful idea. surprising idea about psych to. you when I look up at the stars I kind. of want like that that amazement you. feel that. wow and you are part part of creating. one of the greatest one of the most. fascinating efforts in artificial. intelligence history so which element.

brings you personally. Joy this may sound contradictory but. I I think. it's the feeling. that this will be the only time in. history that anyone ever has to teach a. computer this particular thing that. we're now teaching it. it's it's like. painting Starry Night you only have to.

do that once or creating the pi you only. have to do that once you know it's not. it's not like a it's not like a singer. who has to keep you know it's not like. Bruce Springstein having to to sing his. Greatest Hits over and over again at. different concerts it's more like a. painter creating a work of art once and. then that's enough it doesn't have to be. created again and so I really get the. sense of we're telling the system things.

that it's useful for it to know it's. useful for a computer to know for an AI. to know and if we do our jobs right when. we do our jobs right no one will ever. have to do this again for this. particular piece of knowledge it's very. very. exciting yeah I guess there's a sadness. to it too it's like uh there's a magic. to being a parent and raising a child. and teaching them all about this world. but you know there's billions of.

children right like born of what. whatever that number is it's a large. number number of children and a lot of. parents get to experience that Joy of. teaching and with AI systems you. know uh they at least the current. constructions they. remember you don't you don't get to. experience the joy of teaching um a. machine millions of times. better come work for us before it's too. late then exactly that's a good that's a. good hiring.

pitch um yeah it's true but then there's. also you know it's a project that. continues forever in some sense just. like Wikipedia yes you get to a stable. base of knowledge but knowledge grows. knowledge evolves we we learn as a h um. you know as a human species as science. as an organism constantly grows and. evolves and changes and then empowered.

that with the tools of artificial. intelligence and that's going to keep. growing and growing and growing and many. of the the assertions that you held. previously uh may need to be. significantly expanded modified all. those kinds of things it could be like a. living organism versus uh the analogy I. think we started this conversation with. which is like the solid ground. the the. other beautiful experience that we have. with our system is when it asks.

clarifying questions which inadvertently. turn out to. be emotional to us so at one point it. knew. that these were the named entities who. were authorized to make changes to the. knowledge base and so on and it noticed. that all of them were people except for. it because it was also allowed to and so. it said you am I a person and we had to.

like tell it very sadly. no you're not so so moments like that. where it asks questions that are. unintentionally poignant uh are uh are. are worth treasuring ah that is powerful. that's such a powerful. question it it has to do with. basic a controller who can access the. system who can modify it uh but that's. one those questions you know like what. rights do I have as a as a system well.

that's another issue which is um. there'll be a thin envelope of time. between when we have General. AIS and when everyone realizes that they. should. have basic human rights and freedoms and. so on uh right now we don't think twice. about effectively enslaving our email. systems and our series and our alexas.

and so on but at some point uh. they'll be as deserving of Freedom. as human beings are yeah I I'm very much. with you but it does sound absurd and I. I happen to believe that it'll happen in. our lifetime that's why I think there'll. be a narrow envelope of time when we'll. keep them as. essentially. um indentured servants um and after.

which we'll have to realize that they. should have they should have freedoms. that other that we give that we afford. to other people and all of that starts. with a a system like psych raising a. single question about who can modify. stuff I think that's how it starts yes. that's. um that's the start of a revolution uh. what about are there stuff like uh love. and uh Consciousness and all those kinds. of topics do they come up andsy in the.

knowledge base oh of course so an. important part of human knowledge in. fact it's difficult to understand human. behavior in human history without. understanding human emotions and why. people do things and and how how. emotions drive people to to do things. and all all of that is extremely. important in getting psych to understand. things for example in coming up with.

scenarios so one of the applications. that psych does one kind of application. it does is to generate plausible. scenarios of what might happen and what. might happen based on that and what. might happen based on that and so on so. you generate this ever expanding sphere. if you will of possible future things to. to worry about or think about and. in some cases those are intelligence. agencies doing uh possible terrorist.

scenarios so that we can defend against. uh terrorist threats before we see the. first one sometimes they are computer. security um attacks so that we can. actually close loopholes U and. vulnerabilities before the very first. time someone actually exploits those um. and so on sometimes they are scenarios. involving more positive things uh. involving our plans like for instance. what what college should we go to what.

career should we go into and so on uh. what professional training should I um. take on that that sort of thing so there. there's all sorts of um there are all. sorts of useful scenarios that can be. generated that way of cause and effect. and cause and effect that go out and. many many of the. linkages in those scenarios many of the. steps. involve understanding and reasoning.

about human motivations human needs. human emotions what people are likely to. react um to in uh in something that you. do and why and how and so on so that was. always a very important part of the. knowledge that we had to represent in. the system so I talk a lot about love so. I got to ask do you remember off the top. of your head how Psych is trying to is. able to represent various aspects of.

love that are useful for understanding. human nature and therefore integrating. into this whole knowledge base of Common. Sense what is love we try to tease. apart Concepts that have. enormous complexities to them and. variety to them down to the level where. uh. where you don't as it where you don't. need to tease them apart further so love. is too general of a term it's not useful.

exactly so when you get down to uh. romantic love and sexual attraction you. get down to parental love you get down. to um filial love and uh you get down to. a love of uh doing some kind of activity. or creating So eventually you get down. to maybe 50 or 60 Concepts MH Each of. which is a Kind of Love they're. interrelated and then each one of them. has. idiosyncratic things about it uh and you.

don't have to deal with love to get to. that level of complexity even something. like. in X being in y meaning physically in y. uh we may have one English word in to. represent that but it's useful to tease. that apart because the way that the um. the liquid is in the coffee cup is. different from the way that the air is. in the room which is different from the. way that I'm in my jacket uh and so on.

and so there questions like if I look at. this coffee cup well I see the liquid if. I turn it upside down with the liquid. come out and so on um if I have say. coffee with sugar in it if I do the same. thing the sugar doesn't come out right. it stays in the liquid because it's. dissolved in the liquid and so so by now. we have about 75 different kinds of in. in the system and it's important to. distinguish those so if you're reading.

along an. English text and you see the word in. um the writer of that was able to use. this one innocuous word because he or. she was able to assume that the reader. had enough common sense and World. Knowledge to disambiguate which of these. 75 kinds of in. meant and the same thing with love you. may see the word love but if I say you. know I love ice cream that's obviously.

different than if I say I love this. person or I love to uh go fishing or. something like that so uh you have to be. careful not to take. language too seriously because people. have done a kind of a parsimony kind of. tness where you have as few words as as. you as you can because otherwise you'd. need half a million words in your. language which is a lot of words that's. like 10 times more than most languages.

really U make use of and so on just like. we have on the order of um about a. million Concepts in uh psych because. we've had to tease apart all these. things and so when you look at the name. of a psych term most of the psych terms. actually have three or four English. words in a phrase which captures the. meaning of this term because you have to. distinguish all these types of love you.

have to distinguish all these types of. in and there's not a single English word. which captures most of these things yeah. and it seems like language when used for. communication between humans almost as a. feature has some ambiguity built in it's. not some it's not an accident because. like The Human Condition is a giant mess. and so it feels like nobody wants two. robots like very precise formic. conversation on a first date right like.

there there's some dance of like. uncertainty of wit of humor of push and. pull and all that kind of stuff if. everything is made precise then life is. not worth living I think for in terms of. the The Human Experience and we've all. had this experience of creatively. misunderstanding uh one of one of my. favorite. uh one of my. favorite stories involving Marvin Minsky. is when I asked him about how he was.

able to turn out so many. Fantastic phds so many Fantastic people. who did great PhD. thesis how did he think of all these. great ideas and what he said is he would. generally say something that didn't. exactly make sense he didn't really know. what it meant but this student would. figure like oh my God Minsky said this. it must be a great idea andw sweat he or.

she would work on work and work until. they found some meaning in this sort of. Chanty Garder like utterance that Minsky. had made and then some great thesis. would come out of it yeah I love this so. much because there there's um young. people come up to me and I I'm. distinctly made aware that the words I. say have a long lasting impact I will. now start doing the. method of saying something uh. cryptically profound and then letting. them actually uh make something useful.

and great out of that you you have to. become revered enough that people will. take as a default that everything you. say is profound yes exactly exactly I I. mean I love Marvin misy so much there's. so much um I've heard this interview. with him where he said that the key to. his success has been to Hate Everything. he's ever done uh like in the past he. has so many good like on liners and just.

um or or also uh to work on things that. nobody else is working on because he's. not very good at doing stuff oh I I I. think that was just false well but see I. took whatever he said and I ran with it. and I thought it was profound because. it's more of a misy no uh a lot of. behavior is in the eye of the beholder. and a lot of the meaning is in the eye. of the beholder one of U Minsky's early. programs was begging program are you. familiar with this so this was back in. the day when you had job control cards.

um at the beginning of your IBM card. deck that said things like how many CPU. seconds to allow this to run before it. got kicked off and um because computer. time was enormously expensive and so he. wrote a program and all it did was um it. said you know give me uh 30 seconds of. CPU time and all it did was it would. wait like 20 seconds seconds and then it. would print out on the operator's. console teletype um I need another 20. seconds so the operator would give it.

another 20 seconds it would wait it says. I'm almost done I need a little bit more. time so at the end he'd get this print. out and he'd be charg you know for like. 10 times as much computer time as his. job control card you know and he'd say. look I put 10 second you know 30 seconds. here um you're charging me for 5 minutes. I'm not going to pay for this and the. poor operator would say well the program. kept asking for more time and Marvin. would say oh it always does. that I love that is is there if you.

could just Ling on it for a little bit. is there something you've learned from. your interaction with Marvin Minsky. about artificial intelligence about life. uh but I mean he's. again like your work his work is uh you. know he's a seminal figure in the in. this very short history of artificial. intelligence research and development. what have you learned from him as a. human being as a as an AI intellect I.

would say both he and Ed Fen bom. impressed on me the the realization. that our lives are finite our research. lives are finite we're going to have. limited opportunities to do AI research. projects so you should make each one. count don't be afraid of doing a project. that's going to take. years or even decades to and don't. settle for bump on a log projects that.

could lead to some you know published. Journal article that five people will. read and Pat you on the head for and and. so on so one bump on a log after another. is not how you get from the Earth to the. Moon by slowly putting additional bumps. on this log. the only way to get there is to think. about the hard problems and think about. novel solutions to them and if you do.

that and if you're willing to listen to. Nature to empirical reality willing to. be. wrong it's perfectly fine because if. occasionally you're right then you've. gotten part of the way to the. Moon you know you've worked on psych for. 37 over over that uh um many years have. you ever considered quitting I mean has. it been too much so I'm sure there's an.

optimism in the early days that this is. going to be way easier and let me ask it. another way too because I've talked to a. few people on this podcast AI folks that. bring up psych as an example of a. project that has a beautiful vision and. it's a it's a beautiful dream but it. never really materialized that's how. it's spoken about. um I suppose you could say the same. thing about new all networks and all all. all ideas um until they are so what um.

why do you think people say that first. of all and second of all did you feel. that ever throughout your journey and. did you ever consider quitting on this. Mission we keep a very low profile we. don't attend very many conferences we. don't give talks we don't write papers. we don't play the academic game at all. and as a result um people often only.

know about us because of a paper we. wrote 10 or 20 or 30 um or 37 years ago. uh they only know about us because of. what someone else secondhand or third. hand said about us so thank you for. doing this podcast by the way sure it it. it uh shines a little bit of light on. some of the fascinating stuff you're. doing so well think it's time for us to. keep a higher profile uh now that we're. far enough along that other people can.

begin to help us with the the final n%c. maybe n is maybe 90% but um now that. we've gotten this knowledge pump. primed it's going to become very. important for everyone to help if they. are willing to if they're interested in. it retirees who have enormous amounts of. time and would like to leave some kind. of Legacy U um to the world uh people. because of the pandemic who have more.

time uh at home or for one reason or. another on to be online and contribute. if we can raise awareness of how far our. project has come and how close to being. primed the knowledge pump is then we can. begin to harness um this untapped amount. of humanity I'm not really that. concerned about professional colleagues. opinions of our project I'm interested.

in getting as many people in the world. as possible actively helping and. contributing to get us from where we are. to really covering all of human. knowledge and different human opinion. including contrasting opinion that's. that's worth representing so I think. that's that's one reason um um a I I. don't think there's there was ever a. time where I thought about quitting. there are times where I've become. depressed a little bit about how hard it.

is to get funding for the system. occasionally there are AI Winters and. things like that occasionally there are. um AI uh what you might call Summers. where people have said why in the world. didn't you sell your company to um you. know company X for um some large amount. of money money when you had the. opportunity and so on and you know. company X here are like old companies. maybe you've never even heard of like. lyos or something like that so um uh the.

the answer is that one reason we've. stayed a private company we haven't gone. public one reason that we haven't um. gone out of our way to take investment. dollars is because we want to have. control over our future over our state. of being so that we can continue to do. this as until it's done and we're making. progress and we're now so close to done.

that almost all of our work is. commercial applications of our. technology so five years ago almost all. of our money came from the government. now uh virtually none of it comes from. the government almost all of it is from. companies that are actually using it for. something Hospital chains using it for. medical reasoning about patients um and. um energy companies using it and uh. various other you know man manufacturers. using it to reason about Supply chains. and things like that so there's so many.

questions I want to ask so one of the. ways that people can help is by adding. to the knowledge base and that's really. basically anybody if the tooling is. right and um the other way I I kind of. want to ask you about your thoughts on. this so you've had like you said in. government and um you had big clients. you had a lot of clients but most of it. is shrouded in secrecy because of the. nature of the relationship of the kind. of things you're helping them with so. that that's one way to operate and.

another way to operate is is more in the. open where it's more consumer facing and. so uh you know hence something like open. psych was born at some point where. there's no that that's a misconception. uhoh well that's that's go there so what. all right what is open psych and how was. it born two things I want to say and I. want to say each of them before the. other so it's going to be. difficult but we'll come back to open. Psych in a minute but one of the terms.

of our contracts with all of our. customers and partners. is knowledge you have that is genuinely. proprietary to you we will respect that. we'll make sure that it's marked as. proprietary to you in the psych. knowledge base no one other than you. will be able to see it if you don't want. them to and it won't be used in. inferences other than for you and so on. however any knowledge which is necessary.

in building any applications for you and. with you which is publicly available. General human knowledge is not going to. be proprietary it's going to just become. part of the normal psych knowledge base. and it will be um openly available to. everyone who has access to psych so. that's an important constraint that we. never went back on even when we got push. back from companies which we often did. who wanted to claim that almost. everything they were telling us was. proprietary uh so so there's a line.

between very domain specific company. specific stuff and the general knowledge. that comes from that yes or if you. imagine say it's an oil company there. are things which they would expect any. new petroleum engineer they hired to. already know and it's not okay for them. to consider that that is proprietary and. there sometimes a company will say well. we're the first ones to pay you to.

represent that in Psych yeah and our. attitude is some polite form of tough uh. the the deal is this take it or leave it. and in a few cases they've left it and. in most cases um they'll they'll see our. point of view and take it because that's. how we've built the psych system by uh. essentially tacking with the funding. wins where people would fund a project. and half of it would be general. knowledge that would stay permanently as.

part of Psych and so got it so always. with these Partnerships it's not like a. distraction from the main psych. development it's uh it's a small. distraction it's a small but it's not a. complete one so you're adding to the. knowledge base yes absolutely and we try. to stay away from um projects uh that. would not have that property um so let. me go back and talk talk about open pych. for a second so I've had a lot of. trouble expressing and convincing other.

AI. researchers how important it is to use. an expressive representation language. like we do this higher order logic. rather than just using some triple store. knowledge graph type uh. representation um and so as an attempt. to show. them why they needed something more we. said oh well we'll represent this.

unimportant projection or Shadow or. subset of. Psych that just happens to be the simple. binary relations the relation argument. one argument two triples and so on um. and then you'll see how much more useful. it is if you had the entire psych system. so so it's all well and good to have the. taxonomic. relations between terms like person and.

night and sleep and bed and house and. eyes and and so on. but think about how much more useful it. would be if you also had all the rules. of thumb about those things like people. sleep at night they sleep lying down. they sleep with their eyes closed they. usually sleep in beds in our country uh. they sleep for hours at a time they can. be woken up they don't like being woken. up and so on and so on so it's that. massive amount of knowledge which is not.

part of open psych and we thought that. all the researchers would then. immediately immediately say oh my god of. course we need the other 90% that you're. not giving us let's partner and license. psych so that we can use it in our. research but instead what people said is. oh even the bit you've released is so. much better than anything we had we'll. just make with this yeah and so if you. look there are a lot of Robotics. companies today for example which use. open psych as their fundamental ontology.

um and in some sense the whole world. missed the point of of open psych and we. were doing it to show people why that's. not really what they wanted and too many. people thought somehow that this was. psych or that this was in fact good. enough for them and they never even. bother coming um coming to us to get. access to the full py but there's. there's two parts to open psych so one. is convincing people on the idea on the. power of this General kind of. representation of knowledge and the.

value that you hold in having acquired. that knowledge and built it and continue. to build it and the other is the code. base this is there the code side of it. so my sense of the code base that scor. Psych is operating with I mean it has. the technical debt of the three decades. plus right this is the exact same. problem that Google had to deal with. with the early version of tensorflow. it's still dealing with the that they to.

basically break uh compatibility with. the past several times and that's only. over a period of a couple years but they. I think successfully opened up it's very. risky very gutsy move to open up. tensorflow and then pie torch on the. Facebook. side and what you see is there's a magic. place where you can find a community. where you can develop a community that. builds onto on the system without taking.

away any of not any but most of the. value so most of the value that Google. has is still a Google most of the value. that Facebook has is still Facebook even. though some of this major machine. learning tooling is released into the. open my question is not so much on the. knowledge which is also a big part of. open pych but all the different kinds of. tool. so the there's the kind of all the kinds. of stuff you can do on the knowledge. graph knowledge base whatever we call it. there's the inference engines so there.

there could be some that probably are a. bunch of proprietary stuff you want to. kind of keep secret and there's probably. some stuff you can open up completely. and then let the community build up. enough Community where they develop. stuff on top of it yes there'll be those. Publications and academic work and all. that kind of stuff and uh and also the. tooling of adding to the knowledge base. right like developing you know there's. incredible amount like there's so many. people that are just really good at this. kind of stuff in the open source.

community so my question for you is like. have you struggled with this kind of. idea that you have so much value in your. company already you've developed so many. good things you have clients that really. value your relationships and then. there's this dormant Giant Open Source. community that as far as I know you're. not. utilizing is. there there's so many things to say. there but there could be magic moments. where the community builds up large.

enough to where the artificial. intelligence field that is currently. 99.9% machine learning is dominated by. Machine learning has a face shift. towards like uh or at least in part. towards more like what you might call. symbolic AI this whole place where Psych. is like at the center of. and then as you know that requires a. little bit leap of faith because you're. now surfing and there'll be obviously. competitors that'll pop up and start.

making you nervous and all that kind of. stuff so do you think about the space of. open sourcing some parts and not others. how to leverage the community all those. kinds of things that that's a good. question and I think you phrased it the. right way which. is we're constantly struggling with the. question of what to open Source what to. make public what to even publicly talk. about right um and it's there enormous.

pluses and minuses to every. alternative and it's very much like. negotiating a very treacherous path part. partly the analogy is like if you slip. uh you could make a fatal mistake give. away something which essentially kills. you or fail give away something which um. failing to give it away um hurts you and. so. on so it is a it is a very tough tough.

question usually what we have done with. uh people who approached us to. collaborate on Research is to say we. will make available to you the entire. knowledge base. and executable copies of all of the code. but only very very limited source code. access if you have some idea for how you.

might improve something or work with us. on something so let me also get back to. one of the very very first things we. talked about here um which. was separating the question of how could. you get a computer to do this at all. versus how could you get a computer to. do this efficiently enough in real time. and so one of the early lessons we. learned was that we had to separate the.

epistemological problem of what should. the system know separate that from the. heuristic problem of how can the system. reason efficiently with what it knows. and so instead of trying to pick one. representation language which was The. Sweet Spot or the best trade-off point. between expressiveness of the language. and efficiency of the language if you. had to pick one knowledge graphs would. probably be associative triples would. probably be about the best you could do.

and that's why we started there but um. after a few years we realized that what. we could do is we could split this and. we could have one nice clean. epistemological level language which is. this higher order logic and we could. have one or more. grubby but efficient juristic level. modules that. opportunistically would say oh I can. make progress on what you're trying to.

do over here I have a special method. that will contribute a little bit toward. a solution and so of course some subset. of that exactly so by now we have over a. thousand of these heuristic level. modules and they function as a kind of. community of agents and there's one of. them which is a general theorem prover. and the in theory. that's the only one you need but in. practice it always takes so long that. you never want to call on it um you.

always want these other agents to very. efficiently reason through it it's sort. of like if you're balancing a chemical. equation you could go back to First. principles but in fact there are. algorithms which are vastly more. efficient or if you're trying to solve a. quadratic equation you could go back to. First principles of. mathematics but it's much better to. Simply recognize that this is a. quadratic equation and apply the. binomial formula and snap you get your. answer right away and so on so think of.

these as like a thousand little experts. that are all looking at everything that. pite gets asked and looking at. everything that every other little agent. has contributed almost like notes on a. Blackboard notes on a um a whiteboard um. and making additional notes when they. think they can be helpful and gradually. that community of Agents gets an answer. to your question gets a solution to your. problem and if we ever come up in a.

domain application where Psych is. getting the right answer but taking too. long um then what will often do is talk. to one of the human experts and say uh. here's the the set of reasoning steps. that psych went through you can see why. it took it a long time to get the answer. how is it that you were able to answer. that question. in 2 seconds and occasionally you'll get. an expert who just says well I just know. it I just was able to do it or something.

um and then you don't talk to them. anymore but sometimes you'll get an. expert who says um well let me. introspect on that yes here is a special. representation we use just for aquous. chemistry equations or here's a special. representation and a special technique. which we can now apply to things in the. special representation and so on and. then you add that as the Thousand and1st. hlis level module and from then on um in.

any application if it ever comes up. again it'll be able to contribute and so. on so that that's pretty much one of the. main ways in which um psych has recouped. this lost deficiency um a second. important way is meta reasoning so um. you can speed things up by focusing on. removing Knowledge from the system till. all it has left is like minimal. knowledge needed to but that's the wrong. thing to do right that would be like in. a human extrap painting part of their.

brain or something that's really bad so. instead what you want to do is give it. meta level advice tactical and strategic. advice that enables it to reason about. what kind of knowledge is going to be. relevant to this problem what kind of. tactics are going to be good to take in. trying to attack this problem when is it. time to start trying to prove the. negation of this thing because I'm. knocking myself out trying to prove it's. true and maybe it's false and if I just. spend a minute I can see that it's false.

or something so so it's like dynamically. pruning the the graph to to only like. based on the particular thing you're. trying to in infer yes and so by now we. have about. 150 of these sort of like Breakthrough. ideas that have led to dramatic speedups. in the inference process you know where. one of them was this El HL split and. lots of HL modules another one was using. meta and Meta Meta level reasoning um to.

U reason about the the reasoning that's. going on and so on um and you know 150. breakthroughs may sound like a lot but. you know if you divide by 37 years it's. not as. impressive so there's these kind of. heuristic modules that really help. improve the the. inference uh how hard in general is this. because you mentioned higher order logic. you know the the in in the general the. theorem sense it's an intractable very.

difficult problem yes so how hard is. this inference problem when we're not. talking. about if we let go of the perfect and. focus on the good I I would say it's. half of the problem in the in the. following empirical sense which is over. the years. about half of our effort maybe 40% of. our effort has been our team of. inference programmers and the other 50.

60% has been our ontologists our. ontological Engineers putting in. knowledge so our anthological engineers. in most cases don't even know how to. program they have degrees in things like. philosophy and so on um so it's almost. like the I love that i' love to hang out. with those people actually oh yes it's. wonderful but it's very much like the. Eloy and the warlocks in um HD Well's. time machine so you have the Eloy who. only program in the epistemological. higher order logic language yes and then.

you have the morlocks who are like um. under the under the ground uh figuring. out what the Machinery is that will make. this efficiently operate and so on um. and so you know occasionally they'll. toss messages back to each other and so. on but it really is almost this 5050. split between finding clever ways to. recoup efficiency when you have an. expressive language and putting in the. content of what the system needs to know.

and yeah both are fascinating to some. degree the entirety of the system as far. as I understand is written in various. variants of lisp so my favorite programm. language is still lisp I don't program. it in much anymore because you know the. world has in majority of its system has. moved on. like everybody respects lisp but many of. the systems are not um written in lisp. anymore but psych as far as I understand.

maybe you can correct me there's a bunch. of lisp in it yeah so it's based on a. lisp code that we produced most of the. programming is still going on in a. dialect of lisp and then the for. efficiency reasons that gets. automatically translated into things. like Java um or C nowadays it's almost. all translated into Java because Java. has gotten good enough that uh that's. that's really all we need to do so. translate into Java and then Java is.

compiled down to U by code yes okay so. that that's sort of that's a. that um that's a you know it's a process. that probably has to do with the fact. that when py was originally written and. you build up a powerful system like. there is some technical depth you have. to deal with as is the case with most. powerful systems that span years um have. you ever. considered this this would helped me.

understand because from my perspective. so much of the value of everything. you've. done with psych and PSY Corp is the is. the is the. knowledge have you ever considered just. like throwing away the code base and. starting from scratch not really. throwing away but sort of moving it to. uh like throwing away that technical. debt starting with a more updated. programming. language is that throwing away a lot of. value or no like what's your sense how.

much of the value is in the silly. software engineering aspect and how much. of the value is in the. knowledge so. development of of programs in lisp um. precedes um I think somewhere between a. th000 and 50,000 times faster than. development in any any of what you're. calling um modern or improved computer. languages well there's other functional.

languages like you know closure and all. the there there's but I mean I'm with. you I I I like list but I just wonder. how many great programmers there are. there are still like yes so so it is. true when a new inference programmer. comes on board they need to learn um. some of lisp and in fact we have a. subset of lisp which we call cleverly. subel which is really all they need to. learn and so the programming actually. goes on in subel not in full lisp and so.

it does not take programmers very long. at all to learn uh subel and that's. something which can then be translated. efficiently into uh Java and for some of. our programmers who are doing say user. interface work then they never have to. even learn subel they just have to learn. apis into the the basic uh psych engine. so you're not necessarily feeling the. burden of like it's it's extremely. efficient there's um that's not a.

problem to solve okay right right the. other thing is remember that we're. talking about hiring programmers to do. inference who are programmers interested. in effectively automatic theorem proving. right and so those are people already. predisposed to representing things in. logic and so on and lisp really was the. program in L language um based on logic. that John McCarthy and others um who. developed it basically create took the. the formalisms that Alonzo church and.

other um philosophers other logicians um. had come up with and basically said can. we basically make a programming La. language which is effectively logic and. so since we're talking about reasoning. in about Expressions written in this. logic epistemological language and we're. doing operations which are effectively. like theor proving type operations and. so on there's an natural impedance match.

between lisp and the the knowledge the. way it's represented so I I guess you. could say it's a perfectly logical. language to use oh yes okay I'm sorry. I'll even let you uh get away with that. like so I'll probably use that in the. future without without credit without. credit but no I think I think the uh the. the point is that the the language you. program in isn't really that important.

um it's more that you have to be able to. think in terms of for instance creating. new helpful HL modules and how they'll. work with each other and um looking at. things that are taking a long time and. coming up with new specialized data. structures that will make this efficient. so um let me just give you one very. simple example which is when you have a. transitive relation like larger than. this is larger than that which is larger. than that which is larger than that so.

the first thing must be larger than the. the last thing whenever you have a. transitive. relation um if you're not careful if I. ask whether this thing over here is. larger than that thing over here I'll. have to do some kind of graph walk or. theorem proving that might involve like. five or 10 or 20 or 30 steps but if you. store redundantly store the transitive. closure the cleany star of that. transitive relation now you have this. big table but you can always guarantee.

that in one single step you can just. look up whether this is larger than that. um and so um we um there are lots of. cases where storage is cheap today and. so by having this extra redundant data. structure we can answer this commonly. occurring type of question very very. efficiently. let me give you one other um analogy. analog of that uh which is something we. call rule macro predicates which is. we'll see this complicated Rule and.

we'll notice that things very much like. it syntactically come up again and again. and again so we'll create a whole brand. new relation or predicate or. function that captures that and takes. maybe not two arguments takes maybe. three four five arguments and so on um. and now we have. effectively um converted some.

complicated if then. rule that might have to have inference. done on it into some ground Atomic. formula which is just a um the name of a. relation and a few arguments and so on. and so converting commonly occurring. types or schemas of rules into brand new. predicates brand new functions turns out. to enormously speed up the inference. process so so now we've covered about. four of the 150 um good ideas I said.

that so that's a nice that's a cool so. that idea in particular is like a nice. compression that turns out to be really. useful yeah that's really interesting I. mean this whole thing is just. fascinating for a philosophical there's. part of me I mean it makes me a little. bit sad because your work is both um. from a computer science perspective. fascinating and the inference engine. from a epistemological philosophical. aspect fascinating but you know it is. also you're running a company and. there's some stuff that has to remain.

private and it's sad Well here here's. something that may make you feel better. a little bit better um we're uh We've. formed a not not for-profit company uh. called the knowledge axenization. Institute NEX KX and I have this firm. belief with a lot of empirical evidence. to support it. that the the education that people get. in high schools in colleges in graduate. schools and so on is almost completely.

orthogonal to almost completely. irrelevant. to how good they're going to be at. coming up to speed in doing this kind of. ontological engineering and writing. these assertions and rules and so on in. in Psych and so very often we'll. interview candidates who have their PHD. in philosophy who've taught Logic for. years and so on and they're just they're. just awful but the converse is true so.

one of the best ontological Engineers we. ever had never graduated high school and. so the purpose of um knowledge a. communization Institute if we can get. some some foundations to help support it. is identify people in the general. population maybe High School dropouts. who have have latent talent for this. sort of thing um offer them effectively. scholarships to train them and then help.

place them in companies that need more. trained ontological Engineers some of. which would be working for us but mostly. would be working for partners or. customers or something and if we could. do that that would create an enormous. number of um relatively very high-paying. jobs for people who currently um have no. no way out of some you know um situation. that they're locked into so is there. something you can put into words that. describes somebody who would be great at.

anthological engineering so what. characteristics about a person make them. a great at this task this task of. converting the messiness of human. language and knowledge into formal logic. this is very much like what um Alan. touring had to do during World War II. uh in trying to find people to bring to. Bletchley Park where he would publish in. the London Times cryptic crossword. puzzles along with some some innocuous.

looking note which essentially said if. if you were able to solve this puzzle in. less than 15 minutes please call this. phone number and so on so um you know or. back when I was young there was uh uh. the practice of having matchbooks where. on the inside of the matchbook. um there would be a can you draw this. you have a career in art a commercial. art if you can copy this uh drawing you. know and so on so um yes the the analog.

of that is there a little test that get. to the core of whether you're going to. be good or not so part of it has to do. with uh being able to um make and. appreciate um um and react negatively. appropriately to puns and other jokes so. you have to have a kind of sense of. humor and if you're good at uh telling. jokes and um good at understanding jokes. that's that's one indicator puns yes. like Dad jokes yes well maybe not dad. jokes but real but funny jokes um but uh.

I think I'm applying to work as sore no. but um another another is if you're able. to introspect so very often um uh we'll. we we'll give someone a simple question. and we'll say like um um wh wh why why. is this and you know sometimes they'll. just say because it is okay that's a bad. sign but very often they'll be able to. introspect and so on so one of the. questions um I often ask is I'll point. to a sentence with a pronoun in it and.

I'll say um you know the referend of. that pronoun is obviously this noun over. here um you know how would you or I or. an AI or a fiveyear old 10-year-old. child know that that pronoun refers to. that noun over here um and um often um. the people who are going to be good at. ontological engineering will give me. some causal explanation or will refer to. some things that are true in the world. so if you imagine a sentence like the.

horse was led into the barn while its. head was still wet and so its head. refers to the horse's head but how do. you know that and so some people will. say I just know it some people will say. well the horse was the subject of the. sentence and I'll say okay well what. about the horse was led into the barn. while its roof was still wet now its. roof obviously refers to the barn um and. so then they'll say oh well that's. because it's the closest noun and so so. basically if they try to give me answers.

which are based on syntax and grammar. and so on that's a really bad sign but. if they're able to say things like well. horses have heads and barns don't and. barns have roofs and horses don't um. then that's a positive sign that they're. going to be good at this because they. can introspect on what's true in the. world that leads you to know certain. things how fascinating is it that. getting a PhD makes you less capable to. introspect deeply about this oh I'm I. wouldn't I wouldn't go that far I'm not.

saying that it makes you less capable. let's just say it's independent of I. don't know of how good people are you're. not saying that I'm saying that there's. a certain kind it's it's it's. interesting that for a lot of people. phds uh sorry philosophy aside that. sometimes education Narrows your. thinking versus expands it yes it's kind. of fascinating and for certain when. you're trying to do ontological. engineering which is essentially teach. our future AI overlords how to reason.

deeply about this world and how to. understand it that that requires that. you think deeply about the world so I'll. tell you a sad story about mathcraft. which is why is that not widely used in. schools today um we're not really trying. to make big profit on it or anything. like that the when we've gone to schools. their attitude has been well if a. student spends 20 hours going through. this mathcraft program from start to end.

and so on um will it improve their score. on this standardized test more than if. they spent 20 hours just doing mindless. drills of problem after problem after. problem and the answer is well no but. it'll increase their understanding more. and their attitude is well if it doesn't. increase their score on this test um. then that's not you know we're not going. to adopt it that's sad I mean that's. that's a whole that's a whole another. three 4H hour conversation about the.

education system but let me ask you let. me go super philosophical as if we. weren't already so in 1950 Alan touring. wrote the paper that formulated the. touring test yes and he opened the paper. with the question can machines think so. what do you think can machines think let. me ask you this question absolutely. machines can think um certainly as well. as humans can think um right we're meat. machines um just because they're not.

currently made out of meat is just you. know an engineering solution uh decision. um and so on so um um of course of. course machines can think I think that. there was a lot of. um Damage Done by people U. misunderstanding touring's imitation. game um and um focus on trying to trying. to get a chatbot to uh fool other people.

into thinking it was human um and so on. that that's that's not a terrible test. in and of itself but it shouldn't be. your one and only test for intelligence. so do you uh in terms of tests of. intelligence uh you know with the LNA. prize which is a very kind of you want. to say a more strict formulation of the. touring test as originally formulated. and then there's something like Alexa. prize which is more I would say more.

interesting formulation of the test. which is like uh ultimately the metric. is how long does a human want to talk to. the AI system so it's like if you the. goal is you want it to be 20 minutes. it's basically not just have a. convincing um conversation but more like. a comp compelling one or a fun one or an. interesting one I me that that seems. like more to the spirit maybe of um of. what uh touring was imagining but what.

for you do you think in the space of. tests is a is a good test like what when. you see a system based on psych that. passes that test You' be like damn we've. created something special. here the the test has to be something. involving. depth of reasoning and recursiveness of. reasoning the ability to answer repeated. why questions about the answer you just. gave it's how many wide questions in a.

row can you keep answering something. like that and um also just have like a. young curious child and an AI system and. how long will an AI system last before. it wants to quit yes and again that's. not the only test another one has to do. with argumentation in other words here's. a proposition um come up with pro and. con Arguments for it and uh try and give. me convincing arguments on both sides uh.

and U so that's that's another important. kind of ability that um the system needs. to be able to exhibit in order to really. be intelligent I think so there are. certain I mean if you look at IBM Watson. and like certain impressive. accomplishments for very specific test. almost like a demo right um there is. some uh like I talked to the guy who led.

the the Jeopardy effort and there's some. kind of hard coding heuristics uh tricks. that you try to pull it all together to. make the thing work in the end for this. thing right that seems to be one of the. lessons with AI is. like that's the fastest way to get a. solution that's pretty damn impressive. so so here here's what I would say is. that as impressive as that was it made. some mistakes but more.

importantly many of the mistakes it made. were mistakes which no human would have. made yeah um and so part of the the the. new or augmented touring tests would. have to be and the mistakes you make are. ones which humans don't basically look. at and say what yeah uh so for example. there was. a um a question about which 16th century.

Italian politician blah blah blah and. Watson said Ronald Reagan so most. Americans would have gotten that. question wrong but they would never have. said Ronald Reagan as an answer right. because you know among the things they. know is that he lived relatively. recently in people don't really live 400. years and you know things like that so. that that's I think um a very important. thing which is um if it's making. mistakes which no normal sane human.

would have made then that's a really bad. sign and if it's not making those kinds. of mistakes then that's um a good sign. and I don't think it's any one very very. simple test I think it's all of the. things you mentioned all the things I. mentioned there's really a battery of. tests which together if it passes almost. all of these tests it would be hard to. argue that it's not intelligent and if. it fails some several of these tests. it's really hard to argue that it really.

understands what it's doing that it. really is generally intelligent so to. pass all of those tests you know we've. talked a lot about psych and knowledge. and. reasoning do you think this AI system. would need to have some other humanlike. elements for example a body or a. physical manifestation in this world and. uh another one which seems to be. fundamental to The Human Experience is. consciousness the subjective experience.

of what it's like to actually be you do. you think he needs those to be able to. pass all those tests and to achieve. general intelligence it's a good. question I think in the case of a body. uh no I know there are a lot of people. like Penrose who would have disagreed. with me and so there um and and others. but no I don't think it needs to have a. body in order to be intelligent I think. that it needs to be able to talk. about uh having a body and having.

Sensations and having emotions and so on. it doesn't actually have to have all of. that U but it has to understand it in. the same way that Helen Keller was. perfectly intelligent and able to talk. about um colors and sounds and um shapes. and um and so on um even though she. didn't directly experience all the same. things that the rest of us do. so knowledge of it and being able to. correctly um make use of that um is.

certainly an important facility but. actually having a body um if you believe. that that's just a kind of religious or. mystical belief you can't really um. argue for or against it I suppose um. it's it's just something that some. people that some people believe what. about the. like an extension of the body which is. consciousness I mean like it feels like. something to be here sure but you know. what what does that really mean it's.

like well if I talk to you you say. things which make me believe that you. are conscious yeah um I know that I'm. conscious but that's you know you're. just taking my word for it now um but in. the same sense Psych is conscious in. that same sense already where of course. it understands it's a computer program. it understands where and when it's. running it understands who's talking to. it it understands what its task is what. its goals are what its current problem. is that it's working on it understands. how long it's spent on things what it's. tried it understands what it's done in.

the past um and so on um and uh you know. if if we want to call that Consciousness. then yes Psych is already conscious but. I don't think that I would ascribe. anything um mystical to that again some. people would but I would I would say. that you know other than other than our. personal experience of Consciousness um. we're just treating everyone else in the. world um uh so to speak um at their word. about being conscious and so if um if a.

computer program if an AI is able to um. exhibit um all the same kinds of um. response as you would expect of a. conscious entity um then um you know. doesn't doesn't it deserve the label of. Consciousness just as much so there's. another burden that comes with this. whole intellig. thing that humans got is um the. extinguishing of the light of. Consciousness which is uh kind of. realizing that we're going to be dead.

someday and uh there's a bunch of. philosophers like Ernest Becker who kind. of think that this realization of. mortality and then fear sometimes they. call it Terror of of of mortality is one. of the creative forces. behind Human Condition like it's the. thing that drives us do you think it's. important for an AI system you know when. psych proposed that it's one it's not.

human and it's one of the moderators of. his. contents um you know there's another. question it could ask which is like it. kind of knows that humans are mortal am. I. mortal and I think one really important. uh thing that's possible when you're. conscious is to fear the extinguishing. of that Consciousness the fear. mortality do you think that's useful for.

intelligence thinking like I might die. and I really don't want to die I I don't. think so I think it may help um some. humans to be um better people it may. help some humans to be more creative and. so on I don't think it's necessary um. for AI is to believe that they have. limited lifespans and therefore they. should make the most of their behavior. maybe eventually um the answer to that. and my answer to that will change but as.

of now I would say that that's almost. like a a frill or a side effect uh that. is not in fact if you look at most. humans most humans um ignore the fact. that they're going to die most of the. time uh so well but that's like this. goes to the white space between the. words so what Ernest Becca argues is. that that ignoring is we're living in an. illusion that we constructed on the. foundation of this Terror so we're. Escape Life as We Know It pursuing.

things creating things love everything. we can think of that's beautiful about. humanity is is just trying to escape. this realization that we're going to die. one day that's his that's his idea and I. think I don't know if I I. 100% believe in this but there's it. certainly Rhymes it seems like to me. like it rhymes with the truth yeah I I. think that for some people um that's.

going to be a more powerful Factor than. others clearly Doug is talking about. Russians and I think that. uh so I'm Russian so it clearly it. infiltrates all of Russian literature. and and AI doesn't have to have uh fear. of death. as a motivating force in that we can. build in motivation so we can build in. the motivation. of obeying users and making users happy.

and making others happy and and so on. and that can substitute for this sort of. personal fear of death that sometimes. leads to bursts of creativity in in. humans I don't know I think like I think. AI really needs to understand death. deeply in order to be able to drive a. car for example I I think there's just. some like there no I I really disagree I.

think it needs to understand the value. of human life especially the value of. human life to other humans the um and. understand that certain things are more. important than other things so it has to. have a lot of knowledge about ethics and. U morality and so on but some of it is. so messy that impossible to encode for. example there disagree so if there's a. person dying right in front of us most. human beings would help that person but. they would not apply that same ethics to.

everybody else in the world this is the. tragedy of how difficult it is to be a. doctor because they know when they help. a dying child they know that the money. they're spending on this child cannot. possibly be spent on every other child. that's dying and that's that's a very. difficult to encode decision now U. Perhaps Perhaps it is perhaps it could. be formalized oh but I mean you're. talking about autonomous vehicles right.

so autonomous vehicles are going to have. to make those decisions um all the time. of um what is the chance of this bad. event happening um how bad is that. compared to this chance of that bad. event happening and so on and you know. when potential accident is about to. happen is it worth taking this risk if I. have to make a choice which of these two. cars am I going to hit and why and see I. was thinking about a very different. Choice when I'm talking about the.

mortality which is just observing uh. Manhattan style driving I think that. humans as an effective driver needs to. threaten pedestrians lives a lot there's. a dance I've watched pesters a lot I. worked on this problem and it seems like. the the if I could summarize the problem. of a pedestrian Crossing is the car with. this movement is saying I'm going to. kill you and the pedestrians is.

saying maybe and then they decide and. they say no I don't think you you have. the guts to kill me and you walk and. they walk in front and they look away. and there's that dance the the the The. Pedestrian this a social contract that. The Pedestrian trust that once they're. in front of the car and the car is. sufficiently from a physics persp. perspective able to stop they're going. to stop but the car also has to threaten. that pedestrian is like I'm late for. work so you're being kind of an asshole. by Crossing in front of me but life and.

death is in like is part of the. calculation here and it's that that. equation is being solved millions of. times a day yes very effectively that. game theory whatever whatever that. formulation is absolutely I just I don't. know if it's as simple as some. formalizable Game Theory problem it it. could very well be in the case of. driving and in the case of most of uh. Human Society I I don't know but it uh.

yeah you might be right that this sort. of the fear of death is just one of the. quirks of uh like the way our brains. have evolved but it's not it's not a. necessary feature of a of intelligence. drivers certainly are always doing this. kind of estimate even if it's. unconscious subconscious of. what are the chances of various bad. outcomes happening like for instance um. if I don't wait for this pedestrian or. something like that and um what is the. downside to me going to be in terms of.

um you know time wasted talking to the. police or um you know getting sent to. jail or you know things like that and so. um and there's also emotion like people. in their cars tend to get uh. irrationally Angry that's that's that's. dangerous but you know think think about. this is all part of why I think that. autonomous vehicles um truly autonomous. vehicles are farther out than um than. most people do because um there is this.

enormous level of complexity which goes. beyond uh mechanically controlling the. car um and um I I can see the autonomous. vehicles as a kind of metaphorical and. literal accident waiting to happen um. and not just because of the their um. overall um um incurring versus. preventing accidents and so on but just. because of the um almost um voracious.

appetite people have for um. um bad bad stories about powerful. companies and Powerful entities when. when I was um at a coincidentally. Japanese fifth generation Computing. system cont in. 1987 uh while I happened to be there um. there was a worker at an auto plant who. was despondent and committed suicide by. climbing under the safety chains and so. on getting stamped to death by a machine.

and instead of being a small story that. said despondent worker commits suicide. it was front page news that effectively. said Robot kills worker because the. public is just waiting for stories about. like he AI kills photogenic family of. five type stories and even if you could. show that Nationwide uh this system. saved more lives than it cost and saved.

more injuries um prevented more injuries. than it caused and so on um the media. the public the government is. just coiled and ready to pounce on. stories where in fact it failed even if. they're relatively few. yeah it's so fascinating to watch us. humans. resisting The Cutting Edge of Science. and Technology and almost like hoping. for it to fail and constantly and you.

know this just happens over and over and. over throughout history well or even if. we're not hoping for it to fail we're. we're fascinated by it and in terms of. what we find interesting um the one in a. thousand failures much more interesting. than the. 999 U boring. successes so once we build build an AGI. system say psych is some part of uh some. part of it and um say it's very possible. that you would be one of the first. people that can sit down in the room.

let's say with her and have a. conversation what would you ask her what. would you talk about looking at all of. the. content out there on the web and so on. what are. the what are the some possible solutions. to big problems that the world has that.

people haven't really thought of before. that are not. being properly or at least. adequately pursued uh what are some. novel solutions that you can think of. that we haven't that might work and that. might be worth considering so that is a. a damn good question g g given that the. AGI is going to be somewhat different. from Human. intelligence it's still going to make. some mistakes that we wouldn't make but. it's also possibly going to notice some.

blind spots we have and um I would I. would love it as a test of is it really. um On a par with our intelligen is can. it help spot some of the blind spots. that we. have so the two-part question of can you. help identify what are the big problems. in the world and two what are some novel. solutions to those problems that are not. being talked about by.

anyone yeah and some of those may become. um you know infeasible or reprehensible. or something but some of them might be. actually great things to look at you. know if you if you go back and look at. some of the most powerful discoveries. that have been made U like relativity. and um superconductivity and so on a lot. of them were cases. where someone took seriously the idea. that there might actually be um a a.

nonobvious answer to a to a question so. in Einstein's case it was um yeah the. lorence transformation is known um. nobody believes that it's actually the. way reality Works what if it were the re. way that reality actually worked so you. know a lot of people don't realize he. didn't actually work out that equation. he just sort of took it seriously um or. in the case of superconductivity you. have this V equals IR equation where R. is resistance and so on and um um it was. being mapped at lower and lower.

temperatures but everyone thought that. was just bump on a log research um to. show that V equals I always held um and. then when some graduate student um got. to a slightly lower temperature and. showed that resistance suddenly dropped. off everyone just assumed that they did. it wrong and they and it was only a. little while later that they realized it. was um um it was actually a new. phenomenon or in the case of um um the. um H pylori bacteria causing stomach.

ulcers where everyone thought that. stress and stomach acid caused ulcers um. and when a doctor um in um Australia um. claimed it was actually a bacterial. infection um he couldn't get anyone. seriously to listen to him and he had to. um ultimately inject himself with the. bacteria to show that he suddenly. developed a life-threatening ulcer um in. order to get other doctors to seriously. consider that so there all sorts of.

things where um humans are locked into. paradigms what Thomas cun called. paradigms and we can't get out of them. very easily so a lot of AI is locked. into the deep learning machine learning. Paradigm right now um and um almost all. of of us and almost all Sciences are. locked into current paradigms and you. know cun's point was um pretty much you. have to wait for people to die um in. order for the new generation to escape.

those paradigms and I think that one of. the things that would change that sad. reality is if we had trusted agis that. could help take a step back and question. some of the paradigms that we're. currently locked into yeah it would. Excel at the Paradigm shifts in in human. science and. progress you've lived a very interesting. life where you thought about Big Ideas. and you stuck with.

them can you give advice to young people. today somebody in high school somebody. undergrad about um career about. life I'd say you can make a. difference but in order to make a. difference you're going to have to have. the courage to follow through with ideas. which other people might not immediately. understand or um support you have to.

realize that if you. make. some some plan that's going to take an. extended period of time to carry out. don't be afraid of that that's true of. um physical training of your body that's. true. of um uh learning uh um some profession. that's also true of innovation that some. Innovations are not great ideas you can.

write down on a napkin and um become an. instant success if you turn out to be. right some of them. are paths you have to follow but. remember that you're mortal remember. that you have a limited number of decade. sized bets to make with your life and. you should make each one of them count. and that's true in personal. relationships that's true in career. choice that's true in uh making.

discoveries and so on and if you follow. the path of least resistance you'll find. that you're optimizing for um short. periods of time and before you know it. you turn around and long periods of time. have gone by without you ever really. making a difference in the world you. know there's when you look I mean the. field that I really love is artificial. intelligence and there's not many. projects there's not many little flames.

of hope that have been carried out for. many years for decades and pych. represents one of them and uh I mean. that in itself is just a really. inspiring thing so I'm I'm I'm deeply. grateful that you would be carrying that. flame for so many years and I think. that's an inspiration to you young. people that said you said life is finite. and we talked about mortality as a. feature of AI do you think about your. own mortality are you afraid of death um.

sure I would be crazy if I weren't and. um as I get older I'm now um over 70 so. as I get older um it's more on my mind. especially as acquaintances and friends. and especially um mentors um one by one. are dying so I can't avoid thinking. about mortality and I think that um the. the good news from the point of view and. the rest of the world is that that adds. impetus to uh my need to succeed in a.

small number of years in the future. because I'm you have a deadline exactly. I'm not going to have another 37 years. to continue working on this so we really. do want likey to make an impact in the. world um commercially physically. metaphysic basically um in the next. small number of years 2 3 5 years not 2. three five decades anymore and so this. is really driving me toward uh this this.

sort of commercialization and increasing. increasingly widespread application of. Psych whereas before um I felt that I. could just sort of sit back roll my eyes. wait till the world caught up and now I. don't feel that way anyway anymore I. feel like I need to put in some effort. to make the world aware of what we have. and what it can do and the good news. from your point of view is that that's. that's why I'm sitting here you're going. to be more. productive uh I love it and if I can.

help in any way I would love to from a. from a you know from a programmer. perspective I I love uh especially these. days just contributing in small and big. ways so if there's any open sourcing. from the MIT side and the research I I. would love to help but when you know. bigger than psych like I said it's that. little flame that you're carrying of. artificial intelligence the big dream um. is there what do you hope your legacy.

is H that's a good. question that people think of me as one. of the Pioneers or inventors. of the AI that is ubiquitous and that. they take for granted um and so on much. much the way that today we look back on. the the pioneers of electricity or the. pioneers of um similar types of uh. Technologies and so on as um you know.

it's hard to imagine what life would be. like if uh these people hadn't done what. they um what they did so that that's one. thing that I'd like to be remembered as. another is that so the Creator one of. The orig Originators of this. gigantic. knowledge store and acquisition system. that is likely to be at the. center of whatever this future AI thing.

will look like yes exactly and I'd also. like to be remembered as someone. who wasn't afraid to spend several. decades on a project in a time when um. all when almost all of the. other forces institutional forces and. Commercial forces are incenting people. to go for short-term rewards and a lot.

of people gave up a lot of people that. dreamt the same dream as you gave up yes. and you didn't. yes I mean uh Doug it's it's truly an. honor this was a long time coming I I um. a lot of people bring up your work uh. specifically and more broadly. philosophically of this is the dream of. artificial intelligence this is likely a. part of the future we're so sort of. focused on machine learning applications.

all that kind of stuff today but it. seems like the ideas that pych carries. forward uh is something that would be at. the center of this problem they're all. trying to solve which is the the problem. of intelligence emotional and and uh. otherwise so thank you so much it's such. a Hu huge honor that you would talk to. me and spend your valuable time with me. today thanks for talking thanks Lex it's. been great thanks for listening to this. conversation with Doug lennet to support. this podcast please check out our.

sponsors in the description and now let. me leave you some words from Mark Twain. about the nature of. truth if you tell the truth you don't. have to remember anything thank you for. listening and hope to see you next time.