Richard Craib: WallStreetBets, Numerai, and the Future of Stock Trading | Lex Fridman Podcast #159

the following is a conversation with. richard craig founder of. numerai which is a crowdsourced hedge. fund. very much in the spirit of wall street. bets but where the trading is done. not directly by humans but by artificial. intelligence systems submitted. by those humans it's a fascinating and. extremely difficult machine learning. competition. where the incentives of everybody is. aligned. the code is kept and owned by the people. who develop it. the data anonymized data is very well.

organized and made freely available. i think this kind of idea has a chance. to change. the nature of stock trading and even. just money management in general by. empowering people who are interested in. trading stocks. with the modern and quickly advancing. tools of machine learning. quick mention of our sponsors audible. audiobooks. trial labs machine learning company. blinkist app. that summarizes books and athletic.

greens. all-in-one nutrition drink click the. sponsor links to get a discount and to. support this podcast. as a side note let me say that this. whole set of events around. gamestop and wall street bets has been. really inspiring to me as a. demonstration. that a distributed system a large number. of regular people are able to coordinate. and collaborate. in taking on the elite.

centralized power structures especially. when those elites are misbehaving. i believe that power in as many cases as. possible should be distributed. and in this case the internet as it is. for many cases. is the fundamental enabler of that power. and at the core what the internet in its. distributed nature represents this. freedom of course the thing about. freedom. is it enables chaos. or progress or sometimes both.

and that's kind of the point of the. thing freedom is. empowering but ultimately unpredictable. and i think in the end. freedom wins if you enjoy this podcast. uh subscribe on youtube review it on. apple podcast. follow on spotify support on patreon or. connect with me. on twitter at lex friedman and now. here's my conversation with richard. craig. from your perspective can you summarize. the important events.

around this amazing saga they've been. living through of wall street bets the. subreddit and gamestop. and in general just what are your. thoughts about it from technical to the. philosophical level. i think it's amazing it's like my. favorite story ever. like when i was reading about i was like. this is the best. um and it's it's also you know connected. with my company which we can talk about. but. what i liked about it is like i like. decentralized coordination. and looking at the mechanisms that these.

are wall street bets. users use to hype each other up. to get excited to prove that that they. bought the stock and they're holding. um and and then also to see that. how big of an impact that that. decentralized coordination had. um it really was a big deal well you're. impressed by the distributed uh. coordination the collaboration amongst. like i don't know what the numbers are i. know numero. is looking at the data after all this is.

over and done it'd be interesting to see. like from. uh a large-scale distributed system. perspective. to see how everything played out but. just from your current perspective. what we know is it obvious to you that. such. incredible level of coordination could. happen. where a lot of people come together and. distribute a sense there's an emergent. behavior that happens after that. no it's not at all um obvious and one of. the reasons is the. lack of kind of like credibility to.

coordinate with someone you need to kind. of make. credible contracts or credible claims. so if you have um a username on uh. our wall street bets like some of them. are like deep value. is one of them that's an actual username. by the way we're talking about. there's a website called reddit and. there's subreddits on it. and a lot of people mostly anonymous i. i think for the most part anonymous can. create user accounts and then can then. just. talk on form like style boards you.

should know what reddit is if you don't. know what reddit is check it out. uh if you don't know what red is maybe. go to uh. the ah sub subreddit first aww. with cute pictures of cats and dogs. that's my recommendation anyway. okay yeah that would be a good start to. reddit when you when you get into it. more go to our wall street vets. it gets dark quickly oh we'll probably. talk about that too. uh so so yeah so there's these users. and it's there's no contracts like. you're saying there's no contracts the. the. users are anonymous um but there are.

little things that that do help. so for example if you've posted a really. good investment idea in the past. that exists on reddit as well and it. might have lots of upvotes. um and that's also kind of like giving. credibility to your next. to your next thing um and then they are. also putting up. screenshots uh like this this is the. these. i here's the trades i've made and here's. a screenshot now you could fake the. screenshot. but um but but still it seems like if.

you've got a lot of karma. and you've had a good performance on the. community. it somehow becomes credible enough for. other people to be like you know what. he actually probably did put a million. dollars into this. and you know what i can i can follow. that trade easily. and there's a bunch of people like that. so you're kind of uh integrating all. that information together yourself. to see like huh there's something. happening here and then you jump. onto this little boat of like behavior. like. we should buy the stock or sell the. stock and then another person jumps on.

another. person jumps on and uh all of a sudden. you have. just a huge number of people behaving in. the same direction it's like flock of. whatever birds. exactly what was strange with this one. it wasn't just let's all buy tesla. we love elon we love tesla let's let's. all buy tesla. because that we've heard before right. everybody likes. um tesla well now they do. um so what they did with this in this. case. they're buying a stock that was bad.

they're buying it because it was bad. and that's really weird because that's a. little bit um. too galaxy brain for for a decentralized. community. how did they come up with it how did. they know that was the right one and the. reason they liked it. is because it had really really high. short interest. it had been shorted uh more than its own. uh. float i believe um and so they figured. out that if they all bought this bad. stock. they could short squeeze some hedge.

funds. and those hedge funds would have to. capitulate and buy the stock at really. really high prices. and we should say that shorted means. that these are a bunch of people. when you short a stock you're betting on. the on the. you're predicting the stock's going to. go down and then you will make money if. it does. and then uh what's the short squeeze. it's really that. if you if you are a hedge fund and you. take a big short position. in a in a company um there's a certain. level.

at which you can't sustain holding that. position. right there's no limit to how high a. stock can go but there is a limit to how. low. it can go right so if you short. something you have infinite loss. potential. and if the stock doubles overnight like. gamestop did. um you're putting a lot of stress. on that hedge fund and that hedge fund. manager might have to say. you know what i have to get out of the. trade and the only way to get out. is to buy the bad stock that they don't.

want like they believe will go down. uh so it's an interesting situation. particularly because it's not zero sum. if you say let's make let's all get. together and make a bubble. in watermelons you buy a bunch of. watermelons the price goes up. it comes down again it's a it's a it's a. zero-sum game. if someone's already shorter to stock. and you can make them short squeeze. it's actually a positive sum game so yes. some redditors will make. a lot of money some will lose a lot but. actually the whole group.

will make money and that's really uh. that's really why it's it was such a. clever thing for them to to do. and coupled to the fact that shorting i. mean maybe. you can push back but to me always from. an outsider's perspective. seemed i hope i'm not using too strong. of a ward but it seemed almost unethical. maybe not unethical maybe it's just the. thing to do. it's okay i'm speaking not from an. economics or financial perspective i'm. speaking from.

just somebody who loves i'm a fan of a. lot of people i love celebrating the. success of a lot of people. and this is like the stock market. equivalent of like. haters i know that's not what it is i. know that there's efficient. you you want to have an economy. efficient mechanism for punishing. uh sort of over-hyped overvalued things. that's what shorthand guess is designed. for but just always felt like these. people. are just because they're not just.

betting on the loss of the. company it feels like they're also using. their leverage and power. to manipulate media or just to write. articles. or just to hate on you on social media. and you get. to see that with the almost on so. so this is like the man the. people like hedge funds that we're. shorting are like the. the sort of embodiment of the. evil or just the bad guy the.

overpowerful that's misusing their power. and here's the crowd the people that are. standing up and rising up so. it's not just that they were able to. collaborate. on wall street bets to sort of. effectively make money for themselves. it's also that this is like a symbol of. the people getting together. and fighting the centralized elites the. powerful. and that you know i don't know what your. thoughts are about that in general. at this stage it feels like that's.

really exciting. that people have power. just like regular people have power at. the same time. it's scary a little bit because. you know just studying history people. could be manipulated by charismatic. leaders. and so like uh just like elon right now. is like manipulating. uh encouraging people to buy dogecoin or. whatever uh. the the like that can be good. charismatic leaders and there could be.

bad charismatic legions. and so it's nerve wracking it's a little. bit scary. how much power subreddit can have to uh. destroy somebody because right now we're. celebrating they might be attacking or. destroying somebody that. everybody doesn't like but what if they. attack somebody. that is actually good for this world so. that and that's kind of the. the awesomeness and the. price of freedom is like. it could destroy the world or it can.

save the world but at this stage it. feels like i don't know. overall when you sit back do you think. this was just a positive. wave of emergent behavior is there. something negative about what happened. well yeah the the cool thing is the they. weren't doing anything. the the reddit people weren't doing. anything um exotic. it was an it was a creative trade but it. wasn't exotic. it wasn't it was just buying the stock. okay maybe they bought some options too.

but um it was the hedge fund that was. doing the exotic. thing um so i like that it was. it's hard to say well you know we've got. together. and we've put all pooled all our money. together and now there's a company out. there that's worth more. what's wrong with that yeah right but it. doesn't talk about. you know the motivations which is and. then we destroyed some hedge funds in. the process. is there something to be said about the. the humor. and the i don't know the edginess.

sometimes viciousness of that subreddit. i haven't looked at it. too much but it feels like people can be. quite. aggressive on there uh so is there. what is that is that what is that what. uh freedom looks like. i think it does yeah you definitely need. to let people. the one of the things that people have. compared it to is the occupy wall street. right which is let's say you know some. very sincere. uh liberals like 23 years old.

whatever and they go out with signs and. they they have some kind of case to make. um but this isn't sincere uh. really um it's like um a little bit more. nihilistic. a little bit more yolo um and therefore. a little bit more scary. because who's scared of the who's scared. of the wall street occupy wall street. people with the signs. right nobody but these hedge funds. really are scared i was scared. of the of the wall street bats people. i'm still scared of them.

yeah the anonymity is a bit terrifying. and exciting. yeah i mean yeah i i don't know what to. do with it you know i've been. following events in russia for example. it's like there's a struggle between. centralized power and the distributed. i mean that's the struggle of uh the. history of human civilization right. but this on the internet just that. you can multiply people like. some of them don't have to be real like. you can probably create bots.

like it starts getting me. me as a programmer i start to think like. me is one person. how much chaos can can i create by. writing some bots. yeah and i'm sure i'm not the only. one thinking that uh there's i'm sure. there's the hundreds thousands of uh. good developers out there. listening to this thinking the same. thing and then as that develops. further and further in the next decade. or two what impact does that have on. financial.

markets on just destruction. of uh reputations. of just or politics you know the the. bickering of. left and right political discourse the. dynamics of that being manipulated by. you know the people talk about like. russian bots or whatever. i we're probably in the very early stage. of that right. exactly and uh this is a good example. so do you have a do you have a sense. that most of wall street bets.

folks are actually individual people. right that that's the feeling i have is. there's just individual. maybe young investors just doing a. little bit of an investment but just on. a large scale. yeah exactly the reason i found out what. i've known about. wall street for a while but the reason i. found about gamestop was this. just i met somebody at a party who told. me about it and he was like 21 years old. and he's like it's going to go up 100. in the next one day we're talking about. in last year uh this was probably no. this was yeah a few days ago. yeah it was like maybe um maybe two.

weeks ago or something. um so it was it was already high game. stuff. um but it was just strange to me that. there was someone telling me at a party. uh how to trade stocks who was like 21. years old. um and uh and i started. yeah i started to look into it and um. yeah he and he did make he made. i made 140 in one day uh he was right. and now he's um you know supercharged. he's a little bit wealthier. and now he's gonna look wait for the.

next thing and this decentralized. entity is just gonna get bigger and. bigger and they're gonna together search. for the next thing. so there's thousands of folks like him. and they're going to probably search for. the next thing to attack. people that have power in this world. that sit there with power right now. in government and in finance in. any kind of position are probably a. little bit scared right now. and honestly that's probably a little. bit good. it's dangerous but it's good yeah it. certainly makes you think twice about.

shorting and certainly it makes you. think it's right about putting a lot of. money into a short. like these funds put a lot into one one. or two names. and so it was very very badly risk. managed. do you think shorting is uh can you. speak. at a high level just for your own as a. person is it. good for the world is it good for. markets. i do think that the two kinds of. shorting evil shorting. [Laughter]. and chill shorting okay um evil shorting.

is what melvin capital was doing um. uh and it's like you put a huge position. down. you get all your buddies to also short. it and you start making press. and um and trying to bring this company. down yeah. um and i don't think in some cases. p there's you go out to like fraudulent. companies say this company's a fraud. maybe that's okay like some but but this. they weren't even saying game stuff. they're just saying it's a bad company. and we're gonna bring it to the ground.

bring it to its knees. um a quant fund like numerai. we always have lots of positions and we. never have a position that's like more. than one percent. of our fund so we actually have. right now 250 shorts um i don't know. any of them except for one because. it was one of the meme stocks but. yeah but it we shorting them.

not to make them go we don't even want. them to go down. necessarily yeah that doesn't sound a. bit strange did i say that but. we just want them to to not go up as. much as our longs. right so by shorting. a little bit we can actually go along. more in the things we do believe in. so when we were going long in tesla we. could do it with more money than we had. because we would borrow from banks who. would lend us. money because we had longs and shorts.

because we didn't have market exposure. we didn't have market risk. and so i think that's a good thing. because that means. um you know we can short the oil. companies and go along tesla and make. the future come forward faster. and i do think that's not a bad thing. so we talked about this incredible. distributed system. created by wall street bets and then. there's a platform. which is robinhood which allows the. investors to efficiently as far you can. correct me if i'm wrong but you know.

there's those and there's others and. there's numerai that. allow you to make it accessible for. people to. invest but that said. robin hood was uh in a centralized way. applied its power to restrict trading on. the stock that we're referring to. uh do you have a thought on actually. like all. the things that happened i don't know. how much you were paying attention. uh to sort of the shadiness. around the whole thing do you think it.

was forced to do it or was there. something shady going on. what are your thoughts in general well i. think i i want to see the alternate. history. like i want to see the counter factual. history. of them not doing that not doing it how. bad would it have gotten. for hedge funds how much more damage. could have been done. if the momentum of these short squeezes. could continue. um what happens when there. are short squeezes uh even if um.

they're in a few stocks they affect kind. of all the other shorts too. and suddenly um brokers are saying. things like you need to put up more. collateral. so we had a short it wasn't gamestop. luckily it was blackberry and it went up. like 100 in a day. it was one of these meme stocks super. bad company the ais don't like it. okay they always think it's going down. what's a meme stock. a meme stock is kind of a new term for. these stocks that. catch memetic momentum on reddit.

yeah um and so the meme stocks were. gamestop the biggest one game stonk as. elon calls it. amc um. and blackberry was one nokia was one. so these are high short interest stocks. as well so these are targeted stocks. that. some people say oh isn't it isn't it. adorable that these um. these people are investing money in. these companies that are. you know nostalgic it's like you're.

going to the amc. movie theater it's like nostalgic it's. like no it's not why they're doing it. it's that they had a lot of short. interest that was the main thing. and so they were high chance of short. squeeze. in saying i would love to see an. alternate history do you have a sense. that that. what is your prediction of what that. history would have looked like well. you wouldn't have needed very many more. days of that kind of chaos. to to hurt hedge funds. um i think it's underrated how how.

damaging it could have been. uh because when your shorts. go up your collateral requirements for. them go up. similar to robin hood like we have a. prime broker. that says said to us uh you need to put. up you know like forty dollars. per per hundred dollars of short. exposure. and then the next day they said actually. you have to put up you know. all of it 100 and we were like what. um but if that happens that if that.

happens to. all the short all the commonly held. hedge fund shorts. because they're all kind of holding the. same things if that happens. not only do you have to cover the short. which means you're buying the bad. companies you need to sell your good. companies. in order to cover the short right so. suddenly like. all the good companies all the ones that. the hedge funds like are coming down. and all the all the ones that the hedge. funds hate are going up.

in a cascading way. so i believe that if you could have had. a few more days. of game stock doubling amc doubling. you would have had more and more hedge. fund deleveraging. but so hedge funds i mean they get a lot. of but. they do you have a sense that they do. some good for the world. i mean ultimately so okay first of all. wall street bets itself is. a kind of distributed hedge fund new mri. is a kind of hedge fund.

so like a hedge fund is a very broad. category i mean. like if some of those were destroyed. would that be good for the world. or is it would there be coupled. with the the destroying the evil. shorting would there be just a lot of. pain in terms of investment. in good companies yeah a thing i like to. tell people. if they hate hedge funds is i don't. think you want to re-run. american economic history without hedge. funds. so so on mass they're they're they're.

good yeah. yeah they're good yeah you really. wouldn't want to uh because hedge funds. are kind of like. picking up um they're making liquidity. right in stocks. and so if you'd like if you love venture. capitalists. they're investing in new technology it's. so good you have to also kind of like. hedge funds. because they're the reason venture. capitalists exist. because their companies can have a. liquidity event when they go to the. public markets. so it's kind of essential that we have.

them there are many different kinds of. them. i believe we could maybe get away with. only having an. ai hedge fund. but we don't necessarily need these evil. billions type hedge funds that make the. media and try to kill companies but we. definitely need hedge funds. maybe from your perspective because you. run such an organization. and uh vlad the ceo of robin hood.

sort of had to make decisions really. quickly probably had to wake up in the. middle of the night kind of thing. uh you know and he also had a. conversation with elon musk. on clubhouse which i just signed up for. it was. it was a fascinating one of the great. journalistic uh performances of our time. with the. elon musk surprise for eve how hilarious. would it be if he gets a pulse. and then his wikipedia be like. journalist and uh.

as you know i don't know if you can. comment on any aspects of that but like. if you were vlad how would you do things. differently what are your just. thoughts about his interaction with elon. how he should have played it. differently like i guess there's a lot. of aspects to this interaction one is. about. transparency like how how much do you. want to tell people about really what. went down. there's ndas potentially involved uh. how much on in private.

do you want to push back and say no. you. to centralize power whatever the phone. calls you're getting which i'm sure. he's getting some kind of phone calls. that might not be contractual. like it's not contracts that are forcing. him but he was being. uh what do you call it like pressured. to behave in certain kinds of ways from. all kinds of directions like what. uh what do you take from this whole. situation. i was very excited to see vlad response. i mean it's pretty cool to have him talk.

to. elon right and one of the things that. like struck me in the first like few. seconds of vlad. speaking was like i was like. is vlad like a boomer like. like but here we are like he seemed like. a 55 year old man. talking to a 20 year old yeah elon was. like the 20th. yeah and he's like the 55 year old man. you can see why citadel are nmr buddies. right like you can yeah you can see why. it's like this is a this is a nice. reason. that's not a bad thing it's like he's.

it's like a he's got a respectable. professional attitude well he he also. tried to do like a jokey thing. like no we're not being ageist here. boomer uh but like you like uh like a 60. year old. ceo of bank of america would try to make. a joke for the kids that's what vlad. said exactly. yeah yeah i was like what is this this. guy's like what is he 30. yeah uh and i'm like this is weird. yeah um but i think maybe that's also. what i like about.

elon's kind of influence on american. business is like he's super like. anti the professional right like why. why say why say you know 100 words. about nothing and so i liked how he was. cutting in and saying flat what do you. mean spill the beans bro. yeah so you don't have to be courteous. it's like the first principles thinking. it's like what the hell happened yes and. let's just. talk like normal people the problem of. course is. uh you know for elon uh it's cost them.

what is it tens of millions of dollars. is tweeting like that. but perhaps it's a worthy price to pay. because ultimately there's something. magical about. just being real and honest and just. going off the cuff and making mistakes. and paying for them but just being. real and then moments like this that was. an opportunity for vlad to be that. and he felt like he wasn't do you think. there. do you think we'll ever find out what.

really went down if there was something. shady. underneath it all yeah i mean it would. be sad if nothing shady happened. right but his presence made it shady. sometimes i feel like. that with mark zuckerberg the ceo of. facebook. sometimes i feel like yeah there's a lot. of shitty things that facebook is doing. but sometimes i think he makes it look. worse by the way he presents himself. about those things like i i honestly. think that a large amount of people at.

facebook. just have a huge unstable chaotic system. and they're all. not all but mass are trying to do good. with this chaotic system. but the presentation is like it sounds. like there's a lot of. uh back room conversations that are. trying to manipulate people. uh and there's there's something about. the realness that elon has. that it feels like ceo should have and. vlad had that opportunity. i think mark zuckerberg had that too. when he was younger younger.

and somebody said you got to be more. professional man you can't say. you know lol to an interview. uh and then suddenly he became like this. distant. person that was hot i'd like you'd. rather have him make mistakes. but be honest than be like professional. never make mistakes. yeah one of the difficult tires i think. is like marketing people or like pr. people. is you have to hire people that get the. fact that you can say lol on an. interview.

or i you know take risks. as opposed to what the pr i've talked to. quite a few big ceos and. the people around them are trying to. constantly minimize. risk of like what if he says the wrong. thing what if she says the wrong thing. it's like what be careful it's. constantly like oh. like i don't know and there's this. nervous energy that builds up over time. with larger larger teams. where the whole thing like i visited.

youtube for example everybody had talked. at youtube. incredible engineering an incredible. system but everybody's scared. like let's be uh let's be honest about. this. like madness that we have going on of. huge amounts of video that we can't. possibly ever handle there's a bunch of. hate on youtube. there's this chaos of comments a bunch. of conspiracy theories some of which. might be true. and then just like this mess that we're. dealing with and it's exciting it's. beautiful. uh it's a place where like democratizes.

education all that kind of stuff. and instead they're all like sitting in. like trying to be very polite and saying. like well we're just uh want to improve. the health of our platforms like. it's like this yeah discussion like all. right man let's just be real let's. let's let's both advertise how amazing. this freaking thing is. but also to say like we don't know what. we're doing we have all these nazis. posting videos on youtube. we don't we don't know how to like. handle it and just being real like that.

i suppose that's just the skill maybe it. can't be taught. but over time the whatever the dynamics. the company is it does seem like. zuckerberg and others get. worn down they just get tired yeah uh. they get tired of. not being real of not being real which. is uh sad. so let's talk about numerai which is an. incredible. company uh system idea. i think but good place to start. what is numerai and how does it work.

so numero is the first hedge fund that. gives away. all of its data so this is like probably. the last thing. a hedge fund would do right why would we. give away a data it's like giving away. your edge. um but the reason we do it is because. we're looking for people. to model our data and the way we do it. is by obfuscating the data. so when you get when you look at. numerized data that you can download for. free. it just looks like like a million rows. and of.

numbers between zero and one and you. have no idea what the columns mean. but you do know that if you're good at. machine learning or have done. regressions before you know that i can. still find patents on in this data even. though i don't know. i don't know what the features mean and. and the data itself is time series data. and even though it's obfuscated. anonymized. what is the source data like. approximately what are we talking about. so we are buying. data from lots of different data vendors.

um. and they would also never want us to. share that data. um so we have strict contracts with them. so we only. we only can but it but it's the kind of. data you could never buy yourself. unless you had maybe a million dollars a. year. of budget to buy data so what's happened. with the hedge fund industry is. you have a lot of talented people. who used to be able to trade. and still can trade but now they have.

such a data disadvantage. it would never make sense for them to um. to. to trade themselves but numerai by. giving away this obfuscated data we can. give them a really really high quality. data set that's. that would otherwise be very expensive. and they can use whatever. new machine learning technique they want. to find patterns in that data that we. can use in our hedge fund and so. how much variety is there in underlying. data we're talking about. uh i apologize if i'm using the wrong.

terms but there's. one is just like the stock price the. other there's like options and all that. kind of stuff like the what are they. called order books or. whatever like i is is there. maybe other totally unrelated to. directly to the stock market data. like like the natural language as well. all that kind of stuff. yeah we were really focused on um stock. data that's specific to stocks so um. things like you can have like a p every.

stock has like a p e ratio. for some stocks it's not as meaningful. but every stock has that. every stock has one year momentum how. much they went up in the last year. um but those are very common factors but. we try to get. lots and lots of those factors that we. have for many many years like 15. 20 years history and. and then the setup of the problem is. commonly in quant called like. cross-sectional global equity. you're not really trying to say i want i. i believe this stock will go up.

you're trying to say um the like. relative position of this stock. in feature space uh makes it not a bad. buy. in a in a portfolio so it captures some. period of time. and you're trying to find the patterns. the dynamics captured by. the data of that period of time in order. to make short-term predictions about. what's going to happen. yeah so our predictions are also not. that short we're not really um. caring about things like order books and. and tech data.

not high frequency at all we're actually. holding things for quite a bit longer. so our prediction time horizon is about. one month we end up holding stocks for. maybe like three or four months. so i kind of believe that's a little bit. more like investing. than um then kind of plumbing. like to go long a stock that's mispriced. on one exchange and. shorter on another exchange that's just. arbitrage. but what we're trying to do is really. know know. know something more about the longer.

term future of the stock yeah so from. the patterns from these like periods of. uh time series data you're trying to. understand something fundamental about. the stock not. like about deep value about like with. this big. in the context of the markets like. underpriced overpriced all that kind of. stuff. so like this is about investing it's not. about like. just like you said high frequency. trading which i think is a. fascinating open question for a machine. learning perspective but just to like. sort of build on that.

so you've anonymized the data and now. you're giving away the data. and then now anyone can. try to build algorithms that. make investing decisions on top of that. data or predictions on the top of that. data. exactly and so that that's um. what is that so what does that look like. what's the goal of that what are the. underlying. principles of that so the first thing is. you know. we could obviously model that data. in-house right we can make.

an xg boost model on the data um. and that would be quite good too but. what we're trying to do is by by opening. it. up and letting anybody participate we. can do. quite a lot better than if we modeled it. ourselves. and a lot better on the stock market. doesn't need to be. very much like it really matters the. difference between if you can make. 10 and 12 in an equity market neutral. hedge fund. because the whole usually you're trying.

you're charging two percent. fees so if you can do two percent better. that's like all your fees it's worth it. so. we're trying to make sure that we always. have the best possible model as new. machine learning libraries come out. new new techniques come out they get. automatically synthesized like if. there's a great paper. on supervised learning someone on. numerix will figure out how to use it on. numerized data. and is there an ensemble of. uh models going on or is it always.

or is it more towards kind of like one. or two or three. like best performing models so the way. we decide on how to weight. all of the predictions together is. um by how much the users are staking on. them. oh yeah how much of the cryptocurrency. that they're putting behind their models. so they're saying. i believe in my model tr you can trust. me because i'm going to put. skin in the game um and so we can take. the stake weighted predictions from all. our users. add those together average those.

together and that's a much better model. than any one model. in the in the sum because ensembling a. lot of models together is kind of the. key thing you need to do in investing. too. yeah so you're putting there's the kind. of duality from the user. from the perspective of a machine. learning engineer you're. it's both a competition just a really. interesting difficult machine learning. problem. and it's uh a way to to invest. algorithmically so like here and but.

the the way to invest algorithmically. also is a. way to put skin in the game that. communicates to you. that your uh the the quality of the. algorithm. and also forces you to really uh. be serious about the models that you. build. so it's like everything just works. nicely together like. um i guess one way to say that is the. interests are aligned. exactly okay so uh it's just like poker.

is not not fun when it's like for very. low stakes. but the higher the stakes the more the. dynamics of the system starts playing. out correctly. uh like as a small side note is there. something you can say about which kind. looking at the big broad view of machine. learning today or ai. what kind of algorithms seem to do. good in these kinds of competitions at. this time. is there some universal thing you can. say like.

neural networks suck uh recurring you'll. know or suck transformers suck. or they're awesome like old-school. sort of more basic kind of classifiers. are better all the is there is there. some kind of conclusion. so far that you can say there is there. definitely something. pretty nice about tree models like like. xg. boost um and uh they just seem to work. pretty nicely on this type of data. uh so out of the box if you're trying to. come. 100th in the competition in the.

tournament maybe you would try it use. that. um but what's what's. particularly interesting about the the. problem that um. not many people understand if you're. familiar with machine learning. um this typically will surprise you when. you model our data. so um one of the things that. you look at in finance is you don't want. to be too exposed to any one. risk like even if.

the best sector in the world to invest. in over the last 10 years was. tech you would not does not mean you. should put all of your money into tech. right so though if you train a model it. would say put all your money in stack. it's super good. but um what you want to do is actually. be very careful of how much. of this exposure you have to certain. features. so on numerix what a lot of people. figure out is actually if you train a. model.

on this kind of data you want to somehow. neutralize or minimize. your exposure to these to certain. features which is unusual because if. if you did train um a stop light or stop. street. uh detection uh on computer vision. uh your favorite feature let's say you. could and you have an auto encoder and. it's figuring out okay it's going to be. red and it's got to be white. that's the last thing you want to be you. want to reduce your exposure to.

why would you reduce your exposure to. the thing that's helping you your model. the most. and that's actually this. counter-intuitive thing you have to do. with machine learning on financial data. so reducing it's reducing your exposure. would help you generalize the things. that are. so basically financial data has. a large amount of patterns that appeared. in the past. and also a large amount of patterns that. have not appeared in the past. and so like in that sense you have to. reduce the exposure to red lights. uh to to the color red that's.

interesting but. how much of this is art and how much of. it is science from your perspective so. far. in terms of as you start to climb from. the 100th position to the 95th. in in the competition yeah well if you. do you. make yourself super exposed to uh. one or two features you can have a lot. of volatility. when you're playing numeri you could. maybe very rapidly. rise to be high if you were getting.

lucky yes and that's bit like the stock. market. sure take on massive risk exposure put. all your money into one stock. and you might make 100 but um. it doesn't in the long run work out very. well. and so um the best users are. are trying to stay high for as long as. possible and not not necessarily try to. be. first for a little bit so me.

a developer machine learning researcher. how do i lex freeman participate in this. competition and how do others. i'm sure there'll be a lot of others. interested in participating in this. competition. what are let's see there's like a. million questions but like. first one is how do i get started. well you can go to numer.ai sign up. download the data and on. the data is pretty small in the data.

pack you download. there's like an example script python. script. that just builds a xg boost model very. quickly from the data. and um so in a very short time you can. have an example. model is that a particular structure. like what uh is this model then. submitted somewhere so this. needs to be some kind of structure that. communicates with some kind of api. like how does the whole yeah how does. the your model once you've built it. wants to create a little. little baby frankenstein yeah how does.

it then live in it's okay well we want. you to keep your baby frankenstein at. home. and take care of it we don't want it. okay so we you never. upload your model to us you always um. only giving us predictions so we never. see the code that wrote your model which. is pretty cool yeah that our whole hedge. fund is built. from models where we've never ever seen. the code um. but it's important for the users because. it's their ip. why they want to give it to us that's. brilliant so they've got it.

themselves but they can basically almost. like. license the the predictions from that. yeah um so think about it what some. users do is they set up a. uh compute server and we call numeric. compute it's like a little aws. kind of image and you can automate this. process um so we can ping you we can be. like we need more predictions now. and then you you send it to us okay cool. so that that's. uh is that described somewhere like what.

the preferred is the aws or whether. another cloud platform is there i mean. is there sort of. specific technical things you want to. say that comes to mind that. uh is a good path for getting started so. download the data. maybe play around see if. you can modify the basic uh. the algorithm provided in the. example and then you what set up a. little server on the aws. that then runs this model and takes.

pings and then makes predictions. and uh how does your own money actually. come into play. doing the stake of cryptocurrency. yeah so you don't have to stake you can. start without staking and many users. might. try for months uh without staking. anything at all to see if their model. works on the real life data right and is. not over fit. um but then you can get numeraire.

uh many different ways you can buy it on. um. you can buy some on coinbase you can buy. some on uniswap you can buy some on. binance. um so what what did you say this is uh. how do you pronounce it so this is the. numerai. uh cryptocurrency yeah nmr. nmr what's you just say nmr it is. it is technically called numero numero. i like it yeah but uh and more simple. nmr numero okay so and you could buy it. uh you know basically anywhere yeah so.

it's a bit strange because sometimes. people. are like is this like pay to play right. and it's like. sword it's like yeah yeah you need to. put some money down to show us you. believe in your model. but uh weirdly we we're not selling you. the cr like you can't buy the. cryptocurrency from. us right it's like it's also we never. if you're if you do badly um we destroy. your cryptocurrency. okay that's not good right you don't. want it to be destroyed but.

what's good about it is it's also not. coming to us. right so it's not like we win when you. lose or something like that. like we're the house like we're. definitely on the same team. yes you're helping us make a hedge fund. that's never been done before. yeah so again interests are aligned. there's no uh there's no tension there. at all which is. which is really fascinating you're. giving away everything and then the ip. is owned by. the sort of the the code you never share. the code that's fascinating. um so since i have you here and you said. a hundredth i didn't ask.

out of how many so we'll just. but if i if i then once you get started. and you find this. interesting how do you then. win or do well but also how do you. potentially try to win if this is. something you want to take on seriously. from the machine learning perspective. not from a financial perspective. yeah i think um the first of all you. want to talk to the community people are. pretty open. uh we give out really interesting.

scripts and ideas. for things you might want to try um and. uh but you're also going to need a lot. of compute probably. and so some of the best users are are. you know actually the very first time. someone won. on numerix i would i wrote them a. personal emails like you know you've won. some money. we're so excited to give you three. hundred dollars and then they said i. spend way more on the compute. [Laughter]. um but this is fundamentally a machine. learning problem first i think. is this is one of the exciting things i.

don't know if we'll in. how many ways you can approach this but. really this is. less about kind of. no offense but like finance people. finance minded people they're. also i'm sure great people but it. feels like from the community that i've. experienced these are people who see. finance as a fascinating uh. problem space the source of data but. ultimately they're machine learning. people or.

ai people which is a very different kind. of flavor of community and. i mean i i should say to that uh i'd. love to participate in this. and i will participate in this and i'd. love to hear from other people if you're. listening to this if you're a machine. learning person. you should participate and tell me uh. give me some hints. um how i can do well at this thing. because this boomer uh. i'm not sure i still got it but because. some of it is uh it's like. kaggle competitions like some of it.

is certainly a set of ideas like. research ideas. and like fundamental innovation but i'm. sure some of it is like deeply. understanding getting like an. intuition about the data and then like a. lot of it will be like. figuring out like what works like tricks. i mean. you could argue most of deep learning. research is just tricks on top of tricks. but. there is uh some of it is just the art. of getting to know. how to work on a really difficult. machine learning problem.

and i think what's important the. important difference with something like. a kaggle competition. where they'll set up this kind of toy. problem. and then there will be an out of sample. test like hey you did well out of sample. and this is like okay. cool um but what's cool with nimrod is. you're you're the out of sample is the. real live stock market. we we don't even know like we don't know. the answer to the problem well. we don't like you'll have to find out. live and so we've had. users who've like submitted every week.

for for like. four years um because it's kind of a. interesting we say it's the hardest data. science problem on the planet right. and it sounds maybe sounds like maybe a. bit too much for like a marketing thing. but. it's the hardest because it's the stock. market it's like. literally they're like billions of. dollars at stake. and like no one's like letting it be. inefficient on purpose. so if you can find something that works. on numerous you really have something. that.

that is like working on the real stock. market. yeah because there's like humans. involved in the stock market i mean this. uh you know you could argue there might. be harder data sets like maybe. predicting the weather all those kinds. of things but. the the fundamental statement here is. which i like i was thinking like is this. really the hardest data science problem. and you start thinking about that but. ultimately also boils down to. a problem where the data is accessible. it's made accessible made. really easy and efficient at.

like submitting algorithms so it's not. just. you know it's not about the data being. out there like the weather it's about. making the data super accessible. making a building a community around it. like this is what imagenet did. exactly like it's not just there's. always images. the point is you aggregate them together. you give it a little title. there's a community and that's that was. one of the hardest right for a time. and most important data science problems.

in the world uh because it was. accessible because it was uh. made uh sort of like. there's uh mechanisms by which like. standards and mechanisms about which to. judge your performance all those kinds. of things and numerize actually. step up from that is there something. more you can say about. why from your perspective it's the. hardest uh. problem in the world i mean you said. it's connected to the market so if you. can find. a pattern in the market that's a really.

difficult thing to do because a lot of. people are trying to do it. exactly but there's also the the biggest. one is it's. it's non-stationary time series. we've tried to regularize the data so. you can find. patterns by doing certain things to the. features and the target. but ultimately you're in a space where. you don't. there's no guarantees that the. out-of-sample distributions will. conform to any of the training data. and and every single um.

era which we call on the website like. every single era in the data which is. like sort of showing you the order of. the time. um it's it's even the training data is. has these same. same dislocations and um so. yeah it's and so and then there's yeah. there's so many things that might. um might you might want to try this this. like there's unlimited possible number. of models right. um and so.

by by having it um be. open uh we can at least search that. space. it's zooming back out to the. philosophical you said that numerai is. uh. very much like wall street pets uh. is is there i i think it'd be. interesting to dig in why you think so. i think you're speaking to the. distributed nature of the two. and the power of the people nature of. the two. so maybe can you speak to the.

similarities and the differences. and in which way is nimra more powerful. in which way. is wall street bets more powerful yeah. this is why the wall street bet story is. so interesting to me because it's like. feels like we're connected. yeah um and looking at how just looking. at the forum of wall street vets it's. i was talking earlier about how how can. you make credible claims. you're anonymous okay well maybe you can. take a screenshot how. or maybe you can upvote someone maybe. you can have karma on reddit.

and those kinds of things make this. emerging thing possible. numerai it didn't work at all. when we started it didn't work at all. why. people made multiple accounts they made. really random models and hoped they. would get lucky. and some of them did yes staking. was our like solution to could we. could we make it so that we could trust. we could know which model people. believed in the most.

and we could wait models that had high. stake. more and effectively coordinate this. group of people. to be like well actually there's no. incentive to creating bot accounts. anymore. either i stake my accounts in which case. i should believe in them. because i could lose my stake or i don't. and that's a very powerful thing. uh that having a negative incentive and. a positive incentive. can make can make things a lot better. and staking is like this. is this really nice like key thing about.

blockchain. it's like something special you can do. where they're not even trusting. us with their stake in some ways they're. trusting the blockchain. right um so the incentives like you say. it's about making these perfect. incentives. so that you can have coordination to. solve one problem and. nowadays i i sleep easy. because i have less money in my own. hedge fund. than our users are staking. on their models that's powerful in some.

sense from a human psychology. perspective it's fascinating that the. wall street bets worked at all right. that the the amidst that chaos emerging. behavior. like behavior that made sense emerged it. would be. fascinating to think if numerai style. staking could then be transferred to. places like reddit you know and not. necessarily for financial investments. but. uh like i wish sometimes people.

would you know would have to stake. something. in the comments they make on the. internet yeah. like that's that's the problem with. anonymity is like. anonymity is freedom and power that you. don't have to you can speak your mind. but it's too easy to just be shitty. exactly and so this the i mean you're. making me realize from. like a profoundly philosophical. aspect numerous staking is a really. clean way to solve this problem.

it's it's a really beautiful way of. course it only with numerai currently. works for a very particular problem. right not for human interaction on the. internet but. that's fascinating yeah there's nothing. for to stop people in fact we've open. sourced like the code we use for staking. in a protocol we call erasure. um and any if reddit wanted to they. could even use that code to do. have have enabled staking on um. our wall street pets and they're. actually researching now they've had.

some ethereum grants. on how could they have more crypto stuff. in there in ethereum because wouldn't. that be interesting. like imagine you could um instead of. seeing a screenshot. like guys i promise i will not uh. sell my gamestop we're just going to go. huge. we're not going to sell at all. and here is a smart contract. which no one in the world including me. can.

undo that says my i have staked. uh millions against this claim. um that's powerful and then what could. you do. and of course it doesn't have to be. millions it could be just very small. amount but then. just a huge number of users doing that. kind of stake exactly. that you know that could change the. internet it would change and demand wall. street. they would not they would never have. been able to they would still be short. squeezing. one day after the next every single.

hedge fund collapsing. if we look into the future do you think. it's possible that numerai. style infrastructure where ai systems. backed by humans are doing the trading. is what the entirety of the the stock. market is or the entirety of the economy. is run by basically. this army of ai systems with. high level human supervision yeah the. thing is. that some of them could be could be bad.

actors. um some of the humans no well these. systems could be. tricky so actually i once met a hedge. fund manager this is kind of interesting. he said. um very famous one and he said um. we can see sometimes we can see things. in the market where we. know we can make money but it will mesh. it up. yeah we know we can make money but it. will mess things up. and we choose not to do those things and. on the one hand maybe this is like oh. you're being super arrogant like.

of course you can you can't do this but. maybe he can. and maybe he really isn't doing things. he knows he could do but would. change you know be pretty bad would the. reddit army. have that kind of uh morality. or concern for what they're doing. probably not based on what we've seen. the madness of crowds. there'll be like one person that says. hey maybe.

and then they get trampled over uh. that's. that's the terrifying thing actually. this uh. a lot of people have written about this. is somehow that like. little voice that's human morality gets. silenced when we get into groups and. start chanting. yeah and that's terrifying but like i. think. uh maybe i misunderstood i thought that. um. you're saying ai systems could be. dangerous but you just describe how. humans can be dangerous. so which is safer so i mean one thing is.

uh numero yeah so wall street bets these. kinds of like. these kinds of attacks like it's not. possible to. to model numerized data and then come up. with the idea from the model. let's short screen gamestop right it's. not even framed in that way it's not. like possible to have that idea. so but it is possible for like kind of a. bunch of humans so. i think there's it's numerix could get. very powerful. uh without it being dangerous but.

wall street bets needs to get a little. bit more powerful. and it'll be pretty dangerous yeah well. i mean uh so. this is a good place to kind of think. about mri data today. uh and umri signals and what that looks. like in 10. 20 30 50 100 years you know. like right now i guess maybe you can. correct me but this the data that we're. working with is like a window. it's a you know anonymized obfuscated.

window into a particular aspect a time. period of the market. you know you can expand that more and. more and more and more potentially. you can imagine in different dimensions. to where it encapsulates. all the things that uh where you could. uh include kind of human to human. communication. that was available for like uh to buy. gamestop for example. on on wall street bets so maybe it's a. step back can you. speak to what is numerized signals.

and uh what are the different data sets. that are involved. so with numeric signals um you're still. providing predictions to us. but you can do it from your own data set. so numeraid's all you have to model our. data to come up with predictions. numerous signals is whatever data you. can find. out there you can turn it into a signal. and give it to us. so it's a way for us to import signals.

on data. we don't yet have and. and that's why it's particularly. valuable because it's. going to be signals you're only rewarded. for signals that are. orthogonal to our core signal. so you have to be doing something. uncorrelated and so. strange alternative data tends to have. that property. there isn't too many other signals that. are correlated with. um with uh you know what's happening on.

wall street bets. that's not going to be like correlated. with the price to earnings ratio right. um and we have some users as of recently. as of like a week ago. there was a user that created i think. he's in india. he created a signal. that is scraped from wall street bits. and now we have that signal uh as. one of our signals in thousands. that we use at numerai and the structure. of the signal is similar so this is just.

numbers and time series data. it's exactly and it's just like it's. kind of a you're providing a ranking of. stocks so you just say. give it a one means you like the stock. zero means you don't like the stock. and you provide that for five thousand. stocks in the world. and they somehow converted the the. natural language that's. in the wall street so they've come. exactly so there's they made they open. source this collab notebook. uh you can go and see it and look and so. yeah it's taking that. making a sentiment score and then.

turning it into a rank. sentiment score yeah uh like this stock. sucks so the stock is awesome. uh and then converting that's that's. fast just even looking at that data. would be. fascinating so on the signal side what's. the. vision it's long term what do you see. that becoming. so we want to manage all the money in. the world. that's numerized mission and to get that. we need to have all the data. and have all of the talent like there's.

no way. for the first principles if you had. really good modeling and really good. data. that you would lose right it's just a. question of how much do you need to get. to get really good so numerai already. has some really nice data that we give. out. this year we are 10xing that. and i actually think we'll 10x the. amount of data we have on numerix. every year for at least the next 10. years. wow so it's going to get very big the. data we give out. and signals is more.

data people with any other random data. set. can turn that into a signal and give it. to us and in some sense that kind of. data is the edge cases the weirdness is. the. so you're focused on like the bulk yeah. like the main data and then there's just. like weirdness from all over the place. that just can enter through this. back door exactly and the it's. it's also um a little bit shorter term. so. right the signals are set about a seven. day time horizon.

and the on numeric it's like a 30 day so. it's often for faster situations. you've written about a master plan and. you've mentioned which i love. uh in a similar sort of style of big. style thinking. you would like numerai to manage all of. the world's. money uh so how do we get there. from from yesterday to. several years from now like what's uh.

what is the plan. so you've already started to allude to. it it's like get all the data and get it. uh. yeah all the talent yeah humans. models exactly i mean the important. thing to note there is. what would that mean right and i i think. the biggest thing it means is like uh. if there was one hedge fund um you would. have. uh not so much talent wasted. on all the other hedge funds like it's. super weird how the industry works.

it's like one hedge fund gets a data. source and hires a phd. and another hedge fund has to buy the. same data source and hire a phd. and suddenly a third of american phds. are working at hedge funds. and we're not even on mars and like. so in some ways numerix it's all about. freeing up. people who work at hedge funds to go. work for elon. yeah and and also the people who are. working. on numeraid problem it feels like. a lot of the knowledge there is also.

transferable to other domains. exactly our top one of our top users is. uh he works at nasa. jet propulsion lab yeah and he's he's. like amazing i went to go visit him. there. and it's like he's got like numerous. posters and he's like it's like it looks. like you know. the movies like it looks like apollo 11. or whatever. yeah the point is he didn't quit his job. to join full-time. he's working on getting us to jupiter's. moon that's his mission the europa clip.

emissions. actually literally what you're saying. literally we he's smart enough that we. really want. his intelligence to reach the stock. market because the stock market's a good. thing hedge funds are a good thing. all kinds of hedge funds especially but. we don't want him to quit his job. so he can just do numerous weekends and. that's what he does he just made a model. and it just automatically submits to us. and he's like one of our best users you. mentioned. briefly that stock markets are good for. my sort of outsider perspective.

is there a sense do you think trading. stocks. is uh closer to gambling or is it closer. to. investing sometimes it feels like it's. gambling. as opposed to betting on companies to. succeed and this is maybe connected to. our discussion of shorting in general. but. like from your sense the way you think. about it is it fundamentally still. investing. i do think um i mean it's a good. question. i it's i've also seen lately.

people say this is like speculation is. there too much speculation in the market. and it's like. but all the trades are speculative like. all the trades have a horizon. like people want them to work um. uh so i i i would say that. um and there's certainly a lot of. aspects of. gambling uh math that applies to. investing okay. like one thing you don't do in gambling. is put all your money in one. bet you have bankroll management and.

it's a key part of it. and small alterations to your bankroll. management. might be better than improvements to. your skill. um so there and then there are things we. care about in our fund like we want to. make a lot of independent. bets we talk about it like we want to. make a lot of independent bets because. that's going to be um. a higher sharp than if you have a lot of. bets that depend on each other like all. in one sector. um but but yeah i mean the point the.

the point is that you want the prices of. the stocks to be. to be reflective of of how. of their value of the underlying value. yeah yeah you shouldn't have. there be like a hedge fund that's able. to say well. um i've looked at some data and all of. this stuff's super mispriced. like that's super bad for society if. it's if it looks like that to someone. i guess the the underlying question then. is uh. do you see that the market often like.

drifts away. from the underlying value of companies. and it becomes a game in itself. like with these whatever they're called. like derivatives like the option like. you know um. options and uh shorting and all that. kind of stuff. it's like layers of game on top of the. actual. like what you said which is like the. basic thing that lost your bets was. doing which is like just buying stocks. yeah there are a lot of games that.

people play. that are um in the derivatives market. and. i think a lot of the stuff people. dislike when they look at the history of. of what's happened they hate like credit. default swaps. or collateralized debt obligations like. these are the. these are the kind of like enemies of. 2008. and then the long-term capital. management thing it was like. it was like that 30 times leverage or. something just. that no one like you could just go to um.

a gas station and ask anybody at the gas. station. is it a good idea to have 30 times. leverage and they just say no. the common sense just like went out the. window so. the i yeah i don't i don't i don't. respect. long-term capital management. okay but numero doesn't actually use any. derivatives. unless you call shorting derivative uh. we just we do put money into companies. we and that does help the companies.

we're investing in it's just in little. ways. um we we really did buy tesla and it and. it did. and we were you know we were not um we. played some role in and that's it's. success. super small make no mistake but still i. think that's important. can i ask you uh a pod head question. which is uh. what is money man so. if we just kind of zoom out and look at. i said let's talk to you about.

cryptocurrency which perhaps could be. the future of money. in general how do you think about money. he said numero. the division the goal is to to run. to manage the world's money what is. money. in your view i don't have a a good. answer to that. but it's definitely in my personal life. it's become. more and more uh warped um. and you start to care about the real.

thing. like what's really going on here um. elon has talks about things like this. like what's what is a company. really it's like it's a bunch of people. who like kind of show up to work. together and like. they solve a problem and they might not. be a stock out there that. that that's trading that represents that. what they're doing but it's it's not the. real thing. um and being involved in crypto um. like i would early put in crowd sale of. of ethereum.

and uh and all these other things and. different. crypto hedge funds and things that i've. invested in and it's like. it's just it's just kind of like it. feels like. how i used to think about money stuff is. just just like totally warped. um because you yeah you kind of. you stop caring about like the like. the price and you care about like the. product. so but by the product you mean like the. different mechanisms that money is. exchanged.

i mean money is ultimately a kind of. little you know. on one it's a store of wealth but it's. it's also a mechanism of exchanging. wealth. but that like what what wealth means. becomes a totally different thing. especially with. uh cryptocurrency to where it's almost. like these little contracts these little. agreements these transactions between. human beings that represent. something that's bigger and just like. cash being exchanged 7-11 it feels like. yeah. maybe answer what is finance uh like.

you it's what are you doing when you can. when you have the ability to to take out. a loan. you can bring uh a whole new future into. being. with finance uh if you couldn't get a. student loan. to get a college degree you couldn't get. a college degree like if you didn't have. the money. but now like weirdly you can get it with. and like yeah all you have is this like.

loan which is like so now you can bring. you can bring a different future into. the world and that's how i when i was. saying earlier about if you rerun. american history. economic history without these these. these things like. you're not allowed to take out loans. you're not allowed to have have. derivatives you're allowed to have money. um it would just it just doesn't really. work and it's a really magic thing. how how how much you can do with finance. by kind of bringing the future. forward finance is empowering it's uh we. sometimes forget this but yeah it. enables innovation it enables big risk.

takers and bold. builders that ultimately make this world. better you said you were early. in on cryptocurrency can you give your. high-level overview of. just your thoughts about the past. present and future of. cryptocurrency um yeah so my friends. told me about bitcoin and i. i was interested in um equities a lot. and i was like well. it has no net present uh value it has no. future cash flows. bitcoin pays no dividends um.

so i really couldn't get my head around. it i like. that this could be valuable um. and then i but i did so i didn't feel. like i was early in cryptocurrency. in fact because i was like it was like. 2014 it felt like a long time after. bitcoin. um and then but then i i really liked. some of the things that. uh ethereum was doing it seemed like a. super. visionary thing like i was reading. something that was um. that was just gonna change the world.

when i was reading the white paper. um and i liked the different constructs. you could have inside of an ethereum. that you couldn't have on on bitcoin. like smart contracts and all that. exactly yeah and even the. they were yeah even spoke about. different uh. yeah different constructions you could. have yeah that's a cool dance between. bitcoin and ethereum of. it's in the space of ideas it feels so. young. like i wonder what cryptocurrencies will. look like in the future like if bitcoin.

or ethereum. 2.0 or some version will stick around or. any of those. like who's gonna win out or if there's. even a concept of winning out at all. is there uh a cryptocurrency that you're. especially. find interesting that uh. technically financially philosophically. you think. is is uh something you're keeping your. eye on. well i don't really i'm not looking to. like invest in cryptocurrencies anymore. um but i i.

they are i mean the they're and many are. almost identical. i mean there's not there wasn't too much. difference. um between even ethereum and bitcoin in. some ways right. uh but there are some that i like the. privacy ones i mean i was like. i like z cash for it's like coolness. it's actually. um it's it's like a different kind of. invention. compared to some of the other things. okay can you speak to just briefly to. privacy what is there some mechanisms of.

preserving some privacy of the. so i guess everything is public yeah is. that the problem. the yeah none of the transactions are. private. um and so you know even. like i have some of my i have some. numeraire. and you can just see it in fact you can. go to the website and says like. you can go like etherscan and it'll say. like numerai founder. and i'm like how the hell you guys know. this. so they can reverse engine near whatever. that's called yeah and so they can see.

me move it too. they can see me oh why is he moving it. yeah um. so uh so but yeah zcash then. also when you can make private. transactions you can also play different. games. yes um and it's unclear. it's like what's quite cool about zcash. is i wonder what games are being played. there. no one will know uh so from uh from a. deeply analytical perspective. uh can you describe why dogecoin is. going to. win which it surely will like it it.

very likely will take over the world and. once we expand out into the universe it. will take over the universe. uh or on a more serious note like what. are your thoughts on the recent success. of deutsche coin where. you've spoken to sort of the the meme. stocks the memetics of the whole thing. it feels like the joke. can become the reality. like the the meme the joke has power.

in this world yeah it's fascinating. exactly. it's uh it's it's like. why is it correlated with elon tweeting. about it. it's not just elon alone tweeting right. it's like. elon tweeting and that becomes a. catalyst. for everybody on the internet kind of. like spreading. the joke right exactly the joke of it so. it's it's it's the. the initial spark of the fire for a wall. street. bets type of situation yeah and that's.

fascinating because. jokes seem to spread faster than other. mechanisms. like funny is very effective. at uh captivating the uh like the. discourse on the internet. yeah and i think you can have um like. though i like the one meme. like doge i haven't heard that name in a. long time. yeah um like i think back to that meme. often that's like funny and every time i.

think back to it. there's a little probability that i. might buy some. kind right and so i imagine you should. have millions of people. who have had all these great jokes told. them. and every now and then they reminisce oh. that was that was really funny and then. they're like. let me buy some uh wouldn't that be. interesting if like the entirety. if we travel in time like multiple. centuries. where the entirety of the communication. of the human species.

is like humor like it's all just. jokes like we're high on probably some. really advanced drugs. and we're all just laughing non-stop. it's a weird like. dystopian future of just humor elon has. made me realize uh how. like good it feels to just not take. seriously every once in a while and just. relieve like. the pressure of the world at the same. time the reason i don't.

always like when people finish their. sentences with lol. is like that you don't when you don't. take. anything seriously when everything. becomes a joke. then it feels like uh. that way of thinking feels like it will. destroy the world. it's like i i often think it like will. mim save the world to destroy because i. think. both the possible directions yeah i. think this is a big problem i mean.

america i always felt that about america. a lot of people are telling jokes kind. of all the time. and they're kind of good at it um and. you take someone aside an american. you're like i really want to have a. sincere conversation it's like. hard to even keep a straight face yeah. because um. everything is so there's so much levity. so it's complicated i like how sincere. actually like your twitter. can be you're like i'm in love with the. world yeah. i get so much for it i'm not i'm.

never gonna stop because i realize like. you have to be able to sometimes just be. real and be positive and just be. uh say the cliche things which. ultimately those things actually capture. some fundamental truths about. life yeah but it's it's a dance and i. think elon does a. a good job of that now from an. engineering perspective of being able to. joke but every once you know what. mostly to pull back and be like here's. real problems. let's solve them and so on and then be.

able to jump back to a joke. so it's uh it's ultimately i think i. guess a skill that we. have to learn i uh but i guess your. advice is to invest. everything uh anywhere listening owns. into deutsche coin. that's what i heard from this yeah. exactly yeah. our hedge fund is unavailable uh just go. straight to dogecoin. you're running a successful company and. it's just. interesting because my mind has been in. that space of uh. potentially just being one of the.

millions other entrepreneurs. uh what's your advice on. how to build a successful startup how to. build a successful company. i think that one thing i i do. like and it might be a particular thing. about america but like. there is something about like playing. tell people what you. really want to happen in the world like. don't. stop it it's not it's not gonna make it. um like if you're asking someone to. invest in your company.

don't say i think um maybe one day we. might make a million. dollars um when you actually believe. something else. you actually believe you're actually. more optimistic but you're. toning down your optimism because you. want to appear. um like low risk. but actually it's super high risk if. your company becomes mediocre. because no one wants to work in a. mediocre company no one wants to invest. in mediocrity. um so you should play the real game um.

and obviously this doesn't apply to all. businesses but if you play a. venture-backed. startup kind of game like play for keeps. play to win. go big um and it's very hard to do that. i've always feel like um. i yeah i start you can start narrowing. narrowing your focus. because 10 people are telling you you. know you got to care about this. boring thing that won't matter five. years from now and. you should push back and do the real.

play the real game. so be bold so both i mean there's a. there's an interesting duality there so. there's. the way you speak to other people about. like your plans. and what you are like privately just. in in your own mind and maybe it's. connected with what you're saying about. yeah sincerity as well like if you if. you appear to be. sincerely optimistic about something. that's big or crazy it's putting.

yourself up to be kind of like ridiculed. or something yes. and so if you say my my mission is to um. yeah go to mars it's just so bonkers. that uh. it's hard to say it is but uh. one powerful thing just like you said. is if you say it and you believe it. then actually amazing people uh. come and work with you exactly it's not. just skill but. the the dreams there's something about. optimism that.

uh like that fire that you have when. you're optimistic of. actually having the hope of building. something totally cool something totally. new. that when those people get in a room. together like they can actually do it. yeah yeah there's yeah there's. that's uh and also makes life really fun. when you're in that room. so all that together ultimately. i don't know that's what makes this. crazy ride of a startup really. uh look fun and elon is an example of a.

person who succeeded at that. there's not many other inspiring figures. which is sad. i used to be a google and there's uh. there's something that happens that. sometimes when the company grows. bigger and bigger and bigger where that. kind of ambition kind of quiets down a. little bit. yeah you know google had this ambition. still does of making the world's. information accessible to everyone. and i remember i don't know that's. beautiful. i i still love that dream of.

that you know they used to scan books. but just in every way possible make the. world's information. accessible same with wikipedia every. time i open up wikipedia. i'm just awe inspired by how. awesome humans are man and creating this. together i don't know what the meetings. are over there but. it they it's just beautiful like what. they've created is. is incredible and i'd love to be able to. be part of something like that. and you're right for that you have to be.

bold and there's. and strange to me also i think you're. right that there's how many boring. companies they are. right something i talk about especially. in fintech it's like. why am i excited about this is so lame. like what is this isn't even like. important even if you succeed this is. going to be like. terrible yeah this is not good um and. it's just strange how. people can kind of get fake enthusiastic. about like boring ideas. yeah when there's so many bigger ideas.

that um yeah i mean you read these. things like this company raises money. and it's just like that's a lot of money. for the worst idea i've ever heard. some ideas are uh really big so like i. worked on autonomous vehicles quite a. bit. and there's so many ways in which you. can present that idea to yourself. to the team you work with to just yeah. like to yourself when you're quietly. looking in the mirror in the morning uh. that's really boring. or really exciting like if you're really. ambitious with autonomous vehicles.

there it it changes the nature of like. human robot. interaction it's changing the nature of. how we move forget money if you get all. that stuff it changes like. everything about robotics and ai machine. learning. it changes everything about. manufacturing i mean the cars the. transportation is so fundamentally. connected to cars and if that changes. it you're changing the fabric of society. of movies of. everything and if you go bold and take. risks and go.

be willing to go bankrupt with your. company. as opposed to cautiously you could you. could really change the world and it's. so sad for me to see all these. autonomous companies. thomas vehicle companies they're like. really more focused about. fundraising and kind of like smoke and. mirrors they're really afraid they're. the the entirety of their marketing is. grounded in fear. and presenting enough smoke to where. they keep raising funds so they can. cautiously. use technology of a previous decade or. previous two decades.

to kind of test vehicles here and there. as opposed to do. crazy things in bold and go huge at. scale. to huge data collection i mean um. yeah so that's just an example like the. idea can be big but if you don't allow. yourself. to take that idea and think really big. with it. uh then you're not gonna make anything. happen yeah you're absolutely right. in that so you you've been connected in. in your. work uh with a bunch of amazing people.

how much interaction do you have with. invest with investors like. that whole process is an entire mystery. to me is there some people that just. have influence on the trajectory of. your thinking completely or is it just. this. collective energy that they behind the. company. yeah i mean i i came here and i i. i was amazed how yeah people would i was. only here for a few months and i met. some incredible investors. and i'd almost run out of money and uh.

once they invested i was like i. am not gonna let you down and i was like. okay i'm gonna send them like an email. update every like three minutes. yeah and then they don't care at all so. they kind of wanna. i don't know like so as for some i like. it when it's just like they're always. available to talk. but um a lot of building a business. especially a high-tech business. um there's little for them to add. right there's little for them to add on.

product there's a lot for them to add on. like business development. and if we are doing product research. which is for us. research into the market research into. how to make a great hedge fund. and we do that for years. there's not much to to tell the. investors so they're basically is like. i believe in you there's something i. like the cutter your jib. there's something in in your idea in. your ambition. and your plans that i like and it's. almost like a pat on the back. it's like go go get em kid yeah it is a.

bit like that. uh and that's cool uh that's a good way. to do it i'm glad they do it that way. um like the one in meeting i had which. was like really good with this was. meeting howard morgan uh who's a. actually a co-founder of renaissance. technologies in the like 1980s. and worked with jim simons and um. he he he was in the room and i was. meeting some other guy and he was in the. room. and i was explaining uh how quantitative.

finance works i was like so you know. you they use mathematical models and. then he was like i. yeah i i started renaissance i i know a. bit about this. and then i was like oh my god um. so yeah but then and then i think he. kind of said well. yeah he said well because i was talking. he was working at first round capital. as a partner and they kind of said they. didn't want to invest. um and then i wrote a blog post.

describing the idea and i was like i. really think you guys should invest and. then they end up. oh interesting you convinced them all. that yeah because they're like we don't. really invest in hedge funds and i was. like. you don't see like what i'm doing this. is something so a tech company not a. hedge fund right. yeah numerous brilliant it's when it. caught my eye there's something special. there so i really do. hope you succeed and obviously it's a. risky thing you're taking on. the ambition of it the size of it but i. do hope you succeed. you mentioned jim simons um he comes up.

in another world of mine. really often on the he's just a. brilliant guy. on the mathematic side as a. mathematician but he's also a brilliant. finance hedge fund manager guy have you. gotten a chance to interact with him at. all. have you learned anything from him on. the math. on the finance on the philosophy life. side of things. um i've played poker with him it was. pretty cool it was like um. actually in the show billions they kind. of do a little thing about this.

poker tournament thing with all the. hedge fund managers and that's real life. thing um and they have a lot of like. world series of bracelet. what sort of poke bracelet holders but. it's kind of jim's thing. um and um i met him there. and uh yeah it was kind of brief but i. was just like. he's like oh how do you why are you here. and i was like oh howard sent me you. know he's like. go play this tournament meet some of the. other players. and then um was it texas hold him yeah.

texas hold him tournament yeah. like do you play poker yourself or was. it yeah i do i mean it was crazy and the. on my right was the ceo who's the. current ceo of renaissance. peter brown um and peter mueller. who's a hedge fund manager at pdt. uh and yeah i mean it was just like and. then you know. just everyone and then all these brace. world series like people i know from. like tv. um and robert mercer who's. crazy uh he he's the guy who.

who donated the most money to trump. um and he's just like it's a lot of. personalities character yeah. geez that's crazy um so it's quite cool. how yeah like the. it was really fun and then i managed to. knock out peter muller i have a. uh i got a little trophy for knocking. him out because he was a previous. champion in fact i think he's won the. most and he's won three times. super smart guy um but.

uh but but i will say jim outlasted me. in the tournament um and. they're all extremely good at poker yeah. um but they're also so is a ten thousand. dollar buy-in. um and i was like this is kind of. expensive. uh but it all goes to charity jim's. math charity but then the way they play. they have like rebuys. and like they all do a ton of.

rebuys this is for charity. yeah so um immediately they're like. going all in and i'm like. man like so i end up you know adding. more as well. uh so this is like you couldn't play at. all without doing that. yeah the stakes are highest but you're. you're connected to a lot of these folks. they're. kind of titans of just. uh of economics. and tech in general do you feel a burden. from this you're a young guy.

i did feel a bit out of a place there. like um. the company was quite new and. they also don't speak about things right. so it's not not like going to meet. an uh famous rocket engineer who will. tell you how to make a rocket. they do not want to tell you anything. about how to make a hedge fund. it's like all secretive and that part i. didn't like. um and they were also kind of making fun. of me a little bit. like they would say uh like they'd call.

me like i don't know the bitcoin kid. or yeah um and then they would say even. things like. uh member peter yeah said to me. something like i don't think ai is going. to have. a big um role in finance um and i was. like. hearing this from the ceo of renaissance. was like weird to hear because i was. like of course it will. and he's like but it but he can see he's. i can see it having a really big impact. on things like self-driving cars right. but finance. it's too noisy and whatever and so i. don't think it's like the perfect. application.

and i was like that was interesting to. hear because it's like and that i think. it was that same day. that um libra i think it is. the poker playing ai started to beat. like the humans yeah. so it's kind of funny hearing them like. say oh i'm not sure i could ever. get attack that problem yes and then. that very day is attacking the problem. of the game we're playing well there's a. kind of a magic. magic to um somebody who's exceptionally. successful. looking at you giving you respect but.

also saying that. what you're doing is not going to. succeed in a sense. like they're not really saying it but i. tend to believe. from my interactions with people that. it's a kind of prod to say. like prove me wrong yeah that's. ultimately that's that's how those guys. talk they they see good talent and. they're like. yeah and i think they're also saying. it's not going to succeed. quickly in some way they're like this is. going to take a long time. um and maybe maybe that's good to know.

and certainly ai in in trading. that's one of the most so uh. philosophically interesting questions. about artificial intelligence and the. nature of money. because it's like how much can you. extract in terms of patterns. from all of these millions of humans. interacting. using this methodology of money. it's like one of the open questions in. artificial intelligence in that sense.

you converting into a data set is one of. the biggest gifts. to uh the research community to the. whole anyone who loves. data science and ai this is uh. this is kind of fascinating i'd love to. see where this goes actually i think i. say sometimes. long before agi destroys the world a. narrow intelligence will win all the. money in the stock market. like a wave like it's just a narrow yeah. um. and i wanna i don't know if i'm gonna be. the one who invents that so i'm building.

numerai. to make sure that that narrow ai you. know uses our data. so you're giving a platform to where. millions of people can participate. and do build that narrow ai. themselves people love it when i ask. this kind of question about books. about ideas and philosophers and so on. i was wondering if you had. books or ideas. philosophers thinkers that had an. influence on your life when you were.

growing up. or just uh today that you would. recommend that people check out blog. posts. uh podcasts videos all that kind of. stuff is there something that just kind. of had an. impact on you that you couldn't. recommend a super. kind of obvious one that um. i really what i was reading zero to one. while coming up with numerai it's like i. was like halfway through the book. um and i really do like a lot of the. ideas there and it's it's also about. kind of thinking big and. um uh and also it's like.

peculiar little book it's like why like. this little picture of the hipster. versus unabomber and it's it's a weird. little book so. i like there's kind of like some depth. there in terms of a book on a. if you're thinking of doing a startup. yeah that's a good book a book i like. a lot is um maybe my favorite book. is david deutsch's beginning of infinity. um i just found that so. optimistic it puts you everything you.

read in science. it like makes the world feel like kind. of colder because the. like it's like you know we're just just. coming from evolution. and uh coming from nothing has nothing. should be this way or whatever and. humans are not very powerful we're just. like scum on the earth. and the way david deutsch sees things. and argues that he argues them with the. same rigor. that the cynics often use and then. has a much better conclusion um that.

uh you know some of the statements of. things like you know anything that. doesn't violate the laws of physics. can be solved like so ultimately. arriving in a hopeful. uh like a hopeful yeah without being. like. um a hippie you mentioned kind of advice. for startups is there. in general whether you do a startup or. not you have a device for young people. today. you're like an example of somebody who's. paid their own path. and were i would say exceptionally. successful is there advice.

somebody who's like 20 today 18. undergrad or thinking about going to. college. or in college and so on um that you. would give them. i think uh i often tell young people. don't start companies. is it no don't start a company unless. you're prepared to make it your life's. work. like that's a really good way of of. putting it. and a lot of people think well you know. um this semester i'm gonna take a. semester off and in that one semester. i'm gonna start a company and sell it or.

whatever right and it's just like. what are you talking about it doesn't. really work that way you should be like. super into the idea. so into it that you want to spend a. really long time on it. um is that more about psychology or. actual time allocation like is it. literally. the fact that you need to give a hundred. percent for potentially years. for it to succeed or is it more about. just the mindset that. that's required yeah i mean i think well. any i think yeah you don't want to have. certainly don't have a plan to sell the. company. um like quickly or something what's like.

oh it's like a company that has a very. it's like a big fashion component like. it'll only work now it's like an app. for something um. so yeah i that's that's a big one and. then i also think. something i've thought about recently is. i had a job as a quant. at a fund uh for about two and a half. years. and part of me thinks if i had spent. another two years there.

i would have learned a lot more. and had even more knowledge to to be. where. to basically accelerate how long. numerous took so. the idea that you can sit in an. air-conditioned room. and get free food or even sit at home. now in your underwear. and make a huge amount of money and. learn whatever you want. and get it's just crazy it's such a good. deal. yeah oh that's interesting that's the. case for i was terrified of that like at.

google i thought i would become. really comfortable in that. air-conditioned room. and that i was afraid the quan situation. is. i mean what you present is is really. brilliant that. it's exceptionally valuable the lessons. you learn because you get to you get to. get. paid while you learn from others if you. see that. if you see jobs in in the space of your. passion. that way that it's just an education. it's like the best kind of education.

but of course you have from my. perspective to be really careful. or not to i get get comfortable. relationship then you buy a house or. whatever the hell it is and. then you get you know and then you. convince yourself like well i have to. pay these fees for the car for the. house blah blah blah and then and. there's momentum and all of a sudden. you're on your death bed and there's. grandchildren. and uh you're drinking whiskey and. complaining about kids these days. yeah so i you know that i'm afraid of. that momentum but you're right.

like there's something special about the. education you get. working at these companies yeah and i i. remember on my desk i had the. like a bunch of papers on quant finance. a bunch of papers on optimization. and then the paper on ethereum just on. my desk as well and the white paper and. it's like it's amazing how much how kind. of. and you can learn about so that i also i. think this like idea of like learning. about. intersections of things i don't think. that too many people that know like.

as much about crypto and quant finance. and machine learning. as i do and that's um a really nice set. of three things to know stuff about. and that was because i had like free. time in my job. uh okay let me ask the perfectly. impractical but the most important. question what's the meaning. of all the things you're trying to do so. many amazing things. why what's the meaning of this life of. yours. or ours i don't know humans um.

yeah so have you ever heard people say. asking what meaning of life is is like. asking the wrong question. or something the question is wrong yeah. no that usually people get too nervous. to be able to say that because it's like. your question sucks. i don't think there's an answer it's. like the searching for it. it's like sometimes asking it it's like. sometimes sitting back and looking up at. the stars and being like. huh i wonder if there's aliens up there. if there's.

there's a useful um like uh. palette cleanser aspect to it because it. kind of wakes you up to like. all the little busy hurried day-to-day. activities. all the meetings all the things you like. are part of. we're just like ants a part of a system. a part of another system and. and then when this asking this bigger. question allows you to kind of zoom out. and think about it but this. ultimately i think it's an impossible. thing for a limited capacity like. cognitive capacity to.

to capture but it's fun to listen to. somebody who's exceptionally successful. exceptionally busy now who's also young. like you they ask these kinds of. questions about like. uh death you know do you consider your. own mortality kind of thing. and and life whether that enters your. mind. because it often doesn't it gets it kind. of almost gets in the way. yeah it's amazing how many things you. can like that are trivial that could. like occupy a lot of your mind.

until something until something bad. happens or something. flips you and then you start thinking. about the people you love that are in. your life. and you start thinking about like holy. this ride ends. exactly yeah i i i just had covered. um and i had it quite bad it wasn't what. wasn't really bad. it was just like i also got a. simultaneous like lung. infection so i had like almost like. bronchitis or whatever. i don't even i don't understand that.

stuff but. it felt i started and then you're forced. to be isolated. right and so it's actually kind of nice. it because it's very. it's very depressing yeah and then i've. heard. stories of i think it's shawn parker he. had like all these diseases as a child. yeah and he had to like just stay in bed. for years and then he like made napster. it's like pretty cool um so yeah i had. about 15 days of this. recently just last month and it it feels. like it did shock me into um.

a new kind of uh energy and ambition. were there moments when you were just. like terrified. at the combination of loneliness and. like. you know the thing about covet is like. there's some degree of uncertainty. uh yeah like it feels like it's a new. thing a new monster. that's arrived on this earth and so. you know dealing with it alone a lot of. people are dying. it's like wondering like yeah you do. wonder i mean. for sure and then these there are even.

um. new strains in south africa which is. where i was and maybe i had maybe the. new strain had some. interaction with my genes and i'm just. going to die but ultimately it was. liberating. i loved it oh i loved it i loved that i. got out of it. okay because it also affects your mind. you get confu you get confusion and kind. of. a lot of fatigue and you can't do your. usual tricks of psyching yourself out of. it. so you know sometimes it's like oh man i. feel tired okay i'm just gonna go have. coffee and then i'll be fine. it's like now it's like i feel tired.

cough i don't even want to get out of. bed to get coffee. because i feel so tired and then you. have to confront. uh there's no like quick fix cure and. you're trapped at home. but that so now you have this little. thing that happened to you that was a. reminder that you're mortal. and you get to carry that flag in. in uh in in trying to create something. special in this world. right with new mri listen uh this was. like. one of my favorite conversation because. you were you're the way you think about.

this world of money and just this world. in general is so clear. and you're able to uh explain it so. eloquently. richard that's really fun really. appreciate you talking today thank you. thank you. thanks for listening to this. conversation with richard craig and. thank you to our. sponsors audible audiobooks trio labs. machine learning company blinkist app. that summarizes books. and athletic greens all in one nutrition. drink. click the sponsor links to get a. discount and to support this podcast.

and now let me leave you some words from. warren buffett. games are won by players who focus on. the playing field. not by those whose eyes are glued to the. scoreboard. thank you for listening and hope to see. you next time.