Opinion | If ‘All Models Are Wrong,’ Why Do We Give Them So Much Power? (original) (raw)
[MUSIC PLAYING]
ezra klein
I’m Ezra Klein and this is “The Ezra Klein Show.”
[MUSIC PLAYING]
So before we get started, a bit of housekeeping, promotional kind of thing. There is a sub-Reddit, a page on Reddit for this show. I’m not involved in it. We don’t control it, or moderate it, or anything. But there are a couple of thousand people who like talking about the show over on Reddit. It’s at reddit.com/r/ezraklein.
And they’ve asked me to do an “ask me anything” next Friday the 11th. And I thought I would tell people this is happening in case they want to go participate in it. It’ll be sometime in the morning, Pacific time for me. But if you want to head over there, there’s going to be an opportunity to ask questions and I’ll come for a couple hours and answer them. So that is reddit.com/r/ezraklein.
[MUSIC PLAYING]
So one of the strange things about living in the Bay Area is you’ll be going about your business, talking to people about politics, and parenthood, and how nobody can afford a house, and all of a sudden you’ll run into someone who works in artificial intelligence. And you’ll realize the world, and what is important in it, and what is coming in it looks entirely different to them. Like, they are living across a chasm in expectations from you.
To them, we are on the cusp of a technology that will be more transformational than simply computers and the internet. They’ll be on the level of an Industrial Revolution. It maybe could create a utopia. It could create a dystopia. It could end humanity entirely. And you can dismiss it. Some people do. Sometimes I do. I definitely have the impulse to.
But these are smart people. I know them. And one thing that gives me pause is they are inside something I am outside of. They are seeing inside a technological revolution that is closed to most of us. They are seeing how fast a technology is moving, when, I mean, I try to follow it and I have no idea. And so one of the things I resolved to do this year is get a better handle on what I think of artificial intelligence, what I think it’s going to do to the economy, to society, to our politics, what I think the political approach to it should be. And that means, first and foremost, just understanding what it is, what it’s doing right now.
There’s an important distinction to make here. AI is not just Skynet. It doesn’t mean sentience. It doesn’t mean super-intelligence. To be honest, it’s not clear what it means. There’s an old joke in computer science circles that artificial intelligence is anything a computer can’t do yet. So like before computers can beat humans in chess, well that would be artificial intelligence. Afterwards, well, that’s just machine learning. Chess is a game. It has rules. You can just tell the computer the rules.
So, there’s always an expanding frontier over which, well, that is true intelligence and this is just machine learning. But broadly, we’re talking about machines that can learn and act more autonomously. And this is not a far-off thing. We’re using them now for all kinds of things, from what ads Facebook serves you to where your bail is set after getting arrested for a crime. It is affecting your life, my life, now. It is reshaping our economy and society now.
And it’s moving really, really fast. And even a lot of the people running these systems, they don’t fully understand what they’re doing. And that’s to say nothing of the politicians and regulators who are supposed to be governing them.
So, let’s get started. They’re going to be a few episodes around this theme in the coming months. But Brian Christian is where I wanted to begin. He’s the author of the book, “The Alignment Problem.” There are a lot of good books out there on AI right now, a couple of them from colleagues of mine at The New York Times.
But Christian’s, in my view, is the best book on the technical questions of machine learning written for a general audience. It’s a very, very, very deep work on how machine learning works. And it also ends up being a pretty deep look into how human learning works and the very fraught relationship between the two. Because that’s the fear at the core of all this.
The problems and the possibilities of AI are in a very deep way the problems and possibilities of humanity. They are generated by us. The fear is that it will learn the worst of us. And it will take our mistakes and our dark impulses and reorder society around them. And it will do so for the profits of a few. That’s a really important part of this conversation that often gets missed. The business models of AI, the political economy of AI, it really, really matters. So, I asked Christian to join me for the first of our AI episodes to talk about it. As always, my email is ezrakleinshow@nytimes.com. Here is Brian Christian.
[MUSIC PLAYING]
I want to start with the concept that gives your book its name, just the very idea of an alignment problem. And I was interested to learn from you that it comes from economics, from commentary about capitalism. I actually think it’s a useful place to start before we get to the whole artificial intelligence side of it. So, tell me about the history of alignment problems.
brian christian
Yeah, so the term alignment gets borrowed in 2014 by the computer science community from the economics literature. So, going back to the ‘90s and the ‘80s, economists were talking about, how do you make a value aligned organization where everyone’s pulling in the same direction? Or how do you align someone’s — the subordinate’s incentives with what the manager wants them to do?
And obviously, there’s a huge literature on how this can go horribly wrong. And it also connects to the parenting literature. Every parent has had these experiences of — the example I love is the University of Toronto economist Joshua Gans decided to give his older child, I think $1 or a piece of candy, I forget, every time they helped the younger child use the toilet. And he later discovered that the older child was force feeding water to the younger sibling so that they could use the toilet more times per day.
And so I think this just points to how fundamentally human this problem of incentives really is. And so, this idea of how do you capture, in a system of rewards, or incentives, or goals, or KPIs, or indices what you really want in a way that’s not going to lead itself to some loophole or some kind of horrible side effect that you didn’t intend? That is a much bigger human story than merely the history of AI.
ezra klein
So, there is a moth to the flame of super-intelligent AI that will kill us all dynamic in this conversation. People like to sort of throw the ball away down the field and imagine Skynet. But a lot of what is happening right now is we’re building machine learning into things we currently do, into predicting whether or not a violent offender will re-offend. And so, what kind of bail should they get? Or what kind of parole should they get? Or deciding whether somebody’s going to be a good fit for a job.
And so, can you talk a bit about the ways — the problems you’re looking at, the problems of aligning what we want machines to do and what they actually do, are operating in the here and now, not just the far flung future?
brian christian
Yeah, absolutely. So, this question of are these systems actually doing what we want has this long history. It goes back to 1960. The MIT cyberneticist Norbert Wiener has this famous quote where he’s talking about the “Sorcerer’s Apprentice.” We all know it as the lovable Mickey Mouse cartoon where he tells this broom to fill up a cauldron with water and then ends up almost drowning. Wiener has this quote where he says, this is not the stuff of fairy tales, like this is coming for us.
And the famous quote is, “If we build a machine to achieve our purposes with which we cannot interfere once we’ve started it, then we had better be quite sure that the purpose we put into the machine is the thing we really desire.” And this has continued through the early 20th century, as the thought experiment of the Paperclip Maximizer that turns the universe into paperclips, killing everyone in the process.
But to your point, I don’t think we need these thought experiments anymore. We’re now living with these alignment problems every day. So, one example is there’s a facial recognition data set called Labeled Faces in the Wild. And it was collected by scraping newspaper articles off the web and using the images that came with the articles. Later, this data set was analyzed. And it was found that the most prevalent individuals in the data set were the people who appeared in newspaper articles in the late 2000s.
And so, you get issues like there are twice as many pictures of George W. Bush as of all Black women combined. And so, if you train a model on that data set, you think you’re building facial recognition, but you’re actually building George W. Bush recognition. And so, this is going to have totally unpredictable behavior.
A similar thing in criminal justice. You may think you’re building a risk assessment system to tell you whether someone will re-offend or recidivate, right? But you can’t actually measure crime. You can only measure whether people were arrested and convicted. And so, you haven’t built a crime predictor. You’ve built an arrest predictor, which in 21st century America is a very different thing.
And so, there are many cases like this, large and small. You see the same thing happening in the research community. You think you’re trying to build a program that can win a boat race, and you use as this proxy incentive get as many points in this video game as you can. But maybe getting the most points involves just doing donuts in this little harbor and collecting these little power ups forever. And so, each of these is in its own way an example of this alignment problem. It turns out to be really hard to actually get the system to internalize the goals and the behavior that you actually have in mind.
ezra klein
Can you talk through the story of Amazon trying to build this into their recruiting efforts?
brian christian
Yeah, so Amazon was building a machine learning tool to help them prioritize which resumes should get filtered through when they made a job opening. And famously, they were rating their job candidates on a scale of one to five stars, just like Amazon users rate their products. And in order to do this, they were using what’s called an unsupervised language model. But the basic idea is it looks at resumes for candidates that were hired in the past. And it says, what are the terms that tended to appear on the CVs and resumes of people who we’d hired before? And so, we’ll just kind of up-vote the resumes that look like that.
The problem was that by default this is going to perpetuate any kind of bias or prejudice that existed. So, if your engineering department was mostly male, then you’re going to discover, as the Amazon engineers did, that it is penalizing resumes that have the word women’s in them. So if you played women’s soccer, or if you attended a women’s college, or whatever it might be, the system says that doesn’t look like the kind of language that has appeared on resumes of people we’ve hired in the past. Therefore, we don’t think you should hire this person in the future.
And they ended up penalizing terms like field hockey, or sewing, or things like that, that all kind of skewed towards female applicants. And even it went so far as identifying idioms that were more typical of male engineering candidates, like the use of the word executed. So, it was giving you bonus points if you used this kind of martial speech in your resume. Eventually, they essentially decided to scrap the project, that it was kind of irredeemable.
ezra klein
So, one very tricky quasi-philosophical issue here is whether or not this is, in fact, an alignment problem at all. So, what you’re basically saying is what happens is you turn these algorithms loose and they replicate the way our society actually looks. They look at who Amazon hires and then they learn based on that hiring process. And they say, here’s who you’re probably going to like to hire. An Amazon says, oh, no, no, no, not us, this is how we wanted to hire at all.
But on some level, it is how they were hiring. And maybe somebody who’s less politically correct, to use that — to use that language, might say, no, the machine had it right. And Amazon just doesn’t want to admit what it actually does and what actually works for it. So, these aren’t alignment problems. These are — we’re turning almost too powerful a spotlight on the way our society really functions. But that’s not a problem of the machines.
brian christian
There’s this adage that is famous in statistical circles. And it says, all models are wrong, but some are useful. And I think part of the danger that we have at the current moment is that our models are wrong in the way that all models are wrong, but we have given them the power to enforce the limits of their understanding on the world.
So, one example here being the self-driving Uber that killed a pedestrian in 2018. If you read the National Transportation Safety Board report, you discover a number of interesting things. One was that apparently there was no training data of jaywalkers. It was only ever expecting to encounter someone at an intersection or crosswalk. And so it just wasn’t prepared for how to deal with someone crossing in the middle of the street.
The other thing you find is that it was using this kind of classic machine learning thing of every object belongs into exactly one out of n categories. And this particular woman was walking a bicycle across the street. And so the object recognition system couldn’t make its mind up. It said, OK, well I could see her walking. No, there’s definitely a bike frame. I’m seeing the tires. I’m seeing this little triangular support thing. She’s definitely a cyclist. No, she’s — we can see her walking on the ground. And each time it changed its mind, it had to kind of recompute from scratch whether it thought it was going to hit her or not. And that’s part of what led to the accident.
So, this comes back to this idea that all models are wrong, but we’ve now, in some cases, given them the ability to use effectively lethal force to ensure that the world conforms to their simplified preconception of what the world is. If you live in a world where you don’t think jaywalkers exist, and you can only be a cyclist or pedestrian, and you kill anyone who doesn’t fit into that conceptual scheme, then the world does, in fact, come to resemble the model that you have in your head, but not in the good way.
ezra klein
So, what are we trying to achieve here? Before we get to the question of how do you handle the problem of machine learning algorithms creating disasters that we didn’t intend, what is the promise of these algorithms? They’ve been around for a little while now. We don’t have supercharged economic growth. We’ve not cured cancer. Like, why is there so much promise and investment in this field?
brian christian
I think there’s broadly this idea that we can deploy human level expertise or intelligence at scale for zero marginal cost. I think that’s the broad idea. That if we — not everyone can afford to go to a world class dermatopathologist to decide whether that discoloration on their shoulder needs to be looked at or not. But if you could train a model that’s as good as the best dermatopathologist in the world, then everyone with a smartphone could have access to that level of skill or that level of insight. I think that’s the broad story in terms of why this technology is so attractive.
There’s a well-intentioned societal motive behind some of this. A lot of the ways that hiring has been done in the past is just sort of through the social network of the people that already exist at the company. But that privileges people with certain demographic attributes. They live near the other people or they’re in the same economic class, et cetera, et cetera. And so there’s this meritocratic idea of, no, let’s create a job posting and anyone in the world can apply. But now you have to filter the candidates at a scale that you didn’t have to deal with before.
And so, what do you do? Well, often they turn to machine learning. And as we’ve discussed, there’s these kind of predictably terrible outcomes often. But I think that’s the idea, in terms of your question, what are we really trying to do?
[MUSIC PLAYING]
ezra klein
So, if people are following the conversation at all they maybe there a couple big companies doing this. People hear about Google’s DeepMind. They hear about OpenAI. There’s obviously a lot of work at Tesla. And then, of course, also at Uber on how to do driverless cars. It’s not going to be the case that every small hospital system develops its own remarkable AI player so it can rent out its AI dermatology department.
So, is what’s happening here that a bunch of these different institutions are trying to create the AI interface that other people are going to rent out for their projects? Is what’s happening here that it’s just going to be a few people have it early on and then other people develop it? Like, when we talk about this, who’s going to control this resource and how do others get access to it?
brian christian
I think this question really depends on the level of sophistication of the particular tool. So, on the more spreadsheet end of the spectrum, I think, for example, if you look at criminal justice, pre-trial risk assessment algorithms, many jurisdictions have rolled their own. So, Minneapolis rolled their own risk assessment in the ‘90s, and made a note to themselves to audit it a year later, but then forgot. And then it was in operation for about 15 years before they even thought to check whether it was making accurate predictions or not. So, those sorts of things happen.
On the other hand, yeah, to your point, there is a certain kind of economy of scale. At the moment, a lot of the most performant models are big. They take tens of billions of dollars to train. And so, that’s a certain barrier to entry. I think it’s a really interesting question. Because if you look at the academic literature on AI safety, it’s kind of premised on this almost hobbyist relationship of, there’s a human called H who wants to do something, so he goes to his garage and builds a robot called R. Is R going to do what H wants?
And maybe that’s a useful way for framing some of the actual math, but I don’t think that’s a useful way for thinking about the actual relationship we’re going to have with something like advanced AI. I think it’s more likely that it’s going to be like the OpenAI API or like the user agreement that we have with iOS or Android, where we never really own anything, we’re just kind of subject to the terms and conditions that can change at any moment. So, I think that is much more likely, from my perspective, to be the kind of relationship that we have, especially early on.
ezra klein
But you spent a lot of time with these different companies. I mean, you read through this book, it is a festival of this researcher at DeepMind, and this researcher at OpenAI, and this researcher at Microsoft, and so on. What are they all competing to do? What is their implied long-term business model, after they spend billions of dollars winning this race to create some kind of general or at least very, very powerful AI?
brian christian
Well, that’s a very interesting question. Because if you look at the narrative that’s being told — and this is — yeah, so just use DeepMind and OpenAI as an example. They both tell a story that’s something like, we’re going to solve intelligence and then everything will follow. Once we sort that out, then we can cure cancer, we can solve world hunger, you name it.
And it remains to be seen how the development of AI, and in particular AI safety, would handle something like a kind of late business cycle, revenge of the bean counters, where they say, OK, and why are we spending $40 million on this generic thing that plays chess all day long? So, it does explain why you get some of these contractions. And I am personally very, very interested to see whether safety research stays as robust in the next five years as it has been in the last five years. But that’s the idea. I think at some level, they’re waiting to find the business model down the line.
ezra klein
Well, that worries me, to be honest.
brian christian
Yeah.
ezra klein
I want hold here for a minute, because it’s something your book gestures at, but I always say needs to be a little bit of a deeper conversation here. Because you’ve been talking about AI safety. And for people who aren’t fully into that term, that’s around this alignment problem. Sometimes it’s like long-term, will AI kill us all? Short-term, will it do the things we want it to do? But you have a good line at one point, where you’re imagining just AI that follows us around and helps us make better decisions in the things we need to make decisions on.
And you write, “These computational helpers of the near future, whether they appear in digital or robotic form, likely both, will almost, without exception, have conflicts of interest, the servants of two masters — their ostensible owner and whatever organization created them. In this sense, they’ll be like butlers who are paid on commission. They will never help us without at least implicitly wanting something in return.”
And so, I imagine Google building through DeepMind the winner of the AI race. And I know how Google works. And everywhere I go on the internet, Google is serving me ads that are built on my own personal data and that are trying to get me to either complete a purchase I’ve begun or seem interested in, or trying to get me to make a purchase they think is adjacent to something that I would like. If you had much, much, much, much smarter AI that was much more integrated into my life, that was built on an advertising model, you’re kind of entering a pretty nerve-wracking space of personal manipulation.
And I don’t hear this honestly talked about as much in the AI safety conversation. It’s much more like we’re going to create this amazing thing, but what if it goes wrong, more then we’re going to create this amazing thing, and what if we do wrong?
brian christian
Yeah, and I think this is extremely central. And it goes back to your highlighting of the term alignment was originally an economic term, that Google can build an AI that’s aligned with the interests of Google Corporation that may not be aligned with the interests of the end users or third parties that are affected without even using the software, et cetera, et cetera.
There’s this question of what replaces the right-hand margin. Like, for those of us that remember the internet of the early 2000s, there was this right-hand margin on every website that was full of ads. But you moved from that to an oral interface, where you’re talking to Alexa or you’re talking to your smart home speaker. What is the equivalent of this kind of right margin of space that we can fill?
Is that you ask Alexa what’s the temperature, and it says, oh, it’s 72. By the way, I thought you might be interested in tonight there’s a new show premiering. Like, I don’t know that people have the patience for that. And so, there’s this question of, is the advertising model going to survive the move towards these digital assistants? Is it going to have to be replaced with essentially like product placement/commission driven model, where you say to your smart robot of the future, just get me some toilet paper? And behind the scenes there’s been a huge bidding process for which toilet paper it’s going to get you. And at some level, maybe you care, maybe you don’t.
That’s the kind of thing where I think a lot of the action is going to start to move to that. And so, yeah, you can really ask this question of, is this interface that I’m using actually working for me when I tell it what I want? Probably, mostly not. It’s probably mostly worried about which kind of toilet paper is giving the best commission week or whatever it might be.
ezra klein
Or something deeper than that. I’m very interested in this question of the alignment problem between the end user and the owner with the machine in the middle. Because another possibility here is geopolitical. So we’ve been in a debate over the past year or two in this country over TikTok, which is this remarkable social networking app that is owned by a Chinese company, ByteDance. And there’s been a lot of worries that maybe TikTok is spyware or what is it really doing.
But something very central about TikTok is its underlying algorithm is amazing. If you look at analysis of why TikTok does so well, its ability to intuit what you like through machine learning and feed it to you, it’s absolutely best of class. It needs a lot less information about you than a Facebook or some of these other players do.
Now, you imagine — I mean, China is making tremendous, tremendous investments in AI. Now, you imagine that some of those actually pay off. You build some of that into TikTok. And just on the margin, they’re trying to make you like China more, which maybe is not even the worst thing. We’ve had propaganda efforts in this country forever trying to make people like America more. Why not use your algorithm on your free video app to serve up things that improve cross-cultural communication?
But over time, this stuff becomes really, really, really out of alignment. And it is even hard to know if it’s going on. And I just don’t really know, I guess, what we do about it.
brian christian
Yeah, the question of what we do about it. I mean, there’s two halves to your question, I guess. One is, what’s the end game of this? I am very curious to see whether, for example, Twitter, and Reddit in particular, these kind of quasi anonymous, text-based discussion forums, whether they can actually survive the ability to produce site-specific propaganda at scale.
So, we’re now entering this era of large language models, things like BERT and GPT-3, et cetera, that can produce hand-tailored responses to a given comment thread that wittily reference the previous comments, but maybe have a slight five percent positive skew about — pick your political party. I really don’t know how the idea of anonymous discourse survives. And so, that, I think, is a really open question.
What do we do about it? I think partly there is a transparency issue. And I don’t know what the regulatory framework is going to be. But one of the things that I always want to know is, what is the objective function of the company? What is being optimized for? And ideally, you’d want to have some agency over that.
One of the things I like about Reddit is that there is this little dropdown where you can say, show me the most recent things, show me the most up-voted things, show me the most controversial things. And that’s only a couple of degrees of freedom, but it’s something. You feel like you have your hand on the wheel to some degree. Whereas, when I use other social networks, I’m very aware of the fact that my behavior is sending some kind of training signal back to the mothership. But it’s really not clear what that relationship actually looks like.
And so, I don’t know, for me transparency is the starting point. Just what is it you are trying to do in the first place? Like, what is driving the recommendations that are being sent?
ezra klein
Well, let’s take transparency at two levels. Because the transparency of these algorithms and transparency of what these machines are actually doing when they learn is a huge part of your book. But before we even get to that question of do we know what is happening in them, there is a question of, do we have the right to know what is happening in them? I can’t pop the hood on the Facebook news feed.
brian christian
Yeah.
ezra klein
They don’t give me that option. And they would say, and it’s not a crazy thing to say, the Facebook news feed is our comparative advantage. The algorithm that feeds us is something we spent however much money on. You can’t make us turn that over to the public. Do we need to think of algorithms in some kind of different class than we have thought of a lot of traditional forms of IP?
brian christian
Yeah, in some ways it reminds me a little bit more of financial regulation, where you have to deliberately make the regulation very vague. Because by the time the bill is actually passed, the technology has moved on. And so, the only hope you have any meaningful regulation is to do it on the fly tactically. I mean, that may be the case here.
One of the other things Facebook may say is like, well, we can try to provide you transparency into our algorithm, but we change the algorithm 20 times a day. And we’re constantly A/B testing a thousand different variations on any given user at any given moment. So, what do you mean by “the” Facebook algorithm. You might have transparency one minute and then you refresh the page and now it’s a completely different process affecting results.
So, I think there are huge questions. And I don’t claim to have any idea what the regulatory foothold is here. I do think from the scientific side there’s been a lot of progress on the idea that you can actually constrain the model in a way that makes it intelligible to someone from the outside without sacrificing a lot of performance. So, there’s a really encouraging scientific story. How that actually rolls out into something that’s user facing I think is much less clear.
ezra klein
Talk me through a bit of that scientific story. Because it is interesting. Five years ago, we were — all these things were more rudimentary, but we were much worse at figuring out what was happening inside of them. What got figured out and to what degree, such that we can now know better what it is a machine has learned when it is producing an output?
brian christian
Sure. So, I guess one place to start is if you think about a model, it has three parts. There’s, what are the inputs? What goes on in the middle? And what are the outputs? And so one way to make a model simple is to have fewer inputs. One way to make it simple is to have less stuff going on in the middle.
So, one of the big advances that we’ve seen in machine learning since about 2010, ‘11, ‘12 has been the use of this technology called deep neural networks, which basically just is these simple, rudimentary, mathematical elements that kind of, sort of resemble what goes on in a neuron. It’s like they have a bunch of inputs. And the inputs are just numbers. It’s like one, 0.5, whatever. You add them up. And then if they’re greater than some threshold, you output some other number. Otherwise, you output the zero.
And it turns out that if you have tens of millions of these things stacked into layers, they can do essentially arbitrary tasks. They can tell cats and dogs apart. They can tell cancerous and non-cancerous lesions apart, et cetera. But there’s this real inscrutability. So even if you could pop the hood, you would see like 60 million of these rudimentary things that all have slightly different thresholds of when they output and when they don’t. And so, the question is like, is that level of transparency actually giving you anything?
So that’s been — there’s been two pushes. One push is, do we need to use this inscrutable technology or can we use simpler models — your more classic 20th century statistics? Then the other push has been, OK, for certain applications let’s just say you have to use a giant neural network. Can we actually visualize what’s happening with the interior of this network? Can we actually see, for example, that, OK, layer one has detected this edge? Or layer two is detecting this pattern in this part of the image?
ezra klein
You’re talking here for visual recognition.
brian christian
Yeah, as an example.
ezra klein
Yeah.
brian christian
Yeah. So that’s the two front attack that’s being made, is making simple models competitive with the complex models and making the complex models somehow more intelligible.
ezra klein
I want to put the simple model question to the side, because I take it is almost axiomatic, that if these different players, like DeepMind, get where they want to go, we’re going to be using some very, very, very complicated models and some very, very complicated systems. And if we get to the kind of general artificial intelligence, it’s not going to be a simple model or we’d have it already. So, at some point we’re going to need to know what these programs, searching through all of the information humanity has ever been able to generate, are finding.
And we know that when they start searching they find things we didn’t. We know that, say, the model that is now the best AlphaGo player in the world realized different things playing against itself in AlphaGo than human players had ever realized. And so, how do we see what the model is seeing?
brian christian
Yeah, one way that you can do it would be this idea that’s called perturbation, which is to say — to use your example of Go. You start with a Go board and then you iteratively add and subtract stones to every location on the board. And you see which of those perturbations have the biggest effect in what the model thinks is going on. And you can say, oh, well when I remove this one stone here at the bridge between these two clumps it totally changes its evaluation of who’s winning. Therefore, I can infer that it’s, quote-unquote “focusing” on this area or that this area is salient.
And that type of process can be really helpful. There are many horror stories of researchers finding that the model was essentially focused in completely the wrong area. So, a group of dermatologists built a system to determine whether these marks on your arm are cancerous or not. But when they used these perturbation or saliency methods they discover that actually what the model is looking for is the presence of a ruler next to the lesion. Because the medical textbook images had a ruler next to them for scale. So, it thinks that the ruler is the cancer, things like that. So, that ends up being really helpful.
The other thing that you can do, with Go programs as being an example, is to run the model forward and say, OK, what do you think is going to happen if you take the action that you’re planning to take? And that’s another way to get a sanity check of just if we look through the crystal ball, does the model’s sense of what’s downstream actually track with what makes sense to us?
[MUSIC PLAYING]
ezra klein
So we’ve been talking here about what happens once the machine has learned something. But a lot of your book is about how we are learning to help machines learn and the places we’re taking inspiration from on that. And a lot of where we’re finding some inspiration is actually us. So, could you tell the story just of dopamine and what we have learned dopamine is for in the human mind?
brian christian
Yeah, so in the ‘70s and ‘80s, we were learning a lot about the dopamine system. And we were developing the actual technology to monitor dopamine — individual dopamine neurons in real time and watch them spike. And it was producing a pretty mysterious story. We could see a monkey reach into a little compartment and find a piece of fruit, and boom, there would be this dopamine spike. But the fifth or sixth time, the spike would go away.
And so, what was going on here? And to make a long story short, there was this question of is dopamine encoding our sense of reward? No, not exactly. Is it encoding our sense of surprise? No, not exactly. So, what’s going on? Because we know it’s related to those things, but we can’t really pin down what this signal actually corresponds to.
In parallel, there had been kind of this movement within the computer science AI community called reinforcement learning. And the basic idea was: let’s build systems that can learn to take actions within an environment to get as many points as possible, whether you define — you define the points however you want. So playing chess, you want to capture pieces or whatever it might be. And one of the methods that was successful in the ‘80s for solving this problem is what’s called temporal difference reinforcement learning.
And the basic idea was you take an action, you don’t know if it’s good or bad. You might not have to wait till the end of the entire game in order to know whether you made a mistake. If you suddenly lose your queen, you know that something went wrong. You don’t have to wait till you get checkmated 30 moves later. And so, you can learn from these minute to minute differentials in how well you think things are going. It creates kind of an error signal.
And you can learn — if you think about trying to predict the weather. On Monday, you predict the weekend weather. And by Tuesday, you have a slightly different prediction. You don’t even have to wait and see if you’re right. You can just update and say, OK, my Tuesday prediction is going to be slightly more accurate than Monday. So, my Monday prediction should have been a little bit more like that. And these are called temporal difference errors. It’s kind of you’re learning what a guess should have been from a subsequent guess that you make.
OK, so this was all happening in the computer science departments. So, these kind of neurophysiology data land on the desk of some of the computer scientists, in particular Peter Dayan, who had kind of crossed over into the Salk Institute and was doing some neuroscience work. And when the computer scientists looked at it, they said, oh, this is — the brain’s doing temporal difference learning.
It is learning a guess from a guess. Suddenly, when you find a piece of food that you weren’t expecting, life just got a lot better than you thought it was going to be a moment ago. But if you come to realize that food’s always in the box then life is always about as good as you currently expect it to be, so there’s nothing more to learn.
I just think this is a remarkable story for a couple of reasons. Primarily, it is this idea that computer science and cognitive neuroscience are engaging in this dialogue, this kind of feedback loop where each is now informing the other. And it also, I think, tells us a story about AI. That we are, in my view— this gives us some evidence that we’re not just merely working out some engineering hacks that happen to solve video games or whatever, but that we are actually discovering some of the fundamental mechanisms of intelligence and learning.
We’ve kind of independently found the same mechanism that evolution found. And in fact, there are many different evolutionarily independent temporal differenced mechanisms if you look at bees, if you look at different species. We really are on to, in my view, the philosophical paydirt of artificial intelligence, which is to figure out how our minds work to begin with.
ezra klein
So, I love this. So, basically what is being said here is that dopamine is a way of updating our expectations about the world.
brian christian
Yes.
ezra klein
That we don’t feel good because things got better or worse, we feel good because we are now projecting things to be better or worse. And you say in the book something that has left me thinking about it for quite a while, which is that this helps explain the idea of the hedonic treadmill, the idea that we — that people get used to winning the lottery, they get used to losing a limb. Why does this help explain that?
brian christian
There is this funny connection between the dopamine system and the subjective experience of pleasure and happiness. So, this is obviously a major front in philosophy of mind/neuroscience, is how do we link the physical mechanisms in the brain that we can observe to the feeling of being happy or of liking something?
And to the degree that dopamine is part of that story, it tells us that it feels good to be pleasantly surprised about how promising your life is. But because you are using that signal to learn, it’s like you can’t be permanently pleasantly surprised. Like, eventually you’re going to learn how to actually make accurate predictions. And so some of that pleasure goes away.
And so for me, it tells a story about not just the hedonic treadmill as adults, but the way that evolution has given us this really general purpose learning mechanism, that when you’re one year old, or six months old, or whatever, waving a hand in front of your face, at first it’s really delightful because you don’t expect what’s going to happen. Or you push something off a table and it falls on the ground and you’re delighted because you had no idea what was going to happen. And eventually, you need to get your kicks by playing sports, or by writing academic papers, or writing books, or whatever it is.
And it’s, I think, pretty remarkable that there is this general purpose, take delight in the ways that your predictions are wrong, but also improve your predictions. And this kind of sets up this whole trajectory of our life in a way.
ezra klein
But one thing that made me reflect on is the way that we have alignment problems with ourselves.
brian christian
Oh, yeah.
ezra klein
So take the dopamine function you’re talking about there, one way of describing the hedonic treadmill is that the things we believe in advance will make us happy don’t end up making us happy. So, here we are wandering through our lives, telling ourselves that if we just work so hard and get to this point, we’re going to be happy, and then we’re not. And now dopamine’s not doing something wrong. Like, it’s optimized for fitness and all the other things evolution wanted for us, which is not always happiness.
But there’s a funny way in which we’re sitting here talking about how it’s hard to create the correct reward functions for machines, but we also don’t even really understand day-to-day how to create the correct reward functions for ourselves. And so we’re constantly doing things, like every time I pull out my phone and open up an app because I’m tired, they don’t actually make us happier, but we somehow learn that give us a little hit of dopamine, because well, something is going to happen now that is a little bit better than me being bored and exhausted right here, even though what usually happens is I get annoyed at everybody fighting on Twitter. And so, there’s a way in which there’s a lot of pathos in us trying to teach other beings, sentient or not, organic or not, how to think and how to live when day-to-day I’m not sure we’re so good at ourselves.
brian christian
Yes, I think there’s something really deep here, which is that you could think of humans’ relationship to evolution as having this alignment problem where there are certain things that evolution quote-unquote “wants” us to do that will make us fit to stick around in an environment. That process is really complicated and hard to directly encode into the motivations and desires of people. So, instead we get — we get this weird reward function or these weird sets of incentives where we want to eat chocolate, and have sex, and open our Twitter on our phone, and check our notifications, and all the things that we actually want or are motivated by.
And this really is the problem of reward design. Like, evolution has designed these rewards. And we’ve, in some ways, over-optimized for them. And I guess part of the human condition is realizing that you have some degree of agency. Just because evolution wants you to propagate your genes and do this and that, you don’t have to. You have some degree of agency over what your own goals are. And I think that’s — that’s interesting from a parenting perspective.
So, one of the things Alison Gopnik talks about as a parent is that being a parent is not like building an AI system, because you don’t necessarily have in mind at the beginning the notion of what you want your children to want. I mean, maybe you do within certain parameters, but you also want to give them the leeway to become their own people.
So, yeah, this question of how to behave with respect to a set of rewards that are to some degree kind of hard coded, but you also have kind of control over your environment, you can sort of shape your own reward function to a degree, I think this is intrinsic to the human condition, absolutely.
ezra klein
So one of the things that the research here seems to be doing is giving AI researchers more respect for parts of the human reward system that seemed softer, weirder, more idiosyncratic, more fuzzy. And one of them is very much curiosity.
brian christian
Yes.
ezra klein
Can you talk a bit about what we’ve learned from trying to get computers to play Montezuma’s Revenge?
brian christian
Yeah, this is one of my favorite stories in AI. So, there’s a DeepMind team that came together around 2013 to 2015 to try to build an AI system that could beat not just a single Atari game, but every Atari game using the same generic architecture. And they managed to achieve, I think, some pretty incredible results. It was like 25 times better than a human at video boxing. It was like 13 times better at pinball, et cetera, et cetera.
But there was one game at which this model scored a total of zero points. And that game is called Montezuma’s Revenge. And so, there is this question of why was this one Atari game so difficult for this AI system to beat? And the basic answer is that it has what’s known as sparse rewards.
So, most Atari games, you can essentially just mash buttons until you get some kind of points. And for an AI system, that’s enough to bootstrap the learning process. And you can say, OK, how did I get those points? Let me do a little bit more of that in the future and so on. But in Montezuma’s Revenge you have to execute this huge sequence of really precise movements. Any mistake basically kills you. And only if you do this huge long chain of things correctly do you get even the first points of the game.
And so, the whole premise of this learning algorithm was that you would mash buttons until you got points and then figure out how to get more. But how do you learn if you can never get the first points to begin with? And this is a riddle that is, I think, wonderfully solved by babies. So, the computer science community starts looking over the fence at ideas from developmental psychology. Because, of course, human beings play these games with no problem. So, there’s something going on that enables us to understand how to play these games.
We’ve known since the ‘60s that infants have this really strong novelty drive. In psychology this is sometimes known as preferential looking. So, if you show an infant a toy and then an hour later you give it a choice between that toy and a new toy, they almost always pick the new toy. And this is such a bedrock result that it’s used as a way to study memory, and perception, and things like this in basically newborns.
So, there’s a very fundamental reward, essentially, that people get from seeing new things. The idea was, what if we just plug this novelty reward into our video game system, such that we treat the encountering of new images on the screen as literally tantamount to getting in-game points, just as good as getting in-game points? Suddenly, when you do this, the program has this kind of human-like drive. It wants to explore the limits of the game. It wants to go through the locked door just to see what’s on the other side. It wants to climb the ladder and jump the fence. And that’s what it takes to beat this particular game.
So, that to me, is just another one of these wonderful convergences where some of the insights that we’re getting into these very fundamental drives in human intelligence end up being imported, in a very literal and direct way, into AI software. And then suddenly it can do things it could never do before.
ezra klein
Do you believe we’re going to get super-intelligent general AI? Or do you believe what we’re going to get is sort of like kids with savant-like capabilities in certain things that we need?
brian christian
I think at the limit I see no fundamental principled reason why we’re not going to get some kind of super-intelligent general AI. The question is, what does the road there look like? And I think the road there does look like these weird savant-like, grown-up kids. You can think about GPT-3.
ezra klein
So GPT-3, for people don’t know that is, is OpenAI’s predictive text to artificial intelligence platform. It’s gotten a lot of buzz. It’s one that a lot of people are able to use. So that’s a big place where a lot of people have begun to see how AI could work.
brian christian
It’s essentially like someone who has lived their entire life in a windowless room with an internet connection and has read everything that’s ever been written on the internet, but has no idea what anything actually is, beyond how it’s spoken about. And it turns out you can fake it pretty well, but eventually some of that ignorance will catch up to you.
So, for the foreseeable future, a lot of what I’m worried about with AI is how we’re going to accommodate systems like that. Because to even — in some ways, to even get to worry about the super-intelligent AI, we have to survive the kind of savant, grown-up children AI. And that’s the story of the Sorcerer’s Apprentice. It was just this animated broom that knew nothing except how to pour water. And that was dangerous enough.
ezra klein
So I gave a version of this question to Ted Chiang, the science fiction author, which was, does he think we’re going to get super-intelligent AI? And he said, no, not really, we’re not going to get sentient AI either. Maybe we could, but should we? And then he said, absolutely not. And I thought his reasoning was interesting. And I was thinking about it while I was reading your book. Which is he said, long before we got sentient super-intelligent AI, we would have AI that could really suffer. And given how human beings have treated animals, given how they’ve treated each other, given how they treat machines, we would make this AI suffer on a tremendous level. And as I read your book, there’s a pathos to basically every program you describe.
We have every time embedded it with an unbelievable want for something. A want to see new screens in Montezuma’s Revenge, a want to get points in a video game, a want to be able to fulfill whatever question has been posed to it- a want, a want — we’re creating these little desire machines. And these are often things that it can’t do or often things that we’re going to lose interest in it doing. And I don’t know at what point — I know philosophers think about this, you have to think about the moral weight of this — at what point for a program to not be able to fulfill its wants it is feeling pain.
But it does strike me as well before we’re going to have things that are so intelligent, we have a lot of sympathy for them. And I’m curious how you think about this question.
brian christian
It’s a great question. There is a computer science research group that has the, I think, somewhat tongue in cheek title of People for the Ethical Treatment of Reinforcement Learning Agents. But there are people who absolutely sincerely think that we should start now thinking about the ethical implications of making a program play Super Mario Brothers for four months straight, 24 hours a day.
ezra klein
You talked about one that did Super Mario Brothers, and it’s just caught in this game that has no more novelty. And it’s a novelty seeking robot. And I thought it was so sad.
brian christian
Yeah, it just learns to sit there. Because it’s like, well, why would I jump across this little pipe because it’s just the same old shit on the other side. Like, well, I might as well just do nothing. I might as well just kill myself. And there have been reinforcement learning agents that, because of the nature of the environment, essentially learn to commit suicide as quickly as possible. Because there’s a time penalty being assessed for every second that passes that you don’t achieve some goal. And they can’t achieve it, so they’re like, well, the next best thing is to just like die right now.
And again, it’s like we’re somewhere on this slippery slope. I mean, there is this funny thing for me, where the more I study AI, the more concerned I become with animal rights. And I’m not saying that AlphaGo is equivalent to a factory farm chicken or something like that, necessarily. But going back to some of the things we’ve talked about, the dopamine system, some of these drives that are — the fact that we are building artificial neural networks that at least to some degree of approximation are modeled explicitly on the brain. We’re using TD learning, which is modeled explicitly on the dopamine system. We are building these things in our own image.
And so, the odds of them having some kind of subjective experience, I think, are higher than if we were just writing a generic software. This is the huge question of philosophy of mind, is are we going to if we manage to create something with a subjectivity or not? I’m not sure. But these questions, I think, are going to go from seemingly crazy now to maybe on a par with something like animal welfare by the end of the century. I think that’s not a crazy prediction to make.
ezra klein
Yeah, and then you add in the fact that you can create — it’s not literally an unlimited number, because there’s computing power associated with this, but at some point the idea is this will be simple enough and computing power cheap enough that you can create marginal AI agents at very, very low cost, right? That is how you all of a sudden get all these super low-cost human level laborers. And it does seem a little scary.
I’ve given a lot of attention to this question of what would AI do to us, including well before super-intelligent, just putting people out of work, that kind of thing, and very little to the question of what would we do to it. But I don’t know, I read your book, and as you say, some of these stories, they already make you feel terrible to hear, like the ones of the AI just killing itself because there’s a penalty to doing anything else at this point. And the idea that we wouldn’t know when it was feeling pain, which seems very plausible to me, is pretty profound. So, I don’t know. I don’t know at what point that should actually affect what kind of research we’re doing.
brian christian
Yeah, we have a lot more options than we do with human welfare and animal welfare in terms of, for example, if the agent is this novelty seeking agent that then gets really burned out and bored, could we just like wipe its memory so that, oh, wow, everything is new again and everything is delightful all over again? And it’s sort of living in this weird, kind of “50 First Dates” environment? Is that itself unethical? Or is that the only ethical thing to do at that point? It gets pretty head spinning.
I think there’s also this question of: will there be an ethical imperative to make models simple enough that they don’t have ethical standing? And, I don’t know — there’s a joke that computer scientists make about, you have this household robot. And if you pay extra, you get the version that doesn’t have a subjectivity. So, it’s not suffering. But by default, the cheap one, they couldn’t afford to add that in. So, it will do what you want, but it won’t like it.
ezra klein
This whole — I have to say, this whole part of the conversation, it leaves me feeling real chilled.
brian christian
Yeah, and —
ezra klein
Like, oh, we’re just going to start mind wiping our robot slave helpers because then — I mean, I’m not the most read up in science fiction of anybody you could possibly talk to in a day ...
brian christian
Yeah.
ezra klein
... but I’m read up enough to have read a few stories about ideas like that. And they don’t — I wouldn’t trust us with that kind of power in general. And having — creating a new class of possibly suffering servant workers, but who we’ve paid nothing for, and who we don’t really think — but given, because I am an animal rights person and think what we do to chickens is really appalling, I have no illusions that there is some limit of suffering that we would inflict on a species that we have enough justification to not care about, but whose labor we find useful to us.
Let me ask you about the more near-term cost of this, for human beings that people talk about more. Andrew Yang ran for president and to some degree is running for mayor of New York on the idea that machine learning is going to — and automation will — put people out of jobs. It has a little bit of that quality that the internet had for a while, where you can see it everywhere, but the statistics.
Machine learning, algorithms, computation, automation has gotten way, way better in recent decades. It has obviously put some people out of work. But we do not have a significantly higher rate of unemployment than we did a few decades ago. There are some changes for labor force participation, but not really gigantic ones that I think you can trace to automation. So, why don’t we see 10 or 15 percent unemployment from what we can do now? And will we?
brian christian
To some extent, I’m partial to the idea that we create the problems we then have to solve. Like, anything that makes email easier to send tells you that it’s going to solve the problem of email, because you’re only thinking about it from your own perspective. It’s going to make it easier for you to deal with your inbox. But if everyone has that program, then there’s just more email. And everyone’s still spending the same amount of time. So it’s a treadmill.
There are other treadmills like that. I do think that AI has been a big part of the story of inequality. That if you broadly take the lens of society in the Marxian view, the struggle between labor and capital, AI is the labor that doesn’t — it’s not human. It doesn’t need a wage. It doesn’t advocate for itself. And so, I’m somewhat sympathetic to the idea that we’re permanently tilting this millennia long tension, or at least centuries long tension, in favor of capital. And there’s a lot of deep psychological questions to think about.
The political rhetoric around the economy is framed in terms of jobs, rather than these more primitive things, like good health, good education, whatever. Are jobs going to be the salient rhetorical framing of those things going forward? I’m not convinced that that’s the case. I will be very curious to think about what it means for someone to be in the economy when most of the things that they can do are able to be done by machines better.
So, you think about Amazon Mechanical Turk employs a ton of people. And the tagline of Amazon Mechanical Turk is artificial artificial intelligence, namely people.
ezra klein
It’s so dystopic.
brian christian
Right? I mean, it’s this kind of “Soylent Green” — it’s people. But almost in its very premise, those are the jobs that are ready to get automated away. So, I think it’s interesting to think about what it means to contribute economic value in that kind of society. Right now there are a lot of people whose economic value comes from what they can see, their ability to process information visually or their ability to manipulate objects with their hands.
If you think about sewing, we still don’t really have sewing at scale, even though we have all this advanced manufacturing for steel, et cetera, et cetera. What would it mean to have robots that are dexterous enough to sew? I don’t have an answer to that personally. I think these questions of, does AI kind of lend itself towards this rich get richer type scenario? In my view, yes. And so, I think we have to mitigate that probably politically. But those are huge questions. And I think they’re open questions, from my view.
ezra klein
But, so, it sounds to me like what you’re saying is that the question directly is not jobs so much as it is dignity and status. That you can imagine a world where AI is doing quite a bit. People still have jobs. They’re just a little bit ridiculous. Because you’re operating around the margins of algorithms you don’t understand or you’re just sort of mopping up behind them. And then particularly, if you add in that the people who own the AI are getting all the money out of this, so that people with these now kind of crappy jobs also don’t have this other generator of status and dignity, which is money in our society, then you have a real social standing problem.
And so, one of the questions is like do we understand — we can give a lot of things dignity. Compared to what people were doing 500 years ago, what I do for a living, which is talk to people, and write some stuff about politics on the internet, I am given a lot of status by that in our society. But compared to feeding people at times when people needed to be fed, it’s not that useful of a role. And there are a lot of things like that.
I mean, we choose to have some of these roles that are imbued with dignity, but they really do depend on how we see them socially, how much we give cultural respect to the people in them, and how much we actually pay for them. Teachers get a lot less dignity than I think they deserve compared to, say, investment bankers, because they make so much less money. I think if teachers were paid $500,000 a year, the money, combined with the obvious utility of the role, would make them like society’s most honored people.
brian christian
Yeah. I’m also very struck by — a lot of the things we do for pleasure are the very things that we have sort of automated away. Like, for example, the pastimes of the upper class in Victorian England were fox hunting and gardening, essentially hunting and gathering. That’s like, we’ve built this entire civilization so that you don’t have to hunt and gather. But then the people with the most privilege and the most leisure want to hunt and gather for fun. Because that’s, at some deep level, what we are built to do.
So, yes, I think there’s also questions of status from the perspective of we’re now — everyone is now comparing themselves to the entire global population. I remember I went to a panel at UC Berkeley a few years ago, where it was Jaron Lanier, Neal Stephenson, and Peter Norvig. And they were talking about some of these like long-term economics of AI things.
And Peter Norvig said, it used to be the case that if you made pizza and your pizza was 20 percent better, you might get some super linear amount. You might make 30 percent more money. But nowadays, if you make an email client that’s 20 percent better, you make all the money. And everyone else goes bankrupt or whatever it is. And we just need to sort of fundamentally rethink these questions of standing.
It used to be the case that you could be the best guitar player in a 10-mile radius and that would make you really cool. And nowadays, it’s like, oh, well you’re only the third best guitar player, so why would I watch your YouTube videos when I could watch the best? I wonder if we are going to see a kind of willful parochialism come back as people realize that there are advantages to being essentially the big fish in the small pond. I don’t know exactly what that looks like.
ezra klein
One thing you might say about our society as it currently exists is we have given status to ridiculous things. Maybe because we need to keep people busy, maybe because capitalism rewards somewhat absurd things sometimes, but however it is, the way we actually attach dignity to roles and then train people from the time they’re born to be achievement monsters, trying to get through those roles, is also not a great way of doing society.
And so maybe it’ll be the case that at some point in the kind of post scarcity future, you have automated agents doing a lot of the fundamental work of the economy and people can concentrate on things that, say, the classical philosophers would have thought are closer to the good life. Things that John Maynard Keynes thought we’d be doing when he imagined how rich we’d be, which is like we’ll be painting and thinking about philosophy. Or even just more, I think, prosaically, spending time with our families, and going to the park, and playing sports with our friends, and having a drink with our buddies.
And there will just be more time to enjoy being human. And that won’t be looked down upon. Because the reason it is looked down upon is we have needed to make that a low status, low class activity in order to keep everybody very engaged in this huge economic machine we want to feed.
brian christian
Yeah, I’m very sympathetic to Keynes’s vision of like — in hindsight, you’re like, well, what went wrong? Because we really were on track. And I think about the promise that technology offers society in the broadest terms is to make people happier. But when I think about am I happier as a function of having been born in the 1980s then if I had been born in the 1880s, I don’t think so. I have better dental care. I’m not worried about an abscessed tooth or something like that.
But broadly speaking, I care about my family, my marriage, my friendships. I want to do interesting work and write books, which I could have done 100 years ago. Viewed from that perspective, technology has surprisingly little to offer. I don’t think it’s bringing a lot to the table in terms of addressing the fundamental things that make people happy. Relieving the creature comforts and the physical drudgery associated with them, I think is huge. But we’ve been past that threshold for several generations at this point. And I think people are getting less happy, rather than more.
ezra klein
Well, doesn’t this speak to a possible alignment problem that we were getting at in human beings earlier? I think there’s an endless view that if we lift the condition of scarcity, well then, finally, people will have what they need to be happy. But if you believe that we evolved with our reward function, our dopamine system, and all the rest of it, is optimized for getting us through scarcity, for surviving and reproducing conditions of scarcity, then its absence drives us a little bit crazy. We’re like the machine trapped in a system with no novelty. We need to find something to keep ourselves busy because otherwise we just get listless and a bit lame.
Now, I don’t fully by that. There’s a very interesting discontinuity in research where people are very unhappy when they’re unemployed, but if they then just shift into retirement during that period, they get happier because the status of being retired is a much better cultural status than being unemployed. And there have been many, many human societies that have not worked on neoliberal capitalistic, or for that matter modern communistic or socialistic thought. Hunter-gatherers had a lot more free time than we do.
So, there are different ways of constructing a society. But I do think there’s something possibly to the idea that when you say technology is built to make us happy, fundamentally what you’re saying is it’s going to lift scarcity. And it may not be that human beings are happier, at least beyond a certain point, in a condition of less scarcity. Or at the very least, that that condition takes a lot of adapting to and culturally re-working around in order to get the most out of it.
brian christian
I want to offer a complimentary story to that, which is I think, as some of this work on the dopamine system and intrinsic novelty seeking behavior in infants shows us, not everything is about scarcity. There’s something pleasurable about just visual unpredictability. That’s why we looking at screen savers. It’s why we looking at campfires or moving water. And I think there’s something valuable there too.
And for me, the natural world offers something that’s really an antidote to the world of tech. There’s something that I have come to, through the classic stuff of mindfulness meditation, and just hiking, and being in nature, looking at trees and finding that beautiful, you realize that there’s a lot more psychological sufficiency in the act of just existing in the world, controlling your own attention, letting the world just as it is interest you and surprise you.
And there’s a problem, which is that no one’s making money when you do that. And so, this is kind of a macro version of what we were talking about earlier in our conversation, about all models are wrong. And in some ways, the danger is that the models can reshape reality to become right. In this case, we are creating these built environments in which there is, just frankly, not a lot of visual novelty. Because people live in an apartment with — they can only view the building right next to them, or there’s not any trees near them, or whatever it is.
And so, in order to get that very primitive level of just visual surprise, you have to check your Twitter feed or whatever it might be. And that puts you into this world of status competition. Because that’s how they get you to engage and create the content that other people consume. But if you just walked to your neighborhood park, the park doesn’t require anything from you. And so, I think it is worth bearing in mind. I’m at some level skeptical to the idea that everything we do is about this kind of positional good, this kind of status competition.
When I’m walking through a park, I don’t think, this is really cool because other people aren’t in this park. You appreciate the absence of other people for not kind of getting in the way of your experience of nature, but you don’t think about it as like, this is cool because it is scarce, or I am the victor of this competition to be in this park, or whatever it is. But I get as much from that as I get from Instagram. That’s the funny thing.
ezra klein
I guess that’s a good place to bring it to an end here. So let’s go back to analog for a minute. What are the three books you’d recommend to the audience?
brian christian
Yeah, so, thinking about both what’s coming down the highway and also how do we think about human motivation and human desire. So, the first book that comes to my mind is by Julie Shah and Laura Major. Julie and I were high school classmates. And she’s now an MIT roboticist who works on aerospace manufacturing and all sorts of things. Their book is called “What to Expect When You’re Expecting Robots.” I think it’s a really interesting and persuasive look at the next decade-ish in terms of human- robot interaction.
I’m also thinking about a book by James Carse, who was a professor of religion at NYU, called “Finite and Infinite Games.” And there’s this wonderful backstory to this book, where he is the professor of religion who attended a game theory conference in the ‘80s. And then writes this book, which is religion meets game theory, meets like Wittgenstein’s “Tractatus.”
It’s this very weird, very unique book that is all about what are people really trying to do? And there are certain things that we do to achieve a specific end that we can envision in advance. Other things that we do in this more sort of horizontal, open-ended way to kind of surprise ourselves or prolong an experience. And I think it’s a useful way of actually thinking about living one’s life. It also maps to the problems in AI in a very interesting way.
The third one, coming back to what we were saying about the pleasure of being in the neighborhood park. I’m thinking about a book by my friend Jenny Odell, called “How to Do Nothing: Resisting the Attention Economy.” And it’s on one level a love letter to her neighborhood park and at another level an invitation to think about a world in which most of our activity is directed at some kind of objective.
And again, I think there are surprisingly relevant resonances here with AI. You can’t make an AI system without an explicit objective function that it’s trying to maximize. What does it mean to quote-unquote “do nothing?” And there’s something, I think, powerful about that. Again, both for thinking about what intelligent machines might be like, but also in terms of thinking about these deep questions of human motivation, what makes life enjoyable.
ezra klein
Brian Christian, your book is “The Alignment Problem.” It is fantastic. I highly recommend it. Thank you very much.
brian christian
It’s been a pleasure. Thank you. [MUSIC PLAYING]
“The Ezra Klein Show” is a production of New York Times Opinion. It is produced by Jeff Geld, Roge Karma, and Annie Galvin. Fact checking by Michelle Harris. Original music by Isaac Jones and mixing by Jeff Geld.
[MUSIC PLAYING]