HomeĐời SốngTim urban là ai

Tim urban là ai

07:20, 20/03/2021

PDF: We made a fancy PDF of this post for printing & offline viewing. Buy it here. (Or see a pĐánh Giá.)


We have sầu what may be an extremely difficult problem with an unknown time khổng lồ solve it, on which quite possibly the entire future of humanity depends. — Nichồng Bostrom

Welcome lớn Part 2 of the “Wait how is this possibly what I’m reading I don’t get why everyone isn’t talking about this” series.

Bạn đang xem: Tim urban là ai

Part 1 started innocently enough, as we discussed Artificial Narrow Intelligence, or ANI (AI that specializes in one narrow task lượt thích coming up with driving routes or playing chess), và how it’s all around us in the world today. We then examined why it was such a huge challenge lớn get from ANI khổng lồ Artificial General Intelligence, or AGI (AI that’s at least as intellectually capable as a human, across the board), and we discussed why the exponential rate of technological advancement we’ve seen in the past suggests that AGI might not be as far away as it seems. Part 1 ended with me assaulting you with the fact that once our machines reach human-level intelligence, they might immediately do this:





This left us staring at the screen, confronting the intense concept of potentially-in-our-lifetime Artificial Superintelligence, or ASI (AI that’s way smarter than any human, across the board), và trying to figure out which emotion we were supposed to have on as we thought about that.11← open these

Before we dive inkhổng lồ things, let’s remind ourselves what it would mean for a machine lớn be superintelligent.

A key distinction is the difference between tốc độ superintelligence and quality superintelligence. Often, someone’s first thought when they imagine a super-smart computer is one that’s as intelligent as a human but can think much, much faster2—they might picture a machine that thinks like a human, except a million times quicker, which means it could figure out in five minutes what would take a human a decade.

That sounds impressive, and ASI would think much faster than any human could—but the true separator would be its advantage in intelligence quality, which is something completely different. What makes humans so much more intellectually capable than chimps isn’t a difference in thinking speed—it’s that human brains contain a number of sophisticated cognitive sầu modules that enable things lượt thích complex linguistic representations or longterm planning or abstract reasoning, that chimps’ brains vì chưng not. Speeding up a chimp’s brain by thousands of times wouldn’t bring hyên ổn khổng lồ our level—even with a decade’s time, he wouldn’t be able khổng lồ figure out how to lớn use a set of custom tools lớn assemble an intricate model, something a human could knoông xã out in a few hours. There are worlds of human cognitive function a chimp will simply never be capable of, no matter how much time he spends trying.

But it’s not just that a chimp can’t bởi what we bởi, it’s that his brain is unable lớn grasp that those worlds even exist—a chimp can become familiar with what a human is and what a skyscraper is, but he’ll never be able to understand that the skyscraper was built by humans. In his world, anything that huge is part of nature, period, & not only is it beyond hyên ổn to lớn build a skyscraper, it’s beyond hlặng to realize that anyone can build a skyscraper. That’s the result of a small difference in intelligence chất lượng.

And in the scheme of the intelligence range we’re talking about today, or even the much smaller range aao ước biological creatures, the chimp-to-human unique intelligence gap is tiny. In an earlier post, I depicted the range of biological cognitive sầu capathành phố using a staircase:3


To absorb how big a khuyến mãi a superintelligent machine would be, imagine one on the dark green step two steps above sầu humans on that staircase. This machine would be only slightly superintelligent, but its increased cognitive sầu ability over us would be as vast as the chimp-human gap we just described. And lượt thích the chimp’s incapathành phố to lớn ever absorb that skyscrapers can be built, we will never be able lớn even comprehend the things a machine on the dark green step can vì, even if the machine tried to explain it to us—let alone vì it ourselves. And that’s only two steps above us. A machine on the second-to-highest step on that staircase would be to us as we are lớn ants—it could try for years to teach us the simplest inkling of what it knows và the endeavor would be hopeless.

But the kind of superintelligence we’re talking about today is something far beyond anything on this staircase. In an intelligence explosion—where the smarter a machine gets, the quicker it’s able to increase its own intelligence, until it begins to soar upwards—a machine might take years to rise from the chimp step to lớn the one above it, but perhaps only hours to lớn jump up a step once it’s on the dark green step two above sầu us, & by the time it’s ten steps above us, it might be jumping up in four-step leaps every second that goes by. Which is why we need to realize that it’s distinctly possible that very shortly after the big news story about the first machine reaching human-màn chơi AGI, we might be facing the reality of coexisting on the Earth with something that’s here on the staircase (or maybe a million times higher):


And since we just established that it’s a hopeless activity khổng lồ try to lớn understand the power of a machine only two steps above us, let’s very concretely state once và for all that there is no way to lớn know what ASI will vày or what the consequences will be for us. Anyone who pretends otherwise doesn’t understand what superintelligence means.

Evolution has advanced the biological brain slowly and gradually over hundreds of millions of years, & in that sense, if humans birth an ASI machine, we’ll be dramatically stomping on evolution. Or maybe this is part of evolution—maybe the way evolution works is that intelligence creeps up more & more until it hits the level where it’s capable of creating machine superintelligence, và that level is lượt thích a tripwire that triggers a worldwide game-changing explosion that determines a new future for all living things:


And for reasons we’ll discuss later, a huge part of the scientific community believes that it’s not a matter of whether we’ll hit that tripwire, but when. Kind of a crazy piece of information.

So where does that leave us?

Well no one in the world, especially not I, can tell you what will happen when we hit the tripwire. But Oxford philosopher và lead AI thinker Nichồng Bostrom believes we can boil down all potential outcomes into lớn two broad categories.

First, looking at history, we can see that life works like this: species pop up, exist for a while, and after some time, inevitably, they fall off the existence balance beam và land on extinction—


“All species eventually go extinct” has been almost as reliable a rule through history as “All humans eventually die” has been. So far, 99.9% of species have fallen off the balance beam, và it seems pretty clear that if a species keeps wobbling along down the beam, it’s only a matter of time before some other species, some gust of nature’s wind, or a sudden beam-shaking asteroid knocks it off. Bostrom calls extinction an attractor state—a place species are all teetering on falling into lớn và from which no species ever returns.

And while most scientists I’ve sầu come across acknowledge that ASI would have the ability to lớn skết thúc humans khổng lồ extinction, many also believe sầu that used beneficially, ASI’s abilities could be used to lớn bring individual humans, & the species as a whole, to a second attractor state—species immortality. Bostrom believes species immortality is just as much of an attractor state as species extinction, i.e. if we manage to get there, we’ll be impervious khổng lồ extinction forever—we’ll have sầu conquered mortality và conquered chance. So even though all species so far have sầu fallen off the balance beam và landed on extinction, Bostrom believes there are two sides to lớn the beam và it’s just that nothing on Earth has been intelligent enough yet khổng lồ figure out how to fall off on the other side.


If Bostrom & others are right, và from everything I’ve sầu read, it seems like they really might be, we have two pretty shocking facts khổng lồ absorb:

1) The advent of ASI will, for the first time, open up the possibility for a species to lớn land on the immortality side of the balance beam.

2) The advent of ASI will make such an unimaginably dramatic impact that it’s likely khổng lồ knoông xã the human race off the beam, in one direction or the other.

It may very well be that when evolution hits the tripwire, it permanently ends humans’ relationship with the beam & creates a new world, with or without humans.

Kind of seems like the only question any human should currently be asking is: When are we going lớn hit the tripwire & which side of the beam will we land on when that happens?

No one in the world knows the answer to lớn either part of that question, but a lot of the very smardemo people have put decades of thought inkhổng lồ it. We’ll spend the rest of this post exploring what they’ve sầu come up with.


Let’s start with the first part of the question: When are we going lớn hit the tripwire?

i.e. How long until the first machine reaches superintelligence?

Not shockingly, opinions vary wildly and this is a heated debate aước ao scientists và thinkers. Many, like professor Vernor Vinge, scientist Ben Goertzel, Sun Microsystems co-founder Bill Joy, or, most famously, inventor & futurist Ray Kurzweil, agree with machine learning expert Jeremy Howard when he puts up this graph during a TED Talk:


Those people subscribe lớn the belief that this is happening soon—that exponential growth is at work & machine learning, though only slowly creeping up on us now, will blow right past us within the next few decades.

Others, like Microsoft co-founder Paul Allen, retìm kiếm psychologist Gary Marcus, NYU computer scientist Ernest Davis, & tech entrepreneur Mitch Kapor, believe that thinkers like Kurzweil are vastly underestimating the magnitude of the challenge & believe sầu that we’re not actually that cthất bại to lớn the tripwire.

The Kurzweil camp would counter that the only underestimating that’s happening is the underappreciation of exponential growth, & they’d compare the doubters to lớn those who looked at the slow-growing seedling of the mạng internet in 1985 & argued that there was no way it would amount to anything impactful in the near future.

The doubters might argue baông chồng that the progress needed lớn make advancements in intelligence also grows exponentially harder with each subsequent step, which will cancel out the typical exponential nature of technological progress. And so on.

A third camp, which includes Nick Bostrom, believes neither group has any ground khổng lồ feel certain about the timeline và acknowledges both A) that this could absolutely happen in the near future and B) that there’s no guarantee about that; it could also take a much longer time.

Xem thêm: Fi:Me / Review Sữa Rửa Mặt E100 Nghệ: Ngừa Mụn Và Liền Sẹo Cực Hiệu Quả

Still others, lượt thích philosopher Hubert Dreyfus, believe sầu all three of these groups are naive for believing that there even is a tripwire, arguing that it’s more likely that ASI won’t actually ever be achieved.

So what do you get when you put all of these opinions together?

In 2013, Vincent C. Müller and Nichồng Bostrom conducted a survey that asked hundreds of AI experts at a series of conferences the following question: “For the purposes of this question, assume that human scientific activity continues without major negative sầu disruption. By what year would you see a (10% / 50% / 90%) probability for such HLMI4 to lớn exist?” It asked them lớn name an optimistic year (one in which they believe sầu there’s a 10% chance we’ll have sầu AGI), a realistic guess (a year they believe there’s a 1/2 chance of AGI—i.e. after that year they think it’s more likely than not that we’ll have sầu AGI), and a safe guess (the earliest year by which they can say with 90% certainty we’ll have sầu AGI). Gathered together as one data phối, here were the results:2Median optimistic year (10% likelihood): 2022Median realistic year (1/2 likelihood): 2040Median pessimistic year (90% likelihood): 2075

So the median participant thinks it’s more likely than not that we’ll have AGI 25 years from now. The 90% median answer of 2075 means that if you’re a teenager right now, the median respondent, along with over half of the group of AI experts, is almost certain AGI will happen within your lifetime.

A separate study, conducted recently by author James Barrat at Ben Goertzel’s annual AGI Conference, did away with percentages and simply asked when participants thought AGI would be achieved—by 2030, by 2050, by 2100, after 2100, or never. The results:3By 2030: 42% of respondentsBy 2050: 25%By 2100: 20%After 2100: 10%Never: 2%

Pretty similar to Müller and Bostrom’s outcomes. In Barrat’s survey, over two thirds of participants believe AGI will be here by 2050 and a little less than half predict AGI within the next 15 years. Also striking is that only 2% of those surveyed don’t think AGI is part of our future.

But AGI isn’t the tripwire, ASI is. So when vì the experts think we’ll reach ASI?

Müller và Bostrom also asked the experts how likely they think it is that we’ll reach ASI A) within two years of reaching AGI (i.e. an almost-immediate intelligence explosion), and B) within 30 years. The results:4The median answer put a rapid (2 year) AGI → ASI transition at only a 10% likelihood, but a longer transition of 30 years or less at a 75% likelihood.

We don’t know from this data the length of this transition the median participant would have sầu put at a 1/2 likelihood, but for ballpark purposes, based on the two answers above, let’s estimate that they’d have said 20 years. So the median opinion—the one right in the center of the world of AI experts—believes the most realistic guess for when we’ll hit the ASI tripwire is = 2060.


Of course, all of the above sầu statistics are speculative sầu, and they’re only representative of the center opinion of the AI expert community, but it tells us that a large portion of the people who know the most about this topic would agree that 2060 is a very reasonable estimate for the arrival of potentially world-altering ASI. Only 45 years from now.

Okay now how about the second part of the question above: When we hit the tripwire, which side of the beam will we fall to?

Superintelligence will yield tremendous power—the critical question for us is:

Who or what will be in control of that power, and what will their motivation be?

The answer lớn this will determine whether ASI is an unbelievably great development, an unfathomably terrible development, or something in between.

Of course, the expert community is again all over the board and in a heated debate about the answer to lớn this question. Müller and Bostrom’s survey asked participants khổng lồ assign a probability to lớn the possible impacts AGI would have sầu on humanity và found that the mean response was that there was a 52% chance that the outcome will be either good or extremely good and a 31% chance the outcome will be either bad or extremely bad. For a relatively neutral outcome, the mean probability was only 17%. In other words, the people who know the most about this are pretty sure this will be a huge deal. It’s also worth noting that those numbers refer lớn the advent of AGI—if the question were about ASI, I imagine that the neutral percentage would be even lower.

Before we dive sầu much further inlớn this good vs. bad outcome part of the question, let’s combine both the “when will it happen?” & the “will it be good or bad?” parts of this question inlớn a chart that encompasses the views of most of the relevant experts:


We’ll talk more about the Main Camp in a minute, but first—what’s your deal? Actually I know what your khuyễn mãi giảm giá is, because it was my khuyến mãi too before I started researching this topic. Some reasons most people aren’t really thinking about this topic:

Humans have sầu a hard time believing something is real until we see proof. I’m sure computer scientists in 1988 were regularly talking about how big a deal the internet was likely to be, but people probably didn’t really think it was going lớn change their lives until it actually changed their lives. This is partially because computers just couldn’t do stuff like that in 1988, so people would look at their computer & think, “Really? That’s gonna be a life changing thing?” Their imaginations were limited lớn what their personal experience had taught them about what a computer was, which made it very hard lớn vividly picture what computers might become. The same thing is happening now with AI. We hear that it’s gonmãng cầu be a big khuyến mãi, but because it hasn’t happened yet, and because of our experience with the relatively impotent AI in our current world, we have sầu a hard time really believing this is going khổng lồ change our lives dramatically. And those biases are what experts are up against as they frantically try to lớn get our attention through the noise of collective daily self-absorption.Even if we did believe sầu it—how many times today have sầu you thought about the fact that you’ll spend most of the rest of eternity not existing? Not many, right? Even though it’s a far more intense fact than anything else you’re doing today? This is because our brains are normally focused on the little things in day-to-day life, no matter how crazy a long-term situation we’re a part of. It’s just how we’re wired.

One of the goals of these two posts is to lớn get you out of the I Like khổng lồ Think About Other Things Camp & inkhổng lồ one of the expert camps, even if you’re just standing on the intersection of the two dotted lines in the square above, totally uncertain.

During my retìm kiếm, I came across dozens of varying opinions on this topic, but I quickly noticed that most people’s opinions fell somewhere in what I labeled the Main Camp, and in particular, over three quarters of the experts fell into two Subcamps inside the Main Camp:


We’re gonmãng cầu take a thorough dive inlớn both of these camps. Let’s start with the fun one—

Why the Future Might Be Our Greademo Dream

As I learned about the world of AI, I found a surprisingly large number of people standing here:


The people on Confident Corner are buzzing with excitement. They have sầu their sights phối on the fun side of the balance beam và they’re convinced that’s where all of us are headed. For them, the future is everything they ever could have sầu hoped for, just in time.

The thing that separates these people from the other thinkers we’ll discuss later isn’t their lust for the happy side of the beam—it’s their confidence that that’s the side we’re going to lớn lvà on.

Where this confidence comes from is up for debate. Critics believe sầu it comes from an excitement so blinding that they simply ignore or deny potential negative outcomes. But the believers say it’s naive sầu lớn conjure up doomsday scenartiện ích ios when on balance, giải pháp công nghệ has & will likely kết thúc up continuing khổng lồ help us a lot more than it hurts us.

We’ll cover both sides, và you can form your own opinion about this as you read, but for this section, put your skepticism away & let’s take a good hard look at what’s over there on the fun side of the balance beam—và try to absorb the fact that the things you’re reading might really happen. If you had shown a hunter-gatherer our world of indoor comfort, technology, and endless abundance, it would have sầu seemed lượt thích fictional magic lớn him—we have to be humble enough to acknowledge that it’s possible that an equally inconceivable transformation could be in our future.

Nichồng Bostrom describes three ways a superintelligent AI system could function:6As an oracle, which answers nearly any question posed to lớn it with accuracy, including complex questions that humans cannot easily answer—i.e. How can I manufacture a more efficient car engine? Google is a primitive sầu type of oracle.As a genie, which executes any high-màn chơi comm& it’s given—Use a molecular assembler to lớn build a new and more efficient kind of oto engine—và then awaits its next command.As a sovereign, which is assigned a broad & open-ended pursuit and allowed to operate in the world freely, making its own decisions about how best to lớn proceed—Invent a faster, cheaper, & safer way than cars for humans khổng lồ privately transport themselves.These questions & tasks, which seem complicated to lớn us, would sound lớn a superintelligent system like someone asking you to lớn improve sầu upon the “My pencil fell off the table” situation, which you’d vì chưng by picking it up và putting it back on the table.

Eliezer Yudkowsky, a resident of Anxious Avenue in our chart above sầu, said it well:

There are no hard problems, only problems that are hard to a certain cấp độ of intelligence. Move the smallest bit upwards , & some problems will suddenly move from “impossible” to lớn “obvious.” Move sầu a substantial degree upwards, and all of them will become obvious.7There are a lot of eager scientists, inventors, and entrepreneurs in Confident Corner—but for a tour of the brightest side of the AI horizon, there’s only one person we want as our tour guide.

Ray Kurzweil is polarizing. In my reading, I heard everything from godlike worship of him và his ideas khổng lồ eye-rolling contempt for them. Others were somewhere in the middle—author Douglas Hofstadter, in discussing the ideas in Kurzweil’s books, eloquently put forth that “it is as if you took a lot of very good food and some dog excrement & blended it all up so that you can’t possibly figure out what’s good or bad.”8Whether you like his ideas or not, everyone agrees that Kurzweil is impressive. He began inventing things as a teenager and in the following decades, he came up with several breakthrough inventions, including the first flatbed scanner, the first scanner that converted text to lớn speech (allowing the blind to lớn read standard texts), the well-known Kurzweil music synthesizer (the first true electric piano), and the first commercially marketed large-vocabulary speech recognition. He’s the author of five national bestselling books. He’s well-known for his bold predictions and has a pretty good record of having them come true—including his prediction in the late ’80s, a time when the internet was an obscure thing, that by the early 2000s, it would become a global phenomenon. Kurzweil has been called a “restless genius” by The Wall Street Journal, “the ultimate thinking machine” by Forbes, “Edison’s rightful heir” by Inc. Magazine, và “the best person I know at predicting the future of artificial intelligence” by Bill Gates.9 In 2012, Google co-founder Larry Page approached Kurzweil và asked hyên ổn lớn be Google’s Director of Engineering.5 In 2011, he co-founded Singularity University, which is hosted by NASA và sponsored partially by Google. Not bad for one life.

This biography is important. When Kurzweil articulates his vision of the future, he sounds fully like a crackpot, and the crazy thing is that he’s not—he’s an extremely smart, knowledgeable, relevant man in the world. You may think he’s wrong about the future, but he’s not a fool. Knowing he’s such a legit dude makes me happy, because as I’ve sầu learned about his predictions for the future, I badly want him to be right. And you do too. As you hear Kurzweil’s predictions, many shared by other Confident Corner thinkers lượt thích Peter Diamandis và Ben Goertzel, it’s not hard khổng lồ see why he has such a large, passionate following—known as the singularitarians. Here’s what he thinks is going to happen:


Kurzweil believes computers will reach AGI by 2029 và that by 2045, we’ll have not only ASI, but a full-blown new world—a time he calls the singularity. His AI-related timeline used khổng lồ be seen as outrageously overzealous, và it still is by many,6 but in the last 15 years, the rapid advances of ANI systems have brought the larger world of AI experts much closer to lớn Kurzweil’s timeline. His predictions are still a bit more ambitious than the median respondent on Müller & Bostrom’s survey (AGI by 2040, ASI by 2060), but not by that much.

Kurzweil’s depiction of the 2045 singularity is brought about by three simultaneous revolutions in biocông nghệ, nanocông nghệ, and, most powerfully, AI.

Before we move sầu on—nanogiải pháp công nghệ comes up in almost everything you read about the future of AI, so come inlớn this blue box for a minute so we can discuss it—