Showing posts with label existential risks. Show all posts
Showing posts with label existential risks. Show all posts

February 16, 2011

Anders Sandberg: Why we should fear the paperclipper

Most people in the singularity community are familiar with the nightmarish "paperclip" scenario, but it's worth reviewing. Anders Sandberg summarizes the problem:
A programmer has constructed an artificial intelligence based on an architecture similar to Marcus Hutter's AIXI model...This AI will maximize the reward given by a utility function the programmer has given it. Just as a test, he connects it to a 3D printer and sets the utility function to give reward proportional to the number of manufactured paper-clips.

At first nothing seems to happen: the AI zooms through various possibilities. It notices that smarter systems generally can make more paper-clips, so making itself smarter will likely increase the number of paper-clips that will eventually be made. It does so. It considers how it can make paper-clips using the 3D printer, estimating the number of possible paper-clips. It notes that if it could get more raw materials it could make more paper-clips. It hence figures out a plan to manufacture devices that will make it much smarter, prevent interference with its plan, and will turn all of Earth (and later the universe) into paper-clips. It does so.

Only paper-clips remain.
In the article, Why we should fear the paperclipper, Sandberg goes on to address a number of objections, including:
  • Such systems cannot be built
  • Wouldn't the AI realize that this was not what the programmer meant?
  • Wouldn't the AI just modify itself to *think* it was maximizing paper-clips?
  • It is not really intelligent
  • Creative intelligences will always beat this kind of uncreative intelligence
  • Doesn't playing nice with other agents produce higher rewards?
  • Wouldn't the AI be vulnerable to internal hacking: some of the subprograms it runs to check for approaches will attempt to hack the system to fulfil their own (random) goals?
  • Nobody would be stupid enough to make such an AI
In each case, Sandberg offers a counterpoint to the objection. For example, in regards to the power of creative intelligences he writes,
The strength of the AIXI "simulate them all, make use of the best"-approach is that it includes all forms of intelligence, including creative ones. So the paper-clip AI will consider all sorts of creative solutions. Plus ways of thwarting creative ways of stopping it.

In practice it will be having an overhead since it is runs all of them, plus the uncreative (and downright stupid). A pure AIXI-like system will likely always have an enormous disadvantage. An architecture like a Gödel machine that improves its own function might however overcome this.
In the end, Sandberg concludes that we should still take this threat seriously:
This is a trivial, wizard's apprentice, case where powerful AI misbehaves. It is easy to analyse thanks to the well-defined structure of the system (AIXI plus utility function) and allows us to see why a super-intelligent system can be dangerous without having malicious intent. In reality I expect that if programming such a system did produce a harmful result it would not be through this kind of easily foreseen mistake. But I do expect that in that case the reason would likely be obvious in retrospect and not much more complex.

February 5, 2011

SAI in the material world

Mondolithic Studios
A meme that gets bandied about by Singularity denialists is the assertion that a rogue super-artificial intelligence (SAI) won't be able to act on its desires and make a real impact on the physical world. It's just a really sophisticated computer, goes the thinking; it couldn't actually reach out and touch someone.

This runs contrary to the concerns of those in the Singularity camp who are gravely concerned that an SAI will be both uncontainable and capable of manipulating physical space in a non-trivial way.

I'd like to present a pair of  arguments that will serve as a warning to those who would like to dismiss this possibility. The first is based on a recent technological breakthrough, the second being more of a thought experiment.

Robotic networking and self-replication

RoboEarth is a system that's allowing robots to build on and learn from the experiences of other robots. Think of it as an internet for robots. As it stands, robotics engineers have to teach their bots to navigate and function in the real world. RoboEarth, on the other hand, collects and centralizes information on objects and navigation, and in turn shares this information with other bots. What this means is that any new robot that's connected to this system will have immediate knowledge of its surroundings.

But it doesn't stop there. A recent breakthrough has endowed the TechUnited AMIGO robot with the ability to download all the information it needs for a specific task and then carry out that task. Check out the video below of AMIGO at work:




If this doesn't blow your mind then you're not paying attention. While the task was simple enough, that of autonomously picking up and serving a bottle of water to a person, the potential implications of this are huge. As Joris Peels of iMaterialize clarifies,
If you would combine Robo Earth, with genetic algorithms that automatically design robots and 3D printing you have a very powerful combination. It would be a system that could design a robot based on its experiences, then give that robot all the information it needed to navigate the world and carry out tasks. Anyone could then 3D print this robot anywhere around the world. And the system would be one of continuous learning and itteration with better robots being made every second. We’re still very far away from this but it is these kind of ongoing developments that make me think that I live in the future.

I think we should really consider the implications of this. I know, it sounds a bit sci fi and off piste. But, we will develop a Skynet at one point and we should consider the implications before we do so.
The scenario I'm imagining, as I'm sure are other Singularity-concerned futurists, would see an SAI co-opt this system (or create versions of its own) and begin to fulfill its intentions through a myriad of self-designed, recursively improving, and remotely controlled agents disbursed around the world.

Plenty of room at the bottom

Okay, so there's that example. The next consideration is something a bit more fantastical (relatively speaking): the potential for an SAI to reshape the planet (or significant portions of it) from the molecular scale upwards. Before you tune out, watch this video, Molecular Visualizations of DNA:




What you're seeing in this video is a very small sampling of the kinds of molecular machinery that's capable of arising through the processes of natural selection. What you're not seeing here, however, is the space of all possible molecular machinery that's capable of arising through intentional design. And what you're definitely not seeing here is the space of all possible molecular machinery that's capable of arising through intentional super-intelligent design.

The kinds of molecular machinery that we're familiar with has come about solely for the purpose of maintaining and propagating complex organisms. We're only beginning to imagine the kinds of molecular-scale processes and devices that might be designed to perform other kinds of functions; the design space is massive.

And this is where an SAI comes in. It's easy to imagine a system similar to RoboEarth in which an SAI can design and disburse both macro and micro scale devices. The only limitations facing such a system would be inherent energy and material constraints, other human or SAI-driven countermeasures, and the laws of physics itself.

Okay, what exactly am I imagining? Given free reign, an SAI could potentially re-arrange all matter on the planet. One possibility is that it could turn the Earth into computronium or anything else it wants. Or, it could remove all toxins and other pollutants from the surface and atmosphere. It could turn the planet into a Venusian hell, or a verdant Utopian paradise. Whatever. In all honestly, I can't even really begin to speculate without knowing the intentionality of a Singularity-surviving intelligence. But suffice to say the scope of its impact on the material world needn't be subtle.

For those of us engaged in foresight activities, the risk is in thinking too small on this matter—or in denying it altogether.

December 4, 2010

Maxwell Mehlman: "Extinction by Design: Genetic Engineering and the Future of Human Evolution" [transforming humanity conference]

Maxwell Mehlman presenting "Extinction by Design: Genetic Engineering and the Future of Human Evolution."

Can genetic engineering make things worse? Possibly. Think of unintended consequences visavis global warming. We have no idea if guided evolution would be any worse or better than unchecked evolution. Misguided evolutionary engineering could cause irrevocable harm. What would be worse would be the extinction of the human strain altogether.

The destruction of the human lineage would make it impossible for us to achieve our goals.

Some guidelines:

1. Do not harm children - Parents should not be allowed to genetically engineering their children so that they are exposed to serious risk, serious bodily or mental harm or impairment that is not outweighed by the potential benefit to the child
2. Do not permit state-sponsored, unethical genetic engineering (i.e. eugenics)
3. Do not terminate the human lineage (habitat loss, genetic drift, environmental catastrophe, inbreeding, extremely large body sizes, inability to reproduce, loss of genetic diversity, introduction of rival species)
4. Do not stifle progress toward understanding the universe

November 9, 2010

Clegg's 'Armageddon science'

Physicist Brian Clegg, author of "Armageddon Science: The Science of Mass Destruction", was recently interviewed by Salon. Among the various threats to humanity discussed—a list that includes nuclear weapons, global warming, particle accelerators, nanobots, and apparently autistics—Clegg also discusses the possibility of a technological singularity. But like so many other so-called experts, he gets this one completely wrong. Fair warning, as this next quote is full of fail:
The Singularity came from the ideas of science-fiction writer Vernor Vinge. It's like the Borg in "Star Trek": something that is a combination of living creature and machine. Followers of the Singularity believe the machine will become more and more intelligent, and that this new creature will be the next step. Homo sapiens will either be subjugated or wiped out.

Frankly, I think the time scales are unrealistic. There is no doubt that technology is going to be used to enhance human beings. We've been doing this for a long time. You could say that the dog enhanced human ability; we created dogs from the wolf and used them for our benefit. In the modern sense, we use technology inside our bodies. But technology always jumps in a different direction than you might expect. The people who talk about the Singularity look at computer technology and medical technology that puts stuff in our bodies and assume the technology will continue to progress in the exact same direction. I think we're much more likely to end up with technology that interacts with us rather than becoming a part of us.

The Singularity is a rather crude idea, and the 2040 prediction is ridiculous, especially if you look at how little cybertechnology has actually developed. Ray Kurzeil has this idea of wanting to live forever, and hoped to keep himself going until the technology developed, so the Singularity is a kind of wish fulfillment. Most of the scientific community, frankly, views him as an eccentric. He gives himself blood transfusions and eats vast numbers of supplements every day, and these are things that actually have very little scientific basis for prolonging life. The Singularity is popular because everyone wants to live forever. It's a very appealing idea.
Once again the cult of Ray Kurzweil is invoked, and as is frustratingly typical of these outside commentaries, transhumanism is conflated with the singularity (not to mention the use of ad hominen to dismiss Ray's predictions). Nowhere in this analysis does Clegg discuss the potential for a runaway superintelligence or an "intelligence explosion" outside of human enhancement. So needless to say, his assessment of the singularity risk is grossly understated.

November 4, 2010

Forecasting filters

For you Fermi filter fanatics, be sure to check out Katja Grace's recent honour's thesis, "Anthropic principles agree on bigger future filters":
My main point was that two popular anthropic reasoning principles, the Self Indication Assumption (SIA) and the Self Sampling Assumption (SSA), as well as Full Non-indexical Conditioning (FNC) basically agree that future filter steps will be larger than we otherwise think, including the many future filter steps that are existential risks.
And be sure to check out the insights from Robin Hanson and Michael Anissimov.

October 31, 2010

Anders Sandberg on the Active SETI risk

Anders Sandeberg, one of my favorite transhumanists and ETI theorists, has finally chimed in on the Active SETI debate (i.e. deliberate attempts to contact extraterrestrial intelligences to let them know about our existence). Active SETI has its fair share of detractors, most notably futurist, science fiction author (and occasional Sentient Developments contributor) David Brin who best articulated his concerns in the piece, Shouting at the cosmos: Or how SETI has taken a worrisome turn into dangerous territory. On the other side of the debate are thinkers like Dr. Alexander Zaitsev who believes we should in fact reach out and touch an alien someone.

Not surprisingly, Sandberg's contribution to the debate is unique and provocative. In his article, Inviting invasion: deep space advertisments and planetary security, Sandberg admits that it's hard to assess the risk, but that we might not like the answers:
There are two aspects to extraterrestrial risks: the probability that the signals will be received by somebody, and that we would (afterwards) wish the aliens did not receive them. Stephen Hawking argued that we should be cautious: to him the probability of aliens was relatively high, but he also thought the probability of them being risky was high...This risk might not be a direct invasion threat, but simply dangerous cultural transmissions: in the past some human societies have fared badly when in contact with more advanced societies. Even a radio signal might consist of an information hazard, for example containing infectious ideas or software. The aliens do not even have to be deliberately malicious: many humans would jump at the chance of converting non-believers to their favourite belief system, thinking they do them a great service.

While optimists about SETI tend to think communications would be benign, it is hard to assign a probability to it. The only thing we can say is that we have not seen any alien communications or even signs of them, which suggests that aliens either do not exist, we are not receiving anything from them (e.g. they are too far away or we are listening on the wrong wavelengths) or they are keeping quiet.

From a species survival perspective we should generally prefer the middle answer. Why? If we are the only ones it means that either intelligent life is exceedingly improbable and we are lucky, or that intelligent life is not so uncommon but something wipes it out before it starts to spam the universe. Bad news. If there are aliens and they keep quiet, then they must have a very good and consistent reason. This could again be something positive or neutral (e.g. they are too alien to communicate, they all do not wish to interfere with us) or something bad (e.g. civilizations that remain obvious fall prey to self-replicating weapons). Only the boring middle answer - that we simply cannot communicate for technical or distance reasons - implies safety.
Ouch. And I'm sure Sandberg would agree that the boring answer is also very likely the most improbable answer--particularly given all the recent evidence indicating that the Galaxy may be teaming with Earth-like planets.

September 17, 2010

Turchin: SETI at risk of downloading a trojan horse

Russian physicist Alexey Turchin contends that passive SETI may be just as dangerous—if not more so—than active SETI:


I was fortune enough to be able to talk to Turchin at the Humanity+ Summit at Harvard earlier this year where he clarified his argument to me.

Turchin worries that humanity may be tricked by an out-of-control script that is propagating throughout the Galaxy. This script, which uses a pre-Singularity civilization as its vector, fools its hosts with a lure of some kind (e.g. immortality, access to the Galactic Internet, etc.) who in turn unwittingly build a device that produces a malign extraterrestrial artificial intelligence (ETAI). This ETAI then takes over all the resources of the planet so that it can re-broadcast itself into the cosmos in search of the next victim.

This concept is similar to Carl Sagan's interstellar transportation machine in Contact, except that it would work to destroy our civilization rather than see it move forward.

It's worth noting that this ETAI and its script may be a mutation of some sort, where no civilization was actually responsible for designing the damn thing. It's just a successful replicative schema that's following Darwinian principles.

It's also worth noting that I warned of this back in 2004.

Moving forward, Turchin suggests we raise awareness of the potential problem, change the guidelines for SETI research and consider the prohibition of SETI before we get our own AI.

Turchin's idea sounds ludicrous, but it's one of those crazy things that causes a nervous laugh. I think Turchin's idea needs to be discussed as there may be some merit to it. We need to be careful.

September 6, 2010

Novae produce gamma-rays. Damn.

Bad news: Novae emit gamma-rays.

We've known for a long time that supernovae produce gamma-rays, but until now it was assumed that novae lacked the power to emit such high-energy radiation. This is bad because novae occur at much greater frequency than super- and hypernovae; we are therefore at a much greater risk of being wiped out by a blast of gamma-ray radiation than previously thought.

The Milky Way experiences about 30 to 60 novae per year, with a likely rate of about 40. Roughly 25 novae brighter than about magnitude 20 are discovered in the Andromeda Galaxy each year and smaller numbers are seen in other nearby galaxies.

Contrast that with supernovae which occur about five times every hundred years.

A nova event should not be confused with a supernova. It is a cataclysmic nuclear explosion caused by the accretion of hydrogen onto the surface of a white dwarf star, which ignites and starts nuclear fusion in a runaway manner. A supernova, on the other hand, is a stellar explosion that is more energetic than a nova. Supernovae are extremely luminous and cause a burst of radiation that can outshine an entire galaxy before fading from view over several weeks or months. During this short interval a supernova can radiate as much energy as the Sun is expected to emit over its entire life span. The explosion expels much or all of a star's material at a velocity of up to 30,000 km/s (10% of the speed of light), driving a shock wave into the surrounding interstellar medium.

Though not as powerful as a supernova, novae are still immensely energetic, emitting the equivalent of about 1,000 times the energy emitted by our Sun every year. And now we can add gamma-rays to its list of nasty excretions.

To say that a gamma-ray blast would be bad for us here on Earth would be a gross understatement. Combined with the effects of a cataclysmic stellar explosion, it is one of the most powerful forces in the Universe, able to sterilize massive swaths of the galaxy. Supernovae can shoot out directed beams of gamma-rays to a distance of 100 light years, while hypernovae disburse gamma ray bursts as far as 500 to 1,000 light years away.

As for novae, the explosion creates a hot, dense, expanding shell called a shock front, composed of high-speed particles, ionized gas and magnetic fields. These shock waves expand at 7 million miles per hour—or nearly 1% the speed of light. The magnetic fields trap particles within the shell and whip them up to tremendous energies. Before they can escape, the particles reach velocities near the speed of light. Scientists say that the gamma rays likely result when these accelerated particles smashed into the red giant's wind.

Previous to this discovery, it was known that the remnants of much more powerful supernova explosions can trap and accelerate particles like this, but no one suspected that the magnetic fields in novae were strong enough to do it as well. Supernova remnants endure for 100,000 years and produce radiations that affect regions of space thousands of light-years across.

These explosions produce highly collimated beams of hard gamma-rays that extend outward from a nova or supernova. Any unfortunate life-bearing planet that should come into contact with those beams would suffer a mass extinction (if not total extinction depending on its proximity to the event). Gamma-rays would eat up the ozone layer and indirectly cause the onset of an ice age due to the prevalence of NO2 molecules.

Life on Earth just got that much more tenuous.

August 4, 2010

Martin Rees on our posthuman future—and avoiding catastophe

Thomas McCabe, writing for Kurzweil AI, recently reported on Sir Martin Rees's address to the Long Now Foundation. McCabe notes,
Over the truly long term, our posthuman descendants will become — not just second-generation intelligences — but thousand-generation or million-generation intelligences. He quoted Darwin on how no species can pass its likeness into the distant future unaltered; in a billion years of biological evolution, we’ve gone from bugs to humans, and technological evolution is a lot faster than biological. Our distant descendants will be not just strange, but completely alien to us.
According to Rees, we not only have unprecedented opportunity, but unprecedented responsibility. If the new technologies we build have a high chance of causing civilization-wide catastrophe for the first time in history, we are responsible for actively preventing that from happening, not just trying to predict it or understand it.

Link.

January 27, 2010

Is geoengineering an existential risk?

Well, Milan M. Ćirković and Richard B. Cathcart think it's a distinct possibility. In fact, it may even (partly) explain the Great Silence. Check out the abstract to their article, "Geo-engineering Gone Awry: A New Partial Solution of Fermi's Paradox":
Technological civilizations arising on such planets will be, at some point of their histories or another, tempted to embark upon massive geo-engineering projects. If, for some reasons only very recently understood, large-scale geo-engineering is in fact much more dangerous than previously thought, the scenario in which at least some of the extraterrestrial civilizations in the Milky Way self destruct in this manner gains plausibility. In addition, we speculate on possible reasons, both physical and culturological, which could make such a threat even more pertinent on an average Galactic terrestrial planet than on Earth.
Be sure to read the entire article (PDF). Learn more about geoengineering. And be sure to read Jamais Cascio's article, "It's Time to Cool the Planet."

November 11, 2009

Let’s get metaphysical: How our ongoing existence could appear increasingly absurd

So the Large Hadron Collider has been shut down yet again – this time on account of a bird dropping a piece of a bagel onto some sensitive outdoor machinery. The incident is not expected to keep the LHC out of commission for too much longer, but it represents yet another strange event that has kept the world’s most infamous particle accelerator out of service. In fact, the LHC has yet to function at full operational capacity since its completion over a year ago.

What makes this all the more interesting is that the Hadron Collider has been dubbed by some observers as a doomsday device on account of its unprecedented size and power. A minority of scientists and philosophers believe that the collider could produce a tiny black hole or a strangelet that would convert Earth to a shrunken mass of strange matter.

It's worth re-stating, however, that this is a fringe opinion. Several years ago, Max Tegmark and Nick Bostrom wrote a piece for Nature in which they concluded that a civilization destroys itself by a particle accelerator experiment once every billion years.

Okay, admittedly, one in a billion seems excruciatingly improbable. But not impossible. And it's this 'shadow of doubt' that has got so many people in a tizzy -- especially when considering that this so-called doomsday machine keeps breaking down. Seems awfully convenient, doesn't it? Are we to believe that this is mere co-incidence? Or is there something more to what's going on?

Now, I'm not talking about conspiracies or sabotage, here. Rather, a number of philosophers are making the case that something more metaphysical is going on.

Take, for example, the quantum immortality theory, which argues that you as an observer cannot observe your non-existence, so you will keep on observing your ongoing existence -- no matter how absurd. Aside from a large grain of salt, you also have to buy into the Everett Many Worlds interpretation of quantum mechanics for this to work. As the universe splinters into probability trees, there are new trajectories that are forced into existence by your ongoing presence; in an infinite universe all observations must be made, no matter how improbable.

Now, at any given time we have to assume that we are living in the most probable of all possible habitable worlds. But that doesn't mean it's true -- it's just an assumption given the absence of sampling data. As quantum probability trees diverge, those that tread into more improbable spaces will begin to splinter with less and less frequency and diversity; there will be a limited number of escape routes given absurd and highly complex (but survivable) existence spaces.

All this can lead to some rather bizarre conclusions -- including the thought experiment in which you attempt to obliterate yourself with an atom bomb, only to have some kind of force majeure get in the way that prevents you from acting on your suicide.

It's important to remember that this only works for your ongoing existence. The rest of the world can burn around you; what matters is that you continue to observe the universe.

Okay, back to Hadron. Let's assume for a moment that quantum immortality is in effect and that the LHC is in fact the apocalypt-o-matic. It can therefore be argued that, because we are all collectively put into peril by this thing, we will never get to observe it working properly. There will always be something that prevents the device from doing what it's supposed to be doing -- everything from mechanical failures through to birds dropping bagels on it.

What's even more disturbing, however, is that these interventions could get increasingly absurd and improbable. It may eventually get to the point where we have to sit back and question the rationality of our existence. The world may get progressively screwed up and surreal in order for our personal existence to continue into the future.

One could already make the case that our collective existence is already absurd on account of our possession of apocalyptic weapons, namely the nuclear bomb. We've already come alarmingly close to apocalypse, including the Cuban Missile Crisis and the infamous Stanislav Petrov incident. Would it be unfair of me to suggest that we should probably have destroyed ourselves by now? I would argue that the most probable of Everett Many World Earths have destroyed themselves through nuclear armageddon, but we happen to observe a version of Earth that has not.

This said, our ongoing existence does not seem ridiculously absurd. There are rational and believable reasons that account for our ongoing existence, namely self-preservation and a rigid safety-check system that has prevented a nuclear accident from happening.

But will the same thing be said a few years from now if the Hadron Collider keeps shutting down? What will happen to our sense of reality if stranger and stranger things start to intervene?

And what about the more distant future when we have even more apocalyptic devices, including molecular assembling nanotechnology and advanced biotechnologies (not to mention artificial superintelligence)? It's been said that we are unlikely to survive the 21st Century on account of these pending technologies. But given that there are some probability trees that require our ongoing existence, what kind of future modes will that entail? Will it make sense, or will the succession of improbably survivable events result in a completely surreal existence? Or will our ongoing presence seem rational in the face of a radically altered existence mode -- like totalitarian repression or the onset of an all-controlling artificial superintelligence?

Hopefully I don't need to remind my readers that this is pure philosophical speculation. Metaphysics is often fun (or disturbing as in this case), but it is no substitute for science. I think we should think about these possibilities, but not to the point where it impacts on our daily life and sense of reality.

But I'm sure we'll all want to keep a close eye on that rather interesting particle accelerator in Switzerland.

September 9, 2009

Welcome to the age of weapons containment: A SentDev Classic

Soon after the end of the Cold War, U.S. President George H. W. Bush declared that a new era had opened up in which he hoped that his country would become a “kinder and gentler nation.” Fifteen years later his proclamation appears naĂŻve and gushing with idealism, but his optimism was understandable given the times; the Soviet Union had just collapsed with the Eastern Bloc going down with it, and all without a single shot fired from an American gun. The world, it seemed, had been rebooted and started anew.

Indeed, economic and cultural globalization quickly ensued, ushering in what we now regard as the post-ideological, post-bi-polar geopolitical era. Democracies and capitalism began to take root in areas completely unaccustomed to such institutions. One prominent political theorist, Francis Fukuyama, was so taken by these turn of events that he declared the new era to be the end of history.

But things haven’t turned out exactly as hoped or planned. The weight of history is still very much upon us. The events of September 11, 2001 were a wake-up call of sorts, a not-so-subtle reminder that politically instigated catastrophes are still a real and potent threat. In a world dominated by the hegemonic power of the United States, the world was introduced to the muscle of asymmetric agitation.

To be sure, today’s geopolitical situation is one in which asymmetric threats -- a phenomenon more commonly referred to as terrorism -- are taken to be the most pressing security concerns. Yet this is only part of the story. We live in an era in which conventional warfare between two or more combating nations of roughly equal power is all but behind us. Given the political and economic compatibility of so many nation states, the need and desire for war has waned considerably. As Margaret Thatcher once famously said, democracies “do not go to war with one another.” Moreover, conventional war, with all of today's high-tech tools of destruction, would surely be suicide.

As current events reveal, however, wars are still occurring -- but to call them such might be a stretch. The situation in Iraq and Afghanistan, for example, are the reactionary spasms of occupation. The current regimes in both countries have been established by the conquerors who are now cleaning up the mess of victory, albeit very poorly and haphazardly.

Still, when it comes right down to it, conflicts such as these are still very 'warlike' in their composition. And despite the decline in all-out war between nations, hostilities are still happening. There are several key factors that can account for this ongoing problem.

First, we still live in an era of the sovereign nation state where war is regarded as the self-justified continuation of diplomacy by other means. Second, there is civilization's insatiable appetite for natural resources – a factor that creates volatility in those resource rich areas whose governments are politically and economically at odds with those who desire the resources. Thirdly, and related to the first point, there are ideological reasons for entering into war, whether it be the spread of capitalism, “freedom,” religion or totalitarian ideology. Lastly, and the one I now want to shift attention to, war is a means to prevent a state from developing and using advanced weapons such as nuclear bombs and bio-weapons.

This last point is a relatively new phenomenon, one that I believe will characterize the 21st century.

The dust is finally settling after the collapse of the Cold War and a new era is starting to unfold before us. It is not an era where anyone will afford to be “kinder” or “gentler,” nor will it be an era in which so-called terrorism is the Great Threat (asymmetric threats cease to be asymmetric when the enemy has access to apocalypic technologies). Rather, the 21st century is revealing itself as the Age of Weapons Containment.

There are already strong indicators that this is the case, at least in theory. The U.S. justified its invasion of Iraq under the guise of ferreting out Saddam’s illusory weapons of mass destruction. George W. Bush was successfully able to garner support for an invasion based around a seemingly tangible and dangerous threat. Sure, the reason for war was falsified, but the incident will prove to be an ominous harbinger of things to come – crises that won't involve red herrings.

Since 9/11 the world’s attention has been pre-occupied with threats of hijacked planes and hassles at the airport. At the same time the United States worked obsessively in the Middle East to guarantee its access to oil (an agenda made all the more pressing as China nears superpower status). During this time, however, the North Koreans were busy developing nuclear weapons. Now the U.S. and the international community are scrambling to figure out what to do about it. North Korea is truly a “rogue” nation if there ever was one, with a psychologically unstable and malicious leader at the helm. The question being asked the world over is: how do you take nuclear weapons away from a country once it has acquired them?

The answer is, you can’t. At least not without engaging in a brutal attack that involves nuclear weapons. Frighteningly, the only option may be to allow North Korea to keep their nukes and work to prevent other countries from joining the Nuclear Club. Yet, as the New York Times recently noted, there are at least 40 countries around the world today that have the technological know-how to develop their own nuclear weapons program. The situation seems untenable.

And it’s poised to get worse. Weapons technologies are increasingly set to increase in sophistication, destructive power, and most frightening of all, accessibility. Nuclear weapons are the first of an entirely new set of apocalyptic technologies that include genetically engineered viruses, self-replicating nanotechnology and robotics, and even malign artificial intelligence. As a consequence of these potential threats, one of two things will happen on the geopolitical stage: either agreement will finally be made on the establishment of transnational authorities, or nations will react with unilateral violence against potential threats in an effort to contain the spread of dangerous weapons.

Unfortunately, it may very well be the latter. The U.S. has already set this precedent by virtue of their invasion of Iraq -- an action in which they disregarded the U.N.'s injunctions. While the U.S. worked to prevent the spread of communism during the latter half of the 20th century, they may very well define their 21st century geopolitical role as the country that works to prevent the spread of apocalyptic technologies.

Rather than rely on international bodies, countries with the resolve and military might will react with force when a perceived threat hits the radar. As a potential example, does anyone think for one minute that Israel will stand idly by while Iran develops their own nuclear weapons? How long will Japan and South Korea hold out before they take action against North Korea?

The goal of world federalism seems as far off as ever. The international community cannot get it together and give the United Nations teeth. The United States bears much of the blame. Consequently, nations are waiting until situations become untenable and they're forced to act on their own.

Looking ahead to the future, similar revelations will occur when when bio-labs are detected in suspicious countries, or as nanotechnology and robotics industries mature. As is the case today, only until the situation looks overwhelmingly dangerous will threatened countries react. It will be an era of reactionary efforts to curtail the development and proliferation of extremely dangerous weapons. Beyond a doubt, the United States will beat a unilateral path as it faces each threat, while all the while undermining the global community and shirking its responsibility to help build a powerful international regulatory regime.

But even if transnational agencies can be created, these institutions will still have to face the same issues. Preventing the wide-spread and unchecked accessibility to apocalyptic technologies will redefine the human condition. We may have to live with a multitude of existential threats in perpetuity. This is not a good situation.

In the meantime, all eyes are on North Korea. How the international community deals with this crisis will be a very important precursor to how they will deal with even greater threats in the coming decades.

Originally posted on October 16, 2006.

April 26, 2009

Economist: Safe without the bomb?

The April 11-17 edition of The Economist asks the question: can the world be safe without the bomb? A nuclear-free world may never come about, they argue, but there can be safety in trying:

Nuclear weapons cannot simply be wished away or uninvented. The technology is over 60 years old and the materials and skills needed are widely spread. Still, by infusing his idealism with a dose of realism Mr Obama can do more to create a safer world than simple “Ban the bomb” slogans ever could.

For zero nukes would make no sense if this left the world safe for the sorts of mass conventional warfare that consumed the first half of the 20th century. How many bombs would be needed to prevent that? And with what co-operation and controls to keep these remaining weapons from use? It is hard to say what sort of nuclear future would be more stable and peaceful until you get a lot closer to zero. Happily, the difficult steps needed to get safely to low numbers would all be needed for zero too. Mr Obama’s vision is helpful if it gets people thinking about imaginative ways forward.

Mr Obama is already committed to using the goal of zero to shape his future nuclear plans. Both America and Russia still have far more nuclear warheads than either wants. Even George Bush, no dewy-eyed disarmer, negotiated cuts down to 1,700-2,200 apiece by 2012 (from the 6,000 agreed upon after the cold war had ended) and was ready to go lower. Encouragingly, Mr Obama and his Russian counterpart, Dmitry Medvedev, have agreed that a modest cut will accompany new weapons-counting rules to be fixed by the end of the year, with more ambitious reductions to follow. All the official five except China have been trimming their arsenals too.

Hmm, now where have I heard this before? Oh, yeah -- right here on this blog.

Read the entire article.

March 30, 2009

The perils of nuclear disarmament: How relinquishment could result in disaster

Most everyone agrees that humanity needs to get rid of its nuclear weapons. There's no question that complete relinquishment will all but eliminate the threat of deliberate and accidental nuclear war and the ongoing problem of proliferation.

Indeed, the ongoing presence of nuclear weapons is the greatest single threat to the survival of humanity. To put the problem into perspective, there are currently 26,000 nuclear warheads ready to go -- 96% of which are controlled by the United States and Russia. These two countries alone could unleash the power of 70,000 Hiroshimas in a matter of minutes. In the event of an all-out nuclear war between the U.S. and Russia, it is estimated that as many as 230 million Americans and 56 million Russians would be killed by the initial blasts. The longer term impacts are nearly incalculable, but suffice it to say human civilization would be hard pressed to survive.

Given the end of the Cold War and the establishment of the START agreements, the idea of a deliberate nuclear war seems almost anachronistic. But the potential nightmare of an accidental nuclear exchange is all to real. We have already come very close on several occasions, including the Stanislav Petrov incident in 1983. We are living on borrowed time.

The assertion, therefore, that we need to completely rid ourselves of nuclear weapons appears more than reasonable; our very survival may depend on it. In fact, there are currently a number of initiatives underway that are working to see this vision come true. President Barack Obama himself has urged for the complete eliminate of nuclear weapons.

But before we head down the path to disarmament, we need to consider the consequences. Getting rid of nuclear weapons is a more difficult and precarious proposition than most people think. It's important therefore that we look at the potential risks and consequences.

There are a number of reasons for concern. A world without nukes could be far more unstable and prone to both smaller and global-scale conventional wars. And somewhat counter-intuitively, the process of relinquishment itself could increase the chance that nuclear weapons will be used. Moreover, we have to acknowledge the fact that even in a world free of nuclear weapons we will never completely escape the threat of their return.

The Bomb and the end of global-scale wars

The first and (so far) final use of nuclear weapons during wartime marked a seminal turning point in human conflict: the development of The Bomb and its presence as an ultimate deterrent has arguably preempted the advent of global-scale wars. It is an undeniable fact that an all-out war has not occurred since the end of World War II, and it is very likely that the threat of mutually assured destruction (MAD) has had a lot to do with it.

The Cold War is a case in point. Its very nature as a "war" without direct conflict points to the acknowledgment that it would have been ludicrous to engage in a suicidal nuclear exchange. Instead, the Cold War turned into an ideological conflict largely limited to foreign skirmishes, political posturing and espionage. Nuclear weapons had the seemingly paradoxical effect of forcing the United States and the Soviet Union into an uneasy peace. The same can be said today for India and Pakistan -- two rival and nuclear-capable nations mired in a cold war of their own.

It needs to be said, therefore, that the absence of nuclear weapons would dramatically increase the likelihood of conventional wars re-emerging as military possibilities. And given the catastrophic power of today's weapons, including the introduction of robotics and AI on the battlefield, the results could be devastating, even existential in scope.

So, while the damage inflicted by a restrained conventional war would be an order of magnitude lower than a nuclear war, the probably of a return to conventional wars would be significantly increased. This forces us to ask some difficult questions: Is nuclear disarmament worth it if the probability of conventional war becomes ten times greater? What about a hundred times greater?

And given that nuclear war is more of a deterrent than a tactical weapon, can such a calculation even be made? If nuclear disarmament spawns x conventional wars with y casualties, how could we measure those catastrophic losses against a nuclear war that's not really supposed to happen in the first place? The value of nuclear weapons is not that they should be used, but that they should never be used.

Upsetting the geopolitical balance

Today's global geopolitical structure has largely converged around the realities and constraints posed by the presence of apocalyptic weapons and by the nations who control them. Tension exists between the United States and Russia, but there are limits to how far each nation is willing to provoke the other. The same can be said for the United States' relationship with China. And as already noted, nuclear weapons may be forcing the peace between India and Pakistan (it's worth noting that conventional war between two nuclear-capable nations is akin to suicide; nuclear weapons would be used the moment one side senses defeat).

But should nuclear weapons suddenly disappear, the current geopolitical arrangement would be turned on its head. Despite its rhetoric, the United States is not a hegemonic power. We live in a de facto multi-polar geopolitical environment. Take away nuclear weapons and we get a global picture that looks startlingly familiar to pre-World War I Europe.

Additionally, the elimination of nuclear weapons could act as a destabilizing force, giving some up-and-coming nation-states the idea that they could become world players. Despite United Nations sanctions against invasion, some leaders could become bolder (and even desperate) and lose their inhibitions about claiming foreign territory; nations may start to take more calculated and provocative risks -- even against those nations who used to be nuclear powers.

Today, nuclear weapons are are being used to keep "rogue states" in check. It's no secret that the United States is willing (and even thinking about) bombing Iran as it works to develop its own nuclear weapons and threaten the region, if not the United States itself (Iran will soon have intercontinental ballistic capability; same for North Korea).

It can be said, therefore, that the composition of a nuclear-free world would be far more unstable and unpredictable than a world with nukes. Relinquishment could introduce us to an undesirable world in which new stresses and conflicts rival those posed by the threat of nuclear weapons.

It should be noted, however, that nuclear weapons do nothing to mitigate the threat of terrorism. MAD becomes a rather soft deterrent when "political rationality" comes into question; rationality can be a very subjective thing, as is the sense of self-preservation, particularly when nihilism and metaphysical beliefs come into play (i.e. religious fanaticism).

Nukes could still get in the wrong hands

Even in a world where nuclear weapons are eliminated it would not be outlandish to suggest that fringe groups, and even rogue nations, would still work to obtain the devices. The reasons for doing so are obvious, a grim turn of events that would enable them to take the rest of the world hostage.

Consequently, we can never be sure that a some point down the line, when push comes to shove for some countries or terrorist groups, that they'll independently work to develop their own nuclear weapons.

Dangers of the disarmament process

Should the nuclear capable nations of the world disarm, the process itself could lead to a number of problems. Even nuclear war.

During disarmament, for example, it's conceivable that nations would become distrustful of the others -- even to the point of complete paranoia and all-out belligerence. Countries would have to work particularly hard to show concrete evidence that they are in fact disarming. Any evidence to the contrary could severely escalate tension and thwart the process.

Some strategic thinkers have even surmised that there might be more incentive for a first strike with small numbers of nuclear weapons on both sides, where the attacking nations could hope to survive the conflict. As a result, it's suspected that the final stage of disarmament, when all sides are supposed to dismantle the last of their weapons, will be an exceptionally dangerous time. As a result, disarmament is paradoxically more likely to increase the probability of deliberate nuclear war.

And in addition, concealing a few nukes at this stage could give one nation an enormous military advantage over those nations who have been completely de-nuclearized. This is not as ridiculous as it might seem: it would be all too easy and advantageous for a nation to conceal a secret stockpile and attempt to gain political and military advantages by nuclear blackmail or attack.

Conclusion

I want to make it clear at this time that I am not opposed to nuclear disarmament.

What I am trying to do here is bring to light the challenges that such a process would bring. If we're going to do this we need to do a proper risk assessment and adjust our disarmament strategies accordingly (assuming that's even possible). I still believe that we should get rid of nuclear weapons -- it's just that our nuclear exit strategy will have to include some provisions to alleviate the potential problems I described above.

At the very least we need to dramatically reduce the number of live warheads. Having 26,000 active weapons and a stockpile the size of Mount Everest is sheer lunacy. There's no other word for it. It's a situation begging for disaster.

All this said, we must also admit that we have permanently lost our innocence. We will have to live with the nuclear threat in perpetuity, even if these weapons cease to physically exist. There will never be a complete guarantee that countries have completely disarmed themselves and that re-armament won't ever happen again in the future.

But thankfully, a permanent guarantee of disarmament is not required for this process. The longer we go without nuclear weapons, the better.

March 17, 2009

Transcription: Risks posed by political extremism

Below is a transcription of the talk I gave last year at the IEET's symposium on Building a Resilient Civilization. The title of my presentation was: "Democracy in danger: Catastrophic threats and the rise of political extremism."

If you don't want to read the entire transcription you can always read my summarized version, "Future risks and the challenge to democracy," or just watch the video.

Many thanks to Jeriaska for putting this together.

The world’s democracies are set to face their gravest challenge yet as viable and ongoing political options. George Dvorsky—who serves on the Board of Directors for the Institute for Ethics and Emerging Technologies and Humanity+ while bloging at Sentient Developments—presents on how given these high stakes situations, democratic institutions may not be given the chance to prevent catastrophes or deal with actual crises.

Risks Posed by Political Extremism

dvorsky_01.png

We have had some great ideas today. Jamais, Mike, Martin and others have put together some concrete ways in which we can use the institutions that we have at our disposal and go about dealing with existential risks in a way where we can still live as civilized human beings and not have our lives diminished appreciably.

What I am going to be speaking about today though is another path that we could take. What if we do not set up a resilient civilization? Aside from the obvious result of there being a global-scale catastrophe or outright human extinction, there is the path down to political extremism, which might be a natural consequence of the emergence of existential risks in the first place. There is a double-edged sword here.

dvorsky_02.png

That is essentially the theme that will be at the base level of this discussion. Throughout the 20th century there was a lot of political perturbations and restructuring that happened, largely driven by the maturation of the nation-state and industrial economies. These states had to figure out very quickly how to manage a civilization and redistribute wealth. You had a number of ideological forces coming into play to argue this exact point. It explains a lot of the tensions that did occur in the 20th century, most dramatically in the form of totalitarianism. Meanwhile, the democratic nations, who resisted this radicalism, were working to develop the welfare state and put Keynesian economics into practice. Not everyone had to fall into these radical patterns.

Looking ahead into the 21st century, the politics and restructuring of our political institutions will be driven by the demands of mitigating existential risks. In particular, managing the impacts of disruptive technologies and the threats posed by apocalyptic-scale weapons and ongoing environmental degradation are subjects we absolutely do have to talk about.

This restructuring is already happening. We are living in a post-911 world. That was an example of “superterrorism,” where it was a rather devastating event that impacted not just those immediately involved but on our psychologies and societal sensibilities. We have seen what has happened in the seven years since then. A reaction is happening. Looking into the future, we can certainly anticipate that there will be more of this.

Even the term “existential risks” is slowly starting to seep its way into the popular vernacular. Some of you might have caught during the second debate between John McCain and Barack Obama, McCain actually used the term “existential risks” during the debate, which caught me by surprise. He was not speaking of course of humanwide extinction. He was speaking specifically about the state of Israel and its current situation as it is being confronted with what it perceives to be an existential risk in the form of a nuclear attack from a rather belligerent Iran right now. This I expect not only to enter into our parlance more frequently but to seep itself into public policy in a very real way in terms of our institutions and the ways in which we react to these threats.

dvorsky_03.png

This is a three-part presentation. Existential risks will change the political landscape. There is an ever-growing multiplicity of threats. The next generation of threats are on the horizon. Today, many of us have spoken about what exactly those are going to be. What is worse is that there is an increased chance that there is going to be an increased chance of unchecked development and proliferation. Given the nature of information technology today and the access to information, there is this increased threat that anybody, given the right information and the right resources, can put these threats together. Further compounding that is an increased sense of motivation among some groups, whether done at the individual level or state level, to put these weapons into practice.

dvorsky_04.png

We are collectively speaking here about global catastrophic risks. It does not result in humanwide extinction but it is an event of such devastation that each person on this planet will be impacted in some way. When that happens we will be overnight put into a reactive state. That will be an order of magnitude beyond anything that has happened since World War II, in terms of our entire structure having to be based around this reaction.

Another issue as we are dealing with these catastrophes, maybe politicians will lose faith in the kinds of remedies we are trying to articulate today. Perhaps they may think that we are naive and that a rather more heavy hand is required. They may lose faith in the ability of democracies to deal with these particular problems and look for more extreme measures and more draconian answers to these problems.

dvorsky_05.png

The future may unfortunately not be exactly what we had hoped for. At the end of the Cold War, you had this rather optimistic sense that things were finally going to change for the better. You had this feeling that Western liberal democracy was in the process of triumphing and that free market capitalism was about to envelope the world, you had all this talk about the end of history, and a new world order. However, this has not been the case. The last fifteen years have been replete with violence and has resulted arguably in a more unstable world from a geopolitical perspective. We have hardly reached the sense of a new world order or an end of history.

dvorsky_06.png

In the 21st century, with the introduction of apocalyptic threats, it will not be politics as usual. These times are going to call for more drastic measures. The mere presence of existential risks will result in there being more political extremism. This is a negative feedback loop. Those who feel persecuted will have an added impetus to use weapons of mass destruction.

What kind of challenge does this pose to democracy? Democracy is still the exception and not the rule. Right now, according to some surveys, as little as 45-48% of the world can be classified as being truly free. This was a figure that was down to around 35% as early as 1973. Speaking about cognitive biases today, we may actually be the victims of an ideological bias. We have this idea that democracy is here and here to stay. This may not in fact be the case.

We may actually be living in a rather extraordinary time in human history where the social and technological situation is such that we have reached an equilibrium where we can have the strong democracies and freedoms that we have. We may be entering into a period of social disequilibrium where democracies simply will not be able to withstand the pressure of the threat of existential risks. A big part of what I am speaking about today is the perceived need for extremism. There will be an unprecedented need for social control. That does not mean merely looking at people and wondering what they are doing, but getting them to do something or not to do something.

It can also be not only in anticipation of a disaster happening and getting individuals to work to prevent it from happening but it can be in response to an actual catastrophic event on the scale of World War II, where suddenly everyone in society is mobilized to participate in disaster recovery.

dvorsky_08.png

The second part is defining and anticipating political extremism. What do we mean by political extremism? It is a relative term with no fixed political baseline. It can be used to describe the actions or ideologies of individuals or groups that are outside of a perceived establishment. The views and actions of perceived extremists are typically contrasted with what we would consider moderate opinion. It is also used to describe those groups who violate the sense of there being a common moral standard. Again, it does not have to be a group of individuals outside society, but could be members of your own government.

dvorsky_09.png

Extremists can direct their angst either internally, within their own group, or outside of the confines of the state, or both. They will often be accused of advocating violence against the will of society and the actions are often considered beyond what is necessary. Some might consider the actions of the outgoing administration of the United States in the introduction of the Patriot Act and warrantless wiretapping are extreme measures beyond the call of what is necessary. The term is almost always used as a pejorative. No one declares themselves to be an “extremist.”

dvorsky_07.png

What will give rise to various forms of extremism? When times are good, you are not going to have agitation. You are not going to have calls for more radical political action. When you have economic, environmental and civil strife, that is when things get churned up. There is an old revolutionary credo that the worse things are, the better. That is the only time when people are going to be willing to do something about their situation. Reasons for extremism at the state level are when the ends are felt to justify the means. There is seen to be just cause to implement policies that reduce our civil liberties.

dvorsky_10.png

Radicalism begets radicalism. When you had the emergence of fascism in 1930’s Europe, arguably the reason for it was that it was a reaction to Bolshevism and the threat of Collectivism. You had radicalism establish itself in one spot, and others freaking out about it and deciding to embrace radicalism to counter it. Given prescriptions for the 21st century on how we will address existential risks, you could see the same bipolar stratification happening with different forms of radicalism. I cannot speak to them specifically right now, but perhaps later when I go over particular threats themselves. You could envision a radical progressive group countered by a radical luddite group, for example, as they gain political power.

dvorsky_11.png

There is the issue of future shock, as well. The idea that accelerating change will upset a lot of our psychological sensibilities is something that Émile Durkheim referred to as “anomie.” Because things are changing so quickly, the public does not really know what to grab onto. Their footing is lost, social norms are changing. Today, of course, we are feeling it through such things as gay marriage. Social changes are happening with greater rapidity than they have in the past. I look at things going on at Google with a certain degree of reverence because I simply cannot imagine how they are piecing things together.

How the Nazis took advantage of this in the 1930s was by appealing to people’s sensibility and nostalgia. While on the one hand they would tap into what technology had to offer, they would offer them comfort in values that they could relate to. Accelerating technological change can also give rise to a call for radical action. You also need a psychologically primed populace. A catastrophic event will create a populace that is hysterical or primed for a strong central authority figure to tell them what to do and how to do it. The regime will take advantage of that and scare the populace with threats in hopes of keeping them in control that way. This is a common political tactic. A psychologically primed populace will be both welcoming of and supportive of a regime that comes in and take away those civil liberties.

Despite the previous slide, where I went over conditions, there are drivers that will speak to 21st century politics. How are we going to restructure our politics such that we will avoid wisespread catastrophes and human extinction? We are going to need our political institutions deal with the managing of ongoing disasters. Right now what comes to mind are the environmental disasters that appear to be looming on the horizon. However, one could imagine a pandemic of some sort, a nano-disaster, not at an existential level. Meeting the demands of managing these disasters and dealing with disaster recovery are going to be cost prohibitive to say the least. As usual, the economy is going to be a huge issue in the 21st century.

There are some secondary drivers as well that may again lead to some more sense of instability. That is the emergence of disruptive technologies. That will have profound socio-economic effects that have been somewhat outside the bounds of this symposium. There is nanotechnology, for example, and what it will mean to the manufacturing sector. There is robotics and what it will mean to unemployment. Then there will be dealing with the restructuring of society that will be on the scale of the previous industrial revolution that will happen because of AI.

dvorsky_12.png

Another possibility, one that could be considered extreme, is do-nothingism. That is denial, underestimation and circumvention. Today we are seeing corporate interests interested in obfuscation and disinformation. For their own selfish interests they will see the world burn. Political self-preservation, scared politicians that are simply afraid to do anything, may only be interested in maintaining the status quo. It also might be an issue of human nature. We are victims often of our own psychologies, denial or our inability to grok probabilities, and so on. We may fail to realize a threat is on the horizon.

There is also the possibility of isolationism. Countries that do not want to deal with this will not follow the path of meeting this threat. Also, there is backwardness, simply not comprehending the issue at hand. Look at Africa over the past thirty years now largely being in denial of the AIDS epidemic. That is a massive disaster in its own right.

dvorsky_13.png

Moving on to assessing the various threats, I broke them down into authoritarianism, totalitarianism, paramilitary groups and radical social groups. What are the drivers for an iron-fisted government? I cannot stress enough both the positive and negative injunctions for making people act in a certain way. A state that distrusts and is fearful of its own citizens will pick quick, easy and lazy ways to deal with crisis situations, such as circumventing due process by tweaking the Constitution or putting it aside.

dvorsky_14.pngdvorsky_15.pngdvorsky_16.png

This type of regime can manifest itself through an existing democratic regime like the United States or Canada. It is just a matter of putting the right tools into action to make it happen. For instance, the Nazis were voted into power. It could be a coup, a junta, an occupying force through the establishment of police states and dictatorships. There was a coup d’etat in Pakistan in 1999 with Musharraf taking control of a nuclear-capable country. What can an authoritarian state do to deal with the threat of there being a disaster? It can declare a state of emergency, suspend elections, dissolve the government, ban all criticism and protests, reduce privacy and mobility rights, conduct illegal arrests and torture.

dvorsky_17.png

This would certainly be a threat in terms of the gross diminishment of our rights and civil liberties. It would provoke reactions internally and abroad, working to destabilize the situation even further. I think you could devote an entire symposium to the future of totalitarian threats. It is important to address, simply because it is an existential risk unto itself.

dvorsky_18.png

Why would an authoritarian threat want establish totalitarianism? It is the need for absolute social control, to mobilize the people and get them to think the way that is in the state’s best interest through the imposition of an ideological imperative. This could be a religious imperative, or it could be regime based on getting itself back on its feet after a global catastrophe. I think some of the same technologies that would work to enable totalitarianism in the 21st century would also work to undermine the instantiation of totalitarianism. Namely, communications technologies could be use to hack the system.

This could manifest itself through an existing regime. It might not be through the radical left or right as we know it. It could be the emergence of neo-totalitarianism under new conditions. The political tools would be the same as authoritarianism, but other things that they have at their disposal could be a monopoly on all political activity, on the ideology, on the means of coercion and the means of persuasion. All economic and professional activities of the state would become subject to the state.

January 5, 2009

Global Catastrophic Risks talks now available online

Welcome and IntroductionsWATCH - LISTEN

Anders Sandberg PhD, Oxford University
“Global Catastrophic Risks: An Overview, and Caution about Risk Assessments”
WATCH - LISTEN

Jamais Cascio, IEET Fellow, and research affiliate, Institute for the Future
“Building Civilizational Resilience”
WATCH - LISTEN

Eliezer Yudkowsky, Research Associate. Singularity Institute for Artificial Intelligence
“Cognitive Biases in the Assessment of Risk”
WATCH - LISTEN

Martin Hellman PhD, Professor Emeritus of Electrical Engineering, Stanford University
“Risk Analysis of Nuclear Deterrence”
WATCH - LISTEN

Bruce Damer, CEO, DigitalSpace & Director, Contact Consortium
“Asteroids and Comets: Mitigating Impact Risks and Stepping Stones to a Sustainable Space Program”
WATCH - LISTEN

Mike Treder, Executive Director, Center for Responsible Nanotechnology
“Nanotechnology’s Global Risk and Promises of Resilience”
WATCH - LISTEN

Tad Hogg PhD, Social Computing Laboratory, Hewlett Packard
“Distributed Surveillance with MEMS or Nano-scale Sensors”
WATCH - LISTEN

Kattesh V. Katti PhD, Director, Cancer Nanotechnology Platform, Professor of Radiology, University of Missouri
“Green Nanotechnology: An Economic And Scientific Initiative For the Future Of Human Civilization”
WATCH - LISTEN

Alan Goldstein PhD, CEO of Industrial Nanobiotechnology
“The A-Prize: Tracking The Global Race To Break The Carbon Barrier”
WATCH - LISTEN

J. Storrs Hall PhD, author Beyond AI
“The Weather Machine: Nano-enabled Climate Control for the Earth”
WATCH - LISTEN

George Dvorsky, Director, Institute for Ethics and Emerging Technologies
“Risks Posed by Political Extremism”
WATCH - LISTEN

James J. Hughes PhD, Exec. Director, Institute for Ethics and Emerging Technologies
“Strengthening Transnational Governance to Mitigate Risks”
WATCH - LISTEN

Jamais Cascio, IEET Fellow, and research affiliate, Institute for the Future
“Strategies for Civilizational Resilience in the Face of Global Catastrophic Risks” WATCH - LISTEN