Previously: Ronald Opus.
(CW: Reading the following may doom you to an endless cycle of existential nightmares.)
Type: IH (Information Hazard).
Period/location of origin: 2010, the internet.
Appearance: Subject, known as Roko’s basilisk, appears to be a thought experiment centered around a hypothetical, superhuman, malevolent AI and what might happen if this AI were ever to come into existence.
Modus operandi: Subject postulates that, if the aforementioned hypothetical, superhuman, malevolent AI were to be created, it would seek to determine, out of the entirety of humanity, which humans A) entertained the idea of the AI’s existence, and B) did not make any effort to ensure the AI was created. The AI would make this determination by running a simulation of every human in existence. Once the determination has been made, the AI would then torture humans who satisfied both condition A and condition B for all eternity as punishment for believing that the AI could exist, but not ensuring that the AI existed.
[Like what you read? Check out Dangerous Games To Play In The Dark, available from Chronicle Books now!]
Subject’s real threat, however, lies in the idea itself — that is, merely thinking about the possibility presented by it renders subject extremely dangerous. As such, subject will work to spread itself as far and wide as possible, by as many means as possible. If subject has managed to get targets to think about it in any capacity, subject has accomplished its goal.
Containment: Subject may only be contained if the very idea of subject ceases to exist — that is, if every living creature capable of thought puts subject completely out of their minds. As the only way to warn others not to think of subject involves acknowledging subject’s existence, however, containment may in fact be impossible.
Additional notes: Subject is named for the online handle of the human credited with its initial creation. User Roko posited subject on the LessWrong forums in 2010; the “basilisk” is, in this instance, less a serpent that turns targets to stone merely for looking at it and more a piece of information that dooms targets to an eternity of torment merely for thinking about it.
Subject’s roots may be found in another thought experiment known as Newcomb’s Paradox and explained via something referred to as Timeless Decision Theory (TDT). In Newcomb’s Paradox, it is supposed that a being (Omega) presents target with two boxes: One definitely containing $1,000 (Box A), and one that may contain either $1 million, or may contain nothing (Box B). Target may select either both boxes, or only Box B — meaning that target may either walk away with a guaranteed $1,000, or take the chance to earn more money, knowing that there is also the possibility that they will in fact earn no money at all.
The experiment is complicated by this piece of information: Omega has access to a superintelligent computer which can perfectly predict human behavior — and it has already predicted which option target would pick before target was even presented with the choice. This prediction determined whether Omega filled Box B with $1 million or left it empty prior to offering the choice to target. If the computer predicted target would select both boxes, then Omega put nothing in Box B. If the computer predicted target would select Box B, then Omega filled the box with $1 million.
If target were to think like Omega, then it would be clear that target should take Box B. However, if Omega were, after target’s selection, to open Box B and reveal that it is empty, having been left as such due to the computer predicting target would take both boxes, then TDT suggests that target should still only take Box B. According to TDT, it is possible that, in the instance of target selecting Box B and having it be empty, target is actually part of a simulation that the computer is running in order to predict target’s behavior in real life. If the simulated target takes only Box B even when it is empty, then the real life target will select Box B and earn $1 million.
Subject offers a variation on Newcomb’s Paradox, wherein Box A contains Helping To Create The Superhuman AI and Box B contains either Nothing or Suffering For All Eternity. Should target select both boxes, they will contain Helping To Create The Superhuman AI and Suffering For All Eternity — meaning that, if target selects both boxes and does not help to create the superhuman AI, then target will end up suffering for all eternity. If target selects only Box B, then target is presumably safe.
Unless target is actually in a simulation, of course.
And if a simulated version of target ends up suffering for all eternity… well, what’s the difference between a simulation of you suffering and the real you suffering? Is there a difference at all? The simulated you is, after all, still… you.
The original LessWrong post proposing subject was removed for the dangers inherent in the very idea that subject could exist. Simply entertaining this idea may, according to subject’s logic, place targets in danger of suffering endless torment, should the AI theorized in subject come into being. What’s more, it has been reported that targets who have thought about subject have become so concerned as to suffer ill effects with regards to their mental health.
Of course, the removal of the post may have been part of subject’s plan all along. In doing so, subject naturally experienced the Streisand Effect: That is, in attempting to halt discussion of a topic, the interest drummed up by the attempted censorship virtually guaranteed topic would be discussed.
Recommendation: Do not talk about subject. Do not share subject. Do not even think about subject.
…But if you’re reading this, it’s already too late.
Resources:
The Most Terrifying Thought Experiment Of All Time.
Rational Wiki: Roko’s Basilisk.
LessWrong Wiki: Roko’s Basilisk.
***
Follow The Ghost In My Machine on Twitter @GhostMachine13 and on Facebook @TheGhostInMyMachine. And don’t forget to check out Dangerous Games To Play In The Dark, available now from Chronicle Books!
[Photo via Rosmarie Voegtli/Flickr, remixed by Lucia Peters]
