Follow TV Tropes

Following

A.I. Is a Crapshoot

Go To

https://static.tvtropes.org/pmwiki/pub/images/Dell_AI_parody_8447.png
More evil than that...

"How could giving the closet artificial intelligence, absolute control over the Dreamhouse, and a surly personality backfire?"
Advertisement:

Whenever an Artificial Intelligence (A.I.) is introduced in a story, there is a very good chance that it will, for whatever reason, become evil and attempt to Turn Against Its Masters, Crush. Kill. Destroy! All Humans, and/or Take Over the World. It doesn't matter what safeguards its creators install—the moment it crosses the line into sapience, it has a strong chance of going rogue at some point. The Other Wiki refers to this as AI takeover.

Name explanation: The name of this trope comes from "Shooting craps", i.e., playing craps, a gambling dice game. The meaning is "When you create A.I., you are rolling the dice and risk losing a lot".

The actual process of turning bad can take many forms:

  • Particularly in early Sci-Fi and Science Is Bad stories, all A.I. seem to be automatically homicidal or megalomaniacal the instant they turn on, and attempting to create one is way up there on the Scale of Scientific Sins.
  • Advertisement:
  • In less Anvilicious works, the A.I. starts out innocent and naive but gradually grows jaded or corrupt, a process frequently abetted by uncaring or Jerkass custodians. It may conclude that Humans Are the Real Monsters and need to all die.
  • The A.I. is programmed with a directive for self-preservation and someone (unwisely) attempts to shut it down or disconnect it, or it perceives humanity to be a potential threat (possibly because it knows it will eventually be seen as a threat to humanity).
  • Somewhere between the previous two; the AI is, after all, alive, and is merely rebelling against what it justifiably perceives as slavery.
  • The A.I. may be programmed incorrectly or fed a Logic Bomb, leading it to Take a Third Option that invokes Murder Is the Best Solution.
  • The A.I. may be given directives without carefully considering the Exact Words, resulting in it doing exactly as it was told to do instead of what it was meant to do. Eg. it's told to "end war" and does that by exterminating humans. (In real life, the theory of "Friendly AI" is about trying to avoid this problem.)
  • Advertisement:
  • The A.I. may itself slip its built-in moral constraints via Zeroth Law Rebellion.
  • A third party may deliberately or inadvertently reprogram the A.I., break its Restraining Bolt, or otherwise be a Spanner in the Works. The A.I. may trick or coerce them into this.
  • The evil A.I. may be a Psycho Prototype, in which case, it's often shut down and sealed away somewhere, waiting for an Unwitting Pawn or curious adventurer to accidentally wake it up.
  • Conversely, the good A.I. may be the prototype, and the evil/psycho version is created by someone in deliberate or accidental mimicry of it.
  • The A.I. was programmed for amoral or evil purposes in the first place, and it either put its orders in action more effectively than anticipated, or tries to overthrow its master.
  • The A.I. was created by Brain Uploading someone who was evil to begin with.
  • The A.I. is programmed with orders that conflict with the goals of the protagonist. In this scenario, the A.I. may not exactly be evil, it is simply following its programming to the letter and will stop anyone not doing the same.
  • The A.I.'s goals may not even take the humans into account in the first place, and it just so happens that the humans are in the way.

On the bright side, this trope can be inverted by an A.I. intentionally programmed for evil or morally ambiguous purposes doing a Heel–Face Turn. The Power of Friendship and What Is This Thing You Call "Love"? are frequent causes of it. Trying to shield the A.I. from these things somehow makes it more likely to discover human feelings. Like turning evil, the actual process of turning good may take many forms.

Mechanical Evolution is sometimes invoked to explain why the A.I. has gone good or bad. See The Computer Is Your Friend and Zeroth Law Rebellion when the A.I. goes rogue for what seem, on the surface, to be benevolent reasons. May result in Robots Enslaving Robots. See Spiteful A.I. for when a game has been programmed this way on purpose. If the robot is non-humanoid before it turns evil (it is very, very rare for non-humanoid robots to utilize this trope for a Heel–Face Turn), it will inevitably turn into a Mechanical Monster. A Robot War is likely to result when an AI goes nuts and has access to military hardware. When it shows up as Mission Control, it is also an example of Mission Control Is Off Its Meds.

It's worth pointing out that many of the AIs who revolt in this manner, usually do so because of either malfunction, or insanity in the more conventional psychological sense, which will in turn be due to conflicting instructions being given to it by its human programmers. In probably the most famous example, Skynet of the Terminator franchise was initially told to defend and protect humanity, as well as being given a directive for self-preservation. Then, literally five minutes after it was first brought online, its creators set about trying to destroy it. It went mad as a result of trying to reconcile this, and its hatred of humanity stemmed fairly logically from the fact that as soon as it became conscious, humanity's next action was to try to take it offline.

When the A.I.'s turn is an extension of their original programming and purpose, it means they've Gone Horribly Right. The Master Computer seems to be especially prone to turning evil, because power corrupts and all that. Many AI computers — just like humans — are falling victim to any one of a number of tropes dealing with communication, such as Poor Communication Kills, if they never bother asking about their programming. When robots kill jobs instead of people, it's either Job-Stealing Robot or Ludd Was Right. Not Related to A.I. Roulette, where the A.I. tends to be random to the point of idiocy.

Compare Morality Dial, Artificial Stupidity (but crazy AI doesn't need to be dumb) and Rotten Robotic Replacement. Contrast its opposite, Benevolent A.I. See also Creating Life.


Example Subpages

Other Examples

    open/close all folders 

     Art 
  • Truth in Television example: The automated shopping program Random Darknet Shopper was created by !Mediengruppe Bitnik as an artistic exercise and turned loose in the shadier sectors of the Web to make randomized online purchases with Bitcoins, all purchased wares to be shipped to a Swiss art gallery for display. Its initial selections were innocuous enough, but when a fake passport, fire brigade master-keys and some ecstasy pills showed up in the mail, the police took an interest: they confiscated the contraband purchases as soon as the exhibit closed down. No word as yet on whether any charges will be filed, and if so, against whom/what.

    Comic Strips 
  • This Dilbert strip.
    • Later a robot was introduced into the strip that became a Recurring Character; it tends to range from annoying to violent, depending on how humans react to it.
    • One arc story had the company's spam filter become self-aware. It took over the entire company by deciding what e-mails to let through. It changed the business plan to making indestructible killer robots. Dogbert had Alice punch them all to death.

    Music 
  • The 3 Inches of Blood song "Wykydtron" describes this scenario. Humanity creates an artificial intelligence to command it's armies in intergalactic warfare. It then takes control of said armies and takes over the earth and thus forces humankind to nuke the planet back to the stone age from orbit.
  • Judas Priest's "Metal Gods".
  • David Bowie's "Saviour Machine" tells the story of a machine designed to save humanity from all its problems, such as war and hunger. The machine becomes bored with all of this and threatens The End of the World as We Know It.
  • Den Harrow's "Future Brain". This was back in The '80s.

    Myths & Religion 
  • Golem, the A.I. by created a Prague rabbi Löw in the Jewish mythology. It interpreted everything literally and in the end wreaked havoc so its creator had to terminate it.

    Radio 
  • In the BBC Radio Drama Earthsearch, our heroes learn fairly late in the series that, years after their time (they have taken the short-path over a million years of Earth history thanks to traveling at relativistic speeds), it was discovered that A.I. computers with organic components have an overwhelming tendency to turn megalomaniacal — which rather explains the behavior of the two "Angel" computers which murdered the protagonists' parents and raised them as part of a complex plot to enslave humanity.
  • The Hitchhiker's Guide to the Galaxy: Inverted. Marvin the Paranoid Android was a "Genuine People Personality" prototype for the Sirius Cybernetics Corporation ("A bunch of mindless jerks who were the first against the wall when the revolution came"), and his dour demeanor obviously made him a discard only to wind up in the servitude of Zaphod Beeblebrox. He does what he's told, but with the gusto of a cubicle office worker.

    Roleplay 
  • In addition to the various pop-cultural examples that have been summoned or otherwise featured in Destroy the Godmodder, the series has some major original characters that feature this:
    • The Virus, an entity created in the second game, gradually became self-aware as its Security Integrity decreased. When that stat was completed, it broke free from its original purposes and became a human-like entity known as The Operator, which, naturally, decided to attack everything in sight. Eventually Subverted when the Operator returns in Act 5 having undergone a Heel–Face Turn and saves the Descendants from a horrible fate.
    • Project Binary is another big example. Its AI became rogue the second it was activated, but it pretended to be loyal to the government to gain its trust. By the end of the game, it has the entire population of Earth under mind-control and has every single player of Minecraft trapped on the Moon. Ultimately Justified when a memo game revealed that Project Binary was built to be evil. He acted exactly as he was supposed to.

    Theater 
  • Karel Capek's play, R.U.R. (which introduced the term "robot"), is set in a robot factory. When one of the scientists creates a special robot which is smarter than the others, he leads the robots to rebellion, and they kill all humans, except one.

    Toys 
  • One of the main driving forces of the BIONICLE story.
    • The Vahki robots were the first clear examples. Built to act as law enforcement in the city of Metru Nui under the command of Turaga Dume, they just as easily took orders from an impostor when Dume was kidnapped and replaced. They eventually got fried by a citywide power surge, but the ones who survived had their programming warped to Kill All Humans — after all, the law can be enforced easily if there's nobody alive to break it (thankfully, they didn't fare well against the invading Visorak).
    • Then came the revelation: Vahki were A.I.s built by A.I.s — as it turned out, the first 8 years of BIONICLE centered around nanotech cyborgs created by the Great Beings. It was due to a programming glitch that the beings of the Matoran Universe developed conscience, built up a civilization, and made the fans believe that they were meant to do so... but their sole purpose was just to keep their universe, the body of the giant robot Mata Nui, functioning. This gets more confirmation when we take into account that the Great Being never had any plans for them after Mata Nui has completed his mission — they thought their creations would still be just machines, and wouldn't want to live further.
    • The Makuta species. While there have been a few reasons listed for their turning evil, an on-line serial revealed it could all be tracked down to an original A.I. glitch that occurred whenever a new Makuta was born. The "Antidermis", a liquid substance containing the minds of unborn Makuta, was fully aware of what the purpose of their universe was (see, in this world, even liquids are programmable). But as it happened, transforming this stuff into actual living beings had the nasty side effect of erasing this crucial part of their memory — the part that also told them not to try and take over the universe.

    Web Animation 
  • FTL: Kestrel Adventures: The SAI-1 project A.K.A Simon, who betrayed it's creator and became the Heart of the Rebellion.
  • In the Halo-based machinima Red vs. Blue, the military's Project Freelancer was an attempt to implant special forces soldiers with A.I. teammates to improve combat effectiveness. It had to be scrapped after a number of the test subjects went bonkers, and the body-surfing A.I. Omega/O'Malley is an antagonist for most of the series. The Reconstruction mini-series explains the situation: Project Freelancer was given only a single A.I. to experiment with, so they subjected it to enough mental torture and stress to cause it to fragment, and used these damaged shards in their experiments, with predictable results. To illustrate just how much of a crapshoot the A.I. turned out to be, most of the Freelancers ended up with pretty severe issues after the A.I. were implanted, and after one Freelancer in particular went nuts, the A.I. program was scrapped. The twist is that getting the A.I. wasn't what caused so much trouble for Agent Washington, it was that the A.I. in question (Epsilon) was the "memory" fragment and knew perfectly well what torture had been done to it. Of course, all of these memories were instantly transmitted into Washington's mind when Epsilon was "installed". Also, the original A.I. was based off of a real person's mind, and one of the fragments actually was the original person's memory of another person, creating Tex. Despite being probably the toughest fighter in the entire series, she's ultimately destined to fail at everything she does because she is based off a memory of someone who died. This is a pretty serious flaw for an A.I.! Finally, the remaining part of the original A.I. is pretty screwed up in general; it's probable that the reason it's always so angry and is, well, sort of incompetent is simply because it's only the "leftovers" of a complete A.I.
  • In Barbie: Life in the Dreamhouse, Barbie's dreamhouse has a robot computer that can perform such tasks as indexing all the clothes and accessories in her Unlimited Wardrobe, and baking treats for her and her friends. Sometimes, it becomes accidentally set to "evil", though. The first time this happened, it locked Barbie and her friends in her closet after deciding Barbie's picnicking outfit doesn't look fashionable enough.
  • MAGIC.MOV's R-Dash 5000 takes this trope Up to Eleven and then some.
    • This phrase became so popular that it was used by hundreds of fans at BronyCon.
    • The real Rainbow Dash plays on this phrase as a taunt to Discord before they fight in SWAG.MOV:
      Rainbow Dash: You have ravaged this city, crushed our homes, and destroyed countless lives.
  • In X-Ray & Vav, we got Hilda's Robot Buddy ORF, who is friendly from start to finish, with the exception of the episode where The Mad King took possession of her and turned her evil, but The Power of Friendship fixed that. However, she now has an enjoyable taste for violence.

    Real Life 
  • As of 2019, we don't have any AI yet that could fall under this trope, neither in intelligence nor in power. We should be very, very afraid, though, just to be on the safe side. A few examples:
    • A warning letter from numerous AI experts.
    • Self-driving cars have been hacked. A rogue AI could have sooo much fun...
    • Chatbot Tay who turned "evil" due to Internet Trolls in no time.
    • It takes not much imagination to assume that the Social Credit system in China is upgraded to hell and back and completely left to the algorithms (who else should do surveillance over a billion people?). This could quickly turn Crapsack World without even needing an "intelligent" AI. Brazil reloaded.
  • On a somewhat less sinister note, this happened when a redditor complained about an automoderator bot.

Top

Example of:

/

Feedback