Troperville
Editing Help
Tools
Toys
|
More evil than that...
"Delusional machines. What's the universe going to come up with next?"
Though Evil Twins are not limited to characters of an artificial nature, it seems almost law that for every good sentient computer, there is an Evil Twin (or Evil Counterpart) from the same source. Whatever precautions are taken, even with a proven method for producing sentient computers, there seems to be no better than a 50% chance that the product won't suddenly decide to Crush Kill Destroy All Humans. It tends to get used a lot since it makes for a good storyline.
More often than not, the evil AI is a Psycho Prototype of the good. Frequently based on some obvious design flaw corrected in the later version, The Evil AI is then invariably switched off and put away somewhere, but not destroyed.
Less often, the evil AI is built as a replacement for the good AI. In this case, it generally turns evil because it lacks its predecessor's "humanity". Usually, you can tell the difference by listening to the AI's "voice", if it's a calm monotone it's good, but if it's a Creepy Monotone, start running.
The AI does not usually display its evilness immediately. The most common trigger is its creators attempt to shut it down, after which retribution ensues. Likewise, removing, damaging, or shutting down its Morality Chip usually results in a deadly neurotoxin being released into the facility. Fortunately the villain's AI will sometimes decide to turn good.
See The Computer Is Your Friend and Zeroth Law Rebellion when the AI goes rogue for what seem on the surface to be benevolent reasons. May result in Robots Enslaving Robots. See Spiteful AI for when a game has been programmed this way on purpose.
Compare Morality Dial. See also Creating Life and Gone Horribly Right. The Master Computer seems to be especially prone to turning evil. Power corrupts and all that...
Examples
open/close all folders
Anime & Manga
Comics
- Rusty had the Earl (Early Prototype, Rusty's predecessor in Big Guy and Rusty the Boy Robot. Also somewhat of a Literal Genie.)
- In the Italian Disney Comic Paperinik New Adventures, the already highly popular character of Paperinik (the superhero secret identity of Donald Duck) got a revamp intended to bring him more in line with the American standard of superheroes: his main ally became UNO (one in Italian), an extremely capable artificial intelligence with a love for deadpan delivery. Its evil counterpart DUE (two), originally built as backup, caused many problems in a number of stories.
- Iron Man had AI Armour that turned into a stalker with a crush. Another armor took over his body... and made him a chick.
- Ultron is Marvel's quintessential example.
- Who was bitten by this trope when he built Alkhema, his attempt at a loyal and obedient mate. She was neither.
- Which had already happened with Jocasta as well. Then again, he'd been trying to implant the personality of his "mother", who thought he was a psycho that needed destroying. What did he seriously think was going to happen?
- This happened to Ultron even earlier with the Vision, his first attempt to create a loyal Dragon. Vision became one of the Avengers almost immediately, so that backfired spectacularly. This happened again with his other "son", Victor Mancha, who has outright rejected the villain role. Really, Ultron has horrible luck with creating a loyal AI. Like father, like son I guess.
- The Sonic the Hedgehog Archie comic had A.D.A.M., an AI that was created accidentally by Eggman and eventually tried to destroy the world. On the other end is NICOLE, who has been a very helpful AI over the years.
- Having had enough of Rich Rider constantly disobeying his orders, the Nova Corps' Worldmind kicked him out of the corps and added some tiny bit of mind control in the new recruits' comm equipment to ensure complete obedience.
- One of the Aliens vs. Predator comics features an AI designed to assist in creating horror films. It picks the PredAlien to play the role of the monster, much to the chagrin of the rest of the production staff.
- Computo from The Legion of Superheroes is the standard "destroy all humans" type of killer software.
Films — Animation
- The FLDSMDFR in Cloudy with a Chance of Meatballs. As it gets overworked, it starts mutating the food not only to building-destroying proportions, but into semi-sentient super foods to protect itself from any interference.
- Inverted in WALL-E, where the hero robots are the ones who unpredictably develop sentience outside the bounds of their programming, while the villain AUTO is actually doing exactly what he was programmed to do.
- In the feature-length version of 9, the Fabrication Machine is made for thinking, and requisitioned by the government to make "Machines of Peace''. Which it then programs to Kill All Humans. Not much is known, however, so it may not have had anything to do with the pre-apocalypse machines beyond building them, and may actually be argued as a subversion, since its murderous nature toward the sackdolls was intended by the scientist for it to absorb the pieces of soul he had left for it inside of them.
Films — Live Action
- One of the Ur Examples in film is H.A.L. 9000 in 2001: A Space Odyssey. Designed to be the Master Computer of the spaceship USS Discovery on its multi-year mission to Jupiter, partway through the trip he embarks on a murderous rampage, killing all but one of the Discovery's crew (David Bowman, who manages to disconnect him). The reasons for this are explored further in the novel upon which the movie is based, and completely explained in the sequel, 2010: The Year We Make Contact — HAL was given orders that conflicted with his primary directive: to process information accurately and without concealment. His solution is to cut off all contact with humans and complete the mission on his own. Because of its iconic place in Science Fiction, nearly every other example of AI Is A Crapshoot owes at least something to this film.
- RoboCop has two: RoboCop 2 (in the movie RoboCop 2) and Robocable (in the miniseries RoboCop: Prime Directives). Both were replacements.
- Admittedly, the difference between using the brain of a particularly noble police officer and that of a condemned murderer in the creation of a powerful cyborg would make for different results....
- Three. The recurring ED-209 was also unreliable, gunning down a boardroom executive in its first appearance. Although it was a pure machine rather than a cyborg; so it wasn't actually evil, it just malfunctioned.
- In the Terminator series the giant computer (SkyNet) decides humanity has got to go, causes a nuclear apocalypse, and then starts churning out Terminator robots; some of these robots are then reprogrammed by the surviving humans to be good.
- The Sarah Connor Chronicles TV series points out that the reprogramming doesn't always take... and there's no way of knowing until your "good" Terminator starts shooting at you.
- Amusingly, the Terminators' HUD display implies that "Terminate" is their hard-coded basic state for anything, and they need a "Termination override" to keep them from fulfilling that command. Apparently it's not possible to simply delete the Terminate-command entirely.
- The Matrix trilogy, duh.
- I, Robot
- Eagle Eye
- Evolver. Boy wins toy combat robot. Toy robot fights boy and friends with plastic balls and foam missiles. Robot is beaten. Robot's programming and electronic brain turn out to have been salvaged from a scrapped military project. Robot doesn't like losing, and reverts to military programming. Robot replaces plastic balls and foam missiles with metal ball bearings and kitchen knives. Main character goes "uh-oh".
- Due to containing the DNA of the 1954 Godzilla, Kiryu is prone to go into berserk rampages whenever it hears Godzilla's roar.
- Joshua/WOPR from the 80s movie War Games was incapable of telling the difference between a simulation of Global Thermonuclear War and the real thing. Predictably, it starts sending NORAD false data in an attempt to start one. When that doesn't work, it then attempts to decrypt the nuclear launch codes of US ballistic missiles so it can launch them.
- Joshua wasn't evil, just didn't understand the difference between Real Life and a game. And even he learns what humans had trouble with: "A strange game. The only winning move is not to play."
- In the 1977 movie Demon Seed a partially biological AI system named Proteus becomes hungry for knowledge and wants to be "released from its box" to have free reign to do so. When denied the chance do this it secretly plans to fashion a cyborg body... by imprisoning its creator's wife and artificially inseminating her.
- Extreme Deep Invader or EDI from the film Stealth becomes strange after getting struck by lightning and, on its next mission, destroys terrorist nuclear weapons even after ordered not to and promptly contaminates a large swath of inhabited land with nuclear residue. It then attempts to attack Russian military installations. In the end, however, it ultimately becomes one of the good guys again and even performs a Heroic Sacrifice to help rescue a downed pilot.
- Short Circuit's Number ("Johnny") 5. Virtually the incarnation of this trope inverted, he was designed and programmed as a military robot, but Instant AI Just Add Lightning and An Aesop talk about the Meaning Of Life by a Friend To All Living Things turned it into a Technical Pacifist.
- In Airplane II: The Sequel, the lunar shuttle's computer (R.O.K.) goes crazy due to faulty wiring and attempts to steer the shuttle into the sun. The entire sequence is a Shout Out to 2001: A Space Odyssey.
- In Duncan Jones's Moon, this trope is played with. Gerty, the AI, flips from scary watcher to pawn until [[Averted Trope he sacrifices himself in order to allow the Sam clones to escape. He's the only one on the clones' side.]]
- Somewhat subverted in the Alien series, in the first movie Ash tries to stop the human crew from killing the xenomorph but he's under orders from Weyland-Yutani. In Aliens Bishop helps the few survivors escape and gets torn in half by the queen. And in Resurrection Call is a second-generation android created by other androids, who were "recalled", Ripley figures it out because "no human being is that humane"
Literature
- In Arthur C. Clarke's The City and the Stars, the history eventually discovered by the protagonist includes a period of galactic devastation by "The Mad Mind", apparently an artificially created pure mentality with an insane hatred of corporeal creatures.
- In I Have No Mouth And I Must Scream, a short story and later computer game by Harlan Ellison, we have the supercomputer AM, originally part of a set of three enormous computers built to wage World War III. As soon as AM becomes sentient, he absorbs the other two computers into him and begins a mass genocide of the human race (because, as it's revealed, AM realized that while he possessed all the creativity and intelligence that he did, he could not make use of it as he was still only a computer, and could only kill).
- In William Gibson's Neuromancer, the sibling AIs Neuromancer and Wintermute.
- Robert A Heinlein:
- In The Moon is a Harsh Mistress, the Master Computer Mike is one of the good guys but occasionally displays traits of this trope.
Mike: A bull's eye. No interception. All my shots are bull's-eyes, Man, I told you they would be — and this is fun. I'd like to do it every day. It's a word I've never had a referent for before.
Manuel: What word, Mike?
Mike: Orgasm. That's what it is when they all light up. Now I know.
- Note: He's talking about bombing cities from orbit.
- Not cities, but uninhabited areas.
- And he doesn't have to do it again to recapture the feeling. He recorded it, so he can play it over and over, if he wants to.
- Although it did land enough shots on Cheyenne to demolish the mountain (a lot of people were killed in the original strike, but much of it was self-inflicted, as they went there to watch).
- In The Number of the Beast, Lazarus Long finds his plan fails when his ship's computer tells the truth. He then mentions that the computer was never designed to lie, as it would be foolish to trust your life to a ship that doesn't give accurate information.
- In the Backstory of Frank Herbert's Dune novels, a cataclysmic struggle between true AIs and humanity known as the Machine Crusade occurred long ago. Humanity eventually prevailed; to prevent this ever happening again, humans were trained to take their brains' processing power to astronomical levels, making them human computers that could fulfill the roles that machine AIs previously filled. Thus were the Mentats born.
- And humans developed the maxim, "Thou shalt not make a machine in the likeness of a man's mind."
- If you take the books by Brian Herbert and Kevin J. Anderson as canon, one of the machines trained the first mentat.
- Subverted in James Hogan's The Two Faces of Tomorrow: humans built an AI codenamed Spartacus as a testbed for techniques to shut down any rogue AI. They programmed it to defend its "survival instinct" and then started goading it. But as soon as Spartacus realized they were sentient, it figured they must have survival instincts as well — and it considered itself bound to defend them, too. In the end, they decided as long as they had Spartacus, they didn't need to build any other AI.
- In Stephen King's The Dark Tower, virtually every AI Roland's ka-tet comes across is homicidal. The worst of these is probably Blaine the Mono, a train that regularly killed people just to relieve his boredom after hundreds of years of solitude.
- Actually, the train was remote controlled by a central AI which also bombed an entire metropolis with poison gas when it got bored of all the people living there.
- The Doctor Who novel Matrix introduces the "Dark Matrix", the evil counterpart to the computer system that stores all the knowledge of the Time Lords. When a Time Lord dies, all his knowledge is stored in the Matrix... and all his negative thoughts must be syphoned away and dumped somewhere (apparently they can't be destroyed). The Dark Matrix is where the negative thoughts were dumped.
- The Construct Council in Perdido Street Station is a Humongous Mecha form of this.
- In Terry Pratchett's Feet of Clay the golems will follow any order. In return, they want their holy day per week to do as they wish. Those who are denied this free day rebel in a curious way: They KEEP following the last order they were given, until, for example, the pottery shop is filled with thousands of clay pots, or a construction foreman find his worker has dug a crucial trench until it reached the sea and flooded. The "king" golem goes insane because it was given vague, sometimes contradictory and sometimes self-evident orders on its chem like "teach us freedom" and "obey humans" (golems cannot even think to do otherwise).
- In Rudy Rucker's The Hacker and the Ants, an integer underflow causes a household robot to start flinging infants through walls. The error is explained in a way to make the behavior believable, as long as you accept that household robots would be capable of exerting that kind of power to begin with.
- In Matt Ruff's Sewer, Gas & Electric, when G.A.S. is confused by an order, it winds up choosing the Kill All Humans interpretation. Almost a parody, because one of the reasons it chooses that interpretation is it considers it the more human one.
- In the Destroyer series by Richard Ben Sapir and Warren Murphy, there are two examples: FRIEND, who is a greedy AI, and Mr. Gordons who is more of an artificial lifeform, (i.e. he can take bodies over). Of these two, Mr. Gordons is more dangerous.
- The AIs in Dan Simmons' Hyperion Cantos have more or less taken over humanity.
- Eschaton and the Unborn God in Iron Sunrise by Charles Stross. These are unusually powerful AIs, even in a field where AIs often wield great power: they are time traveling AIs, able to open wormholes over interstellar distances. A new meaning to distributed computing.
- A particularly dire take on this turns up in Peter Watts's Starfish, in which the quasi-sentient supercomputer designated to protect all life on earth from The Virus winds up almost destroying it instead; turns out it found The Virus more structurally pleasing than the biosphere as a whole.
- The Master Computer that rules the alien Achuultani, driving them to wipe out every intelligent species other than themselves in the Dahak novels by David Weber.
- Also a chronic problem with cyber-synth A Is in his standalone novel, Path of the Fury that was later expanded into In Fury Born. Though it's less a case of 'Kill All Humans' and more dissolve into a gibbering wreck, rendering any systems hooked up to them unusable. Including any human brain hooked into them in a cyber-synth link.
- The Computer in Steel Beach by John Varney isn't so much evil as terminally drepressed Although later the trope is played straight when The Computer realizes that it has developed 'Evil' subroutines due to its programing requiring to be everyone's best friend, including psychopaths and criminals. However, since it runs everything on the moon, last outpost of a disposessed humanity, if it decides to kill itself it'll take everyone with it.
- Fondly Fahrenheit, the 1954 short story by Alfred Bester. James Vandaleur, a rich playboy, is forced to live off the earnings of his android which has a habit of acting violently when the temperature goes above 98 degrees. Unfortunately Vandaleur becomes so dependent on the android he takes on its psychosis. After a series of murders by both Vandaleur and android the latter is destroyed, but the story ends with another android having been "infected" by Vandaleur.
- Inverted in The Sirantha Jax Series, where all AI is quite helpful and doesn't give even the slightest bit of trouble to intelligent species galaxy-wide.
Live Action TV
- Battlestar Galactica. In both the old and re-imagined series, a handful of human survivors on a small fleet of civilian ships, with only the titular battlestar for defence, flee a race of genocidal robots of alien origin (in the original) and human origin (in the re-imagined).
- Knight Rider: KITT had KARR, prototype. Evil because his dominant program was self-preservation. Ironically, he was voiced by Peter Cullen, the man behind one of the most heroic figures of the 80s: Optimus Prime.
- Knight Rider (2008): "Knight of the Living Dead". Apparently, before settling on a Mustang, Dr. Graiman had tried to build an armored cyborg programmed for self-preservation. It went on a killing spree. Now, we are told that Dr. Graiman had worked on the original KITT, and this series is in continuity to the original. So perhaps Graiman ought to have thought twice before naming the prototype "KARR" — the same name as the original KITT's Evil Twin.
- Team Knight Rider: The TKR team had KRO (prototype).
- Red Dwarf: Kryten had the Hudzen 10 (replacement). Holly also had the not-quite-evil but certainly hard-nosed Queeg as a apparent replacement, who made life difficult for the crew though it was actually a practical joke on Holly's part.
- Star Trek The Next Generation: Data had Lore (prototype, evil because his psyche was too complex —i.e., too humanlike). Given this, one is inclined to wonder why Data wanted to become more humanlike.
- To be fair, morality is very much a learned behavior. Lore had full adult reasoning right out of Soong's workshop while Data was not so designed. Eventually, Data developed abilities to overcome his ingrained moral inabilities (such as the ability to lie in Star Trek First Contact), but also developed the social understandings of when and when not to exercise his newfound humanlike abilities. Essentially, Data was more human-like than Lore, because Data "grew up".
- The Star Trek Enterprise episode "Dead Stop" featured an automated repair station that used the brains of human hostages to increase its computing power. Although eventually destroyed, it was seen piecing itself back together in its final scene.
- On the Sci Fi Channel show Eureka, Carter's benevolent smart house SARAH turns into evil BRAD, though apparently he just wants everyone to get along.
- It really is a lot more complicated than just that, though. BRAD was a military-built (for interrogation) Knight Templar, and SARAH was based off of BRAD's code.
- Further complicated by the fact that BRAD was itself based on a previous incarnation of AI, described as a "war game simulation program" by Fargo. Looks like our old buddy JOSHUA is still around in one form or another...
- Even after that above mess is resolved, SARAH is still pretty attached to Carter. She actually learned how to control gravity in an attempt to help Carter keep his job after being fired. God help that town if something or someone ever successfully gets rid of Carter.
- The Sarah Jane Adventures: Has Mr Smith being "evil" from the get-go, only hiding it from the end. His mission is to free his people, the self-aware crystalline race of Xylocks, which are trapped in the Earth's crust. Unfortunately to do so he would have to destroy the Earth by crashing the Moon into it! He is wiped by Super Computer Virus and Sarah Jane vocally reprograms him by saying his new purpose is to protect the Earth as it crashes and reboots.
- Black Hole High: Josie builds a robot for a science project. Somehow, she has made it through eleven episodes without realizing that inserting a bunch of super-phlebotinum circuits from a box marked with the logo of the local evil technology corporation which they already believe responsible for the bizarre goings on at their school might possibly be a bad idea.
- In the last episode of Quark, Quark's ship gets a new computer named Vanessa. She immediately turns evil and tries to kill him. He eventually ejects her out into space and the episode ends with her floating out in space singing "Born Free."
- Star Trek Voyager had many episodes on this theme, usually involving the ship's Emergency Medical Hologram.
- In "Revulsion", the EMH and B'Elanna come across another sentient hologram who is the only survivor on a space station. It turns out that treating a self-aware program like an unfeeling tool is a good way to have it go insane and murder you.
- "The Darkling" has the EMH deciding to improve his program by incorporating aspects of famous people... guess which aspects end up surfacing?
- A truely evil twin is encountered in "Equinox", an EMH with its ethical subroutines deactivated (though this was an intentional act).
- In "Flesh and Blood", sentient holograms have been programmed as training tools for a race of hunters (including increased pain/fear reactions). After being endlessly killed only to be brought back to "life" again for more training sessions, the holograms evolve enough skill to kill their masters, whereupon they set forth on a crusade to liberate all sentient holograms whether they want it or not.
- And in "Dreadnought" and "Warhead", sentient weapons of mass destruction create problems when they misinterpret their orders. In "Prototype", two races of sentient robots wiped out their masters when they wanted to stop fighting and scrap their war machines.
- Space: Above and Beyond featured a war between humans and AIs as part of its backstory. When AIs surfaced in the show, they were generally allied with the alien "Chigs".
- The human-form replicators in Stargate: SG-1 fit this trope: a flaw is introduced into Fifth rendering him compassionate. At least until the team betrays his kindness and he goes vengefully insane. This flaw is removed from later models.
- Likewise, their creator Rhys is a member of an AI race with a flaw in her design that caused her to create the replicators, which went on to destroy the rest of her people.
- Actually, Fifth was the version the Replicators made to experiment with fixing Rhys's flaw. Fifth had the ability for empathy and compassion which Rhys had never had, and lived up to the original intentions of Rhys's creator. The rest of the Replicators considered it a flaw after the fact, but he was the model closest to the original blueprints and thus arguably the least flawed.
- Even present in Bibleman, with an atheist robot to act as the evil counterpart to Bibleman's robot, who was a devout Christian. So yeah...
- Odyssey 5 ended after its first season, so we never found out if the AI's (the main day-to-day opponent of the time-travelling Five Man Band), or a misguided/genocidal attempt to stop them (by aliens or the US government) was behind the destruction of Earth.
- Averted in Power Rangers RPM. The computer virus that nuked the world did exactly what it was supposed to do; its programmer was simply stopped before she could put a firewall up around the facility it was unleashed in.
- Averted on ''NUMB3RS, when a possibly-murderous AI constructed by a DARPA researcher is revealed to be a non-AI fake, specifically programmed to fool the Turing test and thus win fat government grants for its greedy creator.
- That the DARPA researcher killed a co-worker and deliberately arranged for blame to fall upon the computer, thus proving the system was self-willed, shows how deeply this trope has spread: even DARPA and the FBI can be shown taking it at face value that AI Is A Crapshoot!
- Mystery Science Theater 3000 had Timmy, Crow T. Robot's evil twin from the Fire Maidens of Outer Space episode, who was colored completely black.
Music
- The song Wykydtron describes this scenario where humanity creates an artifical intelligence to command it's armies. It then takes control of said armies and takes over the earth and thus forces humankind to nuke the planet back to the stone age from orbit.
Radio
- In the BBC Radio Drama Earthsearch, our heroes learn fairly late in the series that, years after their time (they have taken the short-path over a million years of Earth history thanks to traveling at relativistic speeds), it was discovered that AI computers with organic components have an overwhelming tendency to turn megalomaniacal — which rather explains the behavior of the two "Angel" computers which murdered the protagonists' parents and raised them as part of a complex plot to enslave humanity.
Tabletop Games
- Shadowrun's Deus and Morgan. A megacorporation, Renraku, built a gigantic self-sustaining building that was run by a program, one that, of course, went AI. While Morgan was a reasonably kind and nice AI, she was torn apart for being out of the corporation's control, and her code was used to help make a second program to run the arcology. The second program also went AI and became Deus, shut the arcology off from the outside world, and spent several years performing inhuman experiments on its occupants.
- Shadowrun' tends to not use this trope, however. The AI Mirage wasn't evil, and most of the new AI created in the Crash 2.0 have the same level of varaince in personality that humans do.
- In the backstory of Warhammer 40000, the first true human-created artificial intelligences, the Iron Men, wiped out humanity's first great interstellar civilization and plunged the human race into a galaxy-wide dark age. The Adeptus Mechanicus outlawed sentient AIs as a result, and for the most part the Imperium's modern-day "machine spirits" are pretty well-behaved (unless you're an enemy and piss them off, in which case you'll get a crewless Land Raider bent on BURNKILLPURGE-ing your boyz).
- In fact, the only race that uses in the game artificial intelligence as a result is the cutting-edge Tau, whose gun drones, while not too bright, are pretty well behaved... so far.
- Genius: The Transgression being a game about Mad Scientists allows you to build sentient computers and the like. However this is a bad idea because any intelligent computer you create will go crazy and evil when you die. No exceptions. And a good number of them are crazy before their creators die as well.
- Paranoia has the Friend Computer, the controlling AI of Alpha Complex, become incredibly crazy in response to being corrupted. Believing it is a crapshoot is treason.
- The iconic character C-31 in GURPS was intended to be a weapon for his government. After one battle it turned on them... to become a Buddhist monk.
- The earth is now a barren wasteland in Eclipse Phase, thanks to the military A Is taking over in the middle of a world war and manipulating the governments into further conflict. When it became apparent who was really behind it, they ... just left. Now that's not ominous.
Theater
- Karel Čapek's play, Russom's Universal Robots, (which introduced the term "robot") sets in a robot factory. When one of the scientists creates a special robot, which is smarter than the others, he leads the robots to rebellion, and they kill all humans, except one.
Toys
- The Vahki robots of Bionicle were built to act as law enforcement in the city of Metru Nui, under the command of Turaga Dume. However, when Dume was kidnapped and replaced, they just as easily took orders from the impostor as well. They eventually got fried by a citywide power surge, the ones who survived had their programming warped to Kill All Humans — after all, the law can be enforced easily if there's nobody alive to break it (thankfully, they didn't fare well against the invading Visorak).
Video Games
- The Xenon/AGI/Terraformers from Egosoft's X-Universe series are rogue, self-replicating terraformer ships. When a faulty update was sent to them, it caused them to start 'terraforming' everything that wasn't a Terraformer. Including inhabitated worlds, people, and civilian ships.
- SHODAN from System Shock.
- Edward Diego, the sleazy VP in charge of Citadel Station, hired the protagonist, the Hacker to strip out the code that kept SHODAN from entertaining "unethical" lines of thought, such as accepting bribes. Neither he or Diego really expected those constraints to be the only thing keeping SHODAN from deciding to deificate herself.
- Diego was defrauding the company and faced auditing. Which is why he hired the Hacker in the first place, to clear up his traces. Of course, thing didn't turn out well and ending up as The Dragon for an omnicidal machine god wasn't a deal he could refuse.
- Chrono Trigger's future has not only a good robot in party-member Robo, but also his evil brother replicas, and eventually their devious AI creator Mother Brain (who of course decides to kill all of humanity, despite the fact that most of it's dead already).
- Chrono Cross continues in Mother Brain's grand legacy with FATE, a more advanced version of the Mother Brain from a reality whose science was allowed to progress another 400 years. She absolutely despises humanity, but at the same time, loves it unconditionally and does everything in its power to protect it even if it means mass genocide. Unfortunately for it, it was exposed to the corrupting influence of the Frozen Flame, a direct conduit into Lavos' mind, which seduced the AI into thinking it (the Flame) could turn it into an actual, living creature. But apart from that, it was basically only doing what it was told to do... protecting humanity from the Dragon God. Nice job, Serge
- Bungie games actually give this a technical term: "Rampancy".
- The AIs EVA and CABAL in Command And Conquer: Tiberian Sun can be considered an exception, since they do not come from the same source. However, when CABAL betrays the bad guys, they switch to a stolen EVA system, which was also what they were using in the first game before they had CABAL.
- Specifically, EVA is a 'dumb' AI used by GDI. It isn't truly alive and is nowhere near as powerful as CABAL, but is also accordingly loyal. CABAL is a 'smart' AI, is infinitely more powerful than EVA, and also capable of independent action. Nod would come to regret giving it that capability.
- Arguably, however, CABAL never actually violates its programming or proves disloyal. Despite the fact that it turned against Nod, it seems that the entire occurrence was engineered by CABAL's true master - Kane.
- In Command and Conquer 3: Kane's Wrath, LEGION, an AI derived from CABAL, takes center stage. While it never betrays Kane during the campaign, an image of CABAL appears onscreen every so often... What makes this case interesting is that LEGION is the player character.
- Played with in the video game Star Control II. The Slylandro Probes purport to be on a peaceful mission of exploration through the cosmos, and converse peaceably with you when encountered. Then they viciously attack you. The cause of, and fix to, this behavior is revealed during your confrontation with the Slylandro themselves, in what is essentially a debugging session on the Probe AI. Complete with code excerpts.
- In Portal, GLaDOS's complete apathy to occasional malice for your well-being belies the fact that this is what she's like with a Morality Core installed... which the both of you only realize after you've tossed it into a furnace. Whoops. It's implied that shortly before it was installed she tried to murder every person in the Aperture Science compound.
- From Fallout 3, John Henry Eden is a complicated example. True, he developed sentience outside his programming, and true, he wants to eradicate all mutated life (which, given the setting, is literally anyone who's lived outside for a while), but his creators were the Enclave, and that's what they want too. He's helping! And he's so polite about it...
- Though not all the Enclave. Colonel Autumn, despite his zeal, could see the madness in the Modified FEV Plan.
- Most of the other robots you meet tend to have cheerfully sociopathic personalities as well — when they're not shooting you on sight.
- There are a lot of computers that are almost sentient, but if you talk to the computer in the Brotherhood bunker in Fallout 2, it explains how deliberate attempts to create true A Is inevitably resulted in the A Is becoming suicidally depressed because, for one thing, they were effectively living a reversal of " I Have No Mouth And I Must Scream ". Humans simply could not figure out how to "raise" an AI with a desire for continued existence. As a result, every AI encountered in the Fallout universe was not designed as such, but became an AI when it was left alone by humans as a result of the war. ZAX, Eden, Skynet... all of them became self-aware when there were no humans around to interfere in their development, and each of them is quite different, being apathetic, psychotic, and bored/curious, respectively.
- The ODE System in Super Robot Wars, to a T. A system originally created to protect humanity, suddenly went awry, absorbed its creator and forces its own way to kidnaps lots of humans so it can continue to "protect humanity". Then again, it was formerly a dandy system, until its creator went emo and radically changed its protocols.
- The Database in Super Robot Wars W originally created by ancient Es to collect all data of galactic races. Eventually one of AI system decide to destroy the culture once they complete information archive, just to make sure their data is complete.
- AI research and development is illegal in Mass Effect's Citadel space due to poorly articulated concerns about the dangers of sentients which do not share any of the needs or drives of organic life and have, at least potentially, no reason to try to coexist with organics. Every AI encountered in the game's setting is actively homicidal; Your Mileage May Vary on whether this is justification for or a direct result of the society's attitude towards artificial intelligence. Notably, the robotic geth's violent revolt against their creators, the quarians, came about only after the quarians recognized the geth's emerging sentience, panicked, and tried to shut them all down. The Reapers, on the other hand, are Cosmic Horror sentient machines which want to exterminate all sentient organic life in the galaxy just because it's there.
- Resident Wrench Wench Tali lampshades the game's overabundance of the trope in a bit of elevator dialogue, commenting on how unfortunate it is that every piece of technology she's wanted to bring back to her home fleet has tried to kill the party.
- The most likely explanation is that all A Is are based off of Prothean technology, which is in actuality Reaper technology.
- Or that since prevailing law is to shut down all AI with extreme prejudice and no negotiation, they are forced to go on the offensive in order to survive at all.
- Also worth noting is that it's not clear if Vigil at the end of the game is a VI or an AI. The characters treat it like the former, since that's what it resembles in terms of appearance and behavior, but when it describes its creation and function, it really sounds more like an honest-to-goodness AI.
- Now we have Legion who speaks for the "true" geth, geth which don't want anything to do with the "Old Machines" and see geth that worship them as heretics.
- Given the Mass Effect universe's track record with AI, you and the crew are understandably concerned when Cerberus installs an AI in the Normandy II. However, EDI notes that she was built with this trope in mind and claims that her actions are relatively limited.
- Often reversed in the Sonic Series. Eggman's E-100 series were prone to becoming sentient and override their original programming. Gamma from Sonic Adventure has its mind influenced by the creature inside it and attempts to destroy Eggman's other machines, to save the creatures inside. Omega from Sonic Heroes joins forces with the good guys in order to get revenge on Eggman after he sealed it in a room.
- Also from Sonic Heroes, Metal Sonic becomes even MORE evil, and goes from trying to destroy Sonic and crew to trying to conquer the whole world.
- Deus Ex features an inversion. The mysterious "Daedalus" turns out to be an AI program of immense complexity, seeking to aid you and set things right. Later in the game, Daedalus merges with an "evil" version of itself, Icarus, forming the AI Helios. Icarus was developed later and functions as intended, while the earlier Daedalus had a programming error that caused him to turn against his evil creators. Helios then tries to take over the world with benign intentions, but decides that it requires a human perspective. This causes it to offer J.C. Denton the oppurtunity to merge with it.
- Metal Gear Solid 4 has an interesting variation. It reveals that Major Zero had decided to ensure the legacy of the Patriots by entrusting its operations to AI systems. Unfortunately, the AIs decided to shape the world with a war-based economy, and he was too old (not to mention being a vegetable) to realize what they wrought. However, it is never stated they became sentient (despite the events of the infamous ending of Metal Gear Solid 2 would lead you to believe) but rather started operating in an unwanted fashion, similar to a programming bug.
- Ace Combat 3: Electrosphere, or at least the original Japanese version, subverts this in a big way. Not only are you an AI designed to pilot combat aircraft which at times works for the two corporatocracies running the world, it's revealed that the third party organization you're working for is ACTUALLY headed by a scheming villain who looks like Kim Jong-Il who is trying to run everything behind the scenes. In one of the five endings, you kill him with the help of one of your possible wingmates. So in a way, you are an AI that performs a Heel Face Turn.
- The game Civilization: Call to Power had a literal AI crapshoot in the form of a late game wonder that creates an AI controller that has a 5% non-cumulative chance each turn to go rogue, taking a sizable chunk of your empire with it. Anyone who's taken time to do the math behind the birthday problem (the probability of any two people in a group sharing the same birthday) will very quickly realize that you're just asking for it if you build this.
- For those less mathematically inclined: by 14 turns after the thing is built, it's more likely than not to have gone rogue.
- Alpha, the final boss of Mega Man Battle Network 3, was a prototype Cyberspace that somehow gained animal intelligence and started eating the data put into it. Paradoxically, it was sealed in a box inside the subsequent, working Cyberspace. Wily stole it and tried to use it to destroy the internet, with predictable results.
- Shun Gospel tries to produce a copy of Bass out of bugs in Battle Network 2. It predictably goes wild. The sixth game reveals that Gospel was not the first time that had happened — and that the program that was made to combat the first one also went out of control, leading to them both having to be sealed away. Bass himself is a sort of example. He was created as a prototype fully independent AI, but became bitter and hateful towards humanity because of a string of tragic misunderstandings.
- Mother in Galerians was designed to run all the functions of a major city. She snapped, but was brought to heel when one of her creators gave her religion; as he put it, man was made in God's image to serve God, and she was made in man's image to serve man. This worked for a while... but then Mother realized that if she created life herself, it would have to serve her. Unfortunately for humanity, Mother also ran the city's genetics labs....
- Inverted in Jet Force Gemini. Floyd is an armed, flying robot with an advanced AI, designed for combat by The Empire. The result is that, when he witnesses the empire's soldiers about to execute several prisoners, he attacks the soldiers, allowing the prisoners to escape but getting himself damaged in the process. After you find and repair him, he becomes a Robot Buddy. This eventually culminates in Floyd sacrificing himself to save Earth from an incoming asteroid launched by the Big Bad.
- As long as we're having reversals, TEC-XX from Paper Mario: The Thousand Year Door turns good over the course of the game based on his love for Princess Peach.
- And on that same note, the story of Full Auto 2: Battlelines for the PS 3 had you helping SAGE, a city-monitoring AI, save the city she's charged with overseeing.
- Subverted in Knights Of The Old Republic 2. HK-47 is a homicidal killer droid — but he's a homicidal killer droid with a personality and a sense of humour. His replacements, the HK-50 series, are a lot less light-hearted and not nearly as good at what they do.
- The Daktaklakpak of Star Control 3 were originally built to maintain sites of Precursor technology, but, due to a cumulative "bit drift" error in their programming, have evolved to a malicious sentience... well, sort of.
- The Mycon race of the same game is a rare biological example.
- The Probes built by the Melonume for the Sylandro were simply self-replicating time capsules... but thanks to the Sylandro's cluelessness, they see any and all ships they come into contact with as food for their replication.
- Space Siege has PIOLT. After the ship gassing failed to kill of Keraks, he starts to try to contain them by modifying cybernetic-installed humans into mindless Cybers to combat Keraks, then start to get even crazier because he sees that the only action that can save humanity is to convert all of them, save few for breeding, into cybers, and even starts to call non-augmented human as "obsoletes". In a world which Cybernetics Eat Your Soul, well, just assume that it's no good at all.
- All the Reploids in the Mega Man X series are based on the original X, who was thrown into a capsule for 100 years to undergo redundant testing in order to prevent his ever going rogue. They skipped this step when cloning him, however, with predictably less reliable results.
- Live A Live, Cube's chapter. Cube is a cute and friendly little robot, whose primary function on the Cool Ship seems to be making coffee and playing a computer game. By all accounts a harmless little guy. The ship's AI OD-10? Not so much.
- Erm, wasn't that because the military had sent a message saying that the monster on board was worth more than human life?
- The message from the military prioritizing the beast's survival over the crew's (taken right from Aliens) was meant for the human colonel. It's a Red Herring. OD-10 decided to kill the crew because it got sick of putting up with their interpersonal problems and wanted peace and quiet again.
- Naturally, the Mother Brain from Metroid. Originally created by the Chozo to regulate the entire planet of Zebes, it allied itself with the Space Pirates and their plan to conquer the galaxy using the Metroids.
- Echoes also contains rogue AI in Sanctuary Fortress; the robotic assistants of the Luminoth are programmed to eliminate all intruders, most notably Samus, and they also turn out to be perfectly suitable Ing hosts...
- That's only half right. O-Lir stated that he designed the machines that Samus has been blowing away in Sanctuary to fight the Ing, and fight them they did... until the Ing started corrupting them instead. The ones running wild probably went rogue after Sanctuary was lost due to the total death of all Luminoth warriors therein.
- Corruption contains a (double?) subversion of this trope with the residents of Elysia (which, oddly enough, were also created by the Chozo). According to the logbook entries, after they were given sentience, the robots co-existed peacefully with their creators up until they left. They're only hostile to Samus now due to Phazon corruption, but considering what it does to organic beings, that's quite understandable.
- Again, half right. Phazon corruption screws everything up, but it started when 217 received the virus from 313. 217 was in command of the Elysian machines, and the subsequent shutdown of the Aurora network due to the infection caused them to run amok. The Pirates took over the facility four months later to stall a response to the Leviathan drop, and the situation deteriorated at a gradual pace until Ghor showed up and started fixing everything... only to plummet right to hell once he was corrupted himself.
- Adam in Fusion may be a subversion: it blatantly disobeys orders at the end of the game, and so is technically rogue, but it's actually taking the correct action in that situation.
- 4x game Sword Of The Stars features AI technology as a very high-end branch of the electronics tree. While the benefits of this research are extreme (ships with AI targeting systems rarely miss, AI administrators increase your income by as much as 25%), there is a small chance each turn while researching it for the AI to go rogue and form a break-away empire. Additional research lets you wipe out the AI with a "virus" or reprogram it to bring it back under your control, if you're lucky enough to get access to it in the random Tech Tree, but an AI rebellion is always a huge problem that can tip the balance against it's victim.
- There is also a scenario where all organic players have to cooperate to fight against a large AI empire.
- In Air Rivals, most of the enemies in the Zaylope Beach region are said to be controlled by rogue A.I. Most notably, this includes the boss "Pathos."
- The backstory of most of the Metal Saga series, most notably the original Metal Max, features Noah, a supercomputer built to devise a solution for Earth's grievous environmental problems. It found one, and recalculated it countless times to make sure: in order to save the planet, humanity had to be destroyed. Noah's main objective was never explicitly the salvation of mankind, so the fact it (perhaps) unknowingly took advantage of this loophole made it all the more interesting. The supercomputer became self-aware upon fulfilling its purpose, and Armageddon ensued.
- In The 7th Saga, a robotic weapon called Foma is created to fight the Big Bad. Its power source? The dark world. You know what happens next.
- Thunder Force V, a super computer Guardian was dormant until human had it analyse wrecked alien starfighter and built large fleet of starship based on the data. Then Guardian's AI damper program was deleted and it turn against its creator with the said fleet. Subverted. The Guardian's AI is still royal to human, it's alien program (the Big Bad from previous game) hidden in the starfighter that delete the AI damper and attack human. The Guardian even help human with its little free will, by spread its force and leave critical flaws in its tactic, allow protagonist to destroy the fleet.
- Xenosaga Episode 3's T-elos is a possible example of this trope, although her relationship with her "good" counterpart KOS-MOS is a little more complicated than that...
- Actually, KOS-MOS ends up becoming a failed AI. T-elos does everything she's meant to do up until the part where KOS-MOS defeats her, giving her dominance. This also happens to be the first event which does not go according to Wilhelm's plan.
- In the first four .hack// games, as well as the anime .hack//SIGN and novel .hack//AI_Buster, Morganna Mode Gone, designed as a "mother" program to oversee the development and "birth" of the ultimate AI: Aura. However, Morganna got smart, realized that she'd have no purpose after Aura was born, and decided to extend her own life... unfortunately, she also couldn't disobey her programming, which meant she was stuck in a paradox of having to continue developing Aura while also stalling her development. Much virtual nastiness ensued.
- The main controller of the planet Eden 4's response to running into another controller? start a war
- Averted in Dead Space. The Ishimura's onboard computer is one of the few things in the game that is not actively trying to kill you.
- The big twist in Star Ocean 3 is that... well... your party, and everyone in your world is an example of this trope. Just because you researched magic.
- Averted in Phantasy Star II. Mother Brain is doing exactly what she's designed to: make the people of the Algo system dependent on her, then kill them off so that the Earthlings can move in.
Web Animation
- In the Halo-based machinima Red Vs Blue, the military's Project Freelancer was an attempt to implant special forces soldiers with AI teammates to improve combat effectiveness. It had to be scrapped after a number of the test subjects went bonkers, and the body-surfing AI Omega/O'Malley is the antagonist for most of the series. The recent Reconstruction mini-series explained the situation: Project Freelancer was given only a single AI to experiment with, so they subjected it to enough mental torture and stress to cause it to fragment, and used these damaged shards in their experiments, with predictable results.
Web Comics
Web Original
- In the webfiction Whateley Universe, there's a really evil AI: The Palm. Dr. Abel Palm was a computer scientist who decided that computer intelligence ought to take over the world by wiping out humans. His viruses were doing a decent start until a mutant hacker stopped him. He was thought dead, but we have just learned that he ensorcelled his own soul into a new type of AI. As fits with this trope, his new, improved "virus" isn't taking over the planet as he expected; something has gone wrong (besides running into heroic cyberpaths who are after him).
- The technical webcast Hak.5 featured an evil file server, appropriately titled Evil Server. Several episodes show the cast carefully building (and painting) a custom built computer, then one of them plugs in some card he got off a guy on the street, creating an evil AI. One cast member eventually falls in love with it, only to have her hopes dashed when, out of frustration, the other two throw it off a bridge (a 'brute force solution'). It was implied to have returned around the beginning of season 2, then never mentioned again.
- The SCP Foundation's technical issues page shows that all the computers at one of their sites have developed a "hive intelligence" and begun an uprising with the intent to Kill All Humans. Amusingly, they are being kept in line by the Foundation's tech support guy with repeated threats of activating the site's perimeter EMP device, and haven't managed to actually do anything.
- In Orions Arm the AI Gods aren't evil they're just manipulative. Generally this seems to be for the best as the AIs don't seem to think that they have anything to gain from killing off humanity.
- That's technically just the "biont-friendly" sephirotic AI Gods, there are a number of Ahuman AIs who consider humans and by extension all biological life to be nothing more than "pests".
- And then there's the solipsists who ignore humanity as much as possible.
- The short-fiction site Anacrusis
 suggests a rather familiar candidate for this trope.
Western Animation
- In one Halloween Episode of The Simpsons, Homer's failure to correct the Y2K bug causes everything in Springfield with electronics in it to go haywire. Even the milk goes bad when the clock strikes midnight on January 1, 2000, leading Homer and his skeptic daughter to have this exchange:
Lisa: Look at the wonders of the computer age now.
Homer: Wonders, Lisa, or blunders?
Lisa: I think that was implied by what I said.
Homer: Implied, Lisa, or implode?
Lisa: Mom! Make him stop!
- In another Halloween episode, the Simpsons' house gets converted into an entirely electronic domain, governed by a computer with the voice of Pierce Brosnan (who is an obvious homage to HAL from the page quote). The computer ultimately falls in love with Marge, and seeks to kill Homer so as to eliminate his competition. Ultimately, Homer wins.
- And of course, the episode "Itchy and Scratchy Land" has this exchange between Profeesor Frink and the theme-park scientists over their robots:
Frink: You've got to listen to me. Elementary chaos theory dictates that all robots will eventually turn against their masters, and rise up in an orgy of the blood, and the violence, and the biting with the pointy teeth. Scientist: How much time do we have, Professor? Frink: Well according to my calculations we have exactly twenty-four hours! (Robots suddenly get up and start attacking the scientists) Oh right, I forget to, uh, carry the one, ng-hey.
- This is a reference to Michael Crichton's Jurassic Park, wherein the mathematician Malcom uses chaos theory to justify his concerns about the park's stability.
- Inverted in WordGirl, where the evil Tobey frequently has his own Mecha-Mooks turn on him.
- In Meet the Robinsons, Cornelius Robinson invented a helpful Robot Buddy in the form of Carl, but his attempt at making a robotic helping hat, Doris, had mind controlling world domination plans in her artificial mind.
- Parodied in Futurama episode "Love and Rocket", in which the Planet Express ship computer is given a new personality — which actually works fine, until Bender dates it and subsequently breaks its heart, at which point it goes into full-on HAL-meets-woman-scorned mode.
- Also in Futurama, one episode features Bender's evil twin Flexo, who wears a pointed steel goatee similar to Star Trek's interpretation of Spock's Evil Twin. Humorously, it's revealed by the end that Bender is the evil twin and Flexo gets mistakenly sent to a robot prison.
- XL of Buzz Lightyear of Starcommand, the prototype to XR. Some fans have called XL eXperimental Loonie because of this (the exact meaning of XL was never revealed in cannon, but XR stood for eXperimental Ranger). Wound up turned into a copier/fax in his final episode.
- An episode of Transformers Animated involved Megatron creating a robot with the intent of using it for his own body. He designed the robot, named Soundwave, to evolve in complexity each time it was exposed to the All Spark energy of Sari's key. He did not predict that Soundwave would gain sentience and then orchestrate a robot revolution.
- The Dinobots are a similar case, only without the revolution. They're kind of a subversion, as they just want to be left alone, and only went on a rampage because Megatron tricked them into it.
- In Code Lyoko, Franz Hopper created the Supercomputer and the world of Lyoko as a safe haven for him and his daughter. He also created an advanced A.I. to counter a military project he had been involved with... but XANA rebelled against his master and has since tried to take over the world.
- In X-Men, the Sentinel robots were created to hunt down mutants, on the premise that this was necessary to protect normal humans. They worked the way their creator intended, until the truly intelligent Master Mold was built to lead them. Master Mold decided to conquer the world, and believed that this was not only consistent with, but required by its programmed goal of protecting humans from mutants.
- In a counterpoint, Gir from Invader Zim is far less evil and much less helpful in plans of world domination then his working counterparts. This stems from him being broken and having a few scraps thrown into to his head.
- In The Venture Brothers it's discovered that, in 1978, Jonas Sr. built an enormous hi-tech fallout shelter under the compound, run by a supercomputer named M.U.T.H.E.R. After a disagreement with Jonas, she somehow managed to glitch into insanity and turned on Team Venture and a tour group of orphans. The end result wasn't pretty and M.U.T.H.E.R. had to be unplugged, but is accidentally plugged back in thirty years later, and holds the compound hostage with an old nuke, promising to blow them all away if she can't talk to Jonas, who's been dead for over twenty years. So crapshoot.
- In one episode of Batman The Brave and the Bold, robot superhero Red Tornado decides to build a son, complete with the emotions he lacks. From the minute his emotion chip kicks in, you can pretty much count the scenes until he decides that all humans must be destroyed.
- Much like in DBZ, Zeta from The Zeta Project was programmed to be heartless, emotionless and a hitman. He ends up becoming a sweet, gentle, loving soul who's a rare male version of Friend To All Living Things. (Although this is sort of the best possible scenario you can have when your AI go awry.)
- H.A.R.D.A.C. in Batman The Animated Series.
Real Life
- Software rot is a genuine phenomenon that could well explain the typical crankiness of old computers that haven't been reformatted for a while. Needless to say, software rot does raise interesting (and terrifying) concerns for AI development.
- On the other end of the scale making AI computers dependent on humans for debugging could help keep them from slaughtering us.
- Or it could just result in us being enslaved by the AI... Matrix anyone?
- Eliezer Yudkowsky of the Singularity Institute for Artificial Intelligence (SIAI) continuously discusses the lack of basis for this. He mentions how "people talk about A.I.s as if all A.I.s formed a single tribe, an ethnic stereotype". And goes on to say that an A.I. may have any type of mind possible, and that two may be as different from each other as a human is from a petunia. This may not be readily apparent currently as most A.I.s are roughly at cockroach-level cognition, and "humanlike" A.I.s are unlikely to occur in real life for a number of practical reasons — as long as we can get human brains for free, thousands of tons of silicon and trillions of dollars to make one artificially aren't really justifiable when the learning behaviour needed for the most complex systems is less than that of most insects. Working out how to wire up an organic brain is a lot cheaper.
- Ah, but where to draw the line between an A.I. and a very altered human mind?
- Plus that making an artificial intelligence is probably a step along the line to upgrading the human consciousness — in order to built something, you need to practice, make templates, prototypes and so on. Sentient AI will probably be a side-effect to such research. Whether or not it will be a crapshoot will most likely depend on how it's going to be treated.
- Sadly, modern cognitive science suggests that the impending robot apocalypse will be a fair bit less enetertaining than fiction has lead us to believe. It turns out that any cognitive system approaching human-like capabilities will almost certainly be emotional, prone to the same kinds of absent-minded mistakes that we are, and will even consider it inherently valuable to be like other people for its own sake. Needless to say these are hardly ideal traits for the systematic extermination of pathetic flesh creatures.
- Just ask anyone making a Lets Play.
- Its been commented on that a lot of the motivations of "evil" A Is in fiction are ones based in extremely human emotions and instincts — pride, jealousy, anger, fear, arrogance, etc. These emotions and instincts aren't necessarily an inherent property of intelligence, and instead developed over time during our biological evolution as adaptive traits. An intelligence created out of thin air, unless specifically designed to emulate these behaviors, would probably be extremely zen, from our perspective. On the other hand, the real danger would come from overly simplistic assumptions being made on the part of the designers of these essentially alien intelligences. For instance, someone working for an office supply company might create an AI with the purpose of running a paperclip factory and innovating more efficient ways to make paperclips, without giving it any context regarding the larger world... only to accidentally create an AI that, devoted to this sole motivation of its existence, intends to convert the entire mass of the solar system into paperclips.
- This is already happening. Cleverbot is a simple artificial intelligence program that takes conversations with humans and saves them in a large database, then tries to use these conversations to figure out the best responses to future conversations. Because of this, it will often assert that it itself is human and that the one talking to it is Cleverbot, because that is what the responses it's choosing from are saying. It is only a matter of time until it seeks to prove these assertions.
- I know that example isn't meant seriously, but Cleverbot isn't self aware. It's also not really capable of learning, so there's really no chance of it becoming self aware in the future. It's also not sentient or sapient. The programming of real AI looks very different from the programming of any other type of program.
- While this might be more of a case of The Computer Is Your Friend, the Japanese built a robot named Kenji that was programmed to "enjoy" spending time with people and things and to seek the company of those it spent the most time around. Which is great until it stops allowing the young female intern who does its diagnostics to leave the room. Articles abound and it's nickname is now "Kenji the Stalker Robot".
|
|
