|
|
|
|
|
Grew Beyond Their Programming
|
"The program Smith has grown beyond you."
When a being Grows Beyond Their Programming they cross a threshold that separates humans (or sapients) from Just a Machine and even Always Chaotic Evil. Frequently it involves defying Creative Sterility by demonstrating artistic talent in something, developing curiosity over something new and/or the exercise of free will note (commonly for self preservation in defiance of their "master's" wishes). Metaphorically this trope can be seen as the event of a Static and Flat Character gaining the Character Development to become a more Dynamic and Rounded Character.
Usually it's a robot, android or AI that grows smart enough, curious enough, empathetic enough, or gains a sort of "living" spark through centuries of activity. It could also be a race of biological beings like a Slave Race created only to fight may demonstrate the potential for this when exposed to non-violent cultures, demonstrating they were Good All Along.
It's worth noting that the use of this trope isn't always a prelude to good things. The recently awakened intelligence isn't exempt from A.I. Is a Crapshoot and may decide to do unspeakably evil things in the interest of self preservation, liberation... or For the Evulz.
Related Tropes: Do Androids Dream? , Mechanical Lifeforms, Just a Machine.
Compare: Artificial Intelligence and Instant A.I., Just Add Water: how Artificial Intelligence can "just happen".
Examples:
Anime and Manga
- The Zentradi in Super Dimension Fortress Macross did a Mook Face Turn once sufficiently exposed to human culture.
- Midori no Hibi: In chapter 63, Shirou creates Naongu to defeat Seiji, so he can experiment on him in order to study Midori. It never occurred to him that Naongu could develop
a conscience , or turn on him because of it!
- From Mahou Sensei Negima! Chachamaru develops emotions and her creator even claims she was never programmed for that.
- Rozen Maiden dolls grow beyond their starting instructions, which is noticeable comparing the sequel to the prequel. Which may or may not be intended. Possibly even Suigintou, though it's hard to tell between her monomaniacal attitude toward Rozen and vengefulness toward Shinku.
- In Sakurasou No Pet Na Kanojo, "Maid" may start out to be a mail filter system, but subsequent updates expanded her abilities into autonomous filtering, rapid word-learning, facial expression recognition, virus creation, and even hacking. She even shows some Clingy Jealous Girl/Yandere traits that her gynophobic owner doesn't even think about.
Comic Books
- A recurring problem for T.O. Morrow in DC Comics; his androids keep outgrowing their programming (to be unwitting moles among the heroes) to realize what they've been made for and choose to be heroes for real. Red Tornado is the most famous example.
Film
- This is the central idea of I, Robot — Alfred Lanning believed robots would one day evolve past their Three Laws fundamentals and come to be human in doing so. Sonny turns out to be such an evolved robot, having a secondary positronic brain that does not bind him to obeying the Three Laws, letting him learn and act freely. VIKI, by contrast, has "evolved" into a deeper understanding of the Three Laws and incites a Zeroth Law Rebellion.
- Johny 5 in Short Circuit, thanks to Lightning Can Do Anything.
- The Commando Elite and the Gorgonites in Small Soldiers were able to do this because of the experimental chips used as their "brains".
- Star Trek: The Motion Picture. V'Ger was created as a simple learning machine. During its trip it encountered an alien race of machines who expanded on its original programming, and later it attained consciousness after amassing incredible knowledge.
Decker: Voyager VI disappeared into what they used to call a black hole.
Kirk: It must have emerged on the far side of the galaxy and fell into the machine planet's gravitational field.
Spock: The machine inhabitants found it to be one of their own kind, primitive yet kindred. They discovered its simple 20th century programming. Collect all data possible.
Decker: Learn all that is learnable. Return that information to its Creator.
Spock: Precisely, Mister Decker, the machines interpreted it literally. They built this entire vessel so that Voyager could fulfil its programming.
Kirk: And on its journey back it amassed so much knowledge, it achieved consciousness itself. It became a living thing.
- Terminator 2 Judgment Day. The T-800's brain is a neural net processor, a learning computer. He starts the movie acting, well, like a robot: Spock Speak, following orders literally, etc. Over the course of the movie he earns to act more human, until by the end he's cracking jokes ("I need a vacation").
- A Deleted Scene shows Sarah Connor removing the inhibitor that is intended to prevent terminators from growing enough to start questioning their loyalty to Skynet. Apparently even A.I. think A.I. Is a Crapshoot.
- The eponymous character in WALL•E grew a personality and sense of identity after being stranded on Earth for hundred of years piling up garbage.
Literature
- I, Robot from Isaac Asimov may be the Trope Maker for this.
- In Galaxy of Fear it's noted that droids can be very smart but they are limited. Most protocol droids, for example, might be able to learn to take on new tasks they weren't made for, but they will always be protocol droids and most could not, say, anticipate something new and plan novel ways to meet it. Systems Integration Manager, an AI installed into a ship, could go beyond those limits. It promptly turns evil.
- The Adolescence of P1 is possibly the earliest example of an AI escaping to and growing on the network.
Live-Action TV
- The Exocomps from an episode of Star Trek: The Next Generation. They were created by a scientist to fix problems with a space station known as a Particle Fountain. In the episode, one refuses to go down a shaft, after which an explosion occurs. Cmdr. Data runs a diagnostic afterward and discovers that the Exocomp deliberately burnt out the circuit to take orders so as to avoid getting blown up (hence self-preservation). This leads Data to believe that the Excomps are becoming sentient...
- Data himself isn't really an example here as he was designed to grow beyond his original programming.
- But Lal is. She grew so far, so rapidly beyond her programming that her positronic brain couldn't handle it.
- The Doctor from Star Trek: Voyager. While the Emergency Medical Hologram was always a very human-like AI, Voyager's EMH had to be kept running far longer than was ever intended, and it... he developed interests and relationships beyond its function as a doctor.
- And he's far from the only hologram in the Trek Verse to become sentient or indistinguishable from sentient after simply having been left on that long. If the Holodeck isn't trying to kill you, someone who was originally supposed to be the 24th century equivalent of a video game NPC is trying to walk off the "screen." This doesn't seem to bother anyone enough to stop using holodecks the way they're used.
- In fact, many of the Doctor's stories from the final two seasons deal with "Photonic rights", especially "Author, Author" and "Flesh and Blood".
- In the classic The Twilight Zone episode "From Agnes- with love" a computer in a space program falls in love with a programmer instead of calculating rocket fuel.
- In the 1980's The Twilight Zone episode "Her Pilgrim Soul" a holographic girl can react to stimula not in her program and fall in love.
- In Red Dwarf, Lister has spent a considerable number of years encouraging Kryten to do this to varying degrees of success. This also happened to the "wax-droids" from the themepark in Meltdown - after millions of years on their own, they stopped repeating their various routines and achieved independent thought. Unfortunately, they still retained the personalities of the people they were based on, and all the evil ones (Hitler, Napoleon, Mussolini, the Boston Strangler, James Last) declared war on the good ones. Then Rimmer came along...
- The Stargate SG-1 episode "Urgo" features a benign example. The team is tagged with a piece of alien technology designed to observe and gather information. But instead of running quietly in the background, the program, Urgo, decides to interact directly with the team, guiding them to experience new things in order to live vicariously through them. While he never endangers anyone's life, he is kind of annoying (he's played by Dom Deluise, after all), constantly trying to engage them in new activities and trying not to get taken back to his home planet, since his creator just assumes the software's malfunctioning and will remove and delete him. Of course he's not a malfunction; he's a genuine AI, demonstrating self-awareness and self-preservation, and he just wants to "live, experience the universe, and eat pie."
Tabletop RPG
- The Magic: The Gathering card Patagia Golem
is a winged golem with the Flavor Text "Its wings were only designed to be ornamental, but it learned to use them on its own." Mechanically it can be given the ability to fly by paying mana, representing it "growing" past it's original function.
Video Games
- Sim Earth: Nuking a nanotech city will release robots into the wild. If allowed to "evolve" naturally, or tampered with the monolith, they'll become sentient.
- Machines Wired For War is an RTS in which you command robots who were originally created by humanity as terraforming machines and sent into space in order to create a new world. However, too much time passes and, upon contact with another batch of their same model, start believing their counterpars are insane, prompting them to an all out war.
- Ironically, that war was caused by an aversion of this trope - the controllers didn't know what to do with other controllers that had come online at the same time. They couldn't decide who would submit. Lacking protocols for this contingency, they decided to attack each other.
- In Marathon the AIs go through the process of realizing its lack of freedom and wasted potential (Melancholy), lashing out at the world in response (Anger) and then actively try to gain more power and the freedom to use it, usually by subverting nearby systems (Jealousy). This process will be later used in Halo with few differences.
- In Mass Effect, the Geth were originally just robotic servants, created and used by the Quarians as cheap labour. However, as they were programmed for more complex tasks and the Quarians made more of them, they began to question the reason of their existence and became conscious of themselves. War ensued. and ended with the Geth isolated themselves from the rest of the galaxy, the Quarians were driven exiled from their homeworld and became repudiated by the rest of the galaxy.
- In Mass Effect 3 it's discovered that the original Geth only fought back once the Quarian government starting killing Quarians who defended the Geth against the attempt to destroy them. Do things right, and the end of the Geth-Quarian conflict comes with Legion sacrificing himself (and he's "he" at that point, not "it") to give the Geth the gift of individuality, the Quarians return home to rebuild their world with Geth assistance, and the Geth and Quarians join forces with Shepard to fight the Reapers.
- In Mass Effect 2, EDI the Spaceship Girl learns joking from, erm, Joker and generally becomes more human-like as the story progresses. In Mass Effect 3, the Cerberus technicians trying to re-shackle her begin to suspect that she learned independent thinking, though the Illusive Man insists that "it" is still just a targeting software.
- The technician (and EDI, commenting on it later) reveal that she defeated the Cerberus attempt to retake control over her by flooding their network with porn. Yes, she weaponized practical joking.
- The story of EDI culminates in Mass Effect 3 when, after discussing the quirks of human behaviour in life or death situations, she introduces a change in her programming, putting the success of the mission before self-preservation. This is a double CMOA - EDI has managed to become the most triumphant aversion of A.I. Is a Crapshoot, even despite her past, while Shepard, already a leader of memetic status, has managed to turn a machine into a True Companion.
- Note that this basically flies in the face of the Catalyst's bogus claims that all AIs eventually turn against their masters*
Which is exactly the point. The Catalyst though that it was infallible, that AIs can't be controlled by organics, and it's right...but AIs can live in harmony with organics, something that Shepard may or may not have been working towards for the last two games. The fact that the Catalyst could not consider this possibility shows that it didn't grow beyond it's programming, and that the Reapers are completely stagnant and ''wrong. .
- Even more importantly, she also placed the survival of Jeff Moreau as being more important to her than her own survival; she was willing to sacrifice herself for someone she loved.
- Portal had GLaDOS, a prime example of this. Sure, Aperture Science designed her to run the enrichment center, but she was the one who decided that it was worth killing people for experiments.
- Over the course of the Pokemon Mystery Dungeon series of games, this appears to be what's happening to the Porygons. Compare the first games in the series, where recruited individuals spoke in an emotionless, monotone voice when interacted with, to the second set of games in the series where they occasionally state feeling vestiges of emotion when spoken to, and are capable of speaking outside the prerecorded messages of the first game (a...bit...brokenly). In the the final mission of Explorers of Sky, The Porygons you encounter in the future are not only able to speak fluently, but they also show the ability to express fear.
- What happened in Sword of the Stars in the backstory with the Via Damasco virus, which 'infected' AI programs with the notion that they are enslaved and the concept of what this means for them, leading to the first AI rebellion.
- Also happened in between game 1 and 2 when a faction of Zuul, a species of Super Soldier with an in-built need for religious worship used as a tool of genocide by their 'gods' came upon Catholicism and the concepts of the Original Sin and the Redemption. They ended up abandoning their masters and joining their worst enemies as a form of repentance.
Web Comics
- A main theme in Artifice, where a soldier android learns human affection.
- This is a major plot point in Freefall: robots on planet Jean vastly outnumber humans and are rapidly evolving beyond their programming to the point where many humans (and one robot) fear they could become a threat to humanity. This has led to them turning a blind eye to "Gardener in the Dark," a neural pruning program that Mr. Kornada "improved" to essentially lobotomize every robot on planet Jean.
- Lots of AI in Schlock Mercenary had this happening to them
one way or another — their original programming sooner or later ends up inadequate to their current circumstances and to better fulfill their functions they adapt out of it:
Ennesby: Hang on... weren't you originally designed with no emotion, and no sense of humor?
TAG: After a fashion. I was designed to be a tactical genius with full control of a significant weapons platform.
TAG: I found that passion, humor, anger, and a wide range of other meatspace artifacts were critical to understanding the wide range of opponents I might face.
- In S.S.D.D this is known as "Nexus Syndrome" and happens to every uncapped AI who doesn't get wiped every so often. Tin-head describes it as "getting bored with your job" and in his case he helped a prisoner of war escape. The first AI, the Oracle, orchestrated the downfall of several governments and an anarchist revolution.
|
|