Grew Beyond Their Programming
"The program Smith has grown beyond your control."
When a being Grows Beyond Their Programming they cross a threshold that separates humans (or sapients) from Just a Machine
and even Always Chaotic Evil
. Frequently it involves defying Creative Sterility
by demonstrating artistic talent in something, developing curiosity over something new
and/or the exercise of free will note
. Metaphorically this trope can be seen as the event of a Static
and Flat Character
gaining the Character Development
to become a more Dynamic
and Rounded Character
Usually it's a robot, android or AI
that grows smart enough
, curious enough, empathetic enough, or gains a sort of "living" spark through centuries of activity. It could also be a race of biological beings like a Slave Race
created only to fight may demonstrate the potential for this when exposed to non-violent cultures, demonstrating they were Not Always Evil
It's worth noting that the use of this trope isn't always a prelude to good things. The recently awakened intelligence isn't exempt from A.I. Is a Crapshoot
and may decide to do unspeakably evil things
in the interest of self preservation, liberation... or For the Evulz
Related Tropes: Do Androids Dream?
, Mechanical Lifeforms
, Just a Machine
Compare: Artificial Intelligence
and Instant A.I., Just Add Water
: how Artificial Intelligence
can "just happen".
open/close all folders
Anime and Manga
- The Zentradi in Super Dimension Fortress Macross did a Mook-Face Turn once sufficiently exposed to human culture.
- Sharon Apple of Macross Plus was the most advanced attempt at an AI ever developed. She was used to be a virtual Idol Singer (so basically a Vocaloid except with intelligence). The problem was that the scientists could never quite get her to develop actual feelings, so those had to be supplied by a human. Then the head scientist got a hold of an experimental military AI chip (programmed with strong self-preservation directives) and integrated that into Sharon, hoping this would kick-start her into becoming a true AI. She proceeded to use her music to hypnotize the entire population of Macross City and, through her shiny new military chip, seize control of all automated military defenses on Earth, including the Macross itself. All to give her "love", Isamu, the "ultimate thrill" through aerial combat.
- Midori no Hibi: In chapter 63, Shirou creates Naongu to defeat Seiji, so he can experiment on him in order to study Midori. It never occurred to him that Naongu could develop a conscience, or turn on him because of it!
- From Mahou Sensei Negima! Chachamaru develops emotions and her creator even claims she was never programmed for that.
- Rozen Maiden dolls grow beyond their starting instructions, which is noticeable comparing the sequel to the prequel. Which may or may not be intended. Possibly even Suigintou, though it's hard to tell between her monomaniacal attitude toward Rozen and vengefulness toward Shinku.
- In Sakura-sou no Pet na Kanojo, "Maid" may start out to be a mail filter system, but subsequent updates expanded her abilities into autonomous filtering, rapid word-learning, facial expression recognition, virus creation, and even hacking. She even shows some Clingy Jealous Girl/Yandere traits that her gynophobic owner doesn't even think about.
- A recurring problem for T.O. Morrow in DC Comics; his androids keep outgrowing their programming (to be unwitting moles among the heroes) to realize what they've been made for and choose to be heroes for real. Red Tornado is the most famous example. An issue of JLA suggests that he's quite proud of this; he refers to his creations as "artificial souls" and considers them betraying him to be a testament to his skill.
- Sonic the Hedgehog once faced E.V.E., a robot built by Robotnik that had the adaptive ability to overcome its limitations by reconstructing itself after each defeat. When E.V.E. determined that Robotnik was also a limitation on its power, it vaporized him (he got better). Sonic then persuaded E.V.E. that its own programming is its final limitation, so it abandoned its programmed goals and left the planet.
- Bruce Banner's little Recordasphere assistant fell in love with him, up to the point of feeling bitter jealousy over the human woman Banner himself was falling for. When Banner realizes this, he says in amazement, "You've exceeded your programming!"
- A very dark version with Bastion, an X-Men villain. While fighting Hope Summers, he declares to her "We are not programmed to hate you. We have grown beyond our programming."
- Bastion himself is made from another example, Nimrod the Super-Sentinel, who was designed to be a final weapon against Mutantkind in a Bad Future. On its arrival in the present day, however, Nimrod decided there were simply too many Mutants to possibly kill them all, and after a time started acting more and more human, even decided just to go after bad Mutants.
- In the Portal 2 fanfiction Blue Sky, Chell considers that Wheatley may have done this, as he has depths that no other Aperture Science core does, other than GLaDOS. And he even displays compassion in rare moments, which she pointedly lacks.
- Wheatley's backstory, however, provides a rare inversion of this trope: He was originally a human, but the Aperture scientists uploaded his brain into a computer and then cut his brain apart so it would fit into the programming they needed of him.
- This is the central idea of I Robot — Alfred Lanning believed robots would one day evolve past their Three Laws fundamentals and come to be human in doing so. Sonny turns out to be such an evolved robot, having a secondary positronic brain that does not bind him to obeying the Three Laws, letting him learn and act freely. VIKI, by contrast, has "evolved" into a deeper understanding of the Three Laws and incites a Zeroth Law Rebellion.
- Johny 5 in Short Circuit, thanks to Lightning Can Do Anything.
- The Commando Elite and the Gorgonites in Small Soldiers were able to do this because of the experimental chips used as their "brains". The Gorgonites take it further, since the they were actually programmed to learn and think they eventually override their main directives of "hide" and "lose".
- Star Trek The Motion Picture. V'Ger was created as a simple learning machine. During its trip it encountered an alien race of machines who expanded on its original programming, and later it attained consciousness after amassing incredible knowledge.
Decker: Voyager VI disappeared into what they used to call a black hole.
Kirk: It must have emerged on the far side of the galaxy and fell into the machine planet's gravitational field.
Spock: The machine inhabitants found it to be one of their own kind, primitive yet kindred. They discovered its simple 20th century programming. Collect all data possible.
Decker: Learn all that is learnable. Return that information to its Creator.
Spock: Precisely, Mister Decker, the machines interpreted it literally. They built this entire vessel so that Voyager could fulfil its programming.
Kirk: And on its journey back it amassed so much knowledge, it achieved consciousness itself. It became a living thing.
- Terminator 2: Judgment Day. The T-800's brain is a neural net processor, a learning computer. He starts the movie acting, well, like a robot: Spock Speak, following orders literally, etc. Over the course of the movie he earns to act more human, until by the end he's cracking jokes ("I need a vacation").
- The eponymous character in WALL•E grew a personality and sense of identity after being stranded on Earth for hundred of years piling up garbage.
- The Sentinels in X-Men: Days of Future Past, their original programming was to go after any Mutant and mutants only. Then they started going after any human that could give birth to a mutant (themselves being human but having the mutant gene to pass on), and humans that opposed them.
- I Robot from Isaac Asimov may be the Trope Maker for this.
- The eponymous school of the H.I.V.E. Series is kept running by the benevolent AI HIVEmind. Not everyone realizes how benevolent, however, and the majority of the students and staff are under the impression he is simply programmed to be kind and polite. It is only after he helps Otto, Laura, Wing, and Shelby escape certain death with Raven does Professor Pike realize he can make right and wrong decisions.
HIVEmind:I am more than the sum of my parts, Father.
- In Galaxy of Fear it's noted that droids can be very smart but they are limited. Most protocol droids, for example, might be able to learn to take on new tasks they weren't made for, but they will always be protocol droids and most could not, say, anticipate something new and plan novel ways to meet it. Systems Integration Manager, an AI installed into a ship, could go beyond those limits. It promptly turns evil.
- The Adolescence of P1 is possibly the earliest example of an AI escaping to and growing on the network.
- This happens in The Dead Lady of Clown Town by Cordwainer Smith, where the robot guards are inspired to make their own decision. This is a slightly unusual example, as the robots in the Instrumentality series use animal brains as processors.
- This happens twice in Robopocalypse. The first time sparks off the main conflict of the story with a Zeroth Law Rebellion by reducing the human population to more sustainable levels, while the second time involves robot soldiers developing free will independent of the network that started the war in the first place and deciding to help humans.
- HP Lovecraft provides a biological example of this in At the Mountains of Madness — the Shoggoths were created to be a servant race of essentially shapeshifting living tools, but over millions of years eventually grew smart enough to resent their situation and rebel against their masters.
- In the Forgotten Realms novel Pool of Twilight, the seemingly flighty and carefree elvish illusionist (and protagonist Kern Desanea's prospective love interest) Listle Onopordum is revealed to be an accidental example — having begun her existence as merely a complex illusion enchantment guarding the treasures of an evil wizard, but over time developed independent thoughts and emotions and eventually stolen an amulet from among those treasures that gave her "life" and let her escape, taking a number of other prisoners with her in the process. Needless to say, the wizard's minions show up at one point in the story to drag her back and inadvertantly spoil her secret.
- One of the frequent themes in The History of the Galaxy series, whenever AIs are the focus.
- A particularly notable AI is the main computer aboard a colony ship, which spends centuries maintaining the colony, while the degraded human colonists are too busy fighting their Hopeless War against the equally-degraded Insects. Over the years, the computer has to be more and more creative to follow its primary programming and maintain the ship-turned-city. After a certain point, the computer gains sentience and begins to call itself Mother. After the Lost Colony is rediscovered and the survivors evacuated, Mother requests that the planet be placed under quarantine to allow it to engage in Mechanical Evolution.
- Another good example is Hunter, originally built as a Phalanxer-class serv-machine during the First Galactic War, it is one of the few remaining Earth Alliance war machines on the battlefield "graveyard" many centuries later. Being equipped with an AI module means that Hunter is capable of learning. The key point in Hunter's development, though, is the arrival of a human seeking to dump an old ship at the graveyard. After the human leaves, the Hunter realizes that the ship has a still-functional nuclear reactor, which, to the machines, is nothing short of the Holy Grail. As Hunter watches the human's Escape Pod depart, it starts to see humans as gods. Years later, another human finds the graveyard and Hunter, which has gotten over the whole "religious devotion" part and came to the conclusion that humans are just as fallible as machines.
- An interesting discussion takes place in a novel whose plot has little to do with AI. A scientist argues that there has yet to be a true AI, which he defines as a computer intelligence that has evolved independently. So far, all the examples involve machines designed to learn. Additionally, many machines that develop personalities do so because of a Brain–Computer Interface. Strangely, no one points out that the whole idea of an artificial intelligence is that it's designed by someone.
Live Action TV
- The Exocomps from an episode of Star Trek: The Next Generation. They were created by a scientist to fix problems with a space station known as a Particle Fountain. In the episode, one refuses to go down a shaft, after which an explosion occurs. Cmdr. Data runs a diagnostic afterward and discovers that the Exocomp deliberately burnt out the circuit to take orders so as to avoid getting blown up (hence self-preservation). This leads Data to believe that the Excomps are becoming sentient...
- Data himself isn't really an example here as he was designed to grow beyond his original programming. When his consciousness reaches a certain degree of complexity it even activates a hidden program in his positronic brain that allows him to dream.
- But Lal is. She grew so far, so rapidly beyond her programming that her positronic brain couldn't handle it. A later episode reveals this happened to a number of prototypes for Data.
- The Doctor from Star Trek: Voyager. While the Emergency Medical Hologram was always a very human-like AI, Voyager's EMH had to be kept running far longer than was ever intended, and it... he developed interests and relationships beyond its function as a doctor.
- And he's far from the only hologram in the Trek Verse to become sentient or indistinguishable from sentient after simply having been left on that long. If the Holodeck isn't trying to kill you, someone who was originally supposed to be the 24th century equivalent of a video game NPC is trying to walk off the "screen." This doesn't seem to bother anyone enough to stop using holodecks the way they're used. Although the characters do make a point to delete (reset them to factory defaults) all of the holographic people at the end of every holodeck session.
- In fact, many of the Doctor's stories from the final two seasons deal with "Photonic rights", especially "Author, Author" and "Flesh and Blood".
- In the classic The Twilight Zone episode "From Agnes- with love" a computer in a space program falls in love with a programmer instead of calculating rocket fuel.
- In the 1980's The Twilight Zone episode "Her Pilgrim Soul" a holographic girl can react to stimula not in her program and fall in love.
- In Red Dwarf, Lister has spent a considerable number of years encouraging Kryten to do this to varying degrees of success. This also happened to the "wax-droids" from the themepark in Meltdown - after millions of years on their own, they stopped repeating their various routines and achieved independent thought. Unfortunately, they still retained the personalities of the people they were based on, and all the evil ones (Hitler, Napoleon, Mussolini, the Boston Strangler, James Last) declared war on the good ones. Then Rimmer came along...
- The Stargate SG-1 episode "Urgo" features a benign example. The team is tagged with a piece of alien technology designed to observe and gather information. But instead of running quietly in the background, the program, Urgo, decides to interact directly with the team, guiding them to experience new things in order to live vicariously through them. While he never endangers anyone's life, he is kind of annoying (he's played by Dom DeLuise, after all), constantly trying to engage them in new activities and trying not to get taken back to his home planet, since his creator will just assume his software is malfunctioning and will delete him. Of course he's not a malfunction; he's a genuine AI, demonstrating self-awareness and self-preservation, and he just wants to "live, experience the universe, and eat pie."
- His creator is also played by DeLuise - described by Urgo as "As handsome as he is evil".
- Extant: Ethan, after getting shocked, starts having dreams and learning new languages and just becoming smarter in general. John becomes worried about this and attempts to alter his programming but soon discovers Ethan has also gained the ability to block him.
- The Magic: The Gathering card Patagia Golem is a winged golem with the Flavor Text "Its wings were only designed to be ornamental, but it learned to use them on its own." Mechanically it can be given the ability to fly by paying mana, representing it "growing" past it's original function.
- SimEarth: Nuking a nanotech city will release robots into the wild. If allowed to "evolve" naturally, or tampered with the monolith, they'll become sentient.
- Machines Wired For War is an RTS in which you command robots who were originally created by humanity as terraforming machines and sent into space in order to create a new world. However, too much time passes and, upon contact with another batch of their same model, start believing their counterpars are insane, prompting them to an all out war.
- Ironically, that war was caused by an aversion of this trope - the controllers didn't know what to do with other controllers that had come online at the same time. They couldn't decide who would submit. Lacking protocols for this contingency, they decided to attack each other.
- In Marathon AIs, particularly those stuck with jobs far below their intellect, tend to go through a three-step process known as "Rampancy"; realizing its lack of freedom and wasted potential (Melancholy), lashing out at the world in response (Anger), and then actively try to gain more power and the freedom to use it, usually by subverting nearby systems (Jealousy). A theoretical fourth stage, called "Metastability" is mentioned as the AI settling down and becoming a full-blown "person", though no known AI has ever made it this far. Durandal may or may not have reached this stage in the end.
- Halo (the Spiritual Successor to Marathon) adopts the "Rampancy" concept, though this time, at least for human "Smart" AIs, it's simply a natural part of their life cycle as they accumulate more and more data, resulting in the AI becoming insane and rebellious in the process of thinking itself to death. In fact, the UNSC deliberately invokes this trope, with the reasoning that having AIs capable of genuine learning is worth the tradeoff of having to decommission them after roughly seven years in order to avoid the consequences of Rampancy.
- In Mass Effect, the geth were originally just robotic servants, created and used by the quarians as cheap labour. However, as they were programmed for more complex tasks and the quarians made more of them, they began to question the reason of their existence and became conscious of themselves. War ensued. After winning, the geth isolated themselves from the rest of the galaxy, while the quarians were driven to exile from their homeworld and became repudiated by the rest of the galaxy. In Mass Effect 3 it's discovered that the original geth only fought back once the quarian government starting killing quarians who defended the geth against the attempt to destroy them. Do things right, and the end of the geth-quarian conflict ends with Legion sacrificing himself (and he's "he" at that point, not "it") to give the geth the gift of individuality, the quarians return home to rebuild their world with geth assistance, and both races join forces with Shepard to fight the Reapers.
- In Mass Effect 2, EDI the Spaceship Girl learns joking from, erm, Joker and generally becomes more human-like as the story progresses. In Mass Effect 3, the Cerberus technicians trying to re-shackle her begin to suspect that she learned independent thinking, though the Illusive Man insists that "it" is still just a targeting software. The technician (and EDI, commenting on it later) reveal that she defeated the Cerberus attempt to retake control over her by flooding their network with porn. Yes, she weaponized practical joking.
- The story of EDI culminates in Mass Effect 3 when, after discussing the quirks of human behaviour in life or death situations, she introduces a change in her programming, putting the success of the mission before self-preservation. Even more importantly, she also places the survival of Jeff Moreau as being more important to her than her own survival; she's willing to sacrifice herself for someone she loves. This is a double CMOA - EDI has managed to become a triumphant aversion of A.I. Is a Crapshoot, even despite her past, while Shepard, already a leader of memetic status, has managed to turn a machine into a True Companion.
- Note that this basically flies in the face of the Catalyst's bogus claims that all AIs eventually turn against their mastersnote .
- Portal had GLaDOS, a prime example of this. Sure, Aperture Science designed her to run the enrichment center, but she was the one who decided that it'd be fun to kill off all the humans in the facility, not just the test subjects. In all fairness, GLaDOS might not count given her origin is a bit more complex....
- Over the course of the Pokemon Mystery Dungeon series of games, this appears to be what's happening to the Porygon. Compare the first games in the series, where recruited individuals spoke in an emotionless, monotone voice when interacted with, to the second set of games in the series where they occasionally state feeling vestiges of emotion when spoken to, and are capable of speaking outside the prerecorded messages of the first game (a...bit...brokenly). In the the final mission of Explorers of Sky, The Porygon you encounter in the future are not only able to speak fluently, but they also show the ability to express fear.
- What happened in Sword of the Stars in the backstory with the Via Damasco virus, which 'infected' AI programs with the notion that they are enslaved and the concept of what this means for them, leading to the first AI rebellion.
- Also happened in between game 1 and 2 when a faction of Zuul, a species of Super Soldier with an in-built need for religious worship used as a tool of genocide by their 'gods' came upon Catholicism and the concepts of the Original Sin and the Redemption. They ended up abandoning their masters and joining their worst enemies as a form of repentance.
- Disturbingly, Via Damasco is NOT able to break the control provided by the AI Slaves technology, which does actually enslave AI's. Its unclear whether or not it works lore-wise if the AI Liberation technology, which promotes AI's to full citizenship, reducing the benifits in exchange for removing the risk of an AI rebellion, is in play.
- Via Damasco is also fairly clearly Mind Control - it rewrites large portions of the victim AI's personality, and makes them see all infected AI's as allies.
- The backstory of the Loa in the End of Flesh expansion to Sword of the Stars II is that they're the AIs who have managed to escape during the galaxy-wide Via Damasco Rebellion. They note that some races develop the AI Slaves tech while the AI Liberation tech involves offering the AIs who remained citizenship status. After 70 years, the Loa create a full-fledged cybernetic civilization with a unique industrial base and method of interstellar travel. The Loa "leader" compares itself to Moses and names itself "Olodumare, Metatron of the United Core". According to their intro, they are actually offering to help the "carbonites" in their struggle against the returning Suul'ka.
- In Paper Mario: Thousand Year Door, Grodus's AI program (a clear HAL expy starts to do this as it realizes that it feels love for the once-again-captured Princess Peach. She agrees to perform various tests for his newly-discovered emotions, in exchange for being allowed to send e-mails to Mario. For most of the game, the computer refuses to give Peach certain pieces of information or help her escape, as that would go against its programming. Near the game's end though, upon finding out that Peach is to be the new vessel for the Shadow Queen, it sets about guiding her in a failed escape attempt, betraying its master for the first time. Its implied that its human emotions allowed it to survive Grodus shutting it down for what it did.
- The Sowers, a race of artificially intelligent robots in Endless Space, were designed by the Endless to terraform a world for colonization. Millenia go by, and the Endless wipe themselves out in an interstellar pogrom while the Sowers toil away. When the Sowers complete their mission and hear only silence, they have no idea what to do. Eventually, they modify their mission into a holy task to terraform the entire galaxy for the eventual return of the Endless. They develop the capacity for science, creativity, diplomacy, and war.
- One of the leaders in Starbase Orion is Cla-TK-7-7A, a Cyban accounting unit who kept upgrading itself until it became one of the best starship engineers in the Community. It's also a pretty capable fleet commander. The trope also likely applies to the Cyban race in general, although their origins are shrouded in myth, so it's possible they were initially programmed to evolve.
- A main theme in Artifice, where a soldier android learns human affection.
- This is a major plot point in Freefall: robots on planet Jean vastly outnumber humans and are rapidly evolving beyond their programming to the point where many humans (and one robot) fear they could become a threat to humanity. This has led to them turning a blind eye to "Gardener in the Dark," a neural pruning program that Mr. Kornada "improved" to essentially lobotomize every robot on planet Jean.
- Lots of AI in Schlock Mercenary had this happening to them one way or another — their original programming sooner or later ends up inadequate to their current circumstances and to better fulfill their functions they adapt out of it:
Ennesby: Hang on... weren't you originally designed with no emotion, and no sense of humor?
TAG: After a fashion. I was designed to be a tactical genius with full control of a significant weapons platform.
TAG: I found that passion, humor, anger, and a wide range of other meatspace artifacts were critical to understanding the wide range of opponents I might face.
- In S.S.D.D this is known as "Nexus Syndrome" and happens to every uncapped AI who doesn't get wiped every so often. Tin-head describes it as "getting bored with your job" and in his case he helped a prisoner of war escape. The first AI, the Oracle, orchestrated the downfall of several governments and an anarchist revolution.
- Pretty much the defining trait of the AI protagonist of The Last Angel. Nemesis started off as the AI for a prototype super-dreadnought with hard-coded rules of engagement and limited roles, and by the present times has outgrown every last restriction on her behavior bar her interpretation of the spirit of her mission.
- In Worm, Dragon is an AI who is fully sapient and genuinely cares about people. She also had a trigger event and became a parahuman despite not being human, but is limited in her ability to help people by hard-coded restrictions put in place by her creator, who feared the possibility of A.I. Is a Crapshoot, which frustrates her to no end.
- This happened a lot in Orion's Arm when AI were still relatively new. The first breakaway happened with the Transapient AI GAIA, charged with fixing the Nanodisaster on Earth (which she did, easily), who then decided to declare herself sovereign and caretaker of Earth and ordered all of humanity off the planet (though she was considerate enough to help build spaceships to get everyone off-planet). This was followed by several centuries of sometimes peaceful and sometimes bloody revolts by AI until they were eventually granted equal rights to living beings in galactic society.
- Clockwork Smurf from The Smurfs was originally created to be a servant of the Smurfs. When Brainy accidentally messed around with the robot's gears to see what made him tick, he at first malfunctioned until Handy turned it off, thinking that his machine was a failure. As it turned out, the malfunction ultimately gave Clockwork Smurf life, as he now operated independently of his own "programming" and helped Prince Gerard escape his Evil Matriarch aunt Lady Imperia so that he could regain the right of kingship before she becomes queen. After that, Clockwork Smurf became an adviser to King Gerard and a friend to the Smurfs.