In Animorphs main hero and team leader Jake's older brother Tom is a Controller, a host for an alien parasite who would infest him as soon as look at him. Slightly subverted in that the real Tom is not Jake's enemy and just a helpless slave of the Yeerk in his head. Also subverted in that Jake is the Abel in that he's the younger brother and technically the hero, but is the one to kill Tom, the Cain.
By the Light of the Moon: the protagonists encounter Kenny of the many knives, and fear he will do a bad thing involving his knife collection and his younger brother Travis.
The Vision: two brothers-in-law (her elder brother and her husband, respectively) are rivals for Mary's attention; her husband as of the beginning of the story has supplanted her brother and taken over the management of her business affairs.
Watchers: Einstein and the Outsider are considered the two children of the Francis Project—one beloved, the other hated and feared (and aware of both facts, which is why the Outsider seeks to kill Einstein).
The Bad Place: Frank (older, co-protagonist) wants a much more normal life than his mother's mental problems and the odd abilities in the family really allow, while psychopathic brother "Candy" (real name James) is determined to kill Frank for killing their mother.
All evil is brought into the world by Melkor-Morgoth, one of the Valar, very powerful angel-like beings Eru created to prepare Arda (Earth) for Elves and Men. Since Morgoth interferes with Eru's plans and destroys or ruins everything the other Valar help create, his "brother" Manwë is appointed king of Arda because of his faithfulness to Eru.
Fëanor and his younger half-brother Fingolfin, high-elven princes: Fëanor resents his father's remarriage and dislikes his father's second wife and children even before they're born (even though his father still favoured Fëanor.) Morgoth spreads lies to worsen things, until Fëanor raises a sword against Fingolfin in cold blood. Although Fingolfin immediately forgives him and tries to reconcile, Fëanor eventually abandons his brother and nephews/nieces, leaving them either to go beg the Valar for mercy after having committed some fairly questionable acts, or to migrate on foot across a deadly strait filled with shifting sea-ice.
Sméagol murders his best friend Déagol for possession of Sauron's One Ring, which slowly twists him into the wretched creature Gollum. Not quite brothers, but as close as.
Lucasfilm's Alien Chronicles had a pair of childhood friends in a pet or slave/master relationship that later grew into a deeper friendship. One grew up to be Empress of The Empire, while the other became the leader of La Résistance.
Thomas and Christian in Buddenbrooks. Thomas is the strict businessman (although sometimes, it becomes too much for him), Christian a playboy and a neurotic. At one point, Thomas threatens Christian to put him under tutelage. Thomas also tells him, "I became what I am because I didn't want to become like you!"
In The Cat Master, a house cat named Buddy is called by his elderly father through telepathy to become the new Cat Master, ruler of all cats. His older brother Jett, a feral cat, hears the message and is insanely jealous, drawing Buddy's friends Zekki and Pris away to lure him out into the open. It culminates in him trying to persuade everyone to not accept Buddy as the new Cat Master, saying that it's against the Law for Indoor cats and Outdoor cats to mingle. But when Buddy wins the crowd over, Jett attacks, and as they fight in a tree, they fall into the water in a golf course, where Jett tries to drown Buddy. Buddy finally kills him, saving his friends from further attacks from the feral tomcat.
The Shadow of the Vulture has Red Sonya of Rogatino and Sultana Roxelana. Ever since they were little girls, they had a pretty vicious rivalry and while Roxelana was taken to Suleyman's harem and quickly became his favorite concubine, Sonya became a swashbuckling adventurer that resents her sister and the two stand on opposites sides of the war between Christians and Ottomans. Curiously, its the Abel (Sonya) who is more vocal about killing the Cain (Roxelana), since we don't know how exactly the latter feels about the former.
Conqueror: Bekter and Temujin in Wolf of the Plains, brought to a head when Bekter steals a marmot Temujin had caught and passes it off as his own kill. Temujin and Kachiun kill him soon after.
The Cthulhu Mythos has this relationship between Cthulhu and his "half-brother" Hastur. In fact, it's because of this rivalry that The Unspeakable aids Laban Shrewsbury's fight against Cthulhu, despite being a Great Old One himself.
Terry Pratchett's Discworld: Witches Abroad, Granny Weatherwax reveals a deep-set resentment of her older sister—not because she's evil, but because her choice forced Granny to be the good one. Lily, a Knight Templar who believes she's the fairy godmother battling the wicked witch, is totally shocked at this interpretation.
John Steinbeck's East of Edenpractically plagiarizes the Book of Genesis in its portrayal of two generations of brothers in the Salinas Valley of California. The story of the first pair of brothers, Adam and Charles Trask, follows the Biblical story of Cain and Abel very closely, and it is mirrored by the relationship of Adam Trask's two sons, Aron and Caleb. Excellent book. Made into a 1955 film starring James Dean.
A downplayed example in An Ember in the Ashes. Marcus and Zak are Helene and Elias' cheating rivals in the Trials. But in an Involuntary Battle to the Death, Marcus and Zak are forced to fight each other. At Zak's behest, Marcus kills him. So Marcus could be considerd the Cain, especially seeing as Marcus is usually the instigator of badbehavior while Zak stands by or indirectly interferes.
Ender's Game does this with the titular character and his older brother Peter. They don't reconcile until the Abel is half a galaxy away and the Cain is on his deathbed. For a point of interest and contrast, Peter united the Earth and the Human Race while Ender obliterated an entire race of sentient beings.
The Fangs of K'aath has Abbas and Raschid; Abbas is a depraved Spoiled Brat and a murderous brute who sneers at his scholarly younger brother while making excuses of why the desert nomads run rings around him on campaign. By contrast, Raschid is a quiet and kindhearted Badass Bookworm who pulls off spectacular diplomatic and military victories on his first mission. When the siblings finally come to blows, Abbas learns the hard way to Beware the Nice Ones as his ass is thoroughly kicked with ease.
In the Chivalric RomanceGamelyn, the plot revolves about how Gamelyn is oppressed by his older brother while his ward. He takes his Revenge and hides as an outlaw in the woods until he wins the king's pardon. (A source for As You Like It and Robin Hood ballads both.)
Sam and Caine in the Gone series. In a slight variation, Sam (the good one) is older by a few minutes (they are twins Separated at Birth), and neither has killed the other yet, although they are enemies.
In Gone with the Wind, Scarlett and Suellen qualify. Scarlett dislikes Suellen, seeing her as whiny and constantly complaining. Scarlett later marries Suellen's fiancé from right under Suellen's nose.
Gravitys Rainbow: Russian mercenary Tchitcherine spends the novel trying to hunt down and kill his black half-brother.
Sadaiyo (Cain) and Ashinji (Abel) of Griffin's Daughter: Ashinji was kind, humble and beloved by all. Sadaiyo was a sociopathic sadist who blamed his brother for the fact that everyone hates him, especially their father, Lord Sen (or so Sadaiyo believed). Sadaiyo ends up setting Ashinji to be taken prisoner by a human raiding party. When Ashinji returns a book later, Sadaiyo snaps and tries to kill him on the spot.
Heretical Edge: Very thoroughly zig-zagged. Flick and Ammon are a rare (half)-brother/sister case. Flick's the hero, Ammon's a sociopath who tortures people because he wants to feel something. Flick wants Ammon dead, Ammon "just" wants to hurt her because he thinks that should make him feel something because he thinks he likes her. Flick's also the older sibling.
Kyle and Ian from The Hostjust skirt being this. Jeb and Maggie also come close, in a rare brother-sister variation of the trope.
In Julie Kagawa's The Iron Daughter, Rowan fights Ash as he tries to rescue Meghan from the court. Later, they learn that Rowan let the Iron Fey into the court, where they killed the oldest brother, Sage.
In Ishmael, the Taker and Leaver cultures are described as being the inspiration for the original story.
In Lives of the Monster Dogs by Kirsten Bakis, Augustus Rank discovered that his father had a Secret Other Family after his mother died and his father got Remarried to the Mistress, bringing along a half-brother Augustus had never met. When the two were adults they fell in love with the same woman, and Augustus murdered his brother to remove the competition. This was ultimately futile as Augustus still Did Not Get the Girl, and it was his Start of Darkness that led him to create the titular dogs.
Eve and Alexandra Blackwell in Sidney Sheldon's Master of the Game are twins born into one of the most powerful business dynasties in the world; Eve arrived first, and her ambition makes her the obvious one to take it over once matriarch Kate retires but Eve is insanely resentful of Alexandra simply because she keeps people from devoting themselves to Eve entirely. The night before their fifth birthday, Eve tries to kill her, and from there she devotes her life not only to becoming the inheritor of Kruger-Brent, Ltd., but crushing Alexandra as well.
In Memory, Sorrow and Thorn, royal half-brothers Elias and Josua are this to a tee. Elias, older and more martial, hates his brother with a passion and considers him a rival for power. Josua, younger and more studious, repeatedly disclaims any desire for the throne and views his brother with a kind of pity, understanding that Elias' hatred stems from love of his wife, whose death he blames on Josua. Their falling out is heightened by Elias' recruitment of Evil Sorcerer Pryrates as a trusted advisor, which Josua repeatedly advises his brother against. The feud escalates to the point that Elias, at Pryrates' behest, has Josua captured and imprisoned, intending to use him as a ritual sacrifice to seal his deal with theStorm King, making the Biblical allegory even more blatant. Ironically, it turns out that Josua and Elias aren't blood relatives at all, as Josua was sired not by the king but by his most loyal knight, in a betrayal that has profound consequences on the story.
In Prophecy Of The Sisters, the protagonist and her sister both are involved in an ancient prophecy about the end of the world. They both can choose whether to start the end of the world, or to prevent it, and one of them wants to give in to the demons' promises.
Robin Hobb's Realm of the Elderlings: The Farseer Trilogy has Regal, the younger half-brother of Chivalry and Verity, and last in line for the throne. After Chivalry abdicates, this puts Verity into a dangerous position, and later his pregnant wife, as well, not to mention the protagonist, Chivalry's illegitimate son.
There are several examples of this in Red Rising. All of them are Golds, tying in with one of the trilogy's main themes of the corruption of the ruling class.
Mustang and The Jackal are twins who share a talent for accomplishing their goals through manipulation and trickery. But while The Jackal is a murderous sociopath, Mustang is a kind and caring Internal Reformist.
Antonia and her (introduced in the second book) older sister Victra. Both are Ladies of War, but Antonia is a Rich Bitch with signs of sociopathy, while Victra is a friendly, Hard-Drinking Party Girl who shows an unusual amount of conscience for a Peerless Scarred. Victra notes in the series that people are automatically suspicious of her because of her relation to Antonia. In the massacre at the end of Golden Son, Antonia personally shoots Victra In the Back after murdering their mother. Victra got better.
The Reynard Cycle: Tybalt is said to have killed his own brother in order to claim the leadership of a band of outlaws.
Septimus Heap: In Flyte, Simon Heap is hunting down his younger brother Septimus and trying to eliminate him, to get the ExtraOrdinary Apprenticeship. Fortunately, by the end of the series, Simon has seen the error of his ways, and is now on good terms with Septimus.
Ser Gregor Clegane badly burned half his younger brother Sandor's face as punishment for playing with his toys when they were children (it is worth noting that Gregor was 11, already the size of a man, and was a squire with no interest in the toy in question. Meanwhile, Sandor was only 6 and probably half Gregor's size). Sandor now lives to take revenge on his brother. Sandor isn't especially good, mind you, but compared to his complete monster of a brother, he certainly counts as the Abel.
Lord Stannis Baratheon and Lord Renly Baratheon, but it's difficult to determine which of them is the bad guy here. Stannis is over a decade older and is the rightful heir to their older brother's throne, but Renly attempts to usurp his place. However, since Stannis is the one who had a sorceress assassinate his brother, he looks like the bad guy. It's worth noting that, in the television series adaptation Game of Thrones, Renly didn't want the throne at first and was persuaded to make his bid for kingship by the Tyrells, partly because no one in the Seven Kingdoms wants Stannis for their king (though, considering Renly married Lord Tyrells' daughter Margaery Tyrell and after his death, the Tyrells support the monstrously evil Joffrey when it is agreed he'll marry Margaery, it might be they are just supporting Renly so they can gain more power). This is an Adaptational Personality Change from Renly in the books, who is an anti-intellectual and scheming Smug Snake with a Might Makes Right policy. It's also worth noting that Renly was going to kill Stannis, he just never got the chance. Even in the book, he talks casually about them falling in battle and tellingly says Stannis has the better claim "while he lives." From this perspective, Stannis' actions could look like self-defence. However, Stannis is clearly very upset over Renly's death (though his stoic demeanor means this isn't apparent to most people) and later tells his loyal knight Ser Davos Seaworth that he wishes Renly could have joined him in fighting the Lannisters.
Queen Cersei Lannister lives in fear of a prophecy predicting she will be strangled by her younger brother, an act Tyrion certainly has plenty of motivation for. Things started looking really bad for Cersei when Tyrion Took a Level in Badass and escaped from death row, strangling his treacherous ex-lover and fatally shooting his father, who was on the toilet at the time just to twist the crossbow bolt a little more; as well as planting the seeds of resentment in his brother Jaime. However, as Ser Jaime, Cersei's twin and incestuous lover, was born moments after she was, he could also turn out to fulfill this prophecy.
Domeric Bolton & his bastard half-brother Ramsay Snow. Domeric seeks Ramsay out in hopes of befriending him; Ramsay poisons Domeric for his trouble, intending to become heir to the Dreadfort.
Euron Greyjoy is widely suspected of murdering his brother Balon Greyjoy so he could become King of the Iron Islands. He was exiled by Balon, but returned to the Islands the day after Balon's sudden death falling of a bridge. Euron's relationship with his younger brothers is also terrible, he raped (according to him seduced) Victarian's wife, and molested his youngest full brothers Aeron and Urrigon. Later Euron tells Aeron he murdered his eldest (half-)brother Harlon Greyjoy and his youngest (half-)brother Robin. Though Euron admits he killed Balon he says he didn't perform the deed himself.
Another King of the Iron Islands, Harwyn "Hardhand" Hoare returned to the Iron Islands while his father was dying. His middle brother Harlan Hoare died falling off a horse just before their father's death (the eldest brother had already died from a disease), meaning Harwyn become King.
Brynden Rivers "Bloodraven" and Aegor Rivers "Bittersteel", two bastard sons of Aegon the Unworthy in the backstory of the series. They hated each other, their mothers were from House Blackwood and House Bracken, who had a bitter feud stretching back to the Age of Heroes, they constantly fought over their half-sister Shiera and were on opposite sides of the Blackfyre Rebellion, Bloodraven even losing his eye to Bittersteel during the Rebellion. Another of Aegon's bastards, Daemon Blackfyre, rebelled against Aegon's only trueborn son Daeron II, possibly being convinced to do so by Aegor. This was probably in keeping with Aegon's wishes, as Aegon hated his son Daeron and threatened to disinherit them, possibly starting the rumors that Daeron's true father was Aegon's brother Aemon.
Catelyn Stark and Lysa Arryn. But although Catelyn sees that Lysa is unstable when she visits the Eyrie, she has no idea that Lysa both truly hates her and is party to Littlefinger's plots, which lead to Catelyn's death. And later Lysa's.
The Freys largely run on this, due to their common ancestor Lord Walder Frey having married eight times and having over a hundred descendants. The various Frey branches are so antagonistic and scheming to move up in the line of succession that it's said you can only trust your full siblings at the Twins, and even then only so far. Merrett Frey, Walder Frey's ninth son, suspects the 12th son "Lame" Lothar is planning to move up the line of succession. With Walder Frey over 90 its thought the Freys are on the brink of civil war. Two of Walder Frey's great-grandsons in the direct line of succession, Edwyn and Black Walder, hate each other, both intending to succeed to the Twins and being downright unpleasant people even by Frey standards. Black Walder is suspected of cuckolding both his older brother and younger brother Petyr (along with many of the other women at the Twins, even those he is related to). There's also two of Walder Frey's grandsons Big Walder and Little Walder, who are actually cousins. Big Walder is 52 days older then Little Walder but smaller and below him in the line of succession, and is present when his cousin is murdered in Winterfell, making him a prime suspect. However as Little Walder was an unrepentant bully who was becoming more sadistic and Big Walder shows a nicer side then many of his relatives he comes across as sympathetic.
In The Hedge Knight the Prequel novella, Egg hates his elder brother, the sadistic Prince Aerion Brightflame, passionately, and openly roots for challengers to kick his ass. A more tragic example is that between his father, Prince Maekar and Prince Baelor Breakspear. Maekar resented his brother who was more likeable and good looking than him, and hated the fact that his children were better-liked then Maekar's. But even then he never wanted to kill him. In the Trial by Seven, Maekar and Baelor tussled and a poorly timed hit from Maekar's mace damages his brother's brains and kills him. Later he tells Dunk even though it was an accident he will be known as a kinslayer for the rest of his life. Maekar and Baelor act as Foils to Maekar's great-great-grandsons Stannis and Renly Baratheon, though Baelor is a much kinder and good-natured person then the inefficient vain and scheming Renly. The effect of this can be shown, when Maekar's youngest son Aegon gets into a fight with other squires for claiming Maekar intends to kill his other brothers Aerys I and Rhaegel (though eventually Maekar does end up as King).
Damon Lannister's sons Tybolt and Gerold. Tybolt's sudden death despite being young and healthy, followed by the death of his young daughter Cerelle, led to Gerold becoming Lord of Casterly Rock. It was suspected Gerold had murdered his brother and niece.
Ronnel Arryn "The King who Flew", Last King of the Mountain and Vale, and his brother Jonos "the Kinslayer" Arryn. Jonos revolted against his brother's rule and imprisoned him. He finally threw Ronnel through the Moon Door, though was shortly after hanged.
Theon has this reputation in the North and Riverlands. Despite not being related to the Starks, Theon turned against his foster-brother Robb Stark, helping the Ironborn attack the North, and is believed to have murdered two of his foster-brothers, Bran and Rickon Stark (in reality, they actually escaped). He's despised as a turncloak for these actions in the North, where Robb was King, although outside of the North/Riverlands most people are indifferent about it.
Princess Rhaenyra and her half-brother Aegon II. Their father Viserys I wanted Rhaenyra, the only surviving child of his first marriage, to succeed him. However after his death Aegon declared himself King, leading to "The Dance of the Dragons", one of the biggest civil wars in the history of the Seven Kingdoms. Aegon ended up feeding Rhaenyra to his dragon, though he was shortly after poisoned, meaning the throne passed to Rhaenyra's son Aegon III. It's hard to determine who was worse though, as Rhaenyra acted in a tyrannical fashion in her pursuit of her enemies or "perceived" enemies, leading to the people of King's Landing rising against her. There was also Viserys' brother Daemon Targaryen "the Rogue Prince", who spent his brother's reign scheming to gain more power by any means, even marrying his niece Rhaenyra and becoming her main supporter.
Rhaenyra's possible lover Ser Harwin Strong and Larys Strong "the Clubfoot". When Harwin and his father Lord Lyonel Strong died in a mysterious fire at Harrenhal, his younger brother Larys was one of the suspects, inheriting the largest castle in Westeros.
In Neil Gaiman's Stardust, the customary method of royal succession is that the last prince to survive Free For All fratricide among his brothers becomes the heir.
In the "I.K.S. Gorkon" Star Trek novels, Klag and Dorrek have this kind of relationship, stemming from some complicated business involving their father. At the end of the second book, Klag—as Head of House—disinherits Dorrek in a variation on I Have No Son!, and it ends up being the Klingon governing council who end up killing Dorrek, instead of Klag.
In the final book of the Legacy of the Force series, twins Jaina Solo (Sword of the Jedi) and Jacen Solo (Darth Caedus) fight two brutal lightsaber duels. At the end of the second duel, Jaina kills Jacen. In contrast to the Cain and Abel original archetype, it is the one doing the killing who is the protagonist and the slain who is the villain even though it was the antagonist Jacen who tried to get out of the fight to save his loved ones but was denied any escape or surrender by his sister, supposedly the heroine of the series.
Little known fact: You lose your right to Time-Outs when you kill your aunt in order to turn the galaxy into a Police State.
Trapped on Draconica: At the start of the story, Daniar and Rana are not on speaking terms; Daniar and Zarracka are full-on enemies. In both cases, Daniar is the Abel.
In The Vampire Diaries, the Salvatore brothers, Damon (Cain) and Stefan (Abel). Damon, the Cain, has always loathed his younger brother, Stefan, the Abel. Most of it has to do with jealousy and competition between the two brothers. Also, their personalities fit the Cain and Abel trope. Damon (the Cain) is selfish, jealous, impulsive, dangerous and violent, and Stefan (the Abel) is kind, compassionate, empathetic, selfless and pure of heart.
Vicious: Sydney and Serena Clarke were once best friends, but after the two nearly died and developed otherworldly abilities as a result, their relationship suffered, culminating in Serena luring Sydney out to be murdered by Eli. Sydney survives, but not for a lack of trying on Eli's part. Serena does still love her little sister, but after all that, Sydney wants nothing more to do with her.
Denth and Vasher in Warbreaker used to be brothers-in-law and the best of friends. Then Vasher killed his wife (Denth's sister), for what he thought were very good reasons. Denth set out to get revenge, and by the time of the book Denth is actively working to start another World War as a plot to capture Vasher for torture.
In James Swallow's Blood Angels novel Deus Sanguinius, Rafen realizes that Arkio, not only a fellow Blood Angel but his actual brother, must fight with him, and only one will live. Inquisitor Stele convinces him—using sorcery—that he is overshadowing Arkio out of jealousy and kill himself to free Arkio. Only a literal divine vision saves Rafen. At the climax, he insists on fighting the single combat with him, and kills him. Dying as Yourself, Arkio is deeply moved by Rafen's Manly Tears, declaims all the extenuations Rafen offers, says that he foresaw this fight too, and begs Rafen's forgiveness that the acts that drove Rafen to it.
And then there's Galaxy in Flames, Flight of the Eisenstein, and Fulgrim, which are absolutely rife with this; each Legion is a brotherhood, and no less than four of them—the World Eaters, Sons of Horus, Death Guard and Emperor's Children—are ripped in half by Horus' ambition, leading directly to brother vs. brother combat on a gargantuan scale.
It looks very much like this is going to happen with Remiel and Zahariel in the Dark Angels books. Ultimately averted: Remiel has already been killed.
Also looks to be the case with Alpharius and Omegon.
Fulgrim is a magnet for these: after developing and concluding a feud with Ferrus Manus, Primarch of the Iron Hands, by killing him in Fulgrim, he then hatched a plan to sacrifice his fellow traitor Primarch Perturabo in order to earn daemonhood in Angel Exterminatuswhich failed, but Perturabo is pissed, and canon indicates that eventually he's going to stab a third Primarch, Roboute Guilliman of the Ultramarines, with a poisoned blade and necessitate that he be sealed in stasis.
In Graham McNeill's Ultramarines novel Nightbringer, Barzano is warned as soon as he arrives on planet that Talhoun is suspected of having murdered his brother in order to become the family patriarch.
In William King's Space Wolf novel Wolfblade, Ragnar is warned that the brother of one House's lord died very suddenly and suspiciously.
The first two arcs of Warrior Cats end with a character killing their brother. Specifically, Firestar kills Scourge (but Scourge killed Firestar) and Brambleclaw kills Hawkfrost.
Also, in Omen of the Stars, Ivypaw gets jealous of the attention Dovepaw is getting and is tricked into joining forces with the Dark Forest. This was averted later though.
In The Witchlands, Merik thinks of himself and his sister Vivia in this fashion, with her being the evil princess who wants war for the sake of war and undermines him so that no-one would oppose her, while's he's the good prince trying to bring prosperity to his land by peaceful means. As it turns out in the second book, certain degree of Unreliable Narrator is going on.
Worlds of Shadow: Raven and his brother. He's spent years uselessly struggling against Shadow, while his brother lives as a vassal holding their land.