"The Terrible Old Man" by H. P. Lovecraft. Some burglars decided to go for an easy target: that weird old retired sea captain who lives on the edge of town, has a garden full of creepy statues, and talks to a collection of jars with little pendulums inside. It doesn't go well.
In the first chapter of Artemis Fowl, a man tries to pickpocket Butler. Due to Butler being hugely strong, said pickpocket gets his fingers broken.
A flashback in Billy & Howard features this. Three goons decide to pick on Billy's friend. Billy goes nuts (and more than a little racist) and brutalizes all three.
Happens twice in Brent Weeks's The Nightangel Trilogy: first when Azoth's gang try to ambush Durzo Blint, the most accomplished wetboy (magical assassin) ever, and again in the third book when the next generation of child gangers try to ambush Kylar who has taken Durzo's place.
In Polgara the Sorceress, the title character is riding alone through a forest when two bandits attempt to rob/rape her. She calmly states that she is glad that she finally found some food and disguises herself and her horse as monsters with an illusion, sending the pair running.
Eddings subverted this early in The Elenium. Some street thugs decide to mug that guy on the warhorse who just rode into town. Sparhawk tells them he's not interested in playing, as he throws back his cape to reveal armor and broadsword. The thugs decide to go elsewhere.
And then played it straight (but as a non-mugging example) in the sequel, The Tamuli. A character makes a not-quite-audible, but clearly offensive remark about Ehlana, Sparhawk's queen and also his wife. Another character calls for a moment of silence in memory of the loud-mouthed oaf who made the comment; the oaf doesn't get the hint until he's told just exactly who Sparhawk is.
An interesting example in The Way of Kings. Jasnah deliberately sets herself up for this — strolls down a dark alley, decked out in jewels, carrying no visible weapons, etc. When the muggers (and attempted rapists/murderers), do show up, Jasnah naturally destroys them. Shallan is horrified by the moral implications of this baiting, but is also alarmed by Jasnah's unusually passionate hatred of her attackers. It's implied Jasnah may take these little excursions to dangerous parts of town because of a deeply traumatizing time she was mugged when she wasn't a "monster."
In Guards! Guards! , a crook tries to rob what turns out to be a fire-breathing dragon. It doesn't go well.
This trope is also part of why Carrot Ironfoundersson had an uneventful 500-mile journey from the Copperhead Mountains to Ankh-Morpork.
People who are rather more than six feet tall and nearly as broad across the shoulders often have uneventful journeys. People jump out at them from behind a rock then say things like, "Oh. Sorry. I thought you were someone else."
In Reaper Man, a gang tries to rob Windle Poons, who is not only an elderly wizard but one who has recently come back from the dead as a zombie.
In Feet of Clay, a group of crooks tries to rob The Bucket, the local pub for coppers. They bust in only to find it full of off-duty police offers who don't take kindly to having their boozing interrupted. And then, to top it off, they take Angua hostage, presuming that she wouldn't be as dangerous as Carrot or Vimes. No-one told them that she's a werewolf. The audience calmly ignored them (apart from a few who made quips to the effect of "Don't play with your food.").
Some jewel thieves later make the same mistake about Angua in Jingo — they end up confessing to any crime suggested (even in cases where they have to guess what was stolen or lie about their gender) while begging to be let out of the vault.
And in Men at Arms, members of the Assassins' Guild gather in their courtyard to threaten the Watchmen, or possibly kill them "for trespassing" if Vimes won't back off and leave. Then they realize that one of the Watchmen is Detritus... and their elegantly-crafted stilettos and sophisticated poisons will do diddly-squat to a troll, whose skin is solid stone and impervious to anything short of a really enthusiastically-wielded pickaxe and whose bloodstream is already a soup of heavy metals that're nastier than the poisons..
In The Amazing Maurice and His Educated Rodents, a Genre Savvy highwayman robbing the coach Keith, Maurice, and the rats are on goes through a sort of checklist to see if there are wizards, witches, trolls, werewolves, or vampires on the coach (all of which have been the subject of this trope in previous books). Too bad he didn't check for Talking Animals...
It happens twice in Lords and Ladies; first, Casanunda, the "World's Second Greatest Lover", attacks a coach full of wizards. After a display of octarine fire control, the dwarf joins them, whereupon they are attacked by a band of highwaymen — the leader of which is turned into a pumpkin for his trouble, retaining his hat in accordance with the Universal Laws of Humor.
In Carpe Jugulum, a highwayman tries to rob the visiting Count and his family of vampires, and that doesn't end well. Then his ghost tries to rob Death. Death is more amused than anything, complimenting the man on his (relative) vitality in trying to stick to his guns post-mortem.
A few valkyries get mugged by the people they came to pick up. Although that group was Cohen the Barbarian and his horde of elderly (read: with a lifetime's experience of surviving in the most dangerous profession in existence) barbarians.
The New Firm in The Truth end up on (almost) both sides of this trope — they're subject to a (legally sanctioned) mugging attempt by a member of the Thieves' Guild, which does not go well for the thief, and later walk into Biers during business hours and openly try to intimidate a Werewolf into working for them — this would normally be suicidal but thanks to Mr Tulip's almost inhuman ability to intimidate people without trying they manage to walk out alive (but without the werewolf).
In the original past of Night Watch some thieves tried to rob John Keel; he made short work of them. In the modified past, Carcer claims that some thieves tried to attack HIM — "at least, they had some money with them". He was among the thieves who attacked Keel and killed him.
Andy Shank and company trying to beat up Nutt in Unseen Academicals. As an orc, Nutt is much stronger than his size would suggest and could probably dismember them without trying if he wasn't so pacifistic.
Ponder Stibbons reminisces about how his wizardly aptitudes first manifested, when he was a geeky kid and some bullies began beating him up. Nothing like bruises and humiliation to motivate one to start throwing fire around.
There's also a variation in Jingo, where the (relatively) good guys Colon and Nobby try to mug some Klatchians in an alleyway to get their clothes in order to go undercover, but the Klatchians get the better of them and they lose their own clothes instead.
Happens between the Summoning Dark and Vimes in Thud!. A subversion, as Vimes unconsciously drives the ancient hate monster from his mind... In later books, it's still with him, but more or less tame!
Yet another one in Hogfather where Susan Sto Helit stops for a drink in a bar popular among various kinds of supernatural monsters. A drunken bogeyman mistakes her for a human girl slumming among monsters for a thrill and makes increasingly scornful and lewd comments to her, despite the warnings of the bartender, who knows that Susan is only mostly human. The bogeyman gets an unpleasant surprise both in the bar and later when he hides under the beds of the children Susan is caring for as a governess. Perhaps the bartender should have told him whose granddaughter he was messing with.
Susan's father got subjected to this trope first in Mort. Lampshaded by the author: Three men had appeared behind him... They had the heavy, stolid look of those thugs whose appearance in any narrative means that it's time for the hero to be menaced a bit, although not too much, because it's also obvious that they're going to be horribly surprised. They threaten to kill him. Mort is Death's apprentice. It goes about as well as you imagine.
In Going Postal Moist von Lipwig is discussing a special mail coach to Genua. The danger of bandits is brought up, and one of the brothers operating the coaches points out that there aren't any bandits on that road any more, which is good. The other isn't so sure, since they never found out what wiped them out. This is the road through Überwald, no less. (They were probably literally mugging the monster, since Uberwald has vampires, werewolves, and, as of two books after Going Postal, orcs.)
In The Last Hero, a pair of bandits come across an old woman among the Hub's snow, sitting at a fire, knitting, and stuck in the snow next to her is the largest sword they've ever seen.
Intelligent robbers would have started to count up the incongruities here. These, however, were the other kind, the kind for whom evolution was invented.''
We learn that sword belonged to another man with very big feet... whose corpse is lying behind a nearby rock. And so we meet Vena the Raven-Haired, the Discworld counterpart to Xena: Warrior Princess.
In Raising Steam some highwaymen target what they think is just a plain black coach. Sadly for them it contains Lord Vetinari himself (his coat of arms being a black shield upon a black background). His Lordship, one of the best students the Assassin's Guild ever turned out, appreciated the opportunity to stretch his legs during his otherwise tediously boring journey.
Subverted in Lawrence Watt-Evans's Ethshar novel The Misenchanted Sword. The protagonist has a magical sword that won't let him die till he's killed 98 more people, and he wanders back alleys looking for trouble, acting like an old man, a big purse of gold on his hip, expecting this trope. He doesn't run into anyone. Doubly subverted when he later runs into some thugs robbing an elderly lady. Also played straight with said elderly lady.
In one of the Troubleshooters books, someone attempts to mug Jules Cassidy, who's short, gay, and looks like he could be in a boy band. Jules, however, is an FBI agent.
Happens to Jack about once every book, starting in the first installment where he does it on purpose to draw out the mugger who stole a MacGuffin from his client.
A later novel reveals that Jack goes out and gets himself mugged in the park each year, to raise money for the Little League in the form of his would-be muggers' wallets and jewellery.
In Mercedes Lackey's Children of the Night, a shapeshifting souleater vampire who leaves a group and comes back sated is said by the group's leader to have been "trolling for rapists" in the form of an attractive young woman. The doubting member of the group, who is repulsed and uncomfortable about basically murdering random people but needs to feed, thinks this sounds like a good idea, and so he wanders Central Park until a junkie attacks him and is killed.
In Mercedes Lackey's Brightly Burning, Lavan is cornered by a pack of school bullies who pin him down and beat him with a lash. His panic results in a violent breakthrough manifestation of his Firestarting gift that he's the only survivor of.
Lackey likes this trope. In The Last of the Season, what could be more helpless than a cute six-year-old girl holding a teddy bear?
Iron Fist starts with an attempt by Imperial elements to capture or kill the Wraiths; they send a cyborg to the bar where the Wraiths are enjoying themselves, have the cyborg start a fight, and then show up dressed as the local police to arrest everyone. But the Wraiths cotton on to the fact that something's not right. The roster at the time included Runt, Piggy, an expert in hand-to-hand combat, and Phanan, who promptly cut someone's throat with a laser scalpel.
Happens in Shatterpoint. Corrupt police think they can steal Mace Windu's things while he is in customs because he is naked and unarmed. They soon learn that a Jedi Knight does not need clothes or a lightsaber to kick ass.
In Diane Duane's Spider-Man novel The Octopus Agenda, three punks try to assault Venom. With switchblades. Yeah, that doesn't go so well for them.
In Jhereg, young Vlad turned out to be the Monster when accosted by a drunken old Dragaeran sailor. The belligerent Easterner-hater took a header off a cliff, as Vlad scored his first confirmed homicide.
Later, while on the run from the Jhereg, he gets his money robbing bandits. The beginning of Iorich suggests that this is usually a consequence of their failed attempts to rob him.
And in the Viscount of Adrilankha, Piro and company attempt to rob a merchant wagon that turns out to contain his dad and Pel, who'd set the whole thing up with the express intent of getting attacked.
An Italian pickpocket is compelled by an unscrupulous police detective to stage a botched theft from a gentleman whose fingerprints the cop wants to collect covertly. The detective doesn't warn the soon-to-be-late filcher that the target is none other than Hannibal Lecter, with entirely predictable consequences.
In a related variant, the house of Glen Cook's Garrett, P.I. is occasionally broken into, as it's situated in a bad neighborhood. Only out-of-town criminals do so lightly, however, due to the resident Dead Man's vast telekinetic powers and nasty sense of humor.
The Six Sacred Stones. The team crash land in Darkest Africa, and run into a rape gang. Zoe decides to draw their attention while Wizard sneaks out the back with the kids. Did I mention she used to be called Bloody Mary in the Irish Army? Those poor souls didn't stand a chance.
In Maximum Ride, Max takes a stand to protect a girl being threatened by several bigger boys, one of whom is carrying a gun. Max is a genetically engineered hybrid with superhuman strength and agility and has been trained for some time how to use it. The boys basically tell her to bugger off. Cue the buttkicking.
Although to be fair, she did end up getting shot by one of them later. Doesn't tend to work as well when someone has a gun.
In Rainbow Six, three Spanish terrorists attempt to hijack a plane. A plane that has a former SEAL, former SAS major, and former Army Special Forces on board. The end result: three unconscious Spanish terrorists.
In an earlier scene in the same novel, a mugger is assaulting a woman when Kelly happens along and intervenes. In an excess of drug-fueled bravado, the mugger turns on him, only to find himself stone cold dead in a matter of seconds. Ironically, this good deed is what gives the cops a clue about Kelly's identity and modus operandi.
Brigand: (stopping a lone rider) Get down or die. Elminster: (knocking down three men with a spell) I believe a more traditional greeting consists of the words "well met."
At the beginning of Elminster's Daughter, a thief named Narnra Shalace again tries to rob Elminster. He easily fends her off and is about to knock her out when a weird magical interaction reveals that she's his daughter.
In Star Trek: Deep Space Nine novel The Siege, a Psycho for Hire Shapeshifter Meta turns into a girl and befriends a Bajoran girl and her mother. At this point a Cardassian barges in and decides to rape them all, starting with Meta. The Cardassian is ripped apart. From the inside.
A white supremacist hick picks on the super-swordsman Hiro Protagonist. Hiro waits just long enough for the hick to threaten his life so he can decapitate him with just cause.
One reason Hiro prefers his swords is their tendency to avert this trope; skinheads aside, most lowlife types aren't dumb enough pick fights with someone who is obviously carrying a pair of swords. And even the skinhead might not have tried it without a roomful of buddies, not that they improved his life expectancy any.
The Japanese businessman who picks a virtual swordfight with Hiro is another example: while the businessman isn't nearly as good as he thinks, Hiro is awesome in real life and also wrote the code for swordfighting in the virtual world.
The first chapter of a novel bridging a film and its sequel has a Somali pirate attempting to plunder a ship with a black pickup truck and a light green emergency-crew Hummer on deck. Oh, did I mention the films this novel was bridging were the Transformers movies? It specifically notes that one man who survived not only quit piracy, but, for the rest of his life, crossed the street whenever he encountered a pickup truck.
Either inverted or played straight in Oleg Divov's Night Watcher: a vampire decides to stalk a drunken cop, whom the readers already know to be a Badass Normal and The Big Guy. The cop mistakes the fruity vampire for a gay stalker and decides to teach him a lesson before he causes any trouble, preemptively attacking, beating him up and dragging him off to the station. Mind you, the vampire was mostly taken by surprise, and might have turned the tables later, if not for this cop encountering his colleague Captain Kotov along the way; Kotov quickly realizes what's going on and finishes the vampire off.
In Lois McMaster Bujold's novel A Civil Campaign, some well-meaning but clueless relatives are trying to forcibly drag a child away from his unwilling mother. He barricades himself in the bathroom and makes a phone call to a person who invited him to call earlier... Cue one of the biggest Oh Crap moments in literature when they find out that the person on the other end of the call was the Emperor of Barrayar, who is not pleased!
In The Vor Game a opportunistic merc chieftainess thinks she can outwit Gregor and Miles together in order to bring her near the Barryaran throne. She thinks she can be a better Manipulative Bastard then two of the cleverest Vor on Barryar.
'Gregor Vorbarra "Did you think you were dealing with an amateur?"
In Wolfblade, a Warhammer 40,000 novel, while on Terra 3, Space Wolves are out at a pub having a quiet dinner. Some morons try to start a brawl with them. Note, said Wolves are Astartes, are about 8 or 9 feet tall, are superstrong and fast, and were wearing their power armor. Needless to say, the morons got their brawl.
In the Stephen King short story "Popsy", Sheridan, who has been abducting and selling children to pay off his gambling debts, kidnaps a young boy from a shopping mall. Unfortunately for him, the boy is a vampire and manages to break his restraints and turn the tables on his kidnapper just in time for his grandfather, the eponymous Popsy, to come pick him up. The two vampires exsanguinate Sheridan.
Thieves' World book 3 Shadows of Sanctuary, story "Looking for Satan". Wess, a naive young woman just arrived in Sanctuary, goes out for a walk at night. She is attacked by Bauchle Mayne (a criminal her group had run into earlier) and his accomplice. After she knees Bauchle Mayne in the groin and slashes the accomplice with a knife, the accomplice drags Bauchle Mayne away as fast as they can go.
Happens somewhat early on in Jack Higgins's On Dangerous Ground when a Neo-Nazi thug grabs a woman. She responds by kissing him, turning out to be a distraction ploy for her to grab her flick knife tucked up her skirt, which she later uses to cut his face. It is revealed later on that she is a former member of a loyalist paramilitary force in Northern Ireland.
"The Last Defender of Camelot" by Roger Zelazny begins with a trio of muggers picking on a harmless-looking old man who turns out to be the last surviving Knight of the Round Table — and not just any knight, but Sir Lancelot du Lac, who never lost a fight in his entire life.
Happens a few times to Fleming in the Vampire Files series, both in the city and when he's jumped by tramps on his parents' unoccupied farm. A Vegetarian Vampire, he doesn't actually hurt such attackers, just scares the living shit out of 'em.
In The Visitor, the second book in the Animorphs series, a young man attempts to persuade Rachel to get into his car. When the creep doesn't take no for an answer, she scares the living daylights out of him by morphing halfway into an elephant. Considering that she could have completed the morph and stomped him flat or just gone grizzly bear on him, the jerk got off easy.
Marco also does it in The Predator...he begins morphing gorilla and pounds a couple bullies in an alleyway, even though it's quite dangerous morphing like that in public.
Happens on a species level in many science-fiction stories, a noteworthy one being Harry Turtledove's "The Road Not Taken". Aliens, who note that humans are so primitive they don't even have antigravity or FTL (which is so easy to discover that on some planets hunter-gatherers have stumbled onto it) figure it will be a routine invasion. So they march out of their landing ships, arrange themselves in rows, and raise their blackpowder muskets. On 21st century Earth. It is a very short invasion.
1632: Doesn't it sound like a nice recreation for overworked sixteenth-century mercenaries to Rape, Pillage, and Burn around this small peaceful town in which everybody just happens to own and know how to use twentieth century firearms?
In one of the Judge Dredd novels, bad things happen whenever Dredd tries to take a bath including one incident when some thieves attempted to burgle his apartment. Needless to say they ended up serving long sentences.
Curse Of The Wolf Girl has a couple of rather yobbish Alpha Bitches relentlessly bully Agrivex, a young fire-demon currently pretending to be human to attend college, which surprisingly works out all right for them since 'Vex is a sweet natured girl who really doesn't understand nastiness or is prepared to hurt people. Then they try it on 'Vex's friend Kalix. Kalix is a werewolf with anger issues. This works out substantially less well for them.
Montparnasse tries mugging Jean Valjean in Les Misérables. Valjean defeats him easily, gives him a lecture on the dangers of idleness, and then gives him his money. Interestingly, Gavroche then proceeds to steal the money from Montparnasse and gets away with it, proving that when you're sneaky enough, robbing the monster sometimes works.
In Septimus Heap, Ruper Gringe shouts at Simon Heap when the latter is attacking the Dragon Boat Rupert is on, taunting him to fight "like a man". Only being thrown into the water by Nicko Heap saves him from being incinerated by Simon's subsequent ThunderFlash.
After the events of the 12th book, short, blond, cute little Sergeant Murphy has not only battled several monsters along side her wizard best friend Harry Dresden, but just came back from helping him wipe out an entire race of vampires and was on her way to pick up her best friend for a date only to find him dead. Anyone who tries to go against her after her Heroic BSOD, from abusive husbands not appreciating her interference to a bunch of Fomorians muscling in on Chicago territory and slave-trafficking to vampires wanting to blackmail her, very quickly learns why she's still alive after hanging out with one of the most dangerous wizards on the planet for so long.
A humorously-inverted example has a bunch of wannabe dark wizards try to challenge Harry Dresden to a magical duel. They are rather taken aback when he pulls a gun on them.
"'I'm a-fixin' to defend myself', I drawled, Texas-style."
In the third Honor Harrington novel, The Short Victorious War, a Havenite battlecruiser squadron is expecting to smash up a small Manticoran patrol. Instead they drop out of hyper right on top of the dreadnought HMS Bellerophon, which, after her crew gets over their surprise, blows them all to flinders in one broadside. It was a complete fluke: the Bellerophon was on its way back to base and just passing through.
That Is NOT a Good Idea! by Mo Willems A hungry fox sees a nice plump goose in the streets. He asks if she wants to come to dinner (with the intention, of course, of eating her). What he doesn't realize is she sees him the exact same way. The goslings make it a point to reiterate how bad an idea this proposition is. Guess which one of them is, in both senses of the term, in the soup.
Given a pointed aversion in The Lord of the Rings. The hobbits are returning home to the Shire, and stop back at the Prancing Pony Inn in Bree. Fellow travelers complain that the roads have become extremely dangerous and bandit-ridden, which confuses the hobbits, who have not been molested in the slightest. Well, no, you wouldn't be, explain the others — the hobbits are all armed and armored. No robber in his right mind is going to pick on the group that's attired for war.
This happens with newly-crowned King Eugenides in Megan Whalen Turner's The Queen's Thief. Sejanus, the leader among Gen's attendants, conspires to make Gen's life miserable in any number of small ways, from pranks to endless hassle, for months. It makes Gen the butt of a lot of jokes, because what kind of king can't even control his own attendants? By the time Sejanus is done tying his own noose with the rope Gen oh-so-kindly let him have, he's got a life sentence, his brother is exiled, and their noble house, one of the Crown's biggest threats, is all but destroyed.
It also happens in the same book with a trio of assassins. They expect to have an easy time murdering a single, sickly, crippled man with no serious combat training. They do nearly get him, but not before he disarms one, stabs him and another to death with the stolen blade, then kills the third by throwing it.