The Riddler is often treated as slightly less of a threat than most of Batman's gallery because his particular lunacy isn't inherently violent, and he has a compulsion to tell Batman and the police what his plans are (he's tried not to, but he just can't). It's tough to write a Riddler plot that can believably challenge Batman...so many writers don't, essentially writing him as a joke. The difficulty of writing good Riddler stories may also be a factor in the character's recent Heel-Face Turn, wherein he decided to use his genius for puzzles to solve crimes as a (well paid) private detective...at least for NOW...
One issue of The Batman Adventures takes this and runs with it for all it's worth. The Riddler decides to try one last time to beat Batman, vowing that if Batman solves the riddle and defeats him, he'll give up crime forever. The riddle he comes up with really is good, but Batman's busy with multiple other villains and essentially decides to not spend time on the Riddler, and catch him after the fact if necessary. He catches him anyway, completely by chance, and admits as much to the Riddler when asked how he solved the puzzle. Satisfied that he outwitted Batman, even though he got caught, Riddler sings all the way back to Arkham.
In another Adventures book, Riddler found a Really Good hideout, and taunted Batman with riddles about other criminals' planned crimes. Unfortunately, his OCD caused him to structure the riddles as a meta-riddle that led Batman and Robin to him. Initially, he's going to fight them off ... but then he stops and says (more or less) "Take me away. if I did that, I belong in Arkham because I'm really crazy."
Similarly, the fact that the Penguin is perfectly sane may have contributed to his mutation into a gray market white-collar criminal who Batman is grudgingly willing to tolerate as a source of information on the criminal underworld.
The Baffler is a second-rate version of Cluemaster, which makes him a third-rate Riddler.
The Arkham Asylum: Living Hell miniseries introduced several such villains, mixed with Arkham regulars, such as the Junkyard Dog who goes through garbage. Seriously, that's his gimmick. Another included Doodlebug, who paints (although he's a definite example of Not-So-Harmless Villain).
Humpty-Dumpty, who is so delightfully inoffensive that even calling him a villain is a big stretch. Even when one learns that there's a good reason that he's in Arkham, one kinda feels sorry for him; he has an obsession with fixing things by taking them apart and putting them back together again, because his whole life has been a string of disasters, one after another. Unfortunately, his attempts to fix things only make them run worse. His attempts to fix stuff like a subway train, an elevator, and a clock tower have lead to people getting hurt or even killed. And, of course, he murdered his abusive grandmother when he tried to take her apart and put her together again. Despite that, the guy probably saved Batgirl's life when she was trying to apprehend him but injured herself by dislocating her shoulders in a fall (one case where he was able to put something together perfectly).
Condiment King, an absurd parody of gimmick villains, is this trope with a lampshade. Just dangerous enough to be worthy of Batman and Robin's attention, he has at least the potential to be a real threat (think "mustard gas", for just one example). However, in practice, he repeatedly gets defeated in a single page. Because he's an Ineffectual Sympathetic Villain, he keeps getting parole.
Harley Quinn, there are time that you can feel sorry for her, she has spent most of her life chasing The Joker who not only abuses her, but considers her an expendable human shield.
The Marvel Universe's Toad is a classic example of this. He has second-rate powers, a stupid nickname, and an even stupider real name (Mortimer Toynbee). Understandably, he hated himself. However, the first X-Men live action film, with the character played by Ray Park, changed him into a wry villain with more self-respect and redefined powers that are actually scary in their deadliness — managing to become his ownCanon Immigrant (in particular, his Ultimate Marvel incarnation is much more badass than his first one).
He almost revels in his second-rate status, remarking on one occasion that at least it keeps him off the radar of guys like The Punisher.
Not that this makes him particularly successful:
Captain America: What the hell do these kids [the Young Avengers] think they're doing? Spider-Man: Making the Shocker look like an idiot. Which—granted—isn't tough, but is always entertaining.
In one story, a group of heroes are moving through a superhuman prison, vigorously expositing about the potential damage that the villain they're after could cause. In the process, they run by The Shocker's cell.
Wolverine, smirking: "Yeah, we wouldn't want the Shocker to get out - then we'd really be in trouble." The Shocker, arms crossed petulantly: "Shut up!"
He's fallen to the point that he no longer appears on the local news' supervillain alerts even though Stilt-Man did. Desperate, he teams up with a similarly washed-up Hydro-Man to knock over ONE bank and retire. You feel pretty bad for him when Spider-Man not only stops them, but the Shocker accidentally evaporates Hydro-Man and injures himself to the point that his ribs are sticking out of his chest. "You always said I looked like a pincushion..."
But contrary to his reputation, the Shocker actually has a fairly high success rate against Spidey. He once proved himself to be a Not-So-Harmless Villain when he captured Spidey and, in a fit of rage, delivered a No-Holds-Barred Beatdown that nearly killed him. Even The Hood mentioned that he had great respect for the guy.
Another time he teamed up with fellow loser the Trapster (see below), had Spidey at their mercy and only didn't kill him due to suddenly getting a call from their boss informing them that their pay would be doubled if Spidey lived. Ever the pragmatist, Shocker accepted though he remarked that if he killed him, he would "save a fortune on therapy bills".
Since moving to the cast of The Superior Foes of Spider-Man, his status as a massive joke has been played up greatly, though he's more unlucky than incompetent.
Hawkeye once had an enemy (the term should be used loosely) named Oddball who could juggle. He could juggle really well. (Sure, he juggles spheres that contained stuff like tear gas and liquid adhesive that he used like throwing weapons, but it was still pretty lame.) Exactly why this street performer decided to become a criminal was anyone's guess (although he did have a name that may have gotten him beaten up a lot when he was a kid, Elton Healey). To make this worse, he actually formed a team of other villains who used juggling as their MO (seriously, he did) called the Death Throws, each of which specialized in a different type of juggling: Tenpin (juggled clubs), Ringleader (juggled rings), Bombshell (also juggled spheres, but preferred ones that exploded), and Knickknack (who could juggle objects of dissimilar sizes and weights, a difficult trick if you're a performer, yes, but as a villain, still lame). Worst of all, Oddball was eventually killed taking part in the Bloodsport competition in Madripoor (which featured far more competent folks like Wolverine, Mister X and the Taskmaster), in the first round and with a One-Hit Kill, no less, but another guy became the new Oddball. (And for some strange reason, he and the other Death Throws convinced Norman Osborn to hire them during the Dark Reign storyline.)
A LOT of villains that Spider-Woman fought during her solo series fall into this category. There was Hangman (who had a noose as a weapon and little else), Gypsy Moth (who could destroy cloth, and not much else), and Daddy Longlegs (who was very, very tall) but by far the worst was Turner D. Century, a villain who wanted to return society to the cultural and social values that it had before World War I. (In other words, he was a bigot and a chauvinist.) He had no super powers to back this plan up; exactly why it took Spider-Woman a whole issue to apprehend him is a mystery.
Daredevil villain Stilt-Man. A man whose suit of Powered Armor offers some minimal amount of protection while making him very tall. One of the more baffling villains of his era, writers gave up on revamping him into a serious threat a long time ago. Since then, whenever you needed a really pathetic villain to beat up, Stilt-Man was your guy. Eventually, The Punisher killed him. For all that, his wife, Princess Python, was pretty hot, so perhaps Stilt-Man was effective in other areas.
While Stilt-Man may be dead, his legacy lives on in...Lady Stilt-Man! Her first appearance consisted of being mocked by Spider-Man (who thanked her for improving the miserable day he was having), and being defeated by stepping into an open manhole. Even Spider-Man felt sorry for her when she started crying. This change in her next appearance in "Villains for Hire", where she upgraded her armor and Took a Level in Badass.
There was a two issue story about him in a Spider-Man spinoff focusing on the various other characters in Spidey's life, with his idiocy being what makes him so pathetic. However, he ends up becoming super-intelligent via super science and ends up getting the girl and becoming the strongest crime boss in New York, along with figuring out Spider-Man's true identity. He goes back to being dumb, however, when he ends up being miserable by not being able to connect with people anymore.
The Rhino did however apparently gain some notoriety in what may have been his final appearance, in the "Ends of Earth" storyline, Where he held onto Silver Sable so that she would drown with him in a flooding shaft. He may have failed, however, as the psychic Madame Webb has since told Spider-Man that Silver is not dead. (Which could possibly mean that the Rhino survived as well.)
Slyde. A villain who claims that his parents were gunned down by the Incredible Hulk and Captain America, and whose primary mode of attack is the "Slyde Punch", which is just a jab to the ribs. He gets taken down and hauled off to jail with incredible speed. As it happens, he's just a guy going through a midlife crisis who decided to go toe-to-toe with Spider-Man instead of just buying a Corvette or something.
One origin story had him as Jalome Beacher, a chemical engineer at a company later to be revealed a mob front. His main achievement? Making a non-stick chemical that can be applied to virtually anything. When he got fired, Slyde coated a white speed-skater's bodysuit in it. According to That Other Wiki, he could glide at about 30 miles/hr, and the coating made him unable to be directly webbed by Spidey. Pads in his gloves let him hold onto objects so they wouldn't slip out, and his maneuverability was much better than most. His brother Matt, though, was killed by Elektra, and Jalome himself would be killed by Underworld, the nigh-invulnerable hitman of Hammerhead.
Currently, the Slyde outfit is blue and this Slyde is an undercover police officer.
The Kangaroo is another Spider-Man villain that was a big joke. Originally, a boxer named Frank Oliver gained some impressive leaping abilities by studying kangaroos in Australia, but he couldn't cut it as a crook with such skills. Then he gained super-powered leaping abilities from Dr. Jonas Harrow, but even that didn't help him much. Harrow then used a pain-inducing implant to force him to steal a radioactive isotope... Which reduced him to a pile of ashes when he tried. (Harrow was pretty much an Ineffectual Sympathetic Villain himself.) But this got worse. Someone actually admired this guy, and years later, became a new Kangaroo. This one may have fit the Harmless Villain trope more if he weren't so serious about it. After many failed attempts at crime, he seemed to be killed during the Ends of Earth storyline by the villainess Lady Deathstrike.
Without a doubt, the White Rabbit, who Spider-Man has also fought on occasion (well, okay, three times) was one of the worst. According to what she herself says, she was a bored young woman who married an aging millionaire (roughly translated, she was a Spoiled Brat and a trophy wife) who inherited his fortune after he died. (She claims that she "cleverly" killed him and he "died happy", but her stories really aren't very reliable.) Why did she turn to crime? Because she was bored. Seriously, the Lewis Carroll theme worked for Batman's enemy the Mad Hatter, but not for this woman, and the only reason she can get Mooks to work for her is because she can pay them more than they could working for other crooks. The only reason she isn't a Harmless Villain is, as Spider-Man says, she's very likely capable of murder, and could very well kill someone (or maybe herself) in her insane desire to have fun.
In fact, the White Rabbit was such a joke, she was beaten up once by the Grizzly and the Gibbon, two other Ineffectual Sympathetic Villain that Spidey had honestly believed were no threat!
The White Rabbit actually once teamed up with another guy like this called the Walrus (the name was inspired by The Beatles song, apparently). He claimed to have "the proportionate speed, strength, and agility of a walrus". (A walrus is bigger than a man, so that would mean he'd be slower, weaker, and clumsier than his namesake.) Ironically, he does have Super Strength and is incredibly tough and durable (almost Immune to Bullets) and he might make a somewhat competent crook if he just tried harder.
She did manage to become a Not-So-Harmless Villain once, but managed it by dumb luck. While dating the assassin Arcade, the two tried to capture the Black Cat and Wolverine for a Murderworld-style hit. The plan failed and ended with the two villains deposited in the Savage Land, and she stole a goat's head from a shrine, which let to her being Captured by Cannibals. Arcade managed to flee, leaving her to her fate (calling her stupid as he did), but this is where the dumb luck comes in: the goat's head was a sacred relic, and the savage warriors started worshiping her. It was very easy after that for her to gain revenge on Arcade.
Sadly, this success didn't last. Her next appearance was as a member of the Hood's Gang, and who defeated her? Spider-Man's former wife, Mary Jane, who does so defending her boyfriend at a night club.
Pretty much every victim of Scourge (or would-be victim who escaped) has fit the Trope, and a few of them were guys who quit being villains because they knew they were no good at it. Scourge rarely goes after a major player. They've attempted to target a few respectable villains, including the Kraven the Hunter, Hobgoblin, Puppet Master, Cobra, and Diamondback, but failed each time for one reason or another. (Of course, Mark Gruenwald claimed that the whole reason Scourge was designed, originally, was to dispose of villains deemed to be too minor, redundant, or ill-conceived, and he's regretted this since then.)
In fact, those failed attempts and the fact they only target losers most of the time may be proof that the Scourges themselves fit the Trope. In the case of the Hobgoblin the Scourge who marked him had the wrong guy; Flash Thompson had been framed by the real Hobgoblin, whose identity was still a mystery. Even worse, he went after poor Flash when he was in prison, and couldn't defend himself. The Scourge who marked Cobra and Diamondback tried to do so by shooting the fuel tank of a car they were driving; He missed. Of course, aiming at a moving target at long rang might have been too difficult for the Scourges, who usually strike by ambush with a sawed-off shotgun or sub-machine gun using a disguise to get close to a victim; if faced with actual resistance, (say, by someone like the USAgent in his mini-series), any of them can be floored with a single punch.
Though the rat creatures in the comic Bone are quite fearsome in force, the nameless two most commonly seen around the valley where the protagonists live are pretty pathetic on their own. They want to eat the story's protagonist, but they do themselves more harm than anyone else with their bumbling. Their constant bickering over whether to bake the Bones into a quiche is also quite endearing.
A trilogy of junior novel sequels even have them as two of the chosen heroes. Well, more like Token Evil Teammates, really.
Marvel's Porcupine was a rare example of a Heel-Face Turn from this type, although it went wrong. The villain initially created his battlesuit to sell to the military, but for some reason, they weren't interested (perhaps it was because it looked goofy). He became a particularly pathetic supervillain, to the point that when he tried to sell the battlesuit to other villains, they also turned him down. He then turned to the Avengers for assistance, only to be killed by his own costume when he decided to do the right thing and double-cross the Serpent Squad.
The DCU villain Dr. Light started out as a formidable foe capable of taking on the Justice League single-handed, but was a victim of severe Villain Decay in the Bronze Age and Post-Crisis eras, mostly notable for being repeatedly defeated by kids. And while defeat at the hands of the Teen Titans isn't all that shameful, he was also humiliated by Little Boy Blue and the Blue Boys, a team of non-powered pre-teens! That all changed with his rape-the-wife moment in Identity Crisis.
The entire Injustice League, which consisted of Major Disaster, Cluemaster, Clock King, Big Sir, Multi-Man, and Mighty Bruce. Individually, they were talented in some area, if lacking in others. As a group...they're still a bunch of losers. Here's how bad their luck is — while staying in Europe, they happened to attend the same French as a Second Language class as the Justice League. And this was following a bank robbery that was thwarted by the fact that none of them could effectively communicate the idea of "This is a stickup" in French. They ended up pulling a Heel-Face Turn and joined the Justice League as their Antarctica branch.
Bolphunga the Unrelenting, from Green Lantern. A Large Ham villain, notable for using an axe against power-ring wielding space cops, and for attempting to take on Mogo.
Turk, the pettiest of the petty hoods in Harlem, in the Daredevil comics.
The All-New Orb from Ghost Rider is a man with a giant eyeball for a head and a repulsor ray gun. Captain America describes the problem as "nobody takes him seriously enough to put him in an actual cell." He's not quite a Not-So-Harmless Villain since he's yet to prove a proper threat to any superhero thus far, but he's more dangerous than he looks.
In fact, in one story where the Trapster was actually rather competent, he still couldn't win. After beating a couple of crooks senseless who had double-crossed him, he left them trussed up in his paste to let everyone know he had done it. Unfortunately, when the police found the two crooks, they mistook the paste for Spider-Man's webbing, and assumed that the hero had caught them and left them for them to find. (An honest mistake, actually, since Spidey tended to do that a lot, but the Trapster was really upset.)
Astro City has Glue-Gun, an obvious Expy of Paste-Pot Pete. His only major appearance to date showed him invading a superheroes' dinner club, only to be taken out by the busboy he was holding hostage.
Iznogoud the Infamous, the ever-scheming but hapless Grand Vizier to the Caliph, who merely wants "to become Caliph instead of the Caliph."
Captain America villain Batroc the Leaper is one of the most skilled fighters in the Marvel Universe, yet he almost always loses and never gets any respect. Thankfully, the good captain actually seems to like him.
Taken to another level in an issue of Marvel Adventures Avengers, in which Captain America's old enemy tries to reform and ends up inadvertently roping the Avengers into a somewhat amoral scheme to promote an internet dating business.
Batroc is an interesting case, as he's only 'ineffective' when he's fighting Steve one-on-one. When working for someone like Zemo, or fighting other heroes, he can be scarily effective. See his effortless beat down of the super-strong mercenary Paladin and his clashes with Bucky Barnes.
In the above Marvel Adventures example the Avengers all attack Batroc at once and he effortlessly dodges all of their attacks. Captain America is the only one who is actually able to land any punches on him.
It was also noted in a comic about one of Marvel's superprisons that since Batroc's abilities all come from a lifetime of training, people like him are the most dangerous in supervillain prisons, as most of the villains either have their powers sealed or their tech taken away. Another name dropped in that vein is the Kangaroo, of all people.
It's actually shown in a one-shot that Batroc himself knows he's unlikely to ever defeat Cap one-on-one, but the challenge of it is too tough to resist.
Even Batman himself commends Batroc on his speed and skills after defeating him (off-panel) in Volume 4 of JLA-Avengers.
Rainbow Raider in The Flash became this, once going so far as to attend a villainy motivational seminar in a futile effort to stop losing all the time. Neron once sent him an invitation to his upgrades-for-souls meeting just so the Trickster could steal it from him.
Very often all the Flash's Rogues fall in this category.
Larfleeze is implied to be capable of taking on the collected Guardians of the Universe, has natural abilities allowing him to stand his own against entire armies, and has had a billion-year-old enemy create an entire artificial star system as part of a plan to defeat him. He's also so short-sighted and gluttonous that he spent most of recorded history squatting and eating in a tiny corner of a planet that evidently didn't know he was there.
In fact Spider-Man has enough of these that they've several times teamed up as the "League of Losers". Ironically, even though they're so ineffectual that they actually call themselves this name, they generally manage to give Spidey a run for his money every time, to the point that he feels a little bad for himself that he actually took a few punches from guys like The Spot.
The Spot played a role in Mark Waid's Daredevil, in which it seemed like he'd managed to Take a Level in Badass. Nope, it was someone else with the same powers, but who was much more ruthless and inventive in using them. Also he was using the Spot as a power source and DD had to rescue him.
Archie Comics' Sonic the Hedgehog had Evil Sonic/Anti-Sonic. Before his transformation into the more fearsome Scourge, he was nothing more than a Sonic who wore leather and was supposedly badass-type evil. But, with each appearance, he kept getting worse and worse with his lowest point is being karate-chopped by Antoine completely by accident.