Animorphs doesn't start off this way, but it eventually hits this, being a series that's filled with shade of grey, with both Jake and Rachel committing horrific acts. The only characters that end up getting a Happy Ending are Marco and Cassie, with Rachel ending up being dead, Jake having to live with the knowledge that he murderered thousands of defenseless Yeerks, Tobias having to live without Rachel, and Ax being consumed and absorbed prior to the Bolivian Army Ending.
The Bas-Lag Cycle by China Miéville may induce this. The first book's Downer Ending may be tolerable because the main characters are, at least, somewhat decent people and thus inspire some pathos, but by the end of the series the repeated evils Inherent in the System never improve or get foiled. The fact that the only people who are ever seen suffering are the protagonists, and thus the stakes becoming 'all this nameless, faceless evil and injustice is allowed to continue to oppress what few decent people we do meet' may make you wish Spiral Jacobs succeeded in wiping the whole thing off the map in the last book.
Blood And Chocolate suffers from this twice over. On a larger scale, Humans Are the Real Monsters who hate and fear werewolves, while werewolves sometimes provide very good reason why humans hate and fear them. On a smaller scale, any of seven or eight different characters, including the main character and both love interests, could be argued to be the most repulsive character in the book for one reason or another, and of the two characters who are most likeable one's a Straw Feminist who's not treated very seriously and the other gets eviscerated by a supposed friend. The silver lining is that the werewolf female lead and human primary love interest could balance each other out—except that humans really can't trust werewolves, so he winds up trying to kill her.
Aldous Huxley's Brave New World can induce similar reactions to Ethan Frome or Wuthering Heights. The sheer soul-crushing hopelessness of the story combined with the utter depravity of the Crapsack World it portrays has been known to cause severe bouts of depression in readers. In fact, the novel was chastised by critics for precisely this reason upon its initial publication in 1932, and Huxley himself later regretted not offering John the Savage a way out of his dilemma.
In "The Merchant's Tale" of Chaucer's The Canterbury Tales, the merchant's tale of a wife's adultery is supposed to show women as dishonest, but the poisonous way all characters, including the wronged husband are portrayed makes it difficult to sympathize with any of them. This is almost certainly deliberate, as several of Chaucer's characters let their view color their stories and how they tell them.
The Castle by Franz Kafka is about a character, K., who arrives at a village operated by a bureaucracy which is centered in the titular castle, and his endless attempts to find out exactly what the castle does and how to gain influence there and...it gradually turns into a series of interminable, roundabout discussions about the castle and bureaucracy and what K. has to do in order to get anything done, to the point that Kafka himself was unable to finish writing it, making this a case of Darkness-Induced Creator's Apathy.
The Casual Vacancy has a rather mundane premise and characters that, as one reviewer put it, are "fairly horrible or suicidally miserable or dead." Thus, some found it hard to care about the conclusion.
The Child Thief falls into this at the beginning, and for some readers, until its end. The story is billed as a darker retelling of Peter Pan, and it shows—rooting for Peter is all but impossible given his bloodthirsty and careless ways. The real world is not much better in the book—because it focuses on Lost boys and girls, everyone has horrible stories in their past. Although there are a few characters the reader can identify with, they pale in comparison to the horrible people around them rather than providing a real contrast.
Diary of a Wimpy Kid: While it's a stretch to call it outright darkness, the sheer amount of unlikable characters (for instance, the main character Greg already suffers from an unrealistic number of glaring flaws he hardly notices) and sheer amounts of What an Idiot! from all characters moments can really be a bother when trying to find something to root for and keep going. Despite being sold as realistic fiction, the story is set in a world of incompetence where bad things happen to bad people, although this world is interpreted from Greg's obvious Protagonist-Centered Morality.
Dubliners by James Joyce is a book of short stories where every single one (except, perhaps, the last one) is about a disappointing, half-lived life that will probably end in isolation and ignominy. Hope spots are few and far between, and usually swiftly replaced by misery. The only (arguable) exception is the final story, "The Dead", where the private sadness of the protagonist, Gabriel Conroy, is set against the genuine hospitality and generosity of his aunts Kate and Julia and his cousin Mary Jane, whose Christmas party takes up most of the narrative.note The story wasn't written when Joyce was first trying and failing to get the book published, and after being turned down several times he realized with hindsight that he'd been too ungenerous towards his countrymen. "The Dead" was written to redress the balance, and it's less unrelievedly dark than the other stories in the book, only because it's more ambiguous.
Ethan Frome can induce similar reactions as Wuthering Heights. The utter hopelessness of the story has driven many an English Major to the bottle.
Hannibal Lecter novels Hannibal and Hannibal Rising both suffer from this. Almost none of the main characters have any redeeming attributes (sans Clarice, but she's basically a pawn half the time, and in the novel ends up falling for Lecter)—and those who do are mercilessly picked on or forced out of the action. Meanwhile, Hannibal Lecter himself, the murderous psychopath, is practically presented as the hero. Hannibal himself is part of why Hannibal falls into the trope. Even though he is a psychopathic murderer, the vital heart of Silence was the dynamic between Lecter and Clarice; Lecter comes to admire the doggedness and pure, honest nature of her and treats her as kindly as anyone in a crapsack world where seemingly every male treats her horribly. By turning him back into more or less a standard murderer on the loose and severing that mutual respect, it throws Hannibal into this trope, as discussed in great detail by Roger Ebertin his review:
"It misplaces the reason why we liked Hannibal Lecter so much. He was, in the 1991 classic, a good man to the degree that his nature allowed him to be. He was hard-wired as a cannibal and mass murderer, true, but that was his nature, not his fault, and in his relationship with the heroine, FBI trainee Clarice Starling, he was civil and even kind. He did the best he could."
The Hunger Games has an initial premise that is dark enough (children forced to kill one another on national television), but for the first couple of books they're still plenty engaging, with human kindness even in the midst of brutality, and hopes of rebellion and change. The third book goes into a swan-dive down the Sliding Scale of Idealism vs. Cynicism. Katniss' PTSD and emotional disconnect from the world increases (and she's our narrator, which makes for tough reading) and the factions come to seem more and more alike, both of them horrible. Throw in arbitrary and senseless deaths and an ending that seems built to deny any meaningful closure... well, some fans applaud the "realism" of Mockingjay, and others just found it a slog.
Most readers can pinpoint the time when Mockingjay hits its point of no return of ultimate depression: the death of Prim. Why is that? Katniss participates in all of this hellhole of a tournament and rebellion in the first place because she hopes to save Prim, her little sister and the only person whom she's certain she loves. All the sacrifice she does, all trauma that she has to endure because of the games, are annulled abruptly when "The second round of parachutes goes off". The next chapter, Prim is dead and Katniss (and the readers) permanently shuts off for the rest of the book.
Iron Druid Chronicles can invoke this reaction in some people. The main character is an often jerkish Nominal Hero who only gets involved if he is forced to, who is willing to let lesser crimes fall by the wayside in service to what he considers more significant goals, and who is willing to commit any number of those crimes himself if he deems it necessary. His "allies" include an (if Affable) largely amoral vampire (who sells Atticus down the river as soon as it becomes clear that Atticus isn't backing his corner), a pack of werewolves (themselves guilty of a number of more mundane crimes, typically in service of keeping up the Masquerade), and a coven of witches who are of a similar mindset to Atticus, if not worse. Their enemies tend to be Jerk Ass Gods at best, if not outright Omnicidal Maniacs.
This was something Steven Spielberg took great measures to avoid when adapting Jaws, because the original book suffered from this: the protagonists in the novel are so unlikable, selfish, and hypocritical that they make the shark look decent by comparison.
The Kid, the sequel to Push, got hit with this hard. The first book was very dark too, but it at least had some moments of hope and a (somewhat) happy ending. The Kidstarts with Precious dying of AIDS and goes on to focus on her son Abdul, who himself is abused, beaten, and raped to the point that he becomes an abuser himself, murders his girlfriend's parents, and winds up in a mental institution. Suffice to say that many readers who enjoyed Push and rooted for Precious found it very hard to slog through The Kid.
Like Water for Chocolate piles so many disgraces on the protagonist Tita, makes her Love Interest Pedro so spineless and selfish, and reduces the other characters (specially Tita's big sister/Pedro´s wife Rosaura) to such unlikeable assholes... that many readers have given up mid-reading, since it's a chore to read and wonder "what's gonna happen to her now?!"
The Malazan Book of the Fallen doesn't always do this, but it did creep up during Midnight Tides. That novel depicts a war between the Tiste Edur (a nation that keeps humans as slaves and is ruled by an insane emperor who works for an Eldritch Abomination) and Letharas (a brutal, expansionist empire that takes the flaws of capitalism as far as it can without being Played for Laughs). You can't even blame one side for being the ones to initiate the war, since they're both pretty eager for it even before the first blow is struck. Sure, individual characters on both sides of the conflict can be quite sympathetic, but the outcome of the war isn't that suspenseful, 'cause you know you're gonna wind up with a regime of violent, oppressive conquerors either way. The continuation of this plot after the war ends only makes it worse because the victors lack the finesse to control their new subjects, resulting in them slowly being undermined and torn apart by their supposed subjects. Things only improve after a lot of important character deaths when the Tiste Edur give up and leave while Letharas gets a decent ruler.
The Maze Runner series is this to a great degree. While the first book is likeable enough for a young adult fiction, the second takes a sky dive as the protagonist is betrayed by almost all of his friends, who reason that they work For The Greater Good and actually blame him for acting sensible and try to question why is this and that. Then the third book reveals that all the predicament that the experimental subjects have to endure is useless at the very end because there is no cure for the Flare virus, which means that all humanity can do is to let the teenagers start anew in a paradise that comes out of nowhere, which means that the entire trilogy can be averted had the higher powers shed their smugness and actually tell said teenagers what's going on. Adults Are Useless, indeed.
Night Watch is a debatable case. The eponymous Night Watch claims it wants to improve the world, but in practice this means making everyone else think like they do, and their actions include putting Hitler in power. The Day Watch talks of freedom of choice, but they're selfish, hedonistic, and frequently hypocritical. Muggles are snacks, and there's nothing they could conceivably do to influence the situation. On the other hand, it's only the upper ranks that are rotten—both the Night Watch and the Day Watch have good and honorable members in the field. This comes full circle when it is revealed that at the highest levels, the Watches are actually working together; the conflict between them is mostly for the sake of keeping the Others away from normal people, and the Watches often deliberately sabotage their own efforts to overtly influence human society. Their real plans to improve the world are much more subtle and cooperative.
George Orwell's Nineteen Eighty-Four. Winston and Julia are both suspicious, bitter, cowardly and (initially at least) demonstrate few desires beyond basic gratification. They're willing to commit unspeakable atrocities to overthrow one of the most horrific and utterly evil totalitarian governments ever conceived. The story ends with Winston and Julia as lobotomized, dehumanized robots with all their old personalities gone, replaced with soulless nihilism and love of the Party, and they're probably both going to be shot anyway. The Ingsoc government continues to exist to spew propaganda and control, torture and humiliate their own citizens for amusement. Oh, and it's implied that Eurasia and Eastasia are exactly the same as Oceania, so there's nowhere in the world to escape from it. Entirely intentional on Orwell's part, as he volunteered in the Spanish Civil War on the Republican side and quickly learned how authoritarian regimes horribly distort the truth to their own ends, and how often the methods of one's own government can be Not So Different from those of your enemies.note To be precise, when Orwell joined the Republican side, he joined a left-anarchist faction that was genuinely well-intentioned and committed to improving the lives of ordinary Spaniards, but which had the misfortune to be the losers in an intra-factional dispute with the much bigger communist faction, which was essentially controlled by the Soviets and which didn't want any undisciplined anarchists not toeing the line from Moscow.
Donna Tartt's The Secret History sometimes inspires this—it's not uncommon for readers to respond to the revelation that one of these unsympathetic characters will be murdered by thinking "Only one?"
Gillian Flynn's Sharp Objects falls into this at times. Readers are presented with a heavily unlikeable narrator and a whole cast of unlikeable supporting characters. The narrative's overall tone tends towards despair, and in spite of the fact that narrator Camille's story seemingly ends hopefully, with her finally finding a nurturing surrogate family and working towards undoing the damage of her toxic upbringing, it's made very clear that none of the characters will ever truly be happy and that dozens of lives were destroyed beyond repair. Flynn's other books flirt with this, but are generally redeemed with moments of hope, levity, humor, etc.
Robin Wasserman's novel Skinned has a similar problem to Blood and Chocolate, since it initially discusses Fantastic Racism against cyborgs, then applies Cybernetics Eat Your Soul. Do you support genocide, or do you root for inhuman freaks?
The Star Trek Novel Verse is starting to have this effect after the two part novels Plagues of the Night and Raise the Dawn. The series had already gone through a completely legitimate rough patch with the Borg war of Star Trek: Destiny, only for a new cold war with the Typhon Pact to occur. Which was fine until these two books: which destroyed Deep Space Nine, the Bajoran Wormhole and Killed Off for Real about five really popular characters from the TV shows. The books following have increased the canon character body count, and the Federation is starting to collapse. The story has already written itself into Only the Author Can Save Them Now but the stories like the authors just want everyone dead or completely miserable. It's probably one reason why Star Trek Online cherry picks from the novelverse.
A number of Star Wars Legends series are falling into this trope. Starting with New Jedi Order, the books have become progressively darker and everything just seems to be getting worse. Came to a head in Legacy of the Force, which ended with Jacen dying after being hacked to pieces and left in an incinerator, the galaxy under control of Daala, and the galaxy wrecked by yet another pointless war.
A Series of Unfortunate Events: Adults Are Uselessall the time (and this only gets worse as the series goes on), Poor Communication Kills (and, again, this only gets worse), the Baudelaires do not get anything resembling happiness ever within the books themselves (the last book ends with them taking their chances going into the unknown, but there is a big Inferred Holocaust and whatever clues are of them getting a bright future once everything is done are deeply buried amongst endless misery and the Lemony Narrator's ramblings). A similar feeling may come of the enormous barrage of mysteries the series has-the author deliberately invokesThe Chris Carter Effect at its absolute worst (refusing to solve almost all of them), which may drive people into frustration.
A Song of Ice and Fire: Though the series has its share of likable characters, it's also full of death. While initially the death of the "main" character was a refreshing twist, by the end of A Storm Of Swords so many of the characters had been killed off or worse that some readers found it hard to care about the rest of them. Also all the endless, gratuitous war crimes perpetrated by all factions—rapes, skinned children crucified for miles, burning women alive after raping them—not only began to lose their shock value, but made it hard to care about who wins in the end. Some are even convinced that the Big Bad of the series, the White Walkers, are practically saints by comparison, making it difficult to consider them a dire threat. Of course, this is still averted occasionally, since many of the people who end up dying are very loathsome villains (a giant rapist and war criminal spending weeks dying due to a man he killed having poisoned him, a psychotic mercenary who loves maiming being gradually cut to pieces, a sadistic Royal Brat being poisoned and choking to death...). It's frequently shown that just being evil isn't helpful in the long-term. Additionally, and perhaps most importantly, it appears that the Northern Houses are planning to restore the Starks, the de facto good guys, and kill the Boltons, their arch enemies. House Manderly even bakes Bolton allies in pies and state that 'the North remembers' the hideous crimes committed on their kin, allies and the Starks. It's enough to make many fans cry tears of happiness. Simply, hope has returned, which averts this trope a bit.
Deliberately used in The Sound and the Fury: the first brother is severely mentally handicapped, and although hard to hate, his section of the book (written from his perspective) is so confusing that it's hard for many people to identify with him. The second brother is completely insane and lusts after his own sisterwhich turns out to be a lie to protect her, but we don't find this out until later, so you really want to turn away from him. The third brother already has turned away from him, and narrates in a completely comprehensible style, so at first you like him, but then he's a total Jerk Ass who hates everyone around him.
Brought up in Through the Looking-Glass. Tweedledum and Tweedledee recite the poem about The Walrus and the Carpenter. After hearing the poem, Alice wonders about who is the more sympathetic of the two. But the twins points it out that both of the duo ate as many of the oysters as they could. After much thought, Alice concludes that both of them were very unpleasant characters.
Many of the critics and even some of the fans claim that the series suffers this badly in the Love Triangle the third book Eclipse is built around. Edward, Bella, and Jacob all come off as extremely possessive, selfish and emotionally manipulative Jerkasses to the point that some found all possible resolutions to the triangle equally repugnant.
There's also the issue of the Volturi. They are built up as a corrupt government who ruthlessly murder vampires over any slight, forcibly "collect" powerful vampires, and look for excuses to destroy the Cullen family. The problem is, vampires as a whole are a bunch of murderous animals who do nothing but think about their next meal and who to kill to get it. Thus, not only are the Volturi the only ones trying to instill any sense of order into the vampire world, but they're the few actually keeping the vampires from slaughtering humans without restraint. Even the Cullens in no way condemn or try to stop this behavior, which makes their declarations that human lives are valuable come across as rather hollow. Meanwhile, in spite of the fact that they're supposed to be defending humanity, the werewolves apparently slack off a good bit (they do nothing to stop Peter and Charlotte in Midnight Sun, even though there were werewolves transforming at that time), and actively go against their duties when Jacob imprints on Renesmee and puts her before his job as Alpha. This is all best summed up by their actions in Breaking Dawn. The Volturi try to use Renesmee as an excuse to kill the Cullens and their allies, the allies are so kind as to go outside the city limits to slaughter people during their stay, the werewolves do nothing to stop this because the allies are needed to protect Renesmee, and the Cullens loan cars so the allies can find victims more easily.
One problem Twilight has is that all of the characters that are jerks are pushed to the front of the story, often having incredibly fucked up backstories despite being portrayed rather positively (Jasper, Sam), while the ones that aren't are either pushed to the background as fast as possible (Bella's school friends), portrayed negatively despite not really showing any negative traits (Bella's school friends again, Charlie in the early books, and Leah, although her case is more that her negative traits exist but aren't really her fault so much), or made worse in an apparent attempt to make them less sympathetic (Jacob, Charlie again). About the only character in the book that isn't either a massive jerk or made one, but still stays roughly important in the story is Carlisle. And even his morals and motives are somewhat questionable.
The Witcher: The world is filled with monsters, Fantastic Racism at its absolute worst, and a total lack of any sign of change that it makes it hard to get invested in what happens. Really, who do you want to root for? A bunch of back-stabbing racists? The evil empire conquering the known world? Mages and sorceresses playing their own game of world domination? Cruel elven supremacists? Or maybe an ignorant hunter, who kills everyone in his way? Oh, and you know from the start how meaningless everything is, since you are informed before the title page of the first book about the incoming ice age destined to destroy everything. It kinda makes you wonder what the point is of even having a story.
Wuthering Heights can induce this reaction, in the 'too bleak and angsty and without hope' subcategory. (Jasper Fforde's Thursday Next books parody this with an anger management class for the characters, who are warned that their drama has made the story more angsty and angry as time goes by, and they risk going the same way as Titus Andronicus: "Once a gentle comedy of manners, it's now the daftest, bloodiest tale in all of Shakespeare!")
The works of Bret Easton Ellis can have that effect, as pretty much everyone in them is completely shallow, self-absorbed and stupid. Given that this is deliberate, you probably know what you're in for when you start one of his books. Clay in Less Than Zero may be an in-universe example, as eventually he finds himself passively watching horrible things (like his friends face-raping a drugged 12-year-old girl) while saying he just wanted "to see the worst".
Chuck Palahniuk's works definitely fall under this, which causes some to think of him as a nihilist and a shock writer. As an example, Fight Club: it's the road of a man becoming a nihilistic terrorist (via the development of a Split Personality who embodies everything he wants to be) who wants to destroy all of society, casualties barely be damned, because he's sick of people's consumerism, and who manages to raise a secret army who is literally everywhere, even inside the police. Although in the text/movie's defense, it came out long before 9/11 (which made having a character with this kind of mentality and who somewhat successfully manages to carry it out a gigantic no-no), but the comic-book continuation by Dark Horse Comics shows the narrator as being on the fast track to try to fulfill his "urge" again... whether he wants to or not.
Robert Cormier wrote a lot of books that fit easily into this, and a few books that would have avoided this if the likeable characters hadn't died (such as Kate in After The First Death) or been beaten into submission. The best chance his characters are ever given is that the next life might be better than this one (and his later works deny even that.)
The Mayor of Casterbridge. Everyone is, without exception, demonically evil or flat. This is a common complaint about Thomas Hardy. A contemporary reviewer of his work stated that, "[His work] is depressing because he himself is somewhat depressed" and boy does it ever show. Jude the Obscure and Tess of the d'Urbervilles are both unrelentingly depressing Shoot the Shaggy Dog stories. Hardy was to some extent trying to skewer the Victorian values of the day and make the point that it was impossible for good people to survive in such a system, but as the audience it's difficult to not just stop caring about these characters once it's obvious that any Hope Spot will only lead to another horrible disappointment.
Many Tom Holt books suffer from this. The protagonists are sometimes just as cynical, ruthless, selfish, vapid, cowardly, and/or nasty as the erstwhile antagonists. Valhalla and Little People are particularly memorable in this regard.
With a title like Worst. Person. Ever., you'd expect this trope to rear its ugly head. Raymond Gunt is an unrepentant misogynist who brings so much misfortune onto himself, while everyone else around him is just as corrupt and assholish, but evades karma. The novel's relentless heaping of bad luck onto Ray makes it rather difficult to get through. However, judging by Coupland saying he wanted to "damage the reader" with Worst. Person. Ever., it seems this trope was indeed intended.