As a work becomes more popular, the probability of such a group appearing approaches one. Today's media landscape seems to encourage this somewhat, because In a World… where executives and producers seem to believe in Quality by Popular Vote, and because Ratings are so important in determining what keeps getting made and what gets canceled, it's easy for fans of one thing that might be on the verge of getting canned to resent the people who are making another work so popular (and thus keeping it on the air), as they blame those fans for their own favorite work's misfortune.
But others seem to just want to rail against fans simply because they think it makes them superior. This group is comparable to Moral Guardians, only it's their own subjective tastes they are trying to guard. They really love to make up fake stories about how fans are Complaining About People Not Liking the Show. These kinds of people are a big reason for some people's mentality of He Panned It, Now He Sucks!: Fan Haters are always waiting for an opportunity to pounce, and even a single negative critical opinion of something is usually enough to "justify" their "this thing is bad and you should feel bad for liking it" rants. It also contributes to Complaining About People Not Liking the Show: because there are quite a few Fan Haters out there, some fans mistake an honest criticism of a work for Fan Hating and get very defensive.
It is a very odd phenomenon, because after all, different people enjoy different things. And maybe they just enjoy it for one reason or another, not because of their sexual orientation, not because of their intelligence, not because they're old or inexperienced, they just happen to like it. But chances are if you like something that's subject to fan haters, expect one of those comments to be thrown at you. It's incredibly rare (at least, on the internet) to see someone who actually acknowledges that maybe you like different things not because you're mentally ill or homosexual or "haven't seen a 'true' something".
Of course actually being familiar with the works people aren't supposed to like is entirely optional.
Keep in mind that it's also not required to hate the work in order to hate the fans. It's quite possible to hold indifference to or even like a work, and still resent the fandom. Usually this occurs with fandoms that are known for being quite rabid or split down the middle, which can understandably grate on more level headed fans or non-fans. In the case of non-fans, frustration with the fans is a common cause for Hype Aversion and, when the work in question doesn't live up to the hype it was given, Hype Backlash. More often than not, however, Fan Haters will hate both.
Contrast the inverse, Complaining About People Not Liking the Show, where fans of a show will lambaste you for not liking it. This activity has also been known to spawn Fan Haters, though in that case, the hate starts with the fans and spreads to the work. Also, when a fan likes the show, but is a Fan Hater toward a certain character on the show, it's a specific type of Fan Dumb known as The Hate Monger.
