These are what we call the 'YMMV items.' Things that some people find in this work. We call them 'your mileage might vary' because not everyone sees these things in the same way. This starts discussions in the trope lists, a thing we don't want. Please use the discussion page if you'd like to discuss any of these items.
Ensemble Dark Horse: Mary actually became one for some fans after she left the show, mostly because anyone that could incite that much annoyance and whining from the Camdens had to be all right. Especially prominent once Annie stopped bothering to hide that Mary was The Unfavorite, since by then Annie had become a Jerkass.
Fridge Logic: In the episode X-mas two young women are afraid that they don't know enough about the religious aspects of Christmas, and that the Camdens will realize this. Anybody who grew up in the United States could tell you that it is practically impossible to avoid the story of Christmas growing up, especially around Christmas time. Made even more ridiculous when one of the girls doesn't even know what a nativity scene is, and assumes that the people in it are begging for money.
Moral Event Horizon: Many fans thought Annie crossed it when she threw the twenty-something Matt and Lucy and the then underage Simon and Ruthie out of the house and forced them to live in the unfinished (read: no furniture or plumbing) garage apartment until they agreed with her opinion that Mary coming home was a good thing. Ironically, Mary eventually became The Scrappy and general "disappointment" of the Camden family in later seasons and Annie would pretty much cringe if you even mentioned her name.
Pandering to the Base: What some have assumed the show creators to be doing, essentially crafting a show to appeal to a demographic that most other shows have nothing but venom for. Things like the above-mentioned point in Fridge Logic don't make much sense... unless they're less trying to reflect what they see as reality and instead prey on the fears of the religious (that "Christ has been taken out of Christmas"). Considering the sheer amount of Narm, you could almost interpret the show as a Stealth Parody or Take That that kept itself low-key enough to be loved by the people it was actually mocking. It wouldn't be the first time a character meant to mock conservatives ended up being popular with them. See The Simpsons, Family Ties, and All in the Family.
The Scrappy: Kevin with the fans, Mary with the writers. Simon's girlfriend Cecelia (Ashlee Simpson) wasn't too well-received among fans either.
Many viewers dislike Lucy, seeing her as very bland, self centered, and whiny.
Fridge Brilliance (of a sort) kicks in when you realize that they were trying to portray Lucy as growing up to be just like her mother... without quite realizing that a lot of viewers thought her mother was a terrible person.
Ditto for Mary, even Jessica Biel's hotness can't cover up that she's can be a bit of a bitch.
Many dislike Ruthie as well, and see her "cute" antics as bratty.
Any of the non-Camden main cast in later seasons got this because many fans felt that they were unnecessary for a show that's supposed to focus on a family. Ruthie's friend/boyfriend Peter and his mom Paris, Kevin's partner Roxanne, associate pastor Chandler, Martin (though he was later Rescued from the Scrappy Heap when he became a potential Love Interest for Ruthie), Ruthie’s other less liked Love Interest T-Bone, Cecilia’s adopted sister Meredith, Simon's bitchy girlfriend/fiancé Rose and Martin's baby mama Sandy are some prime examples of characters that received a lot of hate from the fandom during their time on the show.
Seasonal Rot: Many fans thought the show started going downhill around the time season 6 started.
The ending of the episode "Lost" has a Crowning Moment of Heartwarming by reuniting a kidnapped child and his family, but shifts into Tear Jerker territory when the usual ending credits are replaced with photos and details of actual missing children.
The whole entire plot of "Nothing Endures But Change," which is made even more heartbreaking when you learn that its plot, concerning Lucy losing a friend in a car accident, is based on the real life death of Beverly Mitchell's (the actress who plays Lucy) friend in a similar accident.
Chandler's breakdown in "Smoking" is also pretty sad, though it may border just a bit too close into narm territory for some.
On the Put on a Bus episode for Mary (Mary was turned into a bit of a troublemaker to facilitate Biel's departure), Simon and Lucy tell her that they care for her very much and want to help her in any way because they look up to her. Then Ruthie's turn comes up, and she basically lays into Mary for being a selfish sister who's let her down and angrily leaves the room. The fact that the What the Hell, Hero? speech came from the youngest, precocious sister was startling and moving (and probably mirrored the writers' feelings as well).