Broken Base: There are three distinctive camps within the fandom: people who consider II the peak and sneer at other instalments, people who play I-IV and sneer at further instalments and people who play I-IV and VI-VIIand sneer onV.
Cult Classic: The Settlers II, in particular its idiosyncratic infrastructure (segmented roads with each carrier relaying wares along their segment), attracted such a cult that it was remade in 3D on its tenth anniversary and also inspired the open-source game Widelands. Ironically, this iconic mechanism was likely influenced by early technical constraints, as it required far less pathfinding work for the computer compared to carriers walking around freely.
Pretty much everyone pretends Heritage of Kings never happened, as it's so different from the rest of the series it can easily qualify for In Name Only instalment. Rather than being, well, Settlers, it's a pretty lackluster Age of Empires clone with unexpectedly strong emphasis on plot and combat.
The older section of the fandom disowns anything after IV, since the gameplay was heavily changed for each following game.
And the even older section of the fandom disowns anything after II, sometimes even including the 10th Anniversary remake.
Catapults in The Settlers II didn't break peace agreements, so you could just use them to bomb through most of the opposition base.
The Amazons from III have a war gong as their war engine. Unlike other war engines, not only this one isn't limited to military buildings (so it can destroy civilian ones, too), it doesn't carry ammunition, instead using mana, thus not requiring reloads. And you are going to have dozens of mana points by the time you will deploy your first gong.
It's the Same, Now It Sucks!: Most of the criticism towards The Settlers IV came down to this, due to it essentially being a slightly more polished version of its predecessor.
It's Short, So It Sucks!: And if that wasn't enough, IV was significantly shorter than III, having merely 12 mission in its main campaign. For comparison, III had 8 missions per faction (netting 24 missions in total) and Amazon expansion had 12 missions for both Amazons and Men and an economic campaign. Even the ancient at IV's premiere II still had 10 missions in its campaign and Gold added a world-conquest mode for another 9 maps, making IV look just lackluster and very brief in comparison.
Sequel Difficulty Drop: In many ways III made your life easier when compared with II, but it's debatable how much of it was difficulty drop and how much quality of life improvements. This is especially notable, since changes introduced in IV over III were purely quality of life, ironing few kinks the previous game had.
Since the road system was removed and all the things became physical objects on the map, transportation was streamlined, with entire group of settlers carrying required materials to the destination and no risk of permanently losing them due to border shift or destruction of a road. And due to object permanence, warehouses aren't pre-requested to have functional transport hubs.
Many jobs no longer required a tool to get worker, and the tools themselves, along with jobs, got simplified (bakers don't require rolling pins, butchers use the same axes as lumberjacks, minters and iron smiths became simply smelters etc.)
Soldier production got streamlined, their promotion system unified for specific unit type (rather than specific soldier) and they now move like in typical RTS, rather than sending spares from existing garrisons.
Catapults were turned into mobile war engines, so they never become obsolete and simply need to reload their ammo, rather than being rebuild wholecloth every time the current building has no more enemy military buildings around.
Campaign level 6 in II: The first level where there is no initial peace treaty with your more powerful, aggressive rivals.
The Settlers IV
Mission 6 "All that glitters" of The Settlers IV: The player is faced with two AI opponents who will attack at regular intervals, but the starting area has very few resources. Focusing on Mana production is a somewhat viable strategy, but even that requires more time than the player may have. And to add insult to injury, the next mission will go as far as claiming that the two enemy factions had no plans to attack the player, while the mission itself has the AI make the first moves (outside of some specific strategies a player might employ).
One of the standalone maps, "Cordoba", may very well be the toughest mission the game has to offer. Even on easy mode, it easily contends with the aforementioned "All that glitters". The player is tasked with withstanding a siege from three sides for 90 minutes. Unfortunately, the AI players start with sizeable and well trained armies, so the only reason they don't crush the player straight away is because the script tells them to wait a little. The timing is not very generous, however. Beating this map on any difficulty requires both luck and the deployment of various cheese tactics.
They Changed It, Now It Sucks!: Some fans of the first two games do not consider everything that came after them true Settlers games and cite the removal of road system to be the ultimate testimony to the decline of the series.
