It seems to me that in the modern usage, "chivalry" specifically only applies to men's behaviour towards women, where "courtesy" has no such restriction. But then again, I am not a native English speaker, so I might be completely mistaken here.
To make a random example, it's common courtesy that if one enters a door and someone else is also about to enter, you hold the door for them after passing. Nothing wrong about that.
However, it is "chivalrous" for a man to make a point to keep the door open for the woman and let her pass first
, even if the man and the woman reach the door at the same time; and there are some men who would consider it emasculating if the woman instead (horror!) held the door for them. This is a bit silly, and I can see why a woman could feel offended by the implication.
Another example: at least over here, it's perfectly normal for friends to pay for each other's drinks and so on. It's simpler than splitting the bill, and well, since if you offer then the next time someone else will, it's not like anyone is always paying. I find that this is a good system. However, if a guy made a point of always
paying for women, even ignoring them when they say that they'd rather pay themselves or split the bill, well, that's creepy and inappropriate. Sadly, it's also the kind of behaviour that some think about when they talk about "chivalry".