If I could take a crack at analyzing this phenomenon, I think a lot of this problem stems from colonialism (as mentioned in the main entry). But the modern iteration of this is caused by the media's interpretation of beauty, And their eurocentric standard of beauty. Which unfortunately caused a lot of unfair resentment, and friction toward lightskin blacks. It arguably also caused a lot of blacks to exploit this to their benefits to the chagrin of some dark skinned blacks. Whether we like it or not the favoritism is there. As a very fair skinned black woman I have witnessed it first hand.
Practical reasons for the Trope
In modelling and live action it might (at least partly) be that people with fairer skin are easier to photograph and film, without proper lighting, a black person gets reduced to a black human shaped blob with visible eyes and teeth, the facial features kinda disappear. Not to deny there are other factors can be behind this, espescially the more social aspects of the trope, but but the more practical aspects of it deserves a mention as well.
This has more to do with the photographer not knowing what he's doing, or not adjusting the lighting on the camera to photograph someone darker-skinned. When this black troper thinks back on the various picture IDs he has had over the years, the ones taken in cities and schools with a healthy black population tend to turn out much better than in places where he was the only black person for miles (in which case he'd often look like the Cheshire Cat... including his current driver's license).
Definition of "black"
This trope may also be related to the tendency to treat blackness as a "dominant gene" — i.e., people define blackness in such a way that if you have one black grandparent and look like it, then you are black. This results in an expansion of the set of people who qualify as black, and it's also relevant that mixed-race blacks are more likely to come from a middle-class background.
That definition is pretty centric to the US and while it is the third most populated country in the world, only about 12% of it are traditionally considered black. The definition of black can vary by a lot depending on who you ask, as well as any perceptions being any "kind" of black brings.
How do class play a part in colourism/intra-racism?
I have noticed that most people like to bring up their social standing when discussing race. People seem to think all fair skin blacks are educated upper middle class law abiding citizens, and darker blacks are lowly uneducated low class criminal ghetto dwellers. Forget how white people view us, how about how other blacks see us?