My gut feeling is that it shouldn't count. If there are many black characters, both heroes and villains, that means that SBM with all its Unfortunate Implications isn't really a thing.
I'd say it can happen, but it's a reason to squint at examples and double-check to make sure, because it's often a sign the trope is being misused.
For Scary Black Man to apply, their blackness has to be connected to them being scary (even if implicitly or unintentionally.) This is less likely to happen in a work with mostly black cast, but it could still occur - imagine a story where a highly-successful black professional is confronted by a huge scary Malcolm Xerox who accuses him of selling out and not being "black enough."
That character could still be a Scary Black Man, even in a work with an all-black cast, because their scariness and blackness are connected in a way that draws on those tropes and stereotypes - the scary black man is portrayed as the "wrong sort" of black person, contrasted with the hero being portrayed as the "right sort", which introduces all the unfortunate implications of the trope and then some.
Edited by Aquillion on Aug 31st 2024 at 2:30:40 AM

Does Scary Black Man still apply if the work in question has a mostly black cast? The description doesn't mention whether it does or not. The example that made me think of this is W'Kabi in Black Panther (who has this trope as an example on his character page). That's a film with a majority black cast, so surely W'Kabi is just a scary man who happens to be black, not this trope. I'd have thought this trope would only apply when there seems to be some link between being 'scary' and 'black', due to one of the few black characters being 'scary'.