Up For Grabs. When a cave man is depicted (especially if it's a character being subject to a "de-evolution ray") they will invariably be the Big Guy, having more muscles and stature than their contemporary descendants (along with lots of hair and a sloping brow, of course).

I know only cursory anthropology, but this is Did Not Do The Research. Our ancestors were much smaller than us, partly due to diet and partly due to genetics.

Occasionally, they are only shown having some Super Strength or being savage fighters, which isn't quite so preposterous.