Where this idea might come from
While this is a False Dichotomy as there's nothing inherently opposing about being strong and being smart and there are plenty of people in Real Life who are both, there is a logic to the idea that there exists a (mild) trade-off between the two traits. If we consider intelligence solely to mean Book Smart, it's not controversial to note that studying, researching, and getting the vast amount of knowledge to be deemed appropriately Book Smart requires a significant amount of time and energy, time and energy in which one is often sitting very still for long periods of time to read and write. That lack of physical activity, if not supplemented by exercising, can lead to physical weakness over time. Geek Physique is this idea taken to its extreme, since being a Geek means dedicating one's time to learning about something in-depth. Though, it should be noted that you can become a "fitness geek," at which point Geek Physique as an assumed character trait doesn't make much sense. Similarly, getting physically fit to the point of being incredibly strong requires a lot of dedication. The time that one could be using to study, read, and research, one is using to develop their muscles and strengths.Even still, this trope ignores that intelligence comes in many forms, people are capabale of multi-tasking and incorporating studying/training on the side in addition to their main area of focus, and the existence of naturally intelligent/naturally strong and fit individuals, who don't need to sink hundreds of hours into studying/training to be what is considered "intelligent" or "strong". It also ignores the fact that learning the best practices to become physically strong and fit often requires intense study and knowledge in its own right in terms of diet, nutrition, human anatomy and musculature, etc.