The Reason noone cares about Ginko's strange clothes, or pale looks, or glass eye, is that he's a MushishiI mean really, just like you wouldn't expect a wizard to look like a normal person (AT THE LEAST they need a Badass Longcoat...)), Mushishi commune with the strange and invisible and always introduces himself as a Mushishi. If he looked normal, people would feel they weren't getting their money's worth.
The series takes place not in the past, but in a post-apocalyptic world.This would explain the more modern appearance of Ginko's Western clothing. (Were this taking pace in the 19th century, he'd be wearing clothing of a different fashion.) Obviously something happened to cause Japan to abandon technology and regress back to a Feudal age. (This "something" could have very well been a disaster of some sort.) The western-style clothing that Ginko wears could have been scavenged from a disaster site or just found, lying abandoned somewhere. (Ginko wears such clothing as a way to symbolize the strange condition that separates him from the rest of his fellow citizens.)
Midichlorians are actually a form of blood-based Mushi that grant their users super powers.Oh wait. This belongs in the Star Wars WMG section....
Ginko only visits one city in the whole series.Everyone looks the same because it's the same people in every episode. Ginko lost touch with reality after losing his memories and his eye. He wanders in circles and thinks he has come to different villages. In reality, he has entered a large city from multiple directions. The city is cursed, and constantly plagued by Mushi. No one recognizes Ginko because they all have undiagnosed mild amnesia from some unidentified Mushi. Those who don't have amnesia play along with Ginko, such as his doctor friend. No telling what a crazy man would do when his delusions are broken. Especially one who controls invisible magic creatures. Thus, this city has the following features: A mountain range, a dry waterfall, a large beach, a rocky peninsula that appears to be an island when approached from the sea, a bamboo forest, a tree forest, large rice farming grounds, and a river.
Ginko is actually Sanji in another life.He still retains his appearance and smoking habit, but he's older and wiser and, because he no longer roams the sea freely, he takes to land instead.
Ginko is high the whole timeHe seems to smoke fairly persistently, but what he's actually smoking is a fairly mild hallucinogen. The mushi aren't real, and the reason he manages to heal people is because he stumbles upon the actual medical method for curing their problems while believing he's dealing with mushi. Everyone just goes along with it, because he has a reputation as an amazing doctor even though it is all fluke.