The TVTropes Trope Finder is where you can come to ask questions like "Do we have this one?" and "What's the trope about...?" Trying to rediscover a long lost show or other medium but need a little help? Head to Media Finder and try your luck there. Want to propose a new trope? You should be over at You Know, That Thing Where.
Quirky Town would cover the locals being used to weird happenings within their town.
I'm not aware of any tropes about a character not wanting to go to a location if they can help it. No place is ever going to agree with everyone. I'm sure many people love New York, but many others might be stressed out by the population density.
If it's something like Eldritch Location or Hyperspace Is a Scary Place, for someone to say "I don't like going to eldritch locations" or "Hyperspace scares me" would just be redundant.
A locations so weird that people with magic or powers don't even want to go there if they can help it. The locals have no problem with it though as they're all ready use to the weird happenings around them.