These are what we call the 'YMMV items.' Things that some people find in this work. We call them 'your mileage might vary' because not everyone sees these things in the same way. This starts discussions in the trope lists, a thing we don't want. Please use the discussion page if you'd like to discuss any of these items.
"Istanbul (Not Constantinople)" was originally a Four Lads song, but most people these days only know the TMBG version.
"Dog on Fire", the instrumental theme song for The Daily Show was originally written and performed by Bob Mould. However, once Jon Stewart became the host a few years into the show's existence, Mould's version was replaced by one performed by They Might Be Giants, which has remained ever since.
"New York City" is a cover of a song by the all-female twee band Cub, whom the Johns were friends with. The original appears on Cub's 1994 album Come Out, Come Out, which had only been released two years prior to TMBG's cover of the song, which helped the fact that more people remembered the cover rather than the original.
"Why Does the Sun Shine? (The Sun Is a Mass of Incandescent Gas)" is a cover of a 1959 song by Tom Glazer. The song was woefully obscure when TMBG covered it and actually features a handful of now-inaccurate facts (notably that the sun is actually made of plasma, not gas). TMBG recorded "Why Does The Sun Really Shine?" a factually accurate new version of the song for their 2009 childrens' album Here Comes Science.
"What Is a Shooting Star?", written by Lou Singer and Hy Zaret (who also wrote the aformentioned "Why Does the Sun Shine").
Epic Riff: "The Lady and the Tiger", in full effect.
"Turtle Songs Of North America," a free download from one of their old sites, consists entirely of weird noises (the "turtle songs") and John Linnell speaking in a creepily calm southern accent about several (made up) North American turtles. There's something a bit unsettling about the all turtles covered. The "Eastern Fighting Turtle" who will attack without provocation from up to five miles away. Then there's the "Tudlow" who's song is both a mating call, and a cry for help. The "Downy Tortuga," makes a noise seems to be repeating the word "unclean." The "Zombie Turtle" which has a hypnotic song that affects humans. And the "Mudflail" who's shell is in a constant state of decomposition.
"Hide Away Folk Family", "Where Your Eyes Don't Go", "Someone Keeps Moving My Chair", "It's Not My Birthday", "Hall Of Heads", "A Self Called Nowhere", "The Bells Are Ringing", "Rat Patrol", "Older", "Ant", "Bastard Wants To Hit Me", "I'm Impressed"...
In case you're wondering, that's a song off of each (non-children's) studio album (and at least one compilation). Yeah, it's one of the band's favorite tropes. And at least half of those have serious Lyrical Dissonance. "Someone Keeps Moving My Chair" is downright upbeat.
Sequel Displacement: By far, even to this day, the most well-known album the band has put out is its third, Flood. This is the album that has "Istanbul (Not Constantinople)", "Particle Man", and "Birdhouse In Your Soul". Since it's the most common gateway into fandom of the band, there's even a Fan Community Nickname for those that bought Flood as their first TMBG album: Floodies.
They Changed It, Now It Sucks: In their early days, the Johns worked by themselves and used double-tracking and drum machines to complete the tracks, and they'd play to recordings live. When they finally decided they needed a backup band, fans were pissed. They got over it eventually, though, and now it's hard for most of them to imagine what it would have been like if that had never happened. Their first band album, John Henry, has a controversial status among fans.