On the pinball side of things, 1989's Black Knight 2000 and 1993's The Twilight Zone were known for having one of the best soundtracks ever for a pinball game. The latter game uses Golden Earring's 1982 hit song "Twilight Zone" as its main theme.
Gyruss (in its initial arcade form) only has one recurring song, a remix of Bach's Toccata and Fugue in D minor, but hearing the song in stereo sound on an '80s (!) -era arcade machine will make you say "Whoa!"
For a game about murdering other assassins to jammin'jazz tunes, the ending theme to The Outfoxies, "Enjoy Life" is as chilled and lovely a piece of jazzy piano as you'll ever find.
The scrolling shooter based on the G.I. Joe: A Real American Hero cartoon had an awesome instrumental mix of the 90's DIC theme as its first level BGM, interspersed with "Got to get tough! Yo Joe!" at the same points where it would be shouted during the opening. They could hardly have picked a better song to psych up a player for the rest of the game and all the gratuitous shooting and explosions involved.
The little-known rhythm game Crackin' DJ has a fairly good soundtrack, but the sequel has an insane soundtrack composed by legendary Sega composers Hiro and Takenobu Mitsuyoshi. Notable highlights include:
"It's All Good", a funky song that teaches you the basics, with excited vocals from Takenobu himself.