Bang Bang (My Baby Shot Me Down) by Nancy Sinatra, over the opening credits. The minimalism of the song only increases the anticipation for the bloodshed ahead. Plus, the guitar is awesome.
Not on the soundtrack, but still awesome, Death Rides a Horse (originally written for the movie of the same name), which plays when the Bride calls out O-Ren Ishii.
Also the re-use of Ennio Morricone's "L'Arena" (originally written for The Mercenary) for the Bride's escape.
Not to mention Morricone's A Silhouette Of Doom. It's called A Silhouette Of Doom and it's written by Ennio Morricone. What more do I need to say!?
Let's not forget Santa Esmeralda's Please Don't Let Me Be Misunderstood, particularly the use of the instrumental break in the song as a lead-in to the Bride's battle with O-Ren Ishii.
That theme that plays when Kim, Zoe, and Abby are chasing Stuntman Mike is all kinds of awesome!
"Jack and Jill" is a really touching song, just a lonely melancholic piano playing in a moment of passiveness. You won't really thing it would be THAT kind of Mood Whiplash after watching the rest of the movie, but...
David Bowie's "Cat People" as used in Inglourious Basterds. Should be completely out of place, but fits perfectly.
If you ever have the chance to have theme music announcing your arrival anywhere, you want it to be "The Verdict". You know it's true. Tarantino and Ennio Morricone cannot be denied.