Letterman is often given credit for being one of those responsible for helping break alternative rock music to the mainstream American public during the 1980's, and many of his most awesome moments involve this genre of music and its related subgenres.
In 1989, the reclusive British band XTC gave an extremely rare live performance - their first of any sort in 7 years - for Letterman's show, performing "King for a Day". It's a song written and sung by Colin Moulding, which allows the band's other lead singer, Reclusive Artist Andy Partidge, to slip into the background a bit more.
On a episode in 2002, Letterman's close friend and frequent musical guest Warren Zevon was the only guest. It was his final public appearance before dying of mesothelioma, which he had been diagnosed with a few months before appearing on the show and would die from in under a year. He gave a lengthy interview with Letterman before giving his final public performance, of his 1978 song "Roland the Headless Thompson Gunner".
Another frequent Letterman guest, Phish, performed a multi-song set on the Ed Sullivan Theater's marquee in 2004. Only one song, "Scents & Subtle Sounds", was taped explicitly for the show, but in true Phish fashion they just kept going after that. The entire 24 minute set can be viewed here. It was the band's last television performance before their split in 2004. Since their reunion in 2009, they have not returned to Letterman's show (yet, anyway)
Letterman became an instant fan of soul-influenced British indie band The Heavy when they played his show in 2010. He loved their performance of "How You Like Me Now" so much that he asked them perform the song again immediately afterwards, the first time Letterman had ever asked an artist to perform an encore. When they came back in 2012 to perform their song "What Makes A Good Man" he again asked them to perform another song immediately afterwards (it was, of course, "How You Like Me Now").
More musically mainstream, but his final NBC show closed with Bruce Springsteen (who Letterman said was the one musical guest he'd always wanted but could never get) doing a knockout version of "Glory Days". He even jumped onto Paul Shaffer's keyboard at one point.
His first show after the 9/11 attacks. No monologue, no music, just him sitting at the desk trying to find some kind of explanation or reason for the tragedy. He did similarly excellent work after Hurricane Sandy and the Sandy Hook shooting in 2012.
Letterman's 2009 on-air confession of having had an affair with one of his staffers and detailing the bizarre extortion plot against him because of it was an oddly triumphant moment, because of the deft way he handled it, even finding some legitimate laughs in the situation. It basically set the gold standard for how public figures ought to deal with scandals: be honest, upfront and genuinely contrite, but be yourself.