Perhaps the ultimate example is "Oh Dance, Baby", where Bret starts singing karaoke, entirely in Korean, about the difficulty of being in love. Or something.
The karaoke number is a reference to an earlier joke in the episode, where Murray got the guys an off-screen "gig" at open-mic karaoke night.
Sometimes this even happens within a song, like with Dave's completely out-of-the-blue rap verse in Sugarlumps.
Interestingly, Mushroom Samba song Prince of Parties is not one: in fact, it is actually probably the only song in the series that is directly referenced in a later episode: "If you party with the party prince, you get two complimentary after-dinner mints!" sings Bret, wearing the costume from Prince of Parties, in the second-season song Sugarlumps.
That's a BLAM referencing another BLAM... so meta...
Alternatively, all the sudden song numbers exist in the same alternate reality...
The wallpaper-skin reference from Freaky is shown to have actually happened shortly after the song ends.
Ear Worm: Too many to list. Everyone has their favourites.
Ensemble Darkhorse: Murray, due to his tendency to dominate almost every scene he appear in.
In the song "Hurt Feelings", Jemaine laments about how his family forgot to call him on his birthday. His next line in the song is "The day after my birthday is not my birthday, mom." At first, this just seems tragically hilarious. However, New Zealand is 17-18 hours ahead of the USA's east coast. Ergo, she actually called him two days after his birthday!
Upon listening again, you may realise that the rhyming structure of the song "Carol Brown" actually foreshadows the episode's ending! Keith was nothing but a thief.
National Stereotypes: New Zealanders making fun of themselves, and New Zealanders making fun of Australians.
Stoic Woobie: Bret and Jemaine seem to experience nothing but failure over the course of the series, yet they're completely nonchalant up until the final episode.
Tearjerker: The hero's misfortunes in "Hurt Feelings" are played for laughs, but the ones in the reprise... not so much.
Played for laughs, sure, but still pretty sad. "The day after my birthday is not my birthday, mum!" Owtch!
The only reason "I'm Not Crying" is so funny is because it's genuinely sad.
After Murray realizes that the Tech Support Blonde he had a crush on, and for whom he was trying to write a song, has left the office, he sadly crosses out "hi" in his work-in progress song and writes "goodbye," segueing into the melancholy "Leggy Blonde."
A bittersweet one: when Murray returns to his old job after quitting to manage a different band, not only is his angry resignation letter still left unopened on his desk, but nobody even realized he was gone. The good news is he can return to his normal life without and baggage, but the bad news is it proves that nobody there cares about him.