There's also "The Pirate King", one of the most infectious songs about Pirates!
Ten words; "I am the Very Modern Model of a Modern Major-General".
From a purely-musical standpoint, this opera contains two brilliant examples of counterpoint writing—creating separate and quite different melodic lines that nonetheless fit together when played simultaneously—in "How Beautifully Blue the Sky" and "When the Foeman Bares His Steel". The first example is even more impressive in that the lines are in two different time signatures. While he's mostly remembered for accompanying comic operas, Sir Arthur was a talented composer in his own right.
Ear Worm: Quite a large number, but by far the most well-known is: "I am the very model of a Modern Major-General..."
Fridge Horror: The policemens' fear of having to face the pirates may seem like cowardice and the girls assumption that they will not come home alive like hastily jumping to conclusions. Until you realize that these are English Bobbies, who were not issued firearms and were going after a band of pirates armed with nothing but truncheons.
The Patter Song "My Eyes Are Fuly Open" was added to the show by a production in the 1970s, lampshaded by Ruth's actress mentioning the song's from Ruddigore. The production was so popular it's now expected.
Mexicans Love Speedy Gonzales: Because of both this play, and Treasure Island's use of the Cornish accent as the pirate accent, the Cornish as a whole happily and willingly engage in antics that turn them into pirates. This carries to such an extent that there is even A Rugby team called the Cornish Pirates.
The melody for "Come, Friends Who Plough The Sea" was later appropriated as the song "Hail, Hail, the Gang's All Here," not (as you will often see claimed) the other way around. Conversely, this was inverted by Kurt Vonnegut in Slaughterhouse-Five, with a reference to people "singing 'Hail, Hail, the Gang's All Here' from The Pirates of Penzance."
The policemen predate the Keystone Kops, but are clearly an influence on them.
Values Dissonance: Frederick's "slave of duty" mindset will tend to strike modern audiences as merely silly. Englishmen of Gilbert's day, though, would have recognized it as a parody of their own code of conduct.