Guide Dang It!: The two biggest contenders are probably the platinum medals and the Whack-a-Mole minigame:
Every optional level has a Platinum Medal hidden in it somewhere, and all 11 are needed for the Infinity +1 Sword. Some of them are hidden in places a savvy gamer would easily think to check, such as down an obvious branch in a path of disappearing platforms, or in a room that (since all the optional levels are mirrored versions of previous stages) in the level's previous incarnation held a treasure chest. Most of them, however, are revealed for stepping in completely arbitrary places, like standing on top of treasure chests or hopping into hollow cubes. You'll probably stumble into one or two, but finding them all without a guide is an exercise in wandering and hopping over every square inch of every optional level.
The Whack-a-Mole minigame requires you to hit 100 moles in 60 seconds. You can get multiple hits on one mole, but without excellent luck, you'll probably max out at 70-80 moles a round. There's a trick to doing it reliably, but said trick is unintuitive: You have to repeatedly jump attack the moles. This actually keeps them from going into their holes and lets you combo them repeatedly for the necessary score.
Most Annoying Sound: Parin's 'boing' sound clip when she jumps, if you are trying to obtain the Crown, will be drilled into your skull.
Climbing the monkey bars in Pimento Park to get the Crown, patiently waiting for 999 rocks to smash agaisnt Spinach Caves' walls to get the Samurai Helm, or even some of the Platinum medals which are VERY well hidden...
There's also the Hoccer minigame. It's a fairly simple 3-on-3 soccer minigame, and you have two minutes to score more goals than the opposing team. However, there are numerous problems: The camera doesn't follow the ball, making it very easy to lose, the ball moves erratically, and it's very hard to hit the ball without doing homing jump attacks that cause it to fly around crazily. Perhaps most annoying of all: The opposing team is lead by Motoro, who's at least three times as big as any other character in the minigame, and he likes to play goalie. If Motoro camps on the opposing goal, good luck getting any shots in.