Polished Port: The 3DS version of the game is aiming to be this, with this blog post explaining how everything is being reworked to accommodate the new system.
That One Achievement: Death Cam, No Deaths. It's the achievement with the lowest completion rate on Steam, and with good reason. You have to play the entirety of Death Came VHS without dying, which is entirely a frustrating affair due to level being a Marathon Level full of combat. The last two rooms in particular start spamming rocket launcher enemies - a weapon that can kill you in two or three hits - and the last one is a room that's on fire, vastly limiting the room you have to navigate to avoid said traps. It doesn't help that you're not allowed to bring weapons in, either.
The Bit Trip: Retro City game in the Arcade. It's necessary to complete all the stages with all the gold collected to unlock the character, but it requires pixel perfect jumps. It will take hundreds of tries.
For the second-rate sprees, "Pick a Fight with Dynamite" is the most frustrating. As soon as Player throws the dynamite, the civilians will scatter, making multiple kills difficult. Furthermore, the dynamite has to destroy vehicles completely, meaning if a car explodes because it's on fire, it doesn't count toward the score. Finally, it's very easy to kill Player with the explosions, so even if the score passes the mark for a gold medal, it won't count if Player dies.
Unwinnable by Insanity: In stage 2 of Final Castle, it's possible to trap yourself by killing the entire Bomb Squad and using up all of your bazooka shots. With nothing to blow up the walls and nothing to kill you, your only option is to save your game and start the whole stage over again.