The puzzle with the painting of Stauf's face. The character is creepy enough to begin with (he's an insane toymaker/millionaire who opens the game's intro sequence by murdering an elderly woman for her purse), but in order to solve the puzzle, you have to cycle through bits of his face that look like lizard skin or demon skin (complete with horns), and when you finally finish it, it looks like Stauf is trying to bite through the painting and attack you).
The fact that you find yourself looking out through the main character's eyes, with no idea what you look like, or why you're in Stauf's mansion, or what the hell the other six people who entered the mansion hard on your heels are doing there.
Coming across a boy in the mansion, and, unless you've done some very thorough searching on the first floor, having no idea who he is or how he entered the place.
That damn creepy children's rhyme that plays after you finish the puzzle in the library. The developers knew they had a winner on their hands with that one - it's the only scene involving the tragic/comic mask cursor that can be replayed more than once.
The woman in the white dress who floats around the second floor hallway. Even after playing through the sequel, we still have no idea who she is or what she's doing there. (Maybe Stauf had a wife - which is Nightmare Fuel in its own right).
Stauf screaming "COME BAAAAAAAAAACK!" when you saved and quit the game.
The Freemason-style pyramid cursor, with that single eye staring at you. Also, for that matter, the throbbing brain cursor. In fact, even the standard skeletal hand was pretty damn creepy when you consider that it was the default cursor. And this was back when most of us were used to seeing the nice, ordinary hand cursor in Myst.
The maze in the dungeon. Every time you hit a dead end, Stauf would inquire, "Feeling....lonely?"
The labyrinth appears to have been designed to maximize paranoia and Claustrophobia. For example, you start walk down a straight corridor that seems to go on forever. There are no distinguishing features to tell you how far you've gone and all the intersections look the same, making it difficult to get your bearings straight. You keeping walking and walking and then suddenly, a wall appears opposite of you through the darkness. Cue a scarySting and Stauf taunting you with "Feeling... lonely?" Congratulations, you've hit a dead end. What's worse is that there are several corridors like this in the labyrinth. In addition, there are also compact, twisting corridors designed to disorient and confuse the player.
Also, can't forget the scene after solving the dollhouse, when you learn that the dolls have childrens' spirits inside. The game's intro, which tells the tale of dolls killing children through illness, was scary enough without all that.
The painting at the top of the staircase? When you click on it, you see the shape of hands outlined and pushing/distorting the painting in all directions, as if someone was trying to escape from inside the painting, before it finally snaps back to looking normal.
The 11th Hour
The Marie ending.
"How about...a Chuck roast?" The capitalization there is not a typo.
How about a freaking tiger advancing on Robin during a cutscene?
How about a flashback wherein Marie's mother, Eileen, is running away from Stauf's mansion, gets her hand stuck between the bars of the locked gate...and she screams bloody murder while the gate eats her hand clean off?
Or slightly later in that video where Eileen is at the doctor's screaming, and raises her arm, you can see the ragged stump, with twitching bits of sinew/muscle and blood running out the artery.