From the trailers, we have Smaug's head slowly coming around the corner, looking directly at Bilbo. It's our first look at the dragon, and it's terrifying.
The sudden flashes of the Eye of Sauron in multiple trailers, especially given what they imply.
Smaug speaking at the end of the second trailer. That Benedict Cumberbatch delivers a truly chilling performance worthy of the dragon is evident with just a few lines. Bonus points for having his voice, which is plenty deep on its own, somewhat altered to the point that it sounds downright demonic.
On top of that, at the beginning of the trailer, Bilbo's voice breaks in fear as he talks to Smaug.
Mirkwood is a scary place even before the spiders show up. The Mind Screw hallucinogenic effect it has on the Company is very unsettling.
The spiders. They actually roar (or as close as you can get). They capture the dwarves quickly and efficiently when Bilbo isn't watching. The idea of being bundled in spider-thread is not pleasant.
Shelob was large and rather bulbous. The Mirkwood Spiders are smaller, but they're leaner, sharper, and somehow more menacing.
Their voices heard when Bilbo wears the Ring are rather unsettling as well. "Feast... feast... FEAST..."
The fact that Bilbo can understand them when he has the Ring on! Their hisses and squeals are translated into menacing words. And the effect lingers for a short while when he takes the Ring off...
Before Bilbo kills it, one of the spiders is just about to take a bite out of Bombur - who's awake and aware of what's going on, and making muffled screams!
What really sells it is the look on Bilbo's face after. He stares at the Ring, completely horrified, and claps his hand over his mouth as if he's ready to vomit, clearly wondering just what the hell is happening to him.
Even more Fridge Horror when you remember that Bilbo lunged at Frodo the same way in Fellowship.
And an extra helping for when Frodo pins Sam with Sting in The Two Towers.
That scene is shot very similarly to the opening of Return of the King, where Smeagol kills Deagol for the Ring, a kind of cinematography Shout-Out.
Much like in The Fellowship of the Ring, we get our Jump Scare from the unlikeliest of sources: when Thranduil starts shouting at Thorin about dragons, some illusion he seems to have over himself fails for an instant, and we get a lovely closeup of his face. Particularly the half that's suddenly a burned, wrecked mess, with a freaky white eye and spots where we can see through to the bone. Thranduil knows dragon fire indeed.
In the extended edition, Gandalf's trip into Dol Guldur provides another one when Thrain attacks him with the obvious intent of killing him, and snarling like a rabid wolf.
The Ring itself frequently proves to be Nightmare Fuel. Resting at Beorn's house, Bilbo takes it out and looks at it and suddenly begins to stroke it, the same way he obsessively does sixty years later at Bag End. And as he does, you can hear the Black Speech far away. Even now after only being in his possession for a short while you can tell Bilbo's starting to fall under its influence.
Made doubly creepy by the fact that Bilbo has no idea what's going on until the aforementioned spider-killing, and the only person who could have warned him (Gandalf) is gone for most of the movie. By the time he realizes that it's corrupting him, he can't bring himself to get rid of it.
The fact that Smaug can smell the difference between the One Ring and the familiar, abundant gold of Erebor.
Smaug: There is something about you... something you carry... something made of gold... but far more precious. (The One Ring begins muttering 'Precious', louder and louder with each repetition, Bilbo clearly jerking from this until he unconsciously takes it off. Smaug reacts to his reappearance, and Bilbo reacts at the fact that he's now visible to the giant dragon, and he wasn't aware that he did that until too late.) Smaug:(interested) There you are, Thief in the Shadows.
Makes you wonder just how coincidental 'precious' was. It gets even worse when you realize Smaug was mere feet away from obtaining the most powerful and evil artifact in Middle-Earth. On some level, Smaug may have even been attracted to the Ring, but Bilbo's (re)appearance distracted him.
He was immune from the Ring's effects, being a Fire Drake of Morgoth. He was hatched corrupt and evil. But the Ring was born from the same source as he was born - Morgoth's fires - and it had a very familiar smell. Which easily allowed him to track Bilbo, even hiding behind a pillar....
Not to mention that he knows Sauron is out there and rebuilding his strength and is willing and eager to burn the world to ashes for him like he did Dale and Erebor if he comes knocking, which makes sense as Smaug is a Fire Drake, meaning that he served Sauron's master, Morgoth. No wonder Gandalf wants Smaug gone!
Gandalf is searching through an old tomb in the High Fells. The tomb consists of a narrow, crumbling walkway that's barely two feet wide, suspended over a ten story drop, which is great if you're afraid of heights. We see that there are nine tombs and they have all been violently opened. Gandalf realizes that nothing broke into the tombs. Something broke out.
He nearly falls twice; once when climbing the steps up a sheer cliff face, and then sliding down the entrance passage and only just managing to pull himself back over the brink. And then, in case your heart wasn't beating fast enough, there's the Jump Scare when a bird flies out of the tomb.
A subtle one where Bilbo finds the keyhole. He calls out to the dwarves who are nowhere in sight after they gave up, then tries to find the dropped key which he accidentally kicks over the cliff. Thankfully Thorin manages to save it, but imagine what would've happened if it did fall all the way down and got lost...
Smaug's first appearance in the movie proper. First, Bilbo sees his eyes and snout emerging from a pile of gold. He is utterly terrified, and backs up behind a pillar to hide himself, but then... on the other side of the pillar, Smaug's tail flexes. The look on Bilbo's face says it all. Not only is the dragon gigantic, but he's WAKING UP!
After Smaug wakes up and starts searching for Bilbo, the next time he moves close to Bilbo, Bilbo stands up, glaring defiantly at Smaug, right up until we cut to a shot of Smaug only centimeters away from Bilbo when he says, "Where are you!" And once he moves too close to Bilbo for comfort, Bilbo turns and runs like hell, drawing Smaug's attention and then Smaug turns in Bilbo's direction with the most terrifying grin ever.
The poison in the arrow that Kíli is shot with. After being left behind, Kíli all but collapses, and a few hours after that he's only capable of writhing and screaming in pain. Extra fuel is added by how the other dwarves still in Laketown must feel—there's nothing they can do to help.
Also, those screams? Sound like someone being tortured. It's no wonder everyone was freaking out.
The Fridge Horror that Bard's kids probably owe their lives to Bofur drinking himself into a stupor in Laketown. If he hadn't missed the boat, Fíli would be the one sent to find kingsfoil, leaving only the elderly Óin and the incapacitated Kíli to fight off the Orcs until Legolas and Tauriel arrived.
Bolg. He's just as big and menacing as his father Azog, and even uglier. His face is held together by iron strips bolted to his skull. And he manages to beat the crap out of Legolas - that puts him in Hero Killer territory.
Smaug stating that he's almost tempted to let Bilbo take the Arkenstone to Thorin, gleefully anticipating how it will corrupt Thorin's heart and drive him mad.
Which leads to very subtle but chilling horror in the scene where Thorin holds Bilbo at swordpoint upon realizing that Bilbo hasn't found the Arkenstone, allowing the rest of Company to rush into Smaug's lair. He is physically threatening the very same hobbit who has saved his and the Company's lives repeatedly, and is willing to throw his lifelong friends and loyal soldiers against a dragon that has obliterated whole armies and cities without suffering as much as a bloody scratch. It is harrowing to see how he slowly succumbs to the very same madness that led to his home's destruction after all his trials and coming so close to his goal.
The subtlety of this lies as much in Thorin's demeanor, just when you think Balin got him to go down to protect Bilbo. He's still acting like Thorin, calm and quiet as can be, and asking in a seemingly reasonable tone where the Arkenstone is, albeit with no visible concern for his friend. Then he uses the sword to block Bilbo's escape, calmly raising it again when the hobbit tries to shove it out of the way, and coldly enough that he's suddenly not Bilbo's comrade at all. Worst of all, there's no visible rage; Thorin is so coldly deliberate that he has to know exactly what he's doing, even though there's nothing reasonable about this at all. If you think about it, Thorin was this close to outright murdering Bilbo until Smaug showed up.
The big reveal of Sauron is quite creepy in its sheer surreality. He manifests as something between a cloud of smoke and a shadow that casts itself on nonexistent surfaces and angles and tries to suffocate Gandalf in his darkness. When it fails, he reveals his humanoid form, which is like the armoured figure from the prologue of the Fellowship, but made of fire. Then the fire spreads from him, leaving him in the middle as the pupil of the Lidless Eye. And we see him again... and again... and again while the movie is practically blastingthe theme of Minas Morgul from the original trilogy. It's no wonder anyone can be driven mad just by looking at him.
The thought of being in Gandalf's position, trapped in Dol Guldur with his staff destroyed and watching armies of Orcs marching out, knowing you can't stop them.
The Company (sans those who stayed behind in Laketown) fleeing Smaug and coming across a chamber filled with the mummified corpses of Dwarven women and children who fled there during Smaug's attack and were trapped. The expressions frozen on the dead and rotting faces are simultaneously heartbreaking and terrifying. Thorin's reaction to it only seals the deal:
Thorin: No. I will not die like this. Cowering, clawing for breath.
The ending with Smaug flying towards Laketown, and the line "I am fire...I am...death." Plus, from there until after the film cuts to black, there's absolutely no music. It's pretty chilling.
Before that, Smaug, still pissed off from chasing the Company, tells Bilbo he knows they came from Laketown, and decides to pay them a visit. A horrified Bilbo actually starts running toward Smaug to stop him from attacking Laketown. Smaug's reaction? He calms down and realizes Bilbo cares about the people in the village. This makes him happy, because he can make Bilbo suffer by destroying Laketown. That is how wrathful Smaug is: destroy a whole town of innocents just to torment one person.
The terrified reactions of those back in Laketown who hear the noise of Smaug bursting out of the mountain, and see the fire. Whether they're dwarves, elves, or men, they can guess what's coming for them. It's particularly bad for Bard, since he's been locked up and is helplessly trying to convince the guards to let him out, while knowing he's only got minutes at best to try and save his children.
Bard:(to the guards) Listen to me! Do you not know what is coming?!
Bilbo pretty much sums it up with the last line in the film.
"I See Fire": a catchy and wistful song playing over the credits to the film - that's all about someone foreseeing Smaug coming to destroy Laketown, that darkness is falling and how they're probably all going to die. With lyrics such as "I hear my people screaming out" and "I see fire and blood in the breeze". There's also the Dissonant Serenity going on; while those rather violent lyrics are being sung, there's just one guitar playing softly in the background in a low-key, mellow manner, emphasizing the creepiness of the song nicely.
Smaug in general. Beyond how big he is note he's officially twice the size of a jumbo jet, making him close to 400 feet long, there's having his lizard like face model his voice actor's facial expressions that's just creepy, especially given how the shots keep emphasizing his huge teeth. The whole context of the scene with him and Bilbo is more frightening than in the book, despite his Badass Boast losing some context, with Biblo having a Freak Out! causing him to take the ring off meaning Smaug DOES see him and the conversation they have is changed from Bilbo making a show of his bravery to Smaug toying around with him trying to have some amusement from his reactions before killing him. And departing from the norm, Smaug's hammy performance does nothing to detract from how frightening he is, especially since the film has actually made him more evil than in the book since his motive to destroy Laketown is less about spite over being robbed and more so that he can make Bilbo suffer, even though he hasn't done anything to him and Smaug actually considered letting him take the Arkenstone.
How utterly silent Smaug can be when he chooses to. If he wasn't covered in gold coins that constantly fall off him the party would have had no idea he was right above them at one point. The idea of something that big and powerful being capable of silently stalking you is a terrifying thing.
The film makes excellent use of Nothing Is Scarier. When Gandalf is skulking around Dol Guldur or Bilbo is sneaking around Erebor, the smothering silence makes you hold your breath waiting for the Orcs or the Necromancer or Smaug to show up any second.
Beorn can be quite scary in bear form. He comes so close to killing the entire company, with them only just closing the door in time. Another few seconds... And he' so powerful that Azog won't attack the Dwarfs when he's in the area.
The scene where brianwashed Thrain chases Gandalf in the extended edition. We're only given brief glimpses for a good long while with him moving past the camera at lightning speed. The only sounds he makes in the scene?: snarling. And then it goes quiet for a moment as Gandalf tries to catch his breath only for Thrain to pounce put of nowhere.