This is based on opinion. Please don't list it on a work's trope example list.
Nightmare Fuel / The Silmarillion
Everything about Morgoth/Melkor. The Satan of Arda, Morgoth engaged in Mind Rape without even having to think about it, and the only sort of pleasure he could experience came from horribly torturing and corrupting Elves and Men. In-universe, lesser Ainur were driven to insanity by his eyes.
Also the fact that Melkor captured many of the Elves and corrupted them, turning them into Orcs. What did he do to them?
Hurin: the mere thought of drawing Morgoth's undivided attention is alarming, but he keeps going at Hurin for years.
Just when you think that the good guys had won the battle, Morgoth unleashed the fires of Thrangorodrim unto them and to the nearby lands. Oh man! This is no joke, in fact this battle called Dagor Bragollach itself is such a nightmare into its own right. People burned left and right as they were not only burned, but consumed by the lava itself.
Ungoliant, as one would expect from a giant spider who brought Shelob into the world. A simple Maia, she drank the sap of the Two Trees and turned into the pure embodiment of darkness and gluttony, and became the Broodmother of all spiders in existence (sorry arachnophobes!). She's never satisfied, and isn't picky about what she eats. She would devour the entire world if given the chance. She almost ate Morgoth himself. And what's worse, the published version of The Silmarillion does not confirm her fate. So...she could still be out there somewhere...always hungry...
What the imperialistic Numenoreans did while under Sauron's instruction to the peoples of Middle-Earth like the Haradrim, the Variags, and the Easterlings. We never get the details, but whatever it was, it so fouled their hearts with hate and pain that centuries later they worshiped and served Sauron just so they could kill Numenor's descendants.
Most likely they were being enslaved and sacrificed to Morgoth, as it is mentioned that Numenorians did that to men of Middle-Earth. Still a terrifying thought.
Ancalagon The Black. So humongously large, so large that he himself broke 3 mountains that are bigger than Mount Everest by his death. He has enough power to melt many of the rings of power. He is basically a walking natural disaster.
Imprisonment on the Isle of Werewolves. Trapped in a dungeon, with werewolves gradually picking the prisoners off. The Lay of Leithian even describes chains that eat their flesh.
Maedhros' torment on Thangorodrim. Morgoth catches him, tortures him, and then hangs him by the wrist from a sheer cliff face. It's not clear how long he was up there, but judging by other events in the timeline his imprisonment lasted for something like twenty years. It's so terrible that he can only beg for death when his best friend Fingon finds him. Then, of course, Fingon has to chop off his hand in order to get him down.