The premise of the book is Nightmare Fuel. Sure, James's insect pals look sweet and cute in the illustrations, but would you like to come face to face with a human sized centipede◊ or spider in real life? They're actually less anthropomorphized than, let's say, Disney's Jiminy Cricket who is essentially a little green-skinned man with no ears.
The premise of the book and the movie has a big one for any parents: Mr. and Mrs. Trotter's sudden deaths after being attacked by a rhinoceros that escaped the London Zoo leave James all alone in the world except for his aunts whom he's presumably never met and who subject him to physical and emotional abuse. The idea of not being able to protect your child from such cruelty might make more than a few parents consider who would raise their children if they were to suddenly die.
James is a young child who quite suddenly loses his mother and father and is taken away from the only home he has ever know. If that isn't pure childhood nightmare fuel, nothing is.
The Old Man is extremely creepy in the book. His first spoken line is: "Come closer to me, little boy. Come right up close to me and I will show you something wonderful." Throughout his meeting with James, he acts rather unhinged and slightly sinister, and it's impossible to guess his real motives for giving James the crocodile tongues — when James loses the bag so the magic works on the peach tree and the bugs instead of on him, there's the sneaking suspicion that he may have escaped a fate more horrible than anything Sponge and Spiker could have done to him.
In the film, the Old Man is heavily toned down; even if a bit of the creepiness remains, he's a lot gentler and is explicitly shown to be benevolent (especially when he reappears, in silhouette towards the end to tell the New Yorkers to "let the boy speak!"), and is revealed to have been the narrator of the story all along. This does not happen in the book.
Even in the book — considering what the crocodiles' tongues did to both the tree and the insects — it is implied that the magic potion would transform James into a giant which would obviously enable him to escape from his monstrous aunts.
The swarm of sharks that comes out of nowhere to attack the peach.
The Cloud Men after the Centipede mocks them. First, they throw their trash at the peach, then they make it hail, then the Centipede gets stuck in place after one Cloud Man dumps quick-drying purple paint all over him. The good news? It's not water-proof.
Ask anyone who watched this movie as a child what the scariest part of the film was; odds are good they'll respond that it was the rhino.
The book at least explained the rhino as one that had escaped from the London Zoo, but in the film adaptation it's portrayed as some sort of ghastly, nightmarish Eldritch Abomination that tragically killed James' parents and remains his greatest fear. And this actually ends up being one of the reasons why it's so scary — the rhino's origin is left unexplained, to the point that it could be anything that your twisted mind can make up.
You don't even have to interpret it as a monster for it to be scary. It may very well have been a storm with winds strong enough to swallow two adults and orphan their child, which isn't even a concept of fantasy.
Even when the rhino isn't on screen, there's always a horrible feeling of dread that's built up around it; outside of flat-out abusing him, James' aunts torment him by promising that the rhino might come back to get him like it got his parents.
The pirate scene has a few instances of obscenely creepy imagery.
Right when James and Miss Spider dive underwater to rescue the Centipede, they encounter another sunken ship on their way down, with a frightening depiction of James' aunts on the bow of the boat (in the place of the traditional mermaid). It's not only excessively creepy, but it's given no explanation whatsoever ( and it's never brought up again. It could be taken as a Foreshadowing of his aunts following James across the Atlantic, but first-time viewers wouldn't know that.
The Nightmare Fuel might be lessened when you realize two of the pirates are played by Jack Skellington and Donald Duck.
The scene at the end where they show up to try and take James and the peach back to England. Besides the fact that they've caught up to him and the dream he's come so close to achieving is in danger of being destroyed, during the whole scene, they're both so pale and sickly looking from being underwater for so long and makeup so badly soaked and smeared that they look like deranged clowns. You can practically feel his fear!
The aunts in the dream look absolutely haunting, but Aunt Spiker is especially worthy of mention: with the exception of her creepily-scowling face, the rest of her body is nothing but bone.
And then of course there's the fact that the aunts mockingly chant "The rhino will get you..." in the background. It starts out as a barely audible whisper, then gets progressively louder until the end, where it sounds as though the aunts are screaming it as the rhino manifests.
The new musical pulls an Adult Fear type of Nightmare Fuel for the bugs. Centipede falls off the peach and James, using one of Spider's ropes, goes after him. The others wait at the edge of the peach and notice the rope has gone slack. Cue the bugs- Ladybug and Grasshopper especially- fearing the worst has happened. It doesn't last long, but for such a brief moment, it drives home the fear of losing both a friend and losing a child!