The premise of the book is Nightmare Fuel. Sure, James's insect pals look sweet and cute in the illustrations, but would you like to come face to face with a human sized centipede◊ or spider in real life? They're actually less anthropomorphized than, let's say, Disney's Jiminy Cricket who is essentially a little green-skinned man with no ears.
The Old Man is extremely creepy in the book. His first spoken line is: "Come closer to me, little boy. Come right up close to me and I will show you something wonderful." Throughout his meeting with James he acts rather unhinged and slightly sinister, and it's impossible to guess his real motives for giving James the crocodile tongues — when James loses the bag so the magic works on the peach tree and the bugs instead of on him, there's the sneaking suspicion that he may have escaped a fate more horrible than anything Sponge and Spiker could have done to him.
In the film, the Old Man is heavily toned down; even if a bit of the creepiness remains, he's a lot gentler and is explicitly shown to be benevolent (especially when he reappears, in silhouette towards the end to tell the New Yorkers to "let the boy speak!"), and is revealed to have been the narrator of the story all along. This does not happen in the book.
Even in the book — considering what the crocodiles tongues did to both the tree and the insects — it is implied that the magic potion would transform James into a giant which would obviously enable him to escape from his monstrous aunts.
Examples from the film:
If there's one thing that can strike terror into the hearts of young children — just like James — then it would be the rhino.
The book at least explained the rhino as one that had escaped from a zoo, but in the film adaptation it's portrayed as some sort of ghastly, nightmarish Eldritch Abomination that tragically killed James' parents and remains his greatest fear. And this actually ends up being one of the reasons why it's so scary — the rhino's origin is left unexplained, to the point that it could be anything that your twisted mind can make up.
Even when the rhino isn't on screen, there's always a horrible feeling of dread that's built up around it; outside of flat-out abusing him, James' aunts torment him by promising that — just like what it did to his parents — the rhino might come out to get him, too.
The pirate scene has a few instances of obscenely creepy imagery.
Right when James and Miss Spider dive underwater to rescue the Centipede, they encounter another sunken ship on their way down, with a frightening depiction of James' aunts on the bow of the boat (in the place of the traditional mermaid). It's not only excessively creepy, but it's given no explanation whatsoever and — worst of all — it's never brought up again.
The scene at the end where they show up to try and take James and the peach back to England. Besides the fact that they've caught up to him and the dream he's come so close to achieving is danger of being destroyed, during the whole scene, they're both so pale and sickly looking from being underwater for so long and makeup so badly soaked and smeared that they look like deranged clowns. You can practically feel his fear!
One of the more unsettling sequences in the film is James's nightmare. He dreams that he's a caterpillar, peacefully biting a peach growing from a branch. But suddenly the music goes from whimsical to ominous as James' aunts appear in their car, launching a pesticide cloud at him. Said cloud eventually manifests itself as the rhino, which then pursues a frightened James. The dream ends with the rhino cornering him underneath a bridge, and just as it rushes into the camera for the kill... James awakes in the nick of time. The whole thing is rendered in creepy cut-out animation that looks more resembles some twisted sketch from Monty Python's Flying Circus.
The aunts in the dream look absolutely haunting, but Aunt Spiker is especially worthy of mention: with the exception of her creepily-scowling face, the rest of her body is nothing but bone.
And then of course there's the fact that the aunts mockingly chant "The rhino will get you..." in the background.