Demonic Spiders: Most definitely the enemies in the Western Tower. All of them are capable of high damage and several of them have a mass attack which can cause confusion resulting in a quick death unless you bought a Calm Kerchief from the village.
The scene where Erio's big sister saves everyone in the Sepulcher after being called upon becomes rather heartwrenching when you get to the bad ending sequel. Erio's been away from home for ten years; this is probably the only contact she's even gotten from him at all in that time. Maybe she was happy to have some word from him even if it was basically "Yeah we go in trouble please bail us out"...and then in the bad ending, he goes and dies. So in the ten years he was away, he only contacted her once, and then he died, without her ever having been able to actually see or talk to him.
Both Erio and Hiante tell Snowe that if Astra ever gets hurt on his watch, they'll kill him. At the time, it just seems like them being overprotective. But in the bad ending sequel it's learned that both of them were dead serious. Hiante tracks Snowe down because he killed Astra and Erio, and it's later discovered that the reason Snowe has a scar across his eye now is because Erio tried to kill him after he stabbed Astra.
Erio at one point wonders how Snowe can still be alive after all that's happened to him, Snowe has trouble keeping warm, and at one point Edgar says "You are not my son! My son is not walking death like you are!" All together, those seemed like rather odd details. But then in Fleeting we learn that Snowe was actually killed and the only thing keeping him going was Xiri's magic.
Paranoia Fuel: In the Sepulcher, Erio discovers that Edgar is blocking his ability to teleport. He warns Hiante that since Edgar figured out how to do that, he might also figure out a way to stop the spells that keep Hiante going.
The Old One is most commonly cited. It doesn't help that he continually summons two flunkies that can completely destroy you if you aren't careful. He got nerfed in version 3 in terms of stats, but his minion summoning is still very frequent and the minions are still faster than a properly leveled party.
The Western Tower boss. All he has to do is cast both Resplendence and Gloom in one turn and it's an instant game over without some serious grinding or some serious luck. Not to mention that you start the fight out with only Hiante and Relenia, and though Astra joins the fight eventually, your party's resident healer never gets involved.
That One Level: The Sepulcher might not have the toughest enemies in the game, but completing it is still fairly difficult. The dungeon locks the player in and cannot be re-entered upon completion, meaning there's a lot of missable content like the flowers for potions and Astra's best sword. After some events, the party will be split and will have a much harder time dealing with enemies until they reunite.
Relenia reaches Iron Woobie status as well once you find out about her daughter.
Erio. He's been away from home for years and has no idea if his sister is mad at him over it or not, and he wasn't treated well when he first arrived, at all. And when Lina discovered he was taking Astra on outings from the tower, Astra lied that she was still having nightmares so they couldn't get rid of him because she was seriously afraid for his life.
Jerkass Woobie: The demon. First he's trapped for a long time, then he gets trapped again, then he tries to befriend the kid he's trapped in because he feels bad for him but they get torn apart by the kid's abusive parents, and then in one of the endings he gets trapped yet again. And I Must Scream is not a fun situation to be in.