Piggy's death. It's made worse by the fact that Piggy had been bullied and beaten ever since he landed on the island, and the only thing he ever wanted was to be rescued, which he was not. He also never stood up to Jack, which he had been planning to do.
Simon's death. He and Piggy were two of Ralph's most loyal allies on the island, and Simon was the first to go. If he hadn't tried to immediately warn the boys about what the beast really was in the midst of the storm and chanting chaos, he wouldn't have been mistaken for the monster itself.
At the end of the book/film, we see one of the littlun's—Percival—who had made a big deal out reciting his name and address at the beginning of the boys' stay on the island, finally approach someone who can help him— the naval officer who comes to rescue them. However, because of the disorganization and neglect on the island, Percival has forgotten his name and can only look up and mouth wordlessly at the man.
Even sadder is Ralph's crying. Despite surviving, he is completely broken by his experiences, his lost innocence, and Piggy's murder.