When you find yourself trying to remember a show (or any works) that's on the tip of your tongue but just out of reach, come here - the collective brain of the TVTropes community can probably help. Post all the details you can remember (examples help). If you're looking for a trope, head over to Trope Finder. Have general questions about tropes? Visit Ask The Tropers!
What about Thou Art the Man?
https://en.wikisource.org/wiki/%22Thou_Art_the_Man%22
Or The Black Cat?
https://en.wikisource.org/wiki/The_Saturday_Evening_Post/The_Black_Cat
If that's not it, here's a list of all of his short stories. I'm sure you'll find it there with patience:
https://en.wikisource.org/wiki/Author:Edgar_Allan_Poe#Individual_stories
The story you remember is likely "The Imp of the Perverse"
. Although the murderer/narrator is not driven to confess by the ghost of his victim, but by his very own "spirit of the Perverse".

We had to read an Edgar Allan Poe work for English class as part of a reading comprehension test, but for the life of me I can't remember what it was called. It involved the narrator committing a murder and getting away with it, only to be tormented by what he believes to be the spirit of the man he murdered. This ultimately culminates into him believing that he was struck down by the spirit and forced into publicly confessing to the crime.