This trope originates with Greek mythology, wherein each mortal has a pair of spirits, which are aspects of themselves, and represent good and evil and actually sit on the character's shoulders. The angel/devil interpretation, however, according to The Other Wiki, originated with Islam in the form of kiraman katibin (literally, "honorable recorders"-their job is to write down a person's good and evil thoughts and deeds).
The earliest mention of the concept in Christian Literature is in the 1st or 2nd century apocrypha "The Shepherd of Hermas". A whole chapter discusses "the two angels that accompany the man : the Angel of Justice and the Angel of Sin".
Judaism has the concept of yetzer (ha)tov, or "good inclination," and yetzer (ha)ra, or "evil inclination."
An anecdote often attributed (falsely) to Native American legend says everyone has a good wolf and an evil wolf battling in his or her heart. The winner will be the wolf that the person feeds.
Biblically, the Book of Job has God and Satan standing, at least metaphorically, on each side of the mortal Job as they bet on how much hardship he can take before he cracks.