Not to mention the photo John gave Reese of Sarah when she was pregnant. Reese states that he always wondered who she's thinking about to make her so sad in it. It's revealed at the end of the film that she was thinking of telling John about his father, Kyle himself.
Don't throw this troper to the hounds for mentioning Terminator 3, but I felt John and Kate's conversation immediately after the cemetery shootout was rather touching, in retrospect. John tries to comfort Kate by saying that some things simply can't be changed, which hits John every bit as hard as the fiancee's death hits Kate (by virtue of, you know, the entire day's events proving beyond a doubt that all the crap he went through in T2 was for nothing). John feels completely helpless, but that he still tries to comfort her despite all of his own hope being lost helps to explain why they're eventually husband and wife.
In the Cemetery scene as a nod to the second movie, John makes a remark on how the Terminator was the closest thing to a father he had.
Combined with one of the biggest Tear Jerker moments ever, John's breakdown when he realizes that the T-800 has to destroy himself to prevent the creation of Skynet. The kid has been through hell the entire movie, and its here that he completely loses it at the thought of losing not just his bodyguard, not just his friend, but the closest thing he has ever had to a father.
The end of the "Salvation" movie, when Marcus offer to sacrifice himself so the doctors can preform a transplant on John.
When John tells the Terminator to reveal himself to the Dysons, he takes their son Danny away to see his room and spare him more trauma. It's especially sweet when you consider that John didn't have much of a childhood himself and would rather not have an innocent child caught up in this any further.
The scene with Marcus and Blair when they're on their way to the Resistance base, and snuggle up by the fire (well, Blair does. Marcus just lets her).
The scene where John laments to The Terminator about his life up to meeting him. It's pretty heartwarming that he can confide to a machine who at the time didn't understand human emotion, but was willing enough to listen.