Clipped this Justifying Edit from the DBZ entry:

Goku was dead once (after Cell), thought to be dead once (Frieza) and just up and left once (end of the series, to train Uub). After both Frieza and Cell, Goku refused attempts to wish him back (why depends on which version you're looking at: manga, original anime or dub). But the fact remains: Twice, he had a chance to return his family and chose not to. And a third time, he spent a decade with his family, only to run off again to train Uub (Why he had to leave home for another decade to do this...)