Two people left behind is one small thing - a huge event like preventing Pearl Harbor would have been catastrophic. Maybe a Japanese submarine would have sunk the Nimitz with an incredibly lucky shot. Maybe a freak up/downdraft would have sent all the planes into the ocean.
The Nimitz inadvertently invoked the circumstances for them to return home.
He decided that a DeLorean would be a better idea.