According to Hugh in a later interview, Stephen Fry had never been able to convincingly fake hitting someone. So when the script called for him to hit Hugh - which happened quite often - he would actually hit him. So that wincing and those cries of pain you hear from Hugh in this show are mostly real.
A lot of the time it's accidental though. A good example of this is in the racing driver sketch. Near the end you can see Stephen's eyes widen in shock, before he folds his arms and looks away.
Executive Meddling: The fourth series was aired on BBC 1 rather than BBC 2 and they mandated the presence of guests on the show—while there had been numerous guests in previous series, they were performers in the sketches rather than interviewed features. Fry and Laurie were not pleased with the change, as is very clear from their absurd interviews and their dripping sarcasm in episode 6 when they were "unable" to find anyone to join them.
Fake American: Hugh Laurie plays an American country singer in one sketch, and he and Stephen play American soldiers saying "ass" a lot in another.