Whitehall is to the British what Pennsylvania Avenue note is to the Americans on other side of The Pond - a common setting for a Government Procedural, like Yes, Minister and House of Cards. Note that Whitehall is a street, not a building. The name 'Whitehall' comes from Whitehall Palace which was home to the English monarchs (and, during the 1650s, the Lord Protectors) from the reign of King Henry VIII until 1698 when it was destroyed by fire. Now all that remains of the original palace is the banqueting house (outside of which Charles I was executed), and a wine cellar hidden away in the basement of the Ministry of Defence. See this map◊ of the major government offices in the neighbourhood.
Merely showing a Whitehall street sign in an Establishing Shot has much the same effect on a British audience as showing The White House, the Pentagon or the U.S. Capitol rotunda has on an American audience - the implied premise is: "What happens next takes place inside a government building."
It should be noted that not all the government buildings are in that specific area — you have some departments in Victoria and the headquarters of SIS (aka MI6), that funky-looking building you see in the 1990s James Bond movies, is in Vauxhall, south of the Thames river. However, most of the particularly important departments — such as the Treasury, the Department of Foreign and Commonwealth Affairs, the Ministry of Defence and the Cabinet Office — can be found in Whitehall.note
If you see a tall clock tower (the one housing Big Ben), the next scene will be inside the Houses of Parliament (though the building itself is technically known as the Palace of Westminster). A huge building, with other office buildings attached, it has miles of corridors and is open for guided tours in the summer (other times you need an MP to arrange it). It's a Royal Palace—so no dying if you're common.
If you see a building with a black door bearing the number '10' on it and a uniformed police officer standing guard outside, then what happens next will be occurring in Number 10, Downing Street, the official residence and offices of the Prime Minister of the United Kingdom. Although it looks like a relatively modest townhouse from the outside, inside it is a really very large network of offices, meeting rooms and the Prime Minister's private quarters. There is actually no keyhole at all and the door only opens from the inside.