The planet Miranda in Serenity is a Shout-Out to William Shakespeare's Tempest and by extension, Huxley's Brave New World. Of course, this is also a reference to real-world naming conventions, as all the moons of Uranus are named after Shakespearean characters, and one moon is in fact named Miranda.
Alan Tudyk's character in V (2009) was impaled by a stake... Sound familiar?
In the pilot, when the crybaby is released, Wash gives a shout out to The Beatles song of that name.
In "Out of Gas", we learn that Serenity's first mechanic was a guy named "Bester." Later we hear Mal call him "Alfred." Alfred Bester was an important author in the post-war, founding era of modern sci-fi (as well as a character in Babylon 5).
In a deleted scene from the pilot episode, Simon's encyclopedia can be heard mentioning a General Richard Wilkins who led the Alliance forces at the Battle of Serenity. Mayor Richard Wilkins was the Big Bad of season 3 of Buffy the Vampire Slayer, also created by Joss Whedon.
"I was aiming for his head." - Avon says exactly the same line with regards to Travis in Blakes Seven.