A Series of Unfortunate Events: An unexpected one, in hindsight: in book number 8, The Hostile Hospital, Violet is compared to Sleeping Beauty. A few years later, her actress Emily Browning would play the lead role in Julia Leigh's "Sleeping Beauty".
The Doctor WhoExpanded Universe novel Prime Time had the Seventh Doctor and his companion Ace attempt to enter a futuristic TV station by posing as an alternative comedian and a children's presenter, the pre-Who careers of the actors who played them.
In Business Unusual, the Doctor reflects that Wilkin, the St Cedd's porter, reminds him of Billy Bunter. In "Shada", Wilkin was played by Gerald Campion, who previously played the title role in the BBC's Billy Bunter of Greyfriars School.
While this might overlap with Expy, the blind Argentine writer Jorge Luis Borges who wrote a story about a library of lost books "appears" as a blind librarian in both The Name of the Rose and in the first book of the Book of the New Sun series. In the former, the character is actually named Jorge de Burgos.
In the Star Trek: Deep Space Nine Relaunch novels, when preparing to insert Iliana Ghemor onto Bajor to replace Kira Nerys, Corbin Entek mentions that the Obsidian Order maintains a database of alien individuals who are similar in appearance to living Cardassians. This is so they can replace those people if they decide it's necessary. Entek mentions that Gul Danar has many different matches in the database, across many species, such as human, Klingon and Romulan. This is a reference to Danar's actor, Vaughn Armstrong, who has played no less than twelve supporting characters of nine different species in Star Trek.
In Star Trek: Articles of the Federation, Chancellor Martok is irritated when Praetor Tal'aura's voice reminds him of his dead wife's. This is an in-joke, as Tal'aura and Martok's wife Sirella were played by the same actress.
Peter David's Star Trek: New Frontier novels feature a complicated one in Morgan Primus: She looks very much like Lwaxana Troi (played by Majel Barrett in the show) and as such also sounds like any Federation ship's computer (voiced by Barrett.) Her mind eventually winds up in Excalibur's computer, which would sound the same but have Morgan's personality, since Morgan is "played by Majel Barrett," as close as a book can get. This can also be a subtle reference to Andromeda, of which Barrett was the executive producer.
It's also implied that Morgan was the original Number One from "The Cage" (also played by, you guessed it, Majel Barrett) given the way she reacts to Shelby being called Number One, the explicit mention of her seeming comfortable in the first officer's chair, and Primus meaning "first".
She also meets Scotty on Risa. Scotty's first reaction is to stare and ask, "Christine?" ...as in Nurse Chapel, yet another way Gene Roddenberry got his wife on the show.
Similarly, again with Peter David (and John de Lancie) in their Star Trek novel I, Q the Ferengi Grand Nagus (who was played by Wallace Shawn on Deep Space 9) repeats - with appropriate alterations - the "classic blunders" line used by Vizzini (who was also played by Wallace Shawn) from The Princess Bride. He also misuses the word "inconceivable", with Data pointing out that it does not mean what he thinks it means.
Similarly, once again with Peter David, his novel Q-Squared had a scene in which Guinan in two alternate universes scream and collapse. In the third, Ten Forward hostess Caryn Johnson goes about her business as usual. Caryn Johnson is the birth name of Whoopi Goldberg, who played Guinan on the show.
Peter David (again?) wrote a new Battlestar Galactica novel called "Sagittarius is Bleeding" where Commander Adama, thinking about Starbuck, says to himself "It's too bad she won't live. But then again, who does?" Edward James Olmos, who plays Adama, said this line in Blade Runner.
In a slight variation, a few of the post-Fleming James Bond novels have Bond mentioning that Sean Connery is his favorite actor.
Ian Fleming himself wrote in one of the last Bond novels that Bond's father was Scottish, in honor of Connery.
Before getting the role of Quark in Deep Space Nine, Armin Shimmerman had a variety of guest spots on The Next Generation, including three different Ferengi, and the face on a Betazed gift box. Years later, he wrote the apocryphal Star Trek novel The 34th Rule, and every single one of his previous characters shows up, ususally prompting some variation on "Say, you look familiar..." from Sisko.
Another Star Trek example: In the Voyager novel The Final Fury, Tom Paris tells Chakotay he has a "nodding acquaintance" with the Starburst Maneuver, and Chakotay "didn't get the reference" - the reference being that Paris's actor, Robert Duncan Mc Neill, played a one-shot character suspiciously similar to Paris in a TNG episode who was kicked out of the academy for performing that maneuver.
A possible one in Harry Potter and the Half-Blood Prince: Professor Slughorn never remembers Ron's name, and at one point calls him Rupert, a possible shout out to Rupert Grint, who plays Ron in the movies.
Word of God says it wasn't. However, in one book, there's a line that mentions "That time Hermione punched Draco" in the third year. Draco was never punched- he was slapped. He was punched in the 3rd film.
In the series of The A-Team novels by Charles Heath, a passer by comments that Hannibal 'looks just like George Peppard'. George Peppard played Hannibal in the TV Series.
In the TV show A Touch of Frost detective Jack Frost is played by comedian David Jason. In the book 'Hard Frost', RD Wingfield, the author, who believed that while David Jason does a very good job as Frost, he isn't his Frost, has Frost's Pointy-Haired Boss Mullet remark about a supermarket opening which will be very good, as the celebrity guest is David Jason!