These are what we call the 'YMMV items.' Things that some people find in this work. We call them 'your mileage might vary' because not everyone sees these things in the same way. This starts discussions in the trope lists, a thing we don't want. Please use the discussion page if you'd like to discuss any of these items.
Some take the view that No. 6 is a Villain Protagonist, destroying Utopia so he can do whatever he wants.
Concerning his MacGuffin resignation; it's believed that those who run the Village do know why he resigned, they simply don't believe it. Flashbacks and dream sequences imply he was routinely involved in murders, and his behavior in the village show's he's clearly excellent at manipulation, even if he wants to be free of it himself. His paranoia starts high and grows over the course of the series. The reason he gave for quitting was as "a matter of conscience", and it's entirely possible that Patriotic Fervor no longer justified the means for him.
If this troper remembers correctly, it's stated early on by one of the Number Twos that they do in fact know why Six resigned, they just want to hear the reason directly from him. It was meant to be just a formality that would lead to other things. Because Six never gave into even that one simple demand, they never got past that most basic phase (or maybe they did, depending on how you interpret the events of the finale).
There is a large contingent of fans who say, given the events in "Fall Out", that the classic call-and answer from the opening credits are being intentionally mis-punctuated, and instead of being read simply as "You are Number Six." in response to the question, "Who is Number One", the iconic line should be read as "You are, Number Six." Meaning that Number Two has been giving the correct answer to the question all along. If true, this would make it the Mind Screw to end all Mind Screws.
Beyond all the above, it's up for grabs whether anyone is a prisoner including Six, a jailer, or if the distinction exists or even matters. At least one Two is a confessed former inmate, but given the context, it may be a lie; several fellow prisoners appear genuinely trying to leave, but they feed into Number Two's plans all the same.
One theory takes this Up to Eleven by saying that the reason Six resigned was because he designed, posited, or discovered the Village in the first place, making him just as much a jailer as a prisoner by implication.
Author's Saving Throw: The comic book sequel, Shattered Visage, reveals that the Gainax Ending of the series finale was a drug-enhanced production filled with actors and staged events, designed to break Number Six's spirit. The comic also returns to the spy-adventure/intrigue genre of the first 12 episodes, exploring how the Village could've existed in the world of espionage. After establishing a level of storytelling more rational than the series finale, the comic focuses on Number Six's character, showing where he is 20 years after the show ended. Fans are split on whether it's a dull, plodding comic that destroys the allegorical madness of "Fall Out," or if the comic rescues the series from incomprehensible nonsense and provides a satisfying epilogue for Number Six.
Of course as to whether it can be considered canon can be in dispute as well, as McGoohan didn't have anything to do with it except to give his opinion as an endorsement (as he didn't own The Prisoner), and he was under contract to not say anything bad about it. He said he "didn't hate it..."
Big Lipped Alligator Moment: The kosho matches (a cross between trampolining and boxing) in It's Your Funeral and Hammer Into Anvil. The bizarre music and absurd outfits only make the scenes stranger.
There were apparently sensible rules to the sport, but it's very hard to tell from what we see.
Bizarro Episode: Basically every single episode after the first 11 - owing to a case of Franchise Zombie. "Bizarre" is relative, but "Do Not Forsake Me Oh My Darling" gives us the first real example in the series. It's immediately followed by "Living In Harmony", in which the entire show (including the iconic opening sequence) is transformed into a Western. The episode after that ("The Girl Who Was Death") turns out to be a bedtime story told by Number 6. The reason for these stories is because the script editor, George Markstein, quit the series and was not replaced. Scripts and story ideas came from random people and places: the Western episode was suggested by a video editor and "The Girl Who Was Death" was an unused script from Danger Man. All this adds to a dissonance of tone and distances the series from exploring life in the Village and Number Six's struggles.
Common Knowledge: When the character of the Prisoner is referenced in other works, it is common to see him placed in his black vest with white piping and the number six lapel pin. This may make serve to make the reference clear, but the original Prisoner took the No. 6 pin off practically as soon as he was given it; he never wore his number willingly, except under extreme duress (like being brainwashed into campaigning enthusiastically for himself in "Free For All").
Crowning Music of Awesome/Ear Worm: Gotta love that title theme. Not that you have much choice. Patrick McGoohan kept asking Ron Grainer to give the original opening theme (demo version here) a little more punch. Grainer responded by punching it Up to Eleven. Seriously, the difference between the first to the second is like the difference between a ride through quiet countryside in a horse-cart, and driving a Lotus Seven along hairpin turns while the waves break against the jagged rocks below.
Just try getting Dem Bones out of your head after Fall Out.
The book which details the production of the series mentions that the roars are a mixture of chanting monks played backwards and a man's scream echoing down a hallway slowed down.
Misaimed Marketing: Many fan sites, as well as the official store in Portmeirion, sell a replica Number 6 badge; you know, like the one McGoohan wore for about two seconds before tearing it off in disgust and declaring he wasn't a number! Though he does wear a "6" rosette for much of "Free For All".
Some places also sell a Number 1 badge, even though no Number 1 badge ever appears in the series.