"Aw, people can come up with statistics to prove anything, Kent. Forfty percent of all people know that!"This well-known saying is part of a phrase often attributed to Benjamin Disraeli and popularized in the U.S. by Mark Twain: "There are three kinds of falsehoods: lies, damned lies, and statistics." Numbers and formulas are supposed to represent "objective scientific data" you cannot deny and examined by intelligent and experienced experts. Now the Consummate Liar wants his forgeries to look undeniably "scientific", so why not use the magic of numbers that the not-so-math-literate masses could never deny? They say that statistics don't lie,note and while that may be true, liars do use statistics. This trope covers all instances of Artistic License Ė Statistics where statistics are used to deceive people as to the truth. The problem is, people do not pay attention to the context, just the numbers. For example, the statement "Brand X is 84% fat-free" sounds good until you realize that this means the food product is 16% fat by weight. Also, "fastest growing" could mean that there used to be one customer and then there were five more, making a five-hundred percent increase. You should also notice Absolute Comparatives: it's fastest growing, but specifically compared to when/what? The whole business of throwing percentages at people in advertising, politics and other forms of propaganda is almost destined for this kind of abuse. Relative measures are more likely to be understood accurately, and thus are less likely to be used in advertising. The bogus uses of statistics are intended to imply a causal link between two elements when they are not linked, the link is questionable, or the link is opposite to what is implied. A beautiful example? "Coca-Cola causes drowning". By looking at statistics on drowning and Coca-Cola sales, you can see a link — more people go swimming on hot days, and more people buy Coke on hot days. Likewise, birth rates per head of population are higher in areas where there are more storks — because birth rates are always higher in rural areas, which is where one finds the Delivery Stork. Correlation does not equal causation; if it does, then we might also conclude that global warming is caused by a decline in pirate population and that 100% of Homo Sapiens who consume dihydrogen monoxide will cease vital functions and decompose. Also be aware of the Law of Very Large Numbers. Any fraction of a very large number is likely to be a large number, no matter how small the fraction is. It is estimated that 2,135,000 Americans have used cocaine (including crack) in the past month. But that's only 0.7% of the population! So, is this a lot of people, or not? You should also be on the lookout for the related effect where things are made more remarkable than they really are. The odds that any given ticket will win a raffle may be very small, but it is certain that one will be a winner. You'd notice being dealt a royal flush in spades at poker, but the odds of it happening are exactly the same as those for being dealt any other hand of five specified cards. Then you can get the kind of statistical abuse in which you are careful to define the question to get you the answer you want. What is the most popular book in the world? Depends if you mean most copies in existence (Quotations from Chairman Mao), most copies ever sold (The Bible) or fastest selling ever (Harry Potter and the Deathly Hallows). Statistics are like studies: who made them and who paid them matters a lot. Want to "prove" that video games cause violence? Get a group of scientists that are already savvy to this and don't mind the lack of ethics. Have them draw from a very small pool of test subjects that are known to display violent behavior. Mental hospitals, prisons, schools for children with behavior disorders, what have you. Do some generic tests that are guaranteed to show up positive, come up with numbers, and presto, instant headline. "Recent test shows 77% of subjects become more violent after playing Mortal Kombat." Most people won't bother with reading the article the whole way through and will just look at the headline. This works with anything from comic books, and rock to watching Brokeback Mountain or voting for specific parties, basically anything. See Push Polling for a specific form of this. Confirmation Bias, or the tendency for people to search out statistics that support their preconceived notions and ignore statistics that don't, is the reason for many of the entries on this list. The forgery mentioned above is also the reason most scientific and medical studies are done double-blind (meaning it's all anonymous, neither the researchers nor the participants know who's in the experimental group nor who's in the control group) and should allow for a chance at being falsified by Real Life (see also The Scientific Method). But you should beware of any advertisements touting a "double-blind" study, especially late-night ads, because they tend to violate the truth-in-advertising laws. In the end, statistics are not lies and statistics don't lie: people lie about the statistic itself or how it is interpreted. Some don't lie, they are simply ignorant, as are most members of the public in terms of statistical interpretation. See Logical Fallacies and Critical Research Failure. Put another way, by baseball announcer Vin Scully:
— Homer Simpson, The Simpsons ("Homer the Vigilante")
"People use statistics the way a drunk uses a lamp post — for support, not illumination."See also Nine out of Ten Doctors Agree, which is much a sub trope to this, and Absolute Comparative, where use of statistics is avoided entirely by comparing the product to nothing. When adding examples, please remember that this trope is not about amusing statistical fallacies, but about using statistics to deceive people as to the truth. For politics-related examples, you can find an abundant amount at PolitiFact (search for any Mostly False or lower verdicts) and Washington Post Fact Checker (3 and above Pinocchios)
open/close all folders
- Nine out of Ten Doctors Agree that the phrase "Nine out of Ten Doctors Agree" has been practically a stock phrase in advertising since the early 20th century.
- "Nine out of ten dentists recommend Trident for their patients who chew gum." The tenth dentist was insistent that his patients never chew gum at all, but surprisingly, Trident didn't want you to know about that.
- One interesting case happened in Portugal, where two ads were being broadcasted on national TV during the same period (and sometimes even in the same commercial break) claiming, respectively, that '90% of dentists use toothpaste X' and '8 out of 10 dentists recommend toothpaste Y to their family'. Together, if you stop to think about it, they imply something is not quite right about those professionals' concern over their own family...
- Or that an awful lot of dentists are unmarried orphans, hence can't recommend it to a family they haven't got.
- In a similar vein, a commercial for Five Hour Energy states that 4 out of 5 doctors wish for their patients who use energy supplements to use low-calorie energy supplements. Think about that: They specify patients who already use energy supplements, meaning they didn't count any doctors who recommend that their patients not use energy supplements at all.
- A Trojan Condoms commercial claims that the United States ranks between two African nations in HIV cases. This means nothing, since the population of the USA is much higher than either of those countries!
- An old advert for Guinness ran with the quote "88.2% of statistics are made up on the spot", attributed to Vic Reeves.
- And even 95.639055252364% of these with ridiculous accuracy. (Source unknown)
- Fletcher Knebel was apparently responsible for "Smoking is the leading cause of statistics", the most famous of which is "100% of non-smokers die".
- In Montreal, there was an ad campaign run by a gum company whose gum came in round shapes instead of the usual square shapes. The ad said, "100% of people who chew square gum die."
- Ever wonder how all car insurance companies manage to to advertise that "people who switch from <competing company> to <our company> save an average of <large amount of money or substantial percentage>"? It's because the sample population "people who switch" is almost entirely composed of "people who are going to save a big chunk of money doing so", or else why would they bother to switch? Since no record is kept of the percentage of people who would not save any money and therefore don't switch, the cited statistic has almost no meaning.
- As the Great Cable TV/Internet/Voice wars began ramping up, they tended to fish from similarly Small Reference Pools for their commercials, typically along the lines of "Of all customers who switched from <company> to <company>, over 53% that switched back did so because they realized they were paying less before." Of course this is said in rapid-fire and/or low-key speech, while the 53% is emblazoned on the screen, making you think that half of the people that switched had come back, instead of that being a sub-statistic OF those that switched back: that figure is never stated.
- A banner ad right here on TV Tropes promotes The Church of Happyology, rhetorically asking, "Why is it the world's fastest growing religion?" Hmmm, maybe because this religion that cannot be named for legal reasons was founded less than 100 years ago, and most major religions are thousands of years old and likely have grown as much as they ever will? Or because they count ad clicks as memberships.
- An ad for an internet phone service claimed that you could save $200 a month by switching, citing a chart showing all the associated fees and pricing. The product for sale listed a price of $25 a month with no installation or other fees, while the sample phone company was cited a price of $120, with $80 in associated fees. Disregarding the fact that you would only save $175 if taken at face value, the price of $120 was for six months of phone service, and the fees would have only been paid once. Looking at just price per month, the internet phone cost more than the sample company.
- A cellular phone service in the US claims to have cheaper phone bills... if you bring your own phone. If you buy a phone from them, you have the option of buying the phone outright, or paying it off in two years, which brings the bill up to what the bigger carriers charge for a similar deal.
- The term "unlimited bandwidth" that internet and cellular phone service providers love to advertise has gotten a lot of bad flak because people have found out yes, there is actually a limit to how much bandwidth you can consume from your provider before they start taking punitive action. Perhaps it was "unlimited" when speeds were so slow you couldn't manage to breach it even if you constantly were downloading something.
- There's a billboard along a freeway in Michigan, advertising a dentist willing to do implants for only a couple hundred dollars. Next to it is a statement saying "Voted best dental office in Michigan". Just below that, in smaller text, is the disclaimer "by our dental staff".
- The German ad for Fisherman's Friend Mint bonbons said nine of nine real men liked Fisherman's Friend bonbons. Shortly after throwing one of their test subjects over board.
Anime and Manga
- An old Archie story had one of the characters becoming a statistics-obsessed nut for the duration of the story, only for Jughead to start citing statistics that horrified them and lead them to run away in fright, at which point Forsythe noted that some ridiculously high percentage of people who quote statistics "make 'em up on the spot!"
- Batman: The Penguin says this quote word-for-word in Detective Comics #684, at the same time pulling a You Have Failed Me on a newly-acquired henchman who, through usage of statistics, "proved" that a broad daylight robbery had a 0% chance of being foiled by Batman.
- Judge Dredd: The Dark Judges famously use the statistic that none of the people they execute will ever commit another felony again as proof that death is the cure for crime.
- In Dilbert, Dogbert once sold "value-priced lottery tickets;" half price, odds of winning only one in ten million less.
Sucker: Hey! This is for yesterday's lottery!
Dogbert: And your point is...?
- Darrell Huff's "How To Lie With Statistics" was printed in the '50s. It's
usually available on eBaystill in print and is a very easy read that shows you all the basics.
- Spoofed by America (The Book), which included a graph on "Growth of Misleading Charts". Two different bar heights represent the same number.
- A book entirely revolving around poking fun of this is Spurious Correlations. A spurious correlation is when two variables appear to be related, but one variable is clearly not causing change in the other. The author used statistical software to find correlations between variables that had nothing to do with each other, like finding out that the divorce rate in Maine is positively correlated with the amount of margarine consumed in the state.
- Yes, Minister:
- There's a very interesting section on this. In a discussion about conscription, Sir Humphrey demonstrates to Bernard how statistics can be obtained which prove both sides of the discussion correct, through the use of leading questions which are not included in the reporting of the survey concerned.
- Another episode combines this with Hypocritical Humor; during one of their many arguments, Hacker brings up some facts to support his point only for Humphrey to superciliously note that Hacker's facts are based on statistics, which are thus unreliable as per this trope. When the argument gets a bit more heated, however, Humphrey begins to cite some statistics that prove his point, only to catch himself and quickly refer to them as 'facts'. Hacker is quick to point out the hypocrisy.
- ABC loves to put out press releases trying to make the previous night's viewership of their shows look good. With their big hits, this isn't so bad; winning the timeslot in total viewers or Adults 18-49 (the demographic used to set ad rates, and thus the figure used to determine whether or not a show gets renewed) is definitely something to be pleased about (unless there's a huge skew between total and A18-49 viewership, but that's another matter). No matter how poorly-watched a show is, however, the ABC PR department can find some figure that sounds good but doesn't actually mean the show's doing well. They frequently put up demographics that aren't really indicative of a show's survival (e.g. women 18-34, or the adults 25-54 demo that only some cable channels use for ad rates), give the amount that the show built on its lead-in (usually when the lead-in was a repeat or another low-rated show, or a repeat of another low-rated show), or claim the the show had the best performance in its timeslot among ABC shows since X weeks/months/years ago (when you look at the absolute ratings, all it says is that ABC's done even worse in the past; this is rarely used for hit new shows since there are usually better statistics for those).
- Programs on Animal Planet are fond of citing how Americans spend more money annually on cat or dog food than on baby food. This is depicted as evidence that Americans pamper their pets like babies, but overlooks several facts: that pets eat pet food for their entire lives, whereas babies only eat baby food for about a year and a half, and that many families have more than one pet at a time, but relatively few have more than one child of an age to eat baby food at the same time. Also, a baby often consumes breast milk from the mother which wouldn't show up in the statistics when calculating the cost of baby food. It also doesn't take into account the many people who make their own baby food, or feed their babies foods like applesauce or bananas that are already nice and soft; pet food is significantly harder to prepare.
- QI is well known for deliberately phrasing questions like this in order to confuse the participants. One question was "What is three times more dangerous than war?" The answer given was work, because three times as many people are killed each year in work-related accidents than die in wars. Now, consider how much time you spent working last year compared to how long you were in a warzone. This prompted unhelpful responses from the panelists: "What if you're a soldier?" "What if you work in a shoe shop, near a war?"
- Monty Python's Flying Circus parodied the use/abuse of meaningless statistics in the sketch "Spectrum":
Man: This bar in this column represents seventeen percent of the population. This one represents twenty-eight percent of the population! And this one represents forty-three percent of the population!Host: Telling figures indeed. But what do they mean to you? What do they mean to me? What do they mean to the average man on the street?
- Parodied in Australian political satire The Dingo Principle:
Listen to this: 40% of people support the Prime Minister and 50% support the Opposition.That's only 90%.Yeah, there was one guy who said the samples weren't big enough to be statistically significant.
- Penn & Teller: Bullshit! point it out by having a man who makes poll research for the Republicans show he can make someone give two different answers to the same question by first asking: "Do you think the government expends too much in health care for immigrants?" The bystander answers "Yes". When he asks: "Would you deny an immigrant the right to treat himself? To give birth in a hospital? Etc..." and other medical services that go well beyond what the governments expends with immigrant health care, the answer now is: "No". Also, they make fun of the guy with his own mathematical wizardry by pointing out: "In this scene, ten cars pass by behind him. One guy from one of the cars shouts saying he sucks. This means that AT LEAST 10% of the American population believes he sucks".
- In an episode of Psych, young Shawn attempts to justify his fear of shark attacks with the statistic that most shark attacks happen in about three feet of water. His dad points out that most swimming in general happens in about three feet of water.
- The Arrogant Worms' song "Don't Go Into Politics" recommends that audiences avoid the fields of politics, science, and music. To support their argument, it points out that people like George Washington, Albert Einstein, and Leonard Bernstein did so, and they're all dead.
- Newspapers love doing this with drug related stories. It's almost impossible to see Ecstasy mentioned in a British newspaper without the qualifier "That killer drug", the supporting statistic is that a dozen people die per year from consuming it. Yet over the course of a year they will rack up an impressive body count in stories about fatal car accidents without ever devolving into calling cars "Those murderous rampaging kill bots" How many Britons drive? How many take E? And how many deaths are caused by legal drugs? - cigarettes, alcohol, over the counter medications, misused prescriptions... And that's not even counting the fact that comparing legal and illegal things is a comparison of apples and oranges. Alcohol was much more dangerous during prohibition as it often wasn't pure and alcohol concentrations were often higher than usual as this was more interesting to smuggle. If Ecstasy were legal people would be educated about its use and avoid harmful interactionsnote , impurities would not be an issue, an accidental overdose from misjudging the dose impossible and it would probably be sold in 25-50mg pills instead of 200mg. Taking those factors into account in the statistics wouldn't help to support the viewpoint of the author, so they are generally left out.
- An Ars Technica article discussed the statistics usually used by software developers to complain about piracy. Specifically, the article pointed out that the statistics most commonly cited are most likely not only bullshit, but old bullshit. Amusingly enough, the image used for the related post on Gamepolitics was a pie chart divided into three sections, marked "Lies/Damned Lies/Statistics".
- The Column 8 column in the Sydney Morning Herald once referenced a statistical correlation between the difficulty of the sudoku on a given day and the price of petrol.
- There is a semi-famous magazine article from 1958 called "The Dread Tomato Addiction" that correlates consumption of tomatoes with everything from death to communism. It can be found here. There is a similar article about bread that can be seen here.
- There is a book produced for people in radio every year that compiles countless statistics about all stations taken from polls. These are used to attract advertisers. The less successful stations who have very few listeners are often forced to hire people who read through the book to get as many favorable statistics as possible, no matter how convoluted they may be. With the huge amount of data in the book, it's possible to say, for instance, that 85% of married men aged some arbitrary amount with income in some arbitrary range and who own a ferret will love your show, even though they represent a tiny proportion of the population. Of course, if you're selling ferret food, that's exactly whom you want to reach.
- The Australian Football League managed to turn an increase in positive drug tests into a decrease in positive drug tests. 
- Two sports teams each only play one game in a season, and that game is against each other. How does the losing team make themselves sound good and their opponents sound bad? They say they only lost one game all season, and that their opponents only won one game all season.
- Similarly in a two-horse race, one comes second and the other next to last.
- A joke about a U.S.-U.S.S.R. athletic event. When the American wins, the Soviet media reads that the Soviet athlete finished second, but the American finished next-to-last.
- A stand-up comedian once said, "There is a direct correlation between being a serial killer, and being born. Show me one murderer who was never born."
- One of Bothered About Dungeons & Dragons (BADD)'s favorite weapons was a list of over 500 Americans they claimed were gamers who had committed suicide in the same year. Thus role playing games somehow cause suicide. Except that even if you take this bogus statistic at face value, 500 suicides a year is still a lower percentage of suicides than clergy and a tiny fraction of the average.
- Patricia Pulling, the leader of this organization, once said in an interview that "8% of the Richmond VA-area population is involved with Satanic worship at some level." When asked where that figure came from, she said that she estimated that 4% of the teenagers and 4% of the adults. She then added them together and got the 8%.
- Another time, BADD cited an increase in suicides corresponding with a major Dungeons and Dragons release. Again, however, there's no evidence that's not simply a coincidence, as similar statistics can be used to prove that the release of a Britney Spears CD caused suicide numbers to jump.
- Saturday Morning Breakfast Cereal tells us how to make anything a victimless crime.
- This strip concludes that the news media got the supposed relationship between cell phones and cancer backwards, i.e., cancer causes cell phones.
- Also this one, which combines it with a dash of Hypocritical Humor:
Newscaster: ... and in science news, according to a new study, 85% of news organizations repeat "new study" press releases without checking whether they're real.
- And one about the significance of 95% confidence when you run 20 tests and only publish the interesting one.
- Another comic points out that the observed mortality rate among humans is only 93%, overthrowing most "100% of people who do X die" statistics by. . .mere statistics.
- Luke Surl expresses this via Visual Pun here.
- The 5 Most Popular Ways Statistics Are Used to Lie to You covers some fallacies commonly used to lie with statistics.
- 8 obvious signs statistics are lying to you.
- Often used in the News Parody Chigüire Bipolar for example in Maduro has 65% of Maduro's popular support, according to polls.
- The Onion does parody this from time to time.
- An article was about a movement to shut down hospitals because "despite rapid advancement in medical technology, the world death rate remains at 100%."
- Another article said that children are universally opposed to children's health care, with responses to the question "Do you want to go to the doctor?" ranging from "NO!!!!" to "inconsolable crying," but no children in favor.
- The Boondocks episode "The Color Ruckus" showed Uncle Ruckus as a child, being homeschooled by his mother, who at one point made this claim about George Washington Carver:
Bunny Ruckus: "George Washington Carver was the man responsible for more peanut allergy deaths than any man in history!"
- An episode of The Simpsons featured Homer forming a vigilante group to fight crime. At one point he recruits Jimbo (who calls the group "the drunken posse") on the basis that he can swing a sack full of doorknobs. Homer later gives an interview to the local news:
Kent Brockman: Mr. Simpson, how do you respond to the charges that petty vandalism such as graffiti is down eighty percent, while heavy sack beatings are up a shocking nine hundred percent?Homer: Aw, people can come up with statistics to prove anything, Kent. Forfty percent of all people know that.
- The Justice League was asked "Maybe you'd care to explain why on your watch, 50% of marriages now end in divorce, and the other 50% end in death!" Aside from the fact that the same was true before the formation of the League, until the end of time, a significant portion of marriages will end in death, as people do have a tendency to die, married or not. That and one can get divorced multiple times - absent resurrection, one can only die once.
- During World War I, helmets were almost withdrawn from British soldiers. When Britain started issuing steel helmets to all soldiers on the western front in 1916, generals began to call for their removal as they increased incidences of headwounds twelvefold and doubled total casualties. The reason? If someone gets hit in the head by some woolly bear or flying frog (German H.E. or rifle grenade) shrapnel and lives it's a "head wound" and if they are unable to fight, the person is a "casualty"; if they die from a bullet in the brain, then they are a "fatality" and so don't appear on casualty statistics. Since helmets let more people survive, the number of head wounds soared. A politician used the numbers to support his position that "helmets are expensive and cause cowardice" (which, as it turns out is exactly the opposite of the kind of behavior helmets encourage, see below), and never explained what it really meant - doubly effective as most people don't know the difference between "casualty" and "fatality". The real justification behind the attempt to withdraw helmets narrowed down to, "all a dead soldier needs is a funeral." A wounded soldier gets dragged out of combat by at least one of his buddies, and then provided weeks, months or even years of medical attention. From a statistical standpoint, adopting helmets drastically increased the effectiveness of enemy weapons - and a lot of WW1 generals genuinely believed in disposable personnel. Luckily, more ethical parties changed the way they recorded casualties, or the helmets would likely have been recalled.
- Likewise, the number of cyclists being treated for head wounds have increased massively since wearing a helmet became more widespread. Of course this is because they previously wouldn't have survived the accident at all.
- Situations like the cyclists mentioned above also can also be attributed to the Peltzman Effect, aka Risk Compensation. This is the effect of a person being aware of greater safety ("I don't have to worry about hitting my head, I'm wearing a helmet!"), and taking greater risks due to the perceived increase in safety. So there actually could be an increase in injuries, due to more people taking greater risks.
- Sick as it was, WW1 quickly devolved into a stalemate, at which point it was clear that the loser would be whichever side ran out of men or money first. With sufficiently large draft pools, conserving resources was more important than conserving manpower.... until 1915, that is, when they began to realise just how shallow their manpower-reserves actually were relative to the demands of the war. By 1917 they were basically doing everything they could to conserve their manpower, to the point that they produced enough helmets to equip even the logistics personnel (i.e. people who might but probably won't be facing shrapnel, bullets, etcetc).
- At some Reform Judaism synagogues, a popular "joke" to lead into the sermon is, "x% of deaths occur in a hospital, x% of deaths occur in a car, x% of deaths happen in the home...[continues on for a while] while there have been only three deaths in a synagogue, and no deaths ever reported while studying Torah! Clearly, the safest passion, therefore, is studying Torah."
- A related strategy was used by US president Richard Nixon to portray marijuana as a gateway drug. His anti-drug team estimated that 80% of marijuana users go on to use cocaine; actually, 80% of cocaine users had started with pot, but only about one in 2,400 marijuana users (just under 0.042%) go on to use cocaine. And related to that, most people willing to take a drug as dangerous as cocaine are willing to take a drug as relatively safe as marijuana. It's the same reason most marijuana smokers have drunk alcohol at some point, and why most alcohol drinkers have drunk something with caffeine in it. Water: the Gateway Liquid.
- One study cited by this article has experts ranking various drugs by their relative danger to individuals and those surrounding them. The original study appears to be fairly legitimate, yet the article commits a statistical sin of simply adding the value of 'harm to self' and 'harm to others' together to rank each drug. There is solid evidence that marijuana itself is very safe if vaped, as opposed to smoked. The same goes for nicotine, which was also found by the same study, commissioned by Public Health England, to be relatively harmless when the pure extract was vaporized. Alcohol is often perceived as less harmful than marijuana and cigarettes, despite the dangerous behaviours it can create.
- When Ronald Reagan's Attorney General Edwin Meese wanted "proof" that pornography was evil, he created the Attorney General's Commission on Pornography. The commission members were a preselected cohort of anti-pornography campaigners. Not surprisingly, they discovered that statistics "proved" that pornography caused crime. However, the 1970 report of the President's Commission on Obscenity and Pornography, which was done by honest researchers and was highly praised for accuracy and honesty, discovered that there was "no evidence to date that exposure to explicit sexual materials plays a significant role in the causation of delinquent or criminal behavior among youths or adults."
- In the heated German censorship debate about blocking sites allegedly containing child pornography, an organization in favor of this censorship law ordered a survey at a market research institute with questions asking if the person taking the survey is against child pornography and in favor of blocking the websites containing it. Over 90% answered 'Yes'. Another survey ordered by an opposing NGO — at the same institute no less — used a slightly different phrasing: Do you agree with blocking the content despite the fact that this content still exists and is easily accessible after the censorship? Over 90% answered with 'No'.
- Many casinos like to advertise their slot machines with lines like "Up To 99% Payout!" to make it sound like the player has a good chance to win. First, "up to" means the payout could be 1% for all you know (although laws usually set a minimum). Secondly, even a 99% payout means that for every $100 you put in the machine, on average, you'll get $99 back, i.e. you still lose. That "99% payout" is also an average that is based on something like one million pulls (plays) on the machine. If you play 100 times in one slot machine, you're not getting a representative sample of that average. These machines work differently in the UK. UK Fun With Prizes are required by law to seek their set hold percentage within a certain number of spins (usually 10,000). To achieve this, they naturally cheat all the time. They also can be, and often are, programmed to go on a suck cycle and take in way more money then they need to, in order to save up for a large series of payouts later.
- A machine may have one payoff rate if you bet a single coin per spin (which most casual gamblers do) and a completely different rate if you bet the maximum coins per spin. For example, the payout for a video poker game might be $1000 for a royal flush if you bet one coin, but $7500 if you bet five. The advertised payout rates assume the player is playing maximum coins, so a casino can have a slot machine that has a completely legal and legitimate payout ratio of one hundred percent and still manage to make money on it.
- Truth in Advertising laws require that if a set of machines is advertised as "Up to 99% Payout," then at least one of them must have 99% payout. Though there may very well be 50 other machines with 10% payout. (And don't expect that the "lucky" machine will be marked in any way, or even that it'll be the same machine each day.)
- The history board on roulette tables gave the illusion that the previous numbers the ball has landed in means that it should have a higher chance of landing on a number not on the board. Except the roulette table has no concept of memory and the ball has an equal chance of landing on the same number as before no matter how many plays were made. If the roulette wheel has any bias at all, it's more likely in favor of the numbers on the board — but don't bet on it, as the gambling industry works very hard to ensure their devices are all truly random.
- A common problem encountered is Simpson's Paradox, best demonstrated by example: Suppose Hospitals 1 and 2 are nearby, but 1 is better equipped for treating people with severe injuries, so proportionally more of the people taken there are badly hurt. It does better at treating badly hurt people than hospital 2, and also does better at treating people who are not badly hurt. However, since people who're badly hurt are more likely to die than people who're not badly hurt whether or not they go to hospital 1 or hospital 2, hospital 1 may still have a higher overall death rate.
Simpson's Paradox is when data shows one trend, but dividing it into categories shows the opposite trend. In the example above, hospital 1 has a higher death rate, but if the patients are split into categories based on severity of injury, it has a lower death rate in each category.
- The same goes with good doctors and bad doctors, as told in the book Super Freakonomics. Good doctors are generally given tougher cases while bad doctors are given easier cases. However, if you look at death rates you see that some doctors have higher death rates, but these are usually the good doctors. Patients with serious cases are more likely to die, so good doctors lose a lot of their patients than, say the doctor who cures hiccups. The lesson is that you can be fairly certain that the doctor you receive at a hospital is competent enough to be assigned to you.
- Italy got Südtirol, which used to be a part of Austria, to be added to their territory after World War II by using this kind of statistics to convince the Americans that the area was mostly populated by Italians. Which it wasn't. To this day, most of Sudtirol's population speaks German for a first language and watches German and Austrian TV, rather than the Italian channels.
- Wolf Blitzer on polling information about the health care debate in American politics:
We did that poll CNN Opinion Research Poll, that said, "You like this health care bill or not like it?" We just assumed, a lot of us, that the people who said they didn't like it didn't like it because it was too much interference, or too much taxes or whatever. But if you take a closer look at people who didnít like it, about 12% of those people who said they didnít like it thought it didn't go far enough. They wanted a single payer option, they wanted the so-called public option, they didnít like not from the right, they didnít like it because it wasnít left or liberal enough. Thatís how you got 50% of the American people who said, "We donít like this plan." But only about 40 or 38% were the ones who said it was too much government interference.
- In the 2004 US Presidential Election, Dick Cheney and John Edwards stated conflicting numbers regarding the Iraq war's casualties... and both men were partially right! Skip down to "90% of casualties".
- One statistic used to justify the creation of The Comics Code Authority was that a large percentage of criminals liked to read comic books, ergo, comic books influenced people to become criminals. Nobody pointed out that they were using the wrong statistic - they should have been asking what percentage of regular comic book readers became criminals.
- Similar was the observation by Frederick Werthram, author of "Seduction of the Innocent", that most juvenile delinquents were comic book readers, which he believed was an indication that comics caused juvenile delinquency. Had he checked what percentage of juveniles overall were reading comics, this would have become meaningless, as during the early to mid-fifties, damn near every kid in America was reading comics.
- The Victorian-era belief that masturbation could drive men insane was derived from a similar error, in that mental asylums reported frequent Dates With Rosie Palms among inmates. The fact that mental illness can impair inhibitions against such behavior wasn't considered, nor the fact that men locked up in an asylum had few other respites from misery, frustration or boredom. Most importantly, nobody had the nerve to ask about the masturbatory practices of men who weren't institutionalized.
- When Anthrocon decided to move from Philadelphia to Pittsburgh, one blogger who protested the move cited that only a tiny number of people of people would be as likely or more likely to attend Anthrocon if it moved to Pittsburgh. The organizers, however, heeded a different statistic: those that lived so far away that the move made little difference.
- badscience.net occasionally shows how statistics get misused. For example, here (on small samples it's quite possible that B isn't significantly different from A or C, but you can put it as "B isn't different from A, C is different from A, so we see that C is different from B", which is wrong) and here (limit the view to one of many multipliers which per se can't prove anything). Unsurprisingly, the areas with traditional relations to snake oil trade suffer most.
- Cryptozoology buffs are fond of citing the fact that new species of animal are still being identified with some frequency, and alleging that this means many other "hidden" species must exist under mainstream scientists' noses. They conveniently overlook the fact that most such species discovered in the last few decades are either found in isolated locales where no biologist had previously looked for new species (caves, obscure jungle canyons, deep-sea ecosystems, tiny isolationist nations), or are "found" when DNA analysis reveals that what had been considered one species is, technically, two (e.g. African forest elephants being genetically distinct from plains elephants). Not to mention the fact that the vast majority of new species are things like insects, small birds and reptiles, or deep sea fish, and not the magnificent megafauna monsters that cryptozoologists crave.
- The previous Iranian government (led by infamous president Ahmadinejad) was notorious for this. In a very obvious example they reported the unemployment to have decreased by 50% while other independent sources (let alone the fact that more people were well ... UNEMPLOYED compared to before) suggested otherwise. Later it was discovered that they had changed the definition of an "employed" person from "one working at least 20 hours a week at a paying job" to "one working at least 2 hours a week". By removing the "paying" condition and cutting the time to one-tenth, they had managed to include people doing a variety of voluntary works and kids helping in family businesses a couple of hours each day (and still only managed to reduce the unemployment percent by 50% which goes to show how messed up their work was).
- They did the same when calculating inflation percent. When every source (from independent economists inside the country to World Bank) was reporting a point to point inflation of above 40% (as was apparent in increased prices everywhere) the official sources reported an inflation of 20% or less. How did they do it? By removing some essential items like rice from item basket (the selection of items whose prices were used to calculate inflation, originally including 300 items, mostly household and food products) and instead adding useless items like unpopular electronic devices (like house alarms) to the basket. This had a two fold effect: 1) It removed the most popular items that naturally experience a larger increase in price during inflation and 2) Added items with little increase in their price, increasing the population without increasing the calculated inflation.
- In fact they were so bad at this that pictures of Ahmadinejad showing graphs (without title or source) and saying "I HAVE PROOF" is now a joke in Iran used to show when someone talks bullshit without evidence to back it up.
- A commonly-cited factoid about the American Revolution is that roughly 1/3 of the residents of the Thirteen Colonies favored independence from Britain, 1/3 opposed it, and 1/3 were undecided or apathetic. The comedy series History Bites (based on the premise: what if TV had been around for 5,000 years) parodied Tom Paine as a spin-doctoring pundit:
TOM PAINE: Only 1/3 of the colonists are opposed to independence. Now, you can't let a minority opinion like that influence public policy!INTERVIEWER: But the same number are in favor of independence.TOM PAINE: But now we're talking half of decided voters, which is essentially a majority. You can't ignore the wishes of half of decided voters!
- The original "statistic" doesn't come from any actual poll anyway... it was an estimation made by John Adams, and he admitted he'd not done any research on that, just that it was his feeling on the matter.
- The Church Of The Flying Spaghetti Monster has semi-famously pointed out the obvious correlation between the decreasing number of pirates worldwide and Global Warming.◊
- Something of a historical subversion: During World War II, the Royal Air Force wanted to add more armor to their planes, but because of weight limits they needed to know which places needed the armor most. So, they examined the planes after they came back and counted how often bullet holes were found in certain areas... and then placed armor in places that showed the fewest bullet holes. This is because they'd spotted the flaw in their sample group; all they had to examine was planes that had come back. The data did not show, as might be assumed at first glance, places that planes were most likely to be shot. It showed places that planes could be shot and still fly home.
- Research into Psychic Powers in the 1970s ram up against this trope when people who (by pure luck) scored well on card-guessing procedures were singled out for re-testing under closer observation. Unsurprisingly, repeated tests found their apparent "powers" didn't work at all the second time around. Had the researchers re-tested the entire population of subjects again, not just the lucky guessers, they'd have found a similar proportion of high-scoring subjects randomly distributed among the volunteers. Instead, the idea that being observed made psychic powers wane was propogated to account for the "mysterious" decline.
- Glurge-y Facebook spam will often end with something to the effect of, "Only 3% of your friends will be brave enough to share this," effectively trying to guilt the reader into helping the spam proliferate.
Of the 99 people polled by Trope Co., 45% said that exactly 80% of this page was insightful, 12.3549966% of the time!