"There are three kinds of falsehoods: lies, damned lies, and statistics."
Numbers and formulas are supposed to represent "objective scientific data" you cannot deny and examined by intelligent and experienced experts. Now the Consummate Liar wants his forgeries to look undeniably "scientific", so why not use the magic of numbers that the not-so-math-literate masses could never deny? They say that statistics don't lie,note and while that may be true, liars do use statistics.
This trope covers all instances of Artistic License Ė Statistics where someone manipulates statistics to present a misleading picture of the truth. The problem is, people do not pay attention to the context, just the numbers. For example, the statement "Brand X is 84% fat-free" sounds good until you realize that this means the food product is 16% fat by weight. Also, "fastest growing" could mean that there used to be one customer and then there were five more, making a five-hundred percent increase. You should also notice Absolute Comparatives: it's fastest growing, but specifically compared to when/what?
The whole business of throwing percentages at people in advertising, politics and other forms of propaganda is almost destined for this kind of abuse. Relative measures are more likely to be understood accurately, and thus are less likely to be used in advertising.
The bogus uses of statistics are intended to imply a causal link between two elements when they are not linked, the link is questionable, or the link is opposite to what is implied. A beautiful example? "Coca-Cola causes drowning". By looking at statistics on drowning and Coca-Cola sales, you can see a link — more people go swimming on hot days, and more people buy Coke on hot days. Likewise, birth rates per head of population are higher in areas where there are more storks — because birth rates are always higher in rural areas, which is where one finds the Delivery Stork. Correlation does not equal causation; if it does, then we might also conclude that global warming is caused by a decline in pirate population and that 100% of Homo Sapiens who consume dihydrogen monoxide will cease vital functions and decompose.
Also be aware of the Law of Very Large Numbers. Any fraction of a very large number is likely to be a large number, no matter how small the fraction is. It is estimated that 2,135,000 Americans have used cocaine (including crack) in the past month. But that's only 0.7% of the population! So, is this a lot of people, or not?
You should also be on the lookout for the related effect where things are made more remarkable than they really are. The odds that any given ticket will win a raffle may be very small, but it is certain that one will be a winner. You'd notice being dealt a royal flush in spades at poker, but the odds of it happening are exactly the same as those for being dealt any other hand of five specified cards.
Then you can get the kind of statistical abuse in which you are careful to define the question to get you the answer you want. What is the most popular book in the world? Depends if you mean most copies in existence (Quotations from Chairman Mao), most copies ever sold (The Bible) or fastest selling ever (Harry Potter and the Deathly Hallows).
Statistics are like studies: who made them and who paid them matters a lot. Want to "prove" that video games cause violence? Get a group of scientists that are already savvy to this and don't mind the lack of ethics. Have them draw from a very small pool of test subjects that are known to display violent behavior. Mental hospitals, prisons, schools for children with behavior disorders, what have you. Do some generic tests that are guaranteed to show up positive, come up with numbers, and presto, instant headline. "Recent test shows 77% of subjects become more violent after playing Mortal Kombat." Most people won't bother with reading the article the whole way through and will just look at the headline. This works with anything from comic books, and rock to watching Brokeback Mountain or voting for specific parties, basically anything. See Push Polling for a specific form of this.
Confirmation Bias, or the tendency for people to search out statistics that support their preconceived notions and ignore statistics that don't, is the reason for many of the entries on this list. The forgery mentioned above is also the reason most scientific and medical studies are done double-blind (meaning it's all anonymous, neither the researchers nor the participants know who's in the experimental group nor who's in the control group) and should allow for a chance at being falsified by Real Life (see also The Scientific Method). But you should beware of any advertisements touting a "double-blind" study, especially late-night ads, because they tend to violate the truth-in-advertising laws.
In the end, statistics are not lies and statistics don't lie: people lie about the statistic itself or how it is interpreted. Some don't lie, they are simply ignorant, as are most members of the public in terms of statistical interpretation. See Logical Fallacies and Critical Research Failure.
Put another way, by baseball announcer Vin Scully:
When adding examples, please remember that this trope is not about amusing statistical fallacies, but about using statistics to misrepresent the truth.
- Nine out of Ten Doctors Agree that the phrase "Nine out of Ten Doctors Agree" has been practically a stock phrase in advertising since the early 20th century.
- "Nine out of ten dentists recommend Trident for their patients who chew gum." The tenth dentist was insistent that his patients never chew gum at all, but surprisingly, Trident didn't want you to know about that.
- One interesting case happened in Portugal, where two ads were being broadcasted on national TV during the same period (and sometimes even in the same commercial break) claiming, respectively, that '90% of dentists use toothpaste X' and '8 out of 10 dentists recommend toothpaste Y to their family'. Together, if you stop to think about it, they imply something is not quite right about those professionals' concern over their own family...
- Or that an awful lot of dentists are unmarried orphans, hence can't recommend it to a family they haven't got.
- In a similar vein, a commercial for Five Hour Energy states that 4 out of 5 doctors wish for their patients who use energy supplements to use low-calorie energy supplements. Think about that: They specify patients who already use energy supplements, meaning they didn't count any doctors who recommend that their patients not use energy supplements at all.
- A Trojan Condoms commercial claims that the United States ranks between two African nations in HIV cases. This means nothing, since the population of the USA is much higher than either of those countries!
- An old advert for Guinness ran with the quote "88.2% of statistics are made up on the spot", attributed to Vic Reeves.
- And even 95.639055252364% of these with ridiculous accuracy. (Source unknown)
- Fletcher Knebel was apparently responsible for "Smoking is the leading cause of statistics", the most famous of which is "100% of non-smokers die".
- In Montreal, there was an ad campaign run by a gum company whose gum came in round shapes instead of the usual square shapes. The ad said, "100% of people who chew square gum die."
- Ever wonder how all car insurance companies manage to to advertise that "people who switch from <competing company> to <our company> save an average of <large amount of money or substantial percentage>"? It's because the sample population "people who switch" is almost entirely composed of "people who are going to save a big chunk of money doing so", or else why would they bother to switch? Since no record is kept of the percentage of people who would not save any money and therefore don't switch, the cited statistic has almost no meaning.
- As the Great Cable TV/Internet/Voice wars began ramping up, they tended to fish from similarly Small Reference Pools for their commercials, typically along the lines of "Of all customers who switched from <company> to <company>, over 53% that switched back did so because they realized they were paying less before." Of course this is said in rapid-fire and/or low-key speech, while the 53% is emblazoned on the screen, making you think that half of the people that switched had come back, instead of that being a sub-statistic OF those that switched back: that figure is never stated.
- A banner ad right here on TV Tropes promotes The Church of Happyology, rhetorically asking, "Why is it the world's fastest growing religion?" Hmmm, maybe because this religion that cannot be named for legal reasons was founded less than 100 years ago, and most major religions are thousands of years old and likely have grown as much as they ever will? Or because they count ad clicks as memberships.
- An ad for an internet phone service claimed that you could save $200 a month by switching, citing a chart showing all the associated fees and pricing. The product for sale listed a price of $25 a month with no installation or other fees, while the sample phone company was cited a price of $120, with $80 in associated fees. Disregarding the fact that you would only save $175 if taken at face value, the price of $120 was for six months of phone service, and the fees would have only been paid once. Looking at just price per month, the internet phone cost more than the sample company.
- A cellular phone service in the US claims to have cheaper phone bills... if you bring your own phone. If you buy a phone from them, you have the option of buying the phone outright, or paying it off in two years, which brings the bill up to what the bigger carriers charge for a similar deal.
- The term "unlimited bandwidth" that internet and cellular phone service providers love to advertise has gotten a lot of bad flak because people have found out yes, there is actually a limit to how much bandwidth you can consume from your provider before they start taking punitive action. Perhaps it was "unlimited" when speeds were so slow you couldn't manage to breach it even if you constantly were downloading something.
- There's a billboard along a freeway in Michigan, advertising a dentist willing to do implants for only a couple hundred dollars. Next to it is a statement saying "Voted best dental office in Michigan". Just below that, in smaller text, is the disclaimer "by our dental staff".
- The German ad for Fisherman's Friend Mint bonbons said nine of nine real men liked Fisherman's Friend bonbons. Shortly after throwing one of their test subjects over board.
- Shizuo in Durarara!! maintains that the series's troll, Izaya, is behind "99% of all the weird crap" that goes on the setting. Sure, Shizuo can get irrational when mad, and will even use statistics and percents to maintain points. ...Did we mention Izaya's a troll? We can only assume that he means every single weird thing.
- An old Archie story had one of the characters becoming a statistics-obsessed nut for the duration of the story, only for Jughead to start citing statistics that horrified them and lead them to run away in fright, at which point Forsythe noted that some ridiculously high percentage of people who quote statistics "make 'em up on the spot!"
- Batman: The Penguin says this quote word-for-word in Detective Comics #684, at the same time pulling a You Have Failed Me on a newly-acquired henchman who, through usage of statistics, "proved" that a broad daylight robbery had a 0% chance of being foiled by Batman.
- Judge Dredd: The Dark Judges famously use the statistic that none of the people they execute will ever commit another felony again as proof that death is the cure for crime.
- Dogbert once sold "value-priced lottery tickets;" half price, odds of winning only one in ten million less.
Sucker: Hey! This is for yesterday's lottery!
Dogbert: And your point is...?
- The Pointy-Haired Boss tells the employees that he's found out that 40% of sick days are taken on Monday and Friday, and declares this to be "unacceptable". Asok laughs until he realizes that the boss is actually that stupid.
- Dogbert once sold "value-priced lottery tickets;" half price, odds of winning only one in ten million less.
- Darrell Huff's "How To Lie With Statistics" was printed in the '50s. It's
usually available on eBaystill in print and is a very easy read that shows you all the basics.
- Spoofed by America (The Book), which included a graph on "Growth of Misleading Charts". Two different bar heights represent the same number.
- A book entirely revolving around poking fun at this is Spurious Correlations. A spurious correlation is when two variables appear to be related, but one variable is clearly not causing change in the other. The author used statistical software to find correlations between variables that had nothing to do with each other, like finding out that the divorce rate in Maine is positively correlated with the amount of margarine consumed in the state.
- Yes, Minister:
- There's a very interesting section on this. In a discussion about conscription, Sir Humphrey demonstrates to Bernard how statistics can be obtained which prove both sides of the discussion correct, through the use of leading questions which are not included in the reporting of the survey concerned.
- Another episode combines this with Hypocritical Humor; during one of their many arguments, Hacker brings up some facts to support his point only for Humphrey to superciliously note that Hacker's facts are based on statistics, which are thus unreliable as per this trope. When the argument gets a bit more heated, however, Humphrey begins to cite some statistics that prove his point, only to catch himself and quickly refer to them as 'facts'. Hacker is quick to point out the hypocrisy.
- ABC loves to put out press releases trying to make the previous night's viewership of their shows look good. With their big hits, this isn't so bad; winning the timeslot in total viewers or Adults 18-49 (the demographic used to set ad rates, and thus the figure used to determine whether or not a show gets renewed) is definitely something to be pleased about (unless there's a huge skew between total and A18-49 viewership, but that's another matter). No matter how poorly-watched a show is, however, the ABC PR department can find some figure that sounds good but doesn't actually mean the show's doing well. They frequently put up demographics that aren't really indicative of a show's survival (e.g. women 18-34, or the adults 25-54 demo that only some cable channels use for ad rates), give the amount that the show built on its lead-in (usually when the lead-in was a repeat or another low-rated show, or a repeat of another low-rated show), or claim the the show had the best performance in its timeslot among ABC shows since X weeks/months/years ago (when you look at the absolute ratings, all it says is that ABC's done even worse in the past; this is rarely used for hit new shows since there are usually better statistics for those).
- Programs on Animal Planet are fond of citing how Americans spend more money annually on cat or dog food than on baby food. This is depicted as evidence that Americans pamper their pets like babies, but overlooks several facts: that pets eat pet food for their entire lives, whereas babies only eat baby food for about a year and a half, and that many families have more than one pet at a time, but relatively few have more than one child of an age to eat baby food at the same time. Also, a baby often consumes breast milk from the mother which wouldn't show up in the statistics when calculating the cost of baby food. It also doesn't take into account the many people who make their own baby food, or feed their babies foods like applesauce or bananas that are already nice and soft; pet food is significantly harder to prepare.
- QI is well known for deliberately phrasing questions like this in order to confuse the participants. One question was "What is three times more dangerous than war?"note The answer given was work, because three times as many people are killed each year in work-related accidents than die in wars. Now, consider how much time you spent working last year compared to how long you were in a warzone. This prompted unhelpful responses from the panelists: "What if you're a soldier?" "What if you work in a shoe shop, near a war?"
- Monty Python's Flying Circus parodied the use/abuse of meaningless statistics in the sketch "Spectrum":
Man: This bar in this column represents seventeen percent of the population. This one represents twenty-eight percent of the population! And this one represents forty-three percent of the population!Host: Telling figures indeed. But what do they mean to you? What do they mean to me? What do they mean to the average man on the street?
- Parodied in Australian political satire The Dingo Principle:
Listen to this: 40% of people support the Prime Minister and 50% support the Opposition.That's only 90%.Yeah, there was one guy who said the samples weren't big enough to be statistically significant.
- Penn & Teller: Bullshit! point it out by having a man who makes poll research for the Republicans show he can make someone give two different answers to the same question by first asking: "Do you think the government expends too much in health care for immigrants?" The bystander answers "Yes". When he asks: "Would you deny an immigrant the right to treat himself? To give birth in a hospital? Etc..." and other medical services that go well beyond what the governments expends with immigrant health care, the answer now is: "No". Also, they make fun of the guy with his own mathematical wizardry by pointing out: "In this scene, ten cars pass by behind him. One guy from one of the cars shouts saying he sucks. This means that AT LEAST 10% of the American population believes he sucks".
- In an episode of Psych, young Shawn attempts to justify his fear of shark attacks with the statistic that most shark attacks happen in about three feet of water. His dad points out that most swimming in general happens in about three feet of water.
- The Wire: A major theme in the series is how organizations will "juke the stats" to make it look like the organization is more successful than it is. The police department will downgrade crimes when filing them to make the crime rate look like it's going down, or start focusing on low-level, trivial arrests to make the conviction rates go up. When Prez gets a job as a teacher, he immediately notices the same tricks being done in standardized testing to make it look like students are performing better. Reality plays almost no part in what the stats suggest.
- The Arrogant Worms' song "Don't Go Into Politics" recommends that audiences avoid the fields of politics, science, and music. To support their argument, it points out that people like George Washington, Albert Einstein, and Leonard Bernstein did so, and they're all dead.
- Newspapers love doing this with drug related stories. It's almost impossible to see Ecstasy mentioned in a British newspaper without the qualifier "That killer drug", the supporting statistic is that a dozen people die per year from consuming it. Yet over the course of a year they will rack up an impressive body count in stories about fatal car accidents without ever devolving into calling cars "Those murderous rampaging kill bots" How many Britons drive? How many take E? And how many deaths are caused by legal drugs? - cigarettes, alcohol, over the counter medications, misused prescriptions... And that's not even counting the fact that comparing legal and illegal things is a comparison of apples and oranges. Alcohol was much more dangerous during prohibition as it often wasn't pure and alcohol concentrations were often higher than usual as this was more interesting to smuggle. If Ecstasy were legal people would be educated about its use and avoid harmful interactionsnote , impurities would not be an issue, an accidental overdose from misjudging the dose impossible and it would probably be sold in 25-50mg pills instead of 200mg. Taking those factors into account in the statistics wouldn't help to support the viewpoint of the author, so they are generally left out.
- An Ars Technica article discussed the statistics usually used by software developers to complain about piracy. Specifically, the article pointed out that the statistics most commonly cited are most likely not only bullshit, but old bullshit. Amusingly enough, the image used for the related post on Gamepolitics was a pie chart divided into three sections, marked "Lies/Damned Lies/Statistics".
- The Column 8 column in the Sydney Morning Herald once referenced a statistical correlation between the difficulty of the sudoku on a given day and the price of petrol.
- There is a semi-famous magazine article from 1958 called "The Dread Tomato Addiction" that correlates consumption of tomatoes with everything from death to communism. It can be found here. There is a similar article about bread that can be seen here.
- There is a book produced for people in radio every year that compiles countless statistics about all stations taken from polls. These are used to attract advertisers. The less successful stations who have very few listeners are often forced to hire people who read through the book to get as many favorable statistics as possible, no matter how convoluted they may be. With the huge amount of data in the book, it's possible to say, for instance, that 85% of married men aged some arbitrary amount with income in some arbitrary range and who own a ferret will love your show, even though they represent a tiny proportion of the population. Of course, if you're selling ferret food, that's exactly whom you want to reach.
- A stand-up comedian once said, "There is a direct correlation between being a serial killer, and being born. Show me one murderer who was never born."
- One of Bothered About Dungeons & Dragons (BADD)'s favorite weapons was a list of over 500 Americans they claimed were gamers who had committed suicide in the same year. Thus role playing games somehow cause suicide. Except that even if you take this bogus statistic at face value, 500 suicides a year is still a lower percentage of suicides than clergy and a tiny fraction of the average.
- Patricia Pulling, the leader of this organization, once said in an interview that "8% of the Richmond VA-area population is involved with Satanic worship at some level." When asked where that figure came from, she said that she estimated that 4% of the teenagers and 4% of the adults. She then added them together and got the 8%.
- Another time, BADD cited an increase in suicides corresponding with a major Dungeons and Dragons release. Again, however, there's no evidence that's not simply a coincidence, as similar statistics can be used to prove that the release of a Britney Spears CD caused suicide numbers to jump.
- Saturday Morning Breakfast Cereal tells us how to make anything a victimless crime.
- This strip concludes that the news media got the supposed relationship between cell phones and cancer backwards, i.e., cancer causes cell phones.
- Also this one, which combines it with a dash of Hypocritical Humor:
Newscaster: ... and in science news, according to a new study, 85% of news organizations repeat "new study" press releases without checking whether they're real.
- And one about the significance of 95% confidence when you run 20 tests and only publish the interesting one.
- Another comic points out that the observed mortality rate among humans is only 93%, overthrowing most "100% of people who do X die" statistics by. . .mere statistics.
- Luke Surl expresses this via Visual Pun here.
- The 5 Most Popular Ways Statistics Are Used to Lie to You covers some fallacies commonly used to lie with statistics.
- 8 obvious signs statistics are lying to you.
- Often used in the News Parody Chigüire Bipolar for example in Maduro has 65% of Maduro's popular support, according to polls.
- The Onion does parody this from time to time.
- An article was about a movement to shut down hospitals because "despite rapid advancement in medical technology, the world death rate remains at 100%."
- Another article said that children are universally opposed to children's health care, with responses to the question "Do you want to go to the doctor?" ranging from "NO!!!!" to "inconsolable crying," but no children in favor.
- From 'Fat, French and Fabulous', "Statistically, the average person has slightly less than two legs." Statistics are fun when you don't use them properly.
- The Boondocks episode "The Color Ruckus" showed Uncle Ruckus as a child, being homeschooled by his mother, who at one point made this claim about George Washington Carver:
Bunny Ruckus: "George Washington Carver was the man responsible for more peanut allergy deaths than any man in history!"
- An episode of The Simpsons featured Homer forming a vigilante group to fight crime. At one point he recruits Jimbo (who calls the group "the drunken posse") on the basis that he can swing a sack full of doorknobs. Homer later gives an interview to the local news:
Kent Brockman: Mr. Simpson, how do you respond to the charges that petty vandalism such as graffiti is down eighty percent, while heavy sack beatings are up a shocking nine hundred percent?Homer: Aw, people can come up with statistics to prove anything, Kent. Forfty percent of all people know that.
- The Justice League was asked "Maybe you'd care to explain why on your watch, 50% of marriages now end in divorce, and the other 50% end in death!" Aside from the fact that the same was true before the formation of the League, until the end of time, a significant portion of marriages will end in death, as people do have a tendency to die, married or not. That and one can get divorced multiple times - absent resurrection, one can only die once.
- During World War I, helmets were almost withdrawn from British soldiers. When Britain started issuing steel helmets to all soldiers on the western front in 1916, generals began to call for their removal as they increased incidences of headwounds twelvefold and doubled total casualties. The reason? If someone gets hit in the head by some woolly bear or flying frog (German H.E. or rifle grenade) shrapnel and lives it's a "head wound" and if they are unable to fight, the person is a "casualty"; if they die from a bullet in the brain, then they are a "fatality" and so don't appear on casualty statistics. Since helmets let more people survive, the number of head wounds soared. A politician used the numbers to support his position that "helmets are expensive and cause cowardice" (which, as it turns out is exactly the opposite of the kind of behavior helmets encourage, see below), and never explained what it really meant - doubly effective as most people don't know the difference between "casualty" and "fatality". The real justification behind the attempt to withdraw helmets narrowed down to, "all a dead soldier needs is a funeral." A wounded soldier gets dragged out of combat by at least one of his buddies, and then provided weeks, months or even years of medical attention. From a statistical standpoint, adopting helmets drastically increased the effectiveness of enemy weapons - and a lot of WW1 generals genuinely believed in disposable personnel. Luckily, more ethical parties changed the way they recorded casualties, or the helmets would likely have been recalled.
- Likewise, the number of cyclists being treated for head wounds have increased massively since wearing a helmet became more widespread. Of course this is because they previously wouldn't have survived the accident at all.
- Situations like the cyclists mentioned above also can also be attributed to the Peltzman Effect, aka Risk Compensation. This is the effect of a person being aware of greater safety ("I don't have to worry about hitting my head, I'm wearing a helmet!"), and taking greater risks due to the perceived increase in safety. So there actually could be an increase in injuries, due to more people taking greater risks.
- Sick as it was, WW1 quickly devolved into a stalemate, at which point it was clear that the loser would be whichever side ran out of men or money first. With sufficiently large draft pools, conserving resources was more important than conserving manpower.... until 1915, that is, when they began to realise just how shallow their manpower-reserves actually were relative to the demands of the war. By 1917 they were basically doing everything they could to conserve their manpower, to the point that they produced enough helmets to equip even the logistics personnel (i.e. people who might but probably won't be facing shrapnel, bullets, etcetc).
- At some Reform Judaism synagogues, a popular "joke" to lead into the sermon is, "x% of deaths occur in a hospital, x% of deaths occur in a car, x% of deaths happen in the home...[continues on for a while] while there have been only three deaths in a synagogue, and no deaths ever reported while studying Torah! Clearly, the safest passion, therefore, is studying Torah."
- A related strategy was used by US president Richard Nixon to portray marijuana as a gateway drug. His anti-drug team estimated that 80% of marijuana users go on to use cocaine; actually, 80% of cocaine users had started with pot, but only about one in 2,400 marijuana users (just under 0.042%) go on to use cocaine. And related to that, most people willing to take a drug as dangerous as cocaine are willing to take a drug as relatively safe as marijuana. It's the same reason most marijuana smokers have drunk alcohol at some point, and why most alcohol drinkers have drunk something with caffeine in it. Water: the Gateway Liquid.
- One study cited by this article has experts ranking various drugs by their relative danger to individuals and those surrounding them. The original study appears to be fairly legitimate, yet the article commits a statistical sin of simply adding the value of 'harm to self' and 'harm to others' together to rank each drug. There is solid evidence that marijuana itself is very safe if vaped, as opposed to smoked. The same goes for nicotine, which was also found by the same study, commissioned by Public Health England, to be relatively harmless when the pure extract was vaporized. Alcohol is often perceived as less harmful than marijuana and cigarettes, despite the dangerous behaviours it can create.
- When Ronald Reagan's Attorney General Edwin Meese wanted "proof" that pornography was evil, he created the Attorney General's Commission on Pornography. The commission members were a preselected cohort of anti-pornography campaigners. Not surprisingly, they discovered that statistics "proved" that pornography caused crime. However, the 1970 report of the President's Commission on Obscenity and Pornography, which was done by honest researchers and was highly praised for accuracy and honesty, discovered that there was "no evidence to date that exposure to explicit sexual materials plays a significant role in the causation of delinquent or criminal behavior among youths or adults."
- In the heated German censorship debate about blocking sites allegedly containing child pornography, an organization in favor of this censorship law ordered a survey at a market research institute with questions asking if the person taking the survey is against child pornography and in favor of blocking the websites containing it. Over 90% answered 'Yes'. Another survey ordered by an opposing NGO — at the same institute no less — used a slightly different phrasing: Do you agree with blocking the content despite the fact that this content still exists and is easily accessible after the censorship? Over 90% answered with 'No'.
- Many casinos like to advertise their slot machines with lines like "Up To 99% Payout!" to make it sound like the player has a good chance to win. First, "up to" means the payout could be 1% for all you know (although laws usually set a minimum). Secondly, even a 99% payout means that for every $100 you put in the machine, on average, you'll get $99 back, i.e. you still lose. That "99% payout" is also an average that is based on something like one million pulls (plays) on the machine. If you play 100 times in one slot machine, you're not getting a representative sample of that average. These machines work differently in the UK. UK Fun With Prizes are required by law to seek their set hold percentage within a certain number of spins (usually 10,000). To achieve this, they naturally cheat all the time. They also can be, and often are, programmed to go on a suck cycle and take in way more money then they need to, in order to save up for a large series of payouts later.
- A machine may have one payoff rate if you bet a single coin per spin (which most casual gamblers do) and a completely different rate if you bet the maximum coins per spin. For example, the payout for a video poker game might be $1000 for a royal flush if you bet one coin, but $7500 if you bet five. The advertised payout rates assume the player is playing maximum coins, so a casino can have a slot machine that has a completely legal and legitimate payout ratio of one hundred percent and still manage to make money on it.
- Truth in Advertising laws require that if a set of machines is advertised as "Up to 99% Payout," then at least one of them must have 99% payout. Though there may very well be 50 other machines with 10% payout. (And don't expect that the "lucky" machine will be marked in any way, or even that it'll be the same machine each day.)
- The history board on roulette tables gave the illusion that the previous numbers the ball has landed in means that it should have a higher chance of landing on a number not on the board. Except the roulette table has no concept of memory and the ball has an equal chance of landing on the same number as before no matter how many plays were made. If the roulette wheel has any bias at all, it's more likely in favor of the numbers on the board — but don't bet on it, as the gambling industry works very hard to ensure their devices are all truly random.
- A common problem encountered is Simpson's Paradox, best demonstrated by example: Suppose Hospitals 1 and 2 are nearby, but 1 is better equipped for treating people with severe injuries, so proportionally more of the people taken there are badly hurt. It does better at treating badly hurt people than hospital 2, and also does better at treating people who are not badly hurt. However, since people who're badly hurt are more likely to die than people who're not badly hurt whether or not they go to hospital 1 or hospital 2, hospital 1 may still have a higher overall death rate.
Simpson's Paradox is when data shows one trend, but dividing it into categories shows the opposite trend. In the example above, hospital 1 has a higher death rate, but if the patients are split into categories based on severity of injury, it has a lower death rate in each category.
- The same goes with good doctors and bad doctors, as told in the book Super Freakonomics. Good doctors are generally given tougher cases while bad doctors are given easier cases. However, if you look at death rates you see that some doctors have higher death rates, but these are usually the good doctors. Patients with serious cases are more likely to die, so good doctors lose a lot of their patients than, say the doctor who cures hiccups. The lesson is that you can be fairly certain that the doctor you receive at a hospital is competent enough to be assigned to you.
- Italy got Südtirol, which used to be a part of Austria, to be added to their territory after World War II by using this kind of statistics to convince the Americans that the area was mostly populated by Italians. Which it wasn't. To this day, most of Sudtirol's population speaks German for a first language and watches German and Austrian TV, rather than the Italian channels.
- Wolf Blitzer on polling information about the health care debate in American politics:
We did that poll CNN Opinion Research Poll, that said, "You like this health care bill or not like it?" We just assumed, a lot of us, that the people who said they didn't like it didn't like it because it was too much interference, or too much taxes or whatever. But if you take a closer look at people who didnít like it, about 12% of those people who said they didnít like it thought it didn't go far enough. They wanted a single payer option, they wanted the so-called public option, they didnít like not from the right, they didnít like it because it wasnít left or liberal enough. Thatís how you got 50% of the American people who said, "We donít like this plan." But only about 40 or 38% were the ones who said it was too much government interference.
- In the 2004 US Presidential Election, Dick Cheney and John Edwards stated conflicting numbers regarding the Iraq war's casualties... and both men were partially right! Skip down to "90% of casualties".
- One statistic used to justify the creation of The Comics Code Authority was that a large percentage of criminals liked to read comic books, ergo, comic books influenced people to become criminals. Nobody pointed out that they were using the wrong statistic - they should have been asking what percentage of regular comic book readers became criminals.
- Similar was the observation by Frederick Werthram, author of "Seduction of the Innocent", that most juvenile delinquents were comic book readers, which he believed was an indication that comics caused juvenile delinquency. Had he checked what percentage of juveniles overall were reading comics, this would have become meaningless, as during the early to mid-fifties, damn near every kid in America was reading comics.
- The Victorian-era belief that masturbation could drive men insane was derived from a similar error, in that mental asylums reported frequent Dates With Rosie Palms among inmates. The fact that mental illness can impair inhibitions against such behavior wasn't considered, nor the fact that men locked up in an asylum had few other respites from misery, frustration or boredom. Most importantly, nobody had the nerve to ask about the masturbatory practices of men who weren't institutionalized.
- When Anthrocon decided to move from Philadelphia to Pittsburgh, one blogger who protested the move cited that only a tiny number of people of people would be as likely or more likely to attend Anthrocon if it moved to Pittsburgh. The organizers, however, heeded a different statistic: those that lived so far away that the move made little difference.
- badscience.net occasionally shows how statistics get misused. For example, here (on small samples it's quite possible that B isn't significantly different from A or C, but you can put it as "B isn't different from A, C is different from A, so we see that C is different from B", which is wrong) and here (limit the view to one of many multipliers which per se can't prove anything). Unsurprisingly, the areas with traditional relations to snake oil trade suffer most.
- Cryptozoology buffs are fond of citing the fact that new species of animal are still being identified with some frequency, and alleging that this means many other "hidden" species must exist under mainstream scientists' noses. They conveniently overlook the fact that most such species discovered in the last few decades are either found in isolated locales where no biologist had previously looked for new species (caves, obscure jungle canyons, deep-sea ecosystems, tiny isolationist nations), or are "found" when DNA analysis reveals that what had been considered one species is, technically, two (e.g. African forest elephants being genetically distinct from plains elephants). Not to mention the fact that the vast majority of new species are things like insects, small birds and reptiles, or deep sea fish, and not the magnificent megafauna monsters that cryptozoologists crave.
- It was once reported that the US was using 250,000 bullets to kill a single insurgent in Afghanistan. The most major problem with this is that they conveniently forgot to explain that this included all the bullets the military was firing, even during training and weapons tests. Never mind mentioning that most bullets go toward suppressing fire or that most casualties in war are victims of bombs and artillery (which this statistic took the care to exclude). It was finally revealed that this number had used a lot of generous rounding to get to that number.
- The previous Iranian government (led by infamous president Ahmadinejad) was notorious for this. In a very obvious example they reported the unemployment to have decreased by 50% while other independent sources (let alone the fact that more people were well ... UNEMPLOYED compared to before) suggested otherwise. Later it was discovered that they had changed the definition of an "employed" person from "one working at least 20 hours a week at a paying job" to "one working at least 2 hours a week". By removing the "paying" condition and cutting the time to one-tenth, they had managed to include people doing a variety of voluntary works and kids helping in family businesses a couple of hours each day (and still only managed to reduce the unemployment percent by 50% which goes to show how messed up their work was).
- They did the same when calculating inflation percent. When every source (from independent economists inside the country to World Bank) was reporting a point to point inflation of above 40% (as was apparent in increased prices everywhere) the official sources reported an inflation of 20% or less. How did they do it? By removing some essential items like rice from item basket (the selection of items whose prices were used to calculate inflation, originally including 300 items, mostly household and food products) and instead adding useless items like unpopular electronic devices (like house alarms) to the basket. This had a two fold effect: 1) It removed the most popular items that naturally experience a larger increase in price during inflation and 2) Added items with little increase in their price, increasing the population without increasing the calculated inflation.
- In fact they were so bad at this that pictures of Ahmadinejad showing graphs (without title or source) and saying "I HAVE PROOF" is now a joke in Iran used to show when someone talks bullshit without evidence to back it up.
- A commonly-cited factoid about the American Revolution is that roughly 1/3 of the residents of the Thirteen Colonies favored independence from Britain, 1/3 opposed it, and 1/3 were undecided or apathetic. The comedy series History Bites (based on the premise: what if TV had been around for 5,000 years) parodied Tom Paine as a spin-doctoring pundit:
TOM PAINE: Only 1/3 of the colonists are opposed to independence. Now, you can't let a minority opinion like that influence public policy!INTERVIEWER: But the same number are in favor of independence.TOM PAINE: But now we're talking half of decided voters, which is essentially a majority. You can't ignore the wishes of half of decided voters!
- The original "statistic" doesn't come from any actual poll anyway... it was an estimation made by John Adams, and he admitted he'd not done any research on that, just that it was his feeling on the matter.
- The Church Of The Flying Spaghetti Monster has semi-famously pointed out the obvious correlation between the decreasing number of pirates worldwide and Global Warming.◊
- Something of a historical subversion: During World War II, the Royal Air Force wanted to add more armor to their planes, but because of weight limits they needed to know which places needed the armor most. So, they examined the planes after they came back and counted how often bullet holes were found in certain areas... and then placed armor in places that showed the fewest bullet holes. This is because they'd spotted the flaw in their sample group; all they had to examine was planes that had come back. The data did not show, as might be assumed at first glance, places that planes were most likely to be shot. It showed places that planes could be shot and still fly home.
- Research into Psychic Powers in the 1970s ram up against this trope when people who (by pure luck) scored well on card-guessing procedures were singled out for re-testing under closer observation. Unsurprisingly, repeated tests found their apparent "powers" didn't work at all the second time around. Had the researchers re-tested the entire population of subjects again, not just the lucky guessers, they'd have found a similar proportion of high-scoring subjects randomly distributed among the volunteers. Instead, the idea that being observed made psychic powers wane was propagated to account for the "mysterious" decline.
- Glurge-y Facebook spam will often end with something to the effect of, "Only 3% of your friends will be brave enough to share this," effectively trying to guilt the reader into helping the spam proliferate.
- When reporting on the decline of a particular caribou herd in the Canadian Arctic (the Bathurst Herd), it's common for reporters and environmental groups to compare the current low numbers to the herd size in 1986 to demonstrate in how bad a shape the herd is currently in. What's never mentioned is that 1986 was the record high number for the herd count, and was 3 to 4 times the average number of animals counted in other years.
- This is commonly done when reporting the unemployment rate in the United States of America. At the end of Barack Obama's administration, unemployment was officially at 4.7%. However, the word "unemployment" is defined differently in the US than it is assumed to be. The US defines "unemployment rate" as "percentage of 18-65 year olds receiving a benefit from the government called 'unemployment compensation'". The assumed definition of unemployment, "percentage of able-bodied 18-65 year olds who do not have a paying job", is called "labor force nonparticipation" and is actually at 37.3%. But that 37.3% is also misleading, for the opposite reason — it includes people who aren't trying to work, such as disabled people, retired people, stay-at-home parents, and people in school. As the population ages and more and more people go to university, this number is bound to increase. Interestingly, out of all of these, the rate of stay-at-home parents is the only one that is expected to decrease. Unemployment rate causes are tricky to be accurate about is what we are trying to get at.
- This is how Barack Obama came to be known as the President with the highest deportation of foreigners in the United States. The common definition of this would have people assume the number derives from "number of people who have entered the United States border and were then removed for any reason." However, the statistic also includes people who attempt to enter at border controls, and are refused (technically, never entered the country). At land crossings, a person trying to enter the United States and is turned away, there is no mechanism to stop them from attempting again until he or she gets into the country... and each one is counted as a separate "deportation" under reporting metrics.
- Somewhat famously the United States' incarceration rate is often mis-attributed as the highest in the world. It is the highest documented in the world, which is a huge difference. In terms of estimated actual incarceration rates the US doesn't even make the top ten since countries like China and Russia simply lie about their incarceration rate.
- All the major types of car reliability statistics are likely to turn into that:
- The owner survey, listing the car owners' voices on reliability, is prone to people not wanting to admit their vehicle is unreliable, or the opposite: owners unfairly bashing their cars.
- The assistance statistics, listing how often a certain model requires roadside assistance, don't account for failures that don't immobilize the car.
- The inspection survey, listing the percentage of cars of certain age failing their inspection or the amount of failures the inspector noticed on an average example of the model, don't account for failures that do not impact the inspection's outcome. Also, expensive cars are more likely to score well, because their owners are more likely to have the money required to keep them in good condition between inspections.
- Whenever people cite Chicago as being a Wretched Hive due to its high number of murders, they conveniently neglect to compare that number to Chicago's population of 2.7 million. Crime rates for a city are measured per 100,000 residents, and Chicago doesn't even crack the top ten most violent cities in America.
- The vegan website Vegan Street claims there's more protein in 100 calories of broccoli (11.1 grams) than in 100 calories of beef (6.4 grams). While true, there's only 31 calories in a cup of broccoli, meaning you'd have to eat over three cups of broccoli to get those 11.1 grams. Conversely, three ounces of beef contains 25 grams of protein. Furthermore, not all proteins are the same; ones from most plants don't have as many of the amino acids the human body needs.
- The Church of Happyology is unsurprisingly quite fond of this trope. The most common is exponentially overstating the number of Happyologists there are. The church often times cites membership statics based on the sale of relevant books, but there are several problems with this. Chiefly, most people who buy Dianetics don't go on to join the church, quite a lot of them are buying the book out of Bile Fascination, and of the actual Happyologists who buy the book, most buy multiple copies.
- Back in 1985, there was a study performed that showed that a woman who was still single at age 40 had only a 2.6% chance of ever getting married. Newsweek even stated that such a woman had a better chance of getting killed by a terrorist. That study is now recognized as severely flawed, for multiple reasons. Although it did draw on US Census data from around that time, it narrowed the sample size far too much. It looked only at white, college-educated women born between the mid-1940's and mid-1950's who had never been married before, so out of 70,000 households, only about 1500 were part of the study. It also used a parametric model, which was meant for making sense of past events, not making predictions about future ones. And it did not take into consideration women who were cohabitating with partners (but not legally married); those women were counted as "single." There were also population conditions endemic to that particular generation, so even if that study had been accurate back in 1985 note , and it wasn't, that data would be obsolete now.
- The idea that if a woman wishes to have a child, she has to do it by age 35 OR ELSE has been studied multiple times. Problem is, almost all of those studies are getting their data, not from a large and well-controlled sample of modern women, but census data about French peasants from The Middle Ages! Few have ever even thought to question that data, but it's problematic for several reasons. First (and most obviously), it comes from an era before germ theory, modern medicine, fertility treatments (such as IVF), hospital births, advances in agriculture and nutrition, the feminist movement, understanding of eggs and sperm, and so much more. Secondly, it only looked at census data, which doesn't explain why few of these women were having children after 35. Sure, it could reflect that women of that time and place were going through menopause or perimenopause sooner than women today do. But it could also reflect a number of other things as well: Maybe these women had fertility problems to begin with, many of which are now treatable. Maybe, since these were peasant farmers, they weren't getting adequate nutrition for regular ovulation or healthy pregnancies. Maybe they were dying early of diseases such as The Black Death, or dying in childbirth. Maybe sex after a certain age, in that time and place, was seen as "unseemly." Maybe their husbands had gone off to war, or had died of diseases/malnutrition/etc. Maybe these women, compared to their younger counterparts, were more likely to use whatever contraceptive methods were available (or abort). We just don't know. What we do know, however, is that more modern data suggests that the decline in a woman's fertility generally happens, not in her mid-30's, but her mid-40's. (And that the risk of a live birth with a birth defect does double: from 0.5% to 1%.) So many women can (and do) become pregnant much later in life than we've been led to believe, either with or without fertility treatments.
- The statistic that 50% of all US marriages end in divorce. In actuality, the divorce rate never rose that high, even when divorce rates spiked in The '70s and early in The '80s (thanks to more and more states implementing "no-fault divorce" laws note ). Not only that, but (for a multitude of reasons), the divorce rate has actually been declining in over the past few decades.
- Moreover, the divorces follow the Pareto's law (80-20 rule). Most of the divorces tend to accumulate to certain people. Someone who has divorced and remarried is more likely to divorce than someone who hasn't divorced ever. Likewise, someone who has had four marriages and divorces is very likely to remarry and re-divorce again rather than find the true love.
- Often used in arguments over the American Civil War to "prove" that it couldn't be about slavery because such a small number of people had slaves (setting aside the fact that even if the small number were true, that conclusion would not follow, as most wars have been fought for the benefit of the ruling class, and the phrase "a rich man's war, and a poor man's fight" is sometimes attributed to a Confederate private). Some will try to claim a ridiculously small number of say, 2%; that's just a straight-up falsehood. Others will get more creative, and that is where statistical manipulation comes in. They'll compare the number of individuals who own slaves in their own name to the population as a whole. Here's why that doesn't work: say a planter has a wife and three children. Of these 5 people, how many own slaves? By this measure, only one (20%), the planter himself, because all the property is in his name. In a more honest calculation that looks only at households, the number of slaveowners jumps up to around 30%, and even that underestimates the economic impact of the institution, i.e., people connected to or who make use of slave labor without necessarily owning any themselves. For example: overseers, partners and employees in corporations that owned slaves (fairly common in transportation and the building trades), the fact that slaves could be rented, etc. Sometimes, a different claim is made to back up the same line: that a slave was a very valuable commodity, often literally compared to a Cadillac or sports car, and thus could only be owned by the ultra-rich. And that's true...of a prime field hand. The fact that most slaves would not qualify as such is ignored; this is the equivalent of claiming that only the ultra-rich can own cars.
- The idea that the average lifespan prior to the Industrial Revolution was only about 35 years. In actuality, plenty of people lived to be well into their 70's and 80's at that time, even without modern medicine and whatnot. The reason the "average" lifespan was so low, was because of high infant mortality. At that time, babies and children had a very high chance of dying before age 5 from disease...but if a child made it past age 5 without succumbing to disease or malnutrition, they had a fairly good chance of making it to old age.
- You may hear a statistic that "One in every _____ women/men will have X cancer". Usually, this is done to promote screenings for said cancer, or as part of "awareness" campaigns. What's usually not said is that this number includes all the diagnoses in that demographic, whether they survive or not, though it's often phrased as how many patients will die from that cancer.
- There's also the "survival rate." As xkcd explains, sometimes a cancer cell or two or three will slip past whatever treatment(s) are given (e.g. chemotherapy, radiation, immunotherapy, whatever), or actually be resistant to it. Which means it may come back, or the tumor may metastasize (that is, pop up elsewhere in the body). The X-year survival rate actually refers to how many years you go without this happening. (For example, if your five-year survival rate is 60%, that means there's a 60% chance your tumor won't come back or metastasize within 5 years...but there's a 40% chance that it will.)
Of the 99 people polled by Trope Co., 45% said that exactly 80% of this page was insightful, 12.3549966% of the time!