TV Tropes Org

Forums

search forum titles
google site search
Total posts: [40]
1
2

Personal Experience vs. Statistical Evidence:

 1 drunkscriblerian, Sat, 26th Feb '11 2:57:56 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
This flies around a lot in the debates here, and I thought we should discuss the matter. Pros and cons of each as I see them;

Statistical Evidence

pros:
  • You can cite your source, so both sides can read the data and draw conclusions from it.
  • If properly applied, scientific method is argument-neutral and as such provides a stable base for both sides of a debate to draw on, at least in theory.

Cons:
  • Often how the data was collected is not shown/glossed over, so it is tough to know whether the data actually applies to an argument or not.
  • Numbers can be twisted around to any purpose the user desires; this was one of the first things I was taught in an "intro to statistics" class, and I think fellow tropers who have been educated in math can back me on that.

Personal Experience

pros:
  • Personal experience is more vivid and nuanced than simple numbers.
  • It reminds us that exceptions exist for any rule, and not to over-generalize.
  • It shows us the difference between theory and practice.

cons:
  • Generally speaking the same source cannot be cited by opposing sides, so deadlocks often result unless the debaters are mature enough to "agree to disagree" when such comes up.
  • Personal experience can be distorted by the individual's own preconceived notions on the matter, rendering the assertion biased.

My opinion on the matter: Personal experience gets irrationally discriminated against, and that attitude encourages the thinking of "I can learn everything I need to know about the world from the comfort of my computer chair". Attempts to bring in personal experience or knowledge into a debate is often met with scorn or derision, or a demand that you "cite your source."

As I said above, numbers can be twisted around to say anything you want. So can personal experience, but that doesn't make either any less valid.

Discuss.
If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
 2 Major Tom, Sat, 26th Feb '11 2:59:21 PM Relationship Status: Barbecuing
Eye'm the cutest!
^ That whole post is pretty much 100% spot on.
Endless Conflict: Every war ends in time, even supposedly this one.
 3 storyyeller, Sat, 26th Feb '11 3:06:12 PM from Appleloosa Relationship Status: RelationshipOutOfBoundsException: 1
More like giant cherries
Welcome back
Life is simple: it has no nontrivial normal subgroups.
 4 drunkscriblerian, Sat, 26th Feb '11 3:28:02 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
@Tom: Glad to see you as well. So, anything I got wrong? And what's your stance on the matter?
If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
 5 Silent Reverence, Sat, 26th Feb '11 3:29:43 PM from 3 tiles right 1 tile up
adopting kitteh
IMHO, statistical evidence is only useful in its entirely when the results it provides can be mapped back without change to individuals. A statistic indicator that the average family has, say, found happiness with a minimum wage of $X currency is of no practical use if the indicator relies on another statistic that the average family has 2.13 children. Unfortunately, it is precisely such statistics that are used to give birth to terribly flawed social management politics.

On the other hand, personal (empirical?) evidence by definition can not have value in and of itself. "This happened to me" is of no practical use; it can only be rendered useful once it is contrasted with a large or distinct enough amount of personal information. "This happened to me, but that other guy did manage to catch the ball because he has longer arms" is pretty useful.

 6 Grain, Sat, 26th Feb '11 3:35:23 PM from South Northwest Earth
Only One Avatar
Is it possible for personal experience to outdo statistical evidence in an argument?
 7 Drunk Girlfriend, Sat, 26th Feb '11 3:36:28 PM from Castle Geekhaven
@Grain: Depends on who's involved with the argument. On the internet, not in the slightest, considering how it's generally held to be bad form to bring anecdote into anything.
"I don't know how I do it. I'm like the Mr. Bean of sex." -Drunkscriblerian
 8 drunkscriblerian, Sat, 26th Feb '11 3:37:43 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
@Grain: If the science is bad enough or the connection tenuous enough in the statistical argument, then yes. I think the best arguments contain a mixture of both.

The weakness of personal experience is mitigated by realizing that your experience isn't the only one that exists, and your reactions to it aren't the only ones possible. Again, it is all in how maturely you apply it.

If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
Always Right
They're both equally important imo. Both of them can give lots of information in their own ways. And both methods can be incredibly biased. But if you do both of them enough time, eventually you find a pattern.

And bias isn't necessarily a bad thing. If you can tell which individuals are known to be biased, you can eventually get a good pattern going from them as well as long as you know what you are doing.
"Every opinion that isn't mine is subjected to Your Mileage May Vary."
 10 Loni Jay, Sat, 26th Feb '11 4:27:01 PM from Australia Relationship Status: Pining for the fjords
There's also personal experience that isn't just one anecdote. Say, people who work in the field under discussion, and while they haven't measured effect exactly, will still notice some definite trends.
Be not afraid...
Often how the data was collected is not shown/glossed over, so it is tough to know whether the data actually applies to an argument or not.
I would call such a thing a "statistic" not "statistical evidence". That's why checking the source is important - there's a reason researches output a side long paper and not just a three line summary.
Numbers can be twisted around to any purpose the user desires; this was one of the first things I was taught in an "intro to statistics" class, and I think fellow tropers who have been educated in math can back me on that.
The same as above also applies here.
If only the raw data (maybe after some processing through well known mathematical methods) is used for the argument, and not the interpretation of that raw data, then it's no problem.
(Actual example I had some time ago:)
Wrong: "Only 2% of women make false rape accusations"
Less wrong: "2% of the women who make rape accusations were threatened with charges if they wouldn't withdraw the accusations"
The data doesn't actually support the conclusion that got used as an argument. In fact all accusations could be true, and those women were falsely forced to withdraw.

Of course, when people rely on statistical arguments they normally only refer to the first form, so...

After trying to defend statistical evidence, I'll try to defend anecdotal evidence:
There's a math joke that goes: "During a lecture a math student tells the professor: 'Sir, I got a counter-example for your statement' 'Doesn't matter, I have two proofs!' "
If someone makes a statement that wouldn't allow something to happen but you experienced it, it means that the other one was wrong - plain and simple. He can try to pull argument after argument, but a single counter-example is sufficient to disprove an absolute statement.

In an argument, statistical evidence and personal experience serve imho two different purposes and they don't really conflict. Using personal experience to "disprove" statistical evidence is missing the point just as much as using statistical evidence to "disprove" personal experience.
I was young and needed a nick.

www.xkcd.com/386/
 12 drunkscriblerian, Sat, 26th Feb '11 5:59:12 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
In an argument, statistical evidence and personal experience serve imho two different purposes and they don't really conflict. Using personal experience to "disprove" statistical evidence is missing the point just as much as using statistical evidence to "disprove" personal experience.

Pretty much what I was trying to get at. However, there does seem to be an irrational level of faith in statistics and published information as it pertains to arguing via the internet, for all that any idiot can post anything he wants to online and call it a "study".

That and (OMG personal experience talking!) I've had my personal experience marginalized a great deal, to the point where I've started returning the favor (e.g. "go get out and live a little before you make such claims, young'un")
If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
Hm... I really would like to know the context of the events.
If you try to use personal experience "instead" of a statistic, it means you are just using a statistic but done badly. In general low sample size (In case of single event it's a sample size of ONE, which is horrible) and no or at least less controlling for different factors (For example your influence on the events, the locations where you tend to be etc etc).

In general you are doing everything a statistic does only worse.

That's why I would like to know the context of the criticism you faced. If you were using your experience as a substitute to a better done statistic, they were imho right on dismissing it. If you used it like "according to you this could never happen but it happened to me" they were wrong.

I was young and needed a nick.

www.xkcd.com/386/
 14 Drunk Girlfriend, Sat, 26th Feb '11 6:17:58 PM from Castle Geekhaven
In general, I tend to find that bringing anecdote into a discussion results in people going "Well, that's not what the statistics say", even if the purpose of the anecdote wasn't to disprove or deny anything, just to illustrate.
"I don't know how I do it. I'm like the Mr. Bean of sex." -Drunkscriblerian
 15 drunkscriblerian, Sat, 26th Feb '11 6:18:26 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
@U (sorry not even going to attempt to spell that): Basically, people were making sweeping generalizations regarding racism and white privilege and I said "No, that's not how it works everywhere because that's not how it works where I live (description)", and they basically said "exception does not prove anything".

Also claiming "sampling bias" when all was trying to say was "well, I don't know what the numbers say but this is my experience on the matter" regarding things I am intimately familiar with.

edited 26th Feb '11 6:18:44 PM by drunkscriblerian

If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
 16 Tuefel Hunden IV, Sat, 26th Feb '11 6:23:01 PM from Wandering. Relationship Status: [TOP SECRET]
Watchmen of the Apocalypse
Both sets of data are useful but have their flaws and limits. Neither should be fully discredited or dismissed either.
"Who watches the watchmen?"
Copy+Paste is your friend for my name ;) (I don't really care though, call me whatever you like ^^)

I think I might even remember that post, was it on tvtropes?
Well, you remember the personal experience = statistic thing I said earlier? If I'm not confusing it you were basically using a whole city or village for your anecdote, which is imho enough data that it shouldn't be dismissed easily. Even if it's a bad (or "not that good") statistic, a model that cannot explain this satisfactory is at least a flawed model.

If the theory doesn't equal the reality, it means the theory is flawed. They should basically have the same results.
But in general people (i.e. WE) like to dismiss anything that doesn't support our point of view - personal experience as a rather weak set of data is just an easy target. Though I would agree that many "studies" that get cited aren't worth their space on the screen, especially because normally only the (biased) interpretations get cited, not the actual data.

edited 26th Feb '11 6:58:37 PM by Uchuujinsan

I was young and needed a nick.

www.xkcd.com/386/
 18 drunkscriblerian, Sat, 26th Feb '11 6:47:31 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
@U: It was in the "Racial Olympics" thread or whatever, though its an argument I've gotten into several times on this forum.

I somewhat agree with you; I'm willing to admit that what's true for me isn't true for everyone (durr), but the complete and utter dismissal of personal experience...yeah, I take issue with that. As I said in the OP, it encourages people to just sit in front of their computers, look up things on the Internet and think they know every damn thing.

And as you said, when someone makes a generalization or absolute statement, popping to say "that isn't true for me" should not get shouted down as much as it does.
If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
I remembered it because I noted how informative it was relative to "normal" anecdotes - I also disagreed with the reaction.
(On a completely irrevelant side note, I hate having to check some vocabulary just to ensure that my German isn't polluting my English too much -.- )
I was young and needed a nick.

www.xkcd.com/386/
 20 drunkscriblerian, Sat, 26th Feb '11 6:55:56 PM from Castle Geekhaven Relationship Status: In season
Street Writing Man
@U: I assumed you were not an English native, because your speech was intelligent despite your spelling. And no, you're perfectly understandable.

Yes, I grok the limitations of personal experience. I just wish others would recognize the utility.
If I were to write some of the strange things that come under my eyes they would not be believed.

~Cora M. Strayer~
Oh my god, "worse" instead of "worth", "the" instead of "that"...

But I have an excuse, it's 4 am here :>

(Ok, I'll stop now with this OT stuff)
I was young and needed a nick.

www.xkcd.com/386/
 22 storyyeller, Sat, 26th Feb '11 7:01:10 PM from Appleloosa Relationship Status: RelationshipOutOfBoundsException: 1
More like giant cherries
After trying to defend statistical evidence, I'll try to defend anecdotal evidence: There's a math joke that goes: "During a lecture a math student tells the professor: 'Sir, I got a counter-example for your statement' 'Doesn't matter, I have two proofs!' "

This may not be the best example, as I'd bet the professor is probably right. All we really know is that at least one of the people made a mistake, and without extra context, it's much more likely to be the student than the professor.

edited 26th Feb '11 7:06:20 PM by storyyeller

Life is simple: it has no nontrivial normal subgroups.
The professor is certainly wrong, because having "two proofs" does jack shit against a counter example. Which basically is the joke. :>
I was young and needed a nick.

www.xkcd.com/386/
 24 storyyeller, Sat, 26th Feb '11 7:05:38 PM from Appleloosa Relationship Status: RelationshipOutOfBoundsException: 1
More like giant cherries
But a "counter example" also does jack shit against a proof.

All we really know is that at least one of them is mistaken. And I don't think it's the professor.

edited 26th Feb '11 7:08:57 PM by storyyeller

Life is simple: it has no nontrivial normal subgroups.
 25 Drunk Girlfriend, Sat, 26th Feb '11 7:07:01 PM from Castle Geekhaven
All I remember from precalculus is how to 2+2 =/= 4.
"I don't know how I do it. I'm like the Mr. Bean of sex." -Drunkscriblerian
Total posts: 40
1
2


TV Tropes by TV Tropes Foundation, LLC is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Permissions beyond the scope of this license may be available from thestaff@tvtropes.org.
Privacy Policy