Useful Notes: The Scientific Method
"There are two kinds of scientific progress: the methodical experimentation and categorization which gradually extend the boundaries of knowledge, and the revolutionary leap of genius which redefines and transcends those boundaries. Acknowledging our debt to the former, we yearn, nonetheless, for the latter."Ah, the scientific method, hated by The Fundamentalist, loved by anyone else. Misrepresenting or outright vilifying it leads immediately into Artistic License territory. If a Straw Vulcan tries to use it, expect the method to be rather unscientific. The actual Scientific Method is rather simple. You'd think this would be hard to mess up, but it is done anyway. The basic process is below:
—Academician Prokhor Zakharov, Sid Meiers Alpha Centauri
- Observe the conditions. (You have to lay down certain ground rules before the rest. )
- Create a research question based on your observations (Such as. "What would happen to Z if I change A?")
- Form a hypothesis. (Make a statement. "Since A behaves in X way, if Y is done Z should happen, because G." Or something of that sort.)
- Make a prediction based on your hypothesis. (This gives you something to test. It also limits the power of your bias to interpret your results to be what you wanted. For example, if you're testing the effectiveness of a cleaning fluid, you can't just clean a mess with it, inspect the result, and decide afterward that what you see is "pretty clean"; making a specific prediction beforehand, with guidelines to determine how you measure things, forces you to interpret whatever really happened.)
- Test it. (This being a rather important step, as it helps give you evidence to support your hypothesis.)
- Repeatability. Despite popular portrayals, no one experiment can or should single-handedly overturn a theory, or create a new one. After all, experiments involve numerous variables and events; the results from just one won't necessarily tell you what you assume they are (and you may have simply made a mistake). By repeating experiments, you get a better sense of whether the phenomenon you're studying occurs throughout the year, with different entities, etc. If it keeps happening, the phenomenon is "repeatable", and the more repeatable it is the more likely it is to be real. (This is one of the hurdles faced by any Einstein Sue whose hypotheses contradict the current consensus.)
- Control. Your experiment may not be very meaningful if you only observe changes in a single subject. It's good to have a "neutral and ordinary" subject, kept as similar to the original as possible, for the sake of comparison. For example, in order to be approved by the United States FDA, a drug has to go through a test wherein one group of people take the drug and another (unknowingly) take a fake pill called a placebo. Without the control group, it would be harder to say whether any improvement was actually caused by the drug, as opposed to the placebo effect (when people feel better because they think they took the drug) or just because people with that condition happen to get better over time anyway. Post hoc ergo propter hoc is the fancy Latin term for the fallacy of assuming, "X happened, then Y happened, therefore X caused Y!"; in English we say, "correlation does not equal causation." Fictional science falls prey to the fallacy with depressing frequency, and in real life, control is one means of fighting it and keeping in touch with reality. One everyday situation in which non-scientists use "control subjects" is diagnosing problems with a piece of technology. E.g., if a DVD isn't playing properly, you might keep everything the same except for switching the DVD player with a different one, or trying a different DVD, etc, until you isolate the cause.
- Consilience. Whatever your findings are, they ought to match those of other scientists in other fields, or else something somewhere is askew. A geneticist comparing bird DNA to reptile DNA should have similar results as a zoologist comparing their bones, because the theory of evolution predicts that these types of data match up. Thus science is more cooperative between different disciplines rather than the popular portrayal of being competitive (an example of such a portrayal is Hard on Soft Science). When concilience isn't happening, it raises some flags, as with the apparent contradictions between the current models of relativity and quantum mechanics. This can be frustrating, but it also makes many scientists excited for the opportunity to fill a hole in our understanding and maybe win a Nobel prize or two.
- Falsifiability. This one can be counter-intuitive. Falsifiability is not about whether the theory is false, but rather concerns whether it can be verified experimentally at all. For example, a claim that an ancient Japanese coin is buried somewhere in the soil of Mars could hypothetically be proven (if we found the coin), but it is for all practical purposes impossible to disprove ("the soil of Mars" being essentially an infinite resource that we can never finish digging up; the coin could always be just in the next shovelful). It works in reverse too: the claim that "All Women Love Shoes" is disproven by the existence of just one woman who hates, is indifferent towards or "merely" likes shoes, but impossible to prove—even if you manage to ask every single woman currently alive (and they all love shoes), there's still the ones who are already dead or yet to be born.
Another example of an unfalsifiable experiment is mentioned in Lies, Damned Lies, and Statistics. Want to "prove" that video games cause violence? Get a group of scientists that are already savvy to this and don't mind the lack of ethics. Have them draw from a very small pool of test subjects that are known to display violent behavior. Mental hospitals, prisons, schools for children with behavior disorders, what have you. Do some generic tests that are guaranteed to show up positive (therefore unfalsifiable in the first place by definition), come up with numbers, run the study with No Control Group, and presto, instant headline. "Recent test shows 77% of subjects become more violent after playing Mortal Kombat." Most people won't bother with reading the article the whole way through and will just look at the headline. This works with anything from comic books, and rock to watching Brokeback Mountain or voting for specific parties, basically anything.
If you cannot conceive of a condition that your theory might be disproven, then chances are your theory is a Tautology -a type of Circular Reasoning. Faking, confirmation bias, Moving the Goalposts, God Of The Gaps, unfalsifiable theories being used in Lies, Damned Lies, and Statistics, and such have made the chance of falsifiability an important consideration in making a hypothesis. Notice how this connects to the "make a prediction" step in the scientific method; you need to make sufficiently specific predictions for your hypothesis at least partially to give Reality room to disprove you and keep everything more realistic.