Needs Help: Critical Research Failure

Critical Research Failure has long had the same problem that resulted in It Was His Sled getting an Example Sectionectomy: it is defined by what "everyone knows", or at least, in this case, everyone with "a high school education (or less) and/or a cursory knowledge of the subject". That opens the door for Fan Myopia to set in, as people in different places may have different bits of common knowledge, and people with more than "a cursory knowledge of the subject" lose sight of what people who don't know as much know. It's somewhat surprising that CRF hasn't gone down the same path as It Was His Sled, because it has the added problem of being inherently negative. Combine the two, and it's not surprising that whenever I encounter it on a YMMV page (especially for nonfiction works and Video Review Shows), it seems to basically amount to Complaining About Inaccuracies You Don't Like. Older examples, at least on the page itself, do tend to be things that are reasonably common knowledge, but now it's not even clear to me that people are trying to stick to things "everyone knows" let alone justify them as such; in that sense CRF seems to have inherited the role of the old Did Not Do The Research page, except that knowing the actual definition can make an accusation of Critical Research Failure seem harsher than may have been intended if the troper wasn't as aware of the definition. Even discounting that, a number of examples both on and off the page come off as nerds whining that a work didn't get something about their favorite topic right.

It doesn't help that the definition itself has always been so vague - coming from a time when our notion of what a trope was and what it took for a trope to leave YKTTW was much looser than now - that for much of its history, the page was arguably more defined by a note at the end of the description imploring tropers against misuse than by the description itself. In the earliest form the Wayback Machine has, it read as follows:

Less than a year later, the note was less detailed: ...and by 2012, the note had been reduced to an almost completely useless statement asking to make sure examples are really examples. Later that year it was restored to a more effective state, but then a 2014 TRS thread removed it entirely in an attempt to clean up the description, the only major change the opening paragraph has undergone since the page's inception. The major gist of that thread was to remove references to a fairly specific situation that had never been treated as definitive, a generous interpretation of which might be that the work has to be speaking with authority, giving off the sense that it actually does know what it's talking about even as it's obvious that it doesn't (which would border on Dan Browned But More So), and which the OP of the thread claimed made "every single non-in universe example wrong" (something the still-existing comparison to Cowboy BeBop at His Computer seems to back up, and which Madrugada seemed to think was the case in this 2011 Trope Talk thread). Unfortunately, the description that came out of that thread is still vaguer than it might appear at first glance, and merging the last incarnation of the note's description of the standard of commonality of knowledge required means it just adds to the confusion: What is the operative part of that description? Is it that the inaccuracy in question is so easily spotted? Or is it how obvious it is that it was made up without even any regards for research? Or is it just how egregious it is? The laconic - "a blatantly obvious factual error concerning common knowledge" - suggests the former, but can be interpreted as the latter, and evidence to back up all these interpretations, and others besides, can be found throughout the examples. (Ironically, the original YKTTW suggests the specific situation removed in 2014 may have been the key to making a far more workable and enforcable page than what actually resulted.)

In any case, it's always been impossible to determine what examples are misuse and which aren't. Even if I didn't know something before it was pointed out in the article, I'm in no position to judge how many people would know such a thing, whether the "average person" in the original note or someone with a "high school education and/or a cursory knowledge of the subject" in the current description; it's entirely possible I should have known it and forgot, especially if I know little to nothing about the topic in question, or even that it's entirely natural that I'd be one of today's lucky 10,000. (After all, the creators of the work in question certainly didn't know it!) Nonetheless, there are a distressingly high number of examples that I didn't know before going through them (and it's worth noting that despite what the 2014 version of the note said, the vast majority of the examples do feel the need to explain what makes them inaccurate), and several other cases that seem to be misuse no matter what:

• Not surprisingly, in the absence of the original note's injunction against Justifying Edits, several examples have Justifying Edits that completely obviate their presence on the page (in some cases implying that the error is intentional, and not necessarily in the sense that they're supposed to be in-universe) and should have just been removed.
• Some examples are cases of continuity nitpicking within a series, rather than inaccuracies about the outside world.
• Several examples fall under tropes outside the Artistic Licence series that depict common Hollywood means of depicting various things, like Hollywood Evolution, or misconceptions widespread enough to have tropes on them like 90% of Your Brain. These tropes are not about inaccuracies in various fields in general, but specific inaccuracies that are common across media and may mislead many viewers into thinking that they're true. Perhaps more disturbingly, generally only one or two works get singled out for these errors (though "humans are the only ones who kill their own kind" gets called out several times), meaning they get singled out unfairly for an error that's quite widespread.
• I already mentioned "if you have to explain it it's not a CRF", but several examples go into large amounts of legalese or technical detail that shouldn't be necessary if the error in question is obvious to someone with only "a cursory knowledge of the subject". One example, particularly egregiously, calls itself a "slightly more esoteric" CRF, an obvious contradiction in terms.
• Some examples are more about the author than the audience. Some talk about the author admitting that they got something wrong, but where the thing in question isn't necessarily obvious. At least two examples are about people admitting to not doing any research but with comparatively less emphasis on what they got wrong, one of them not even specifying what they got wrong.
• Some examples seem to emphasize how easily the error could have been corrected than whether the error was particularly egregious. Others seem to be about how much the error undermines the entire work. Still others are about how the truth is the exact opposite of what the work claims, even if that truth isn't necessarily well-known.
• Not on the page itself, but on YMMV.Todd In The Shadows, there's a "CRF" entry about how Todd discovers and mentions that the subject of a One Hit Wonderland entry actually had a second hit. That's the exact opposite of not doing the research, so by the standards of the last bullet point, whoever added that entry pulled a Critical Research Failure about Critical Research Failures.
• Even when examples do try to justify their presence on the page, they're questionable. Twice the page tries to claim that "any child", or at least "any dinosaur-crazed eight-year-old", could tell you the dinosaurs died out 65 million years ago. Now I wasn't as dinosaur-crazy as some kids, but I'd be surprised if most kids that weren't both as intelligent and as dinosaur-crazy as Calvin would be able to tell you exactly how many years ago the Mesozoic ended by heart; certainly I couldn't. At least in the one case, where Godzilla (1954) tries to claim dinosaurs lived two million years ago, I can recognize that's close enough to the time of humans not to be right, but if Jimmy Neutron: Boy Genius tells me the dinosaurs died out 200 million years ago, I won't have any reason to dispute it.
• On occasion, the entire standard breaks down because the standard would have been different at the time the work was made. This is especially egregious for what is cited as the oldest example on the page, The Song of Roland, where the entry admits the authors would have had no way of knowing any better than what they put down on the page, even though their error is obvious to a modern reader. Similarly, as I noted at the start, the standard would be different in different places as well, and that seems to be how The Road To El Dorado justifies its placement on the page against English-speakers who might not be as familiar with the history of the period depicted, despite the makers of the film and the primary intended audience both being in the latter category.
• One example talks about the description of the game on the box not matching the actual game contained within or the other documentation in the box, which isn't even a case of "not doing the research" so much as two parts of the same company not talking with one another.

I'm not sure what's to be done about this: cut it entirely? Redefine it to be In Universe Examples Only? Tighten up the definition in other ways to make it clearer and more enforcable? Restore the injunction against misuse in some way? Something else entirely?

edited 23rd Jul '17 3:18:43 AM by MorganWick

Opening but with a caveat: This Audience Reaction is inherently dependent on the standard of knowledge of the troper adding it. That is simply the nature of the deed and not a problem. Also, unlike It Was His Sled where spoilerage is an additional issue the subjectivity is the only aspect.

If memory serves, years ago I did examine this trope for the possibility to make it IUEO but decided that the amount of work needed was unjustifiable.

As for the definition, the laconic is for once correct.

"For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled." - Richard Feynman

I just did a quick reading of this page myself. It comes across more as trivia to me than an audience reaction, since a large chunk of the examples are clearly not "common knowledge", and are simply details that don't follow reality in a story that are presumably meant to, but would be hugely different if they followed reality (that sometimes had to be rigorously studied to prove inaccurate).

I think part of the problem is that "common knowledge" isn't as real or as defined as we like to believe it is as a society, as TV Tropes' own Common Knowledge page proves as its very purpose. In other words, trying to have examples based off of what's common knowledge makes it where essentially anything can be added to the page by anyone, much like the issue I think Mainstream Obscurity has (although that was more an inevitability, while common knowledge has gotten to that point long ago).

For instance, from this very thread, I remember the whole "dinosaurs dying off 65 million years ago" thing practically being a Catchphrase when I was a kid. I don't think about it much now, but it actually does surprise me that someone got that wrong, or I should say, that it's not as common a piece of knowledge as I thought based on my own experiences.

edited 24th Jul '17 12:53:37 PM by Jokubas

Relevant question: Why the late Did Not Do The Research was nuked? Depending of the answer, if Critical Research Failure is being treated as Did Not Do The Research II...

edited 3rd Aug '17 6:48:08 PM by MagBas

I believe that Did Not Do The Research was deleted because it was actually meant to be an INDEX, not a trope. Of course, its unclear name led to it being misused all the time, so it was moved to Consistency under "Lack of external consistency".

Per the Trope Epitaph: "Turned into a virulent complaining based Pothole Magnet, used solely to bitch about which works failed the most at realism, which is pointless on a wiki about fiction regardless."

It sounds like Did Not Do The Research was a more complainy version of Artistic License.

Off hand, I'd say Critical Research Failure be renamed "Research Failure" removing the "everyone knows" subjectivity (it would cease to be YMMV) and and just apply it to any such errors, leaving it as objective as Series Continuity Error.

The question is how/if we can keep it from decaying the same way Did Not Do The Research.

I'd say the difference between Research Failure and Artistic License is that the latter are Intentional and/or Necessary Weasels for the sake of plot. My one question is how do we keep Research Failure distinct from Series Continuity Error, Continuity Snarl, Canon Defilement, and such?

Worth noting that the Artistic License tropes were themselves all renamed from You Fail X Forever, which implied that they weren't intentional, and are still used for both intentional and unintentional examples (with their descriptions patiently explaining the truth), in part because it's hard to tell which is which without Word of God.

DNDTR towards the end of its life.

edited 3rd Aug '17 8:10:56 PM by MorganWick

Series Continuity Error and the like are errors with respect to a work's continuity. Artistic License and the like usually reflect errors with respect to reality. I think consistency has a similar explanation.

The idea that the cutting of Did Not Do The Research caused misuse to migrate to this page is interesting but needs some evidence in form of a comparison between pre-DNDTR cut Critical Research Failure and post-DNDTR cut CRF.

"For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled." - Richard Feynman

Well, when I look at CRF from about the same time as the DNDTR link I gave, I see a much larger proportion of in-universe examples, and more of the out-of-universe examples are justifiable, but you might want to take a look for yourself.

Well, yeah, it seems like the trope quality decayed then.

Wonder what the ideal solution would be. Restore Did Not Do The Research as a Trivia trope - factual inaccuracies are pretty much Trivia - so that people use that instead?

"For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled." - Richard Feynman

Potentially yeah but we would have to make it clear that inaccuracies that are trivia are minor things and have nothing to do with the plot, inaccuracies to make the story and such work should be Artistic License.

I'm not sure I understand why Artistic License examples should be removed if the example isn't about the effect on the plot. There are plenty of subtropes already for background effects rather than inaccuracies needed to make the story work.

Link to TRS threads in project mode here.

Because people search on the Web for what they want to see or read, I think it's fair to say that tropers who have "cursory knowledge of the subject" won't go out of their way to add examples to the page. I mean, if they only have cursory knowledge of the subject, then it's fair to say that they aren't particularly interested in the subject. But tropers who know quite a bit more about the subject probably have more interest in it and are therefore more likely to add examples to the page. My point being that the page is likely to skew in favor of those tropers who know more about the subject.

If we agree that this problem for Critical Research Failure was caused, at least in part, by the removal of Did Not Do The Research, then I think it makes more sense to keep DNDTR cut. It would just move the problem, rather than solve it. Even if we name it something else, I doubt these — for lack of a more concise term — educated tropers would not notice its creation.

Concerning the "factual inaccuracies are pretty much trivia" point, I disagree. Those errors are clearly in the work, and I think whatever Did Not Do The Research could cover is already adequately covered by either Artistic License (did not do the research on RL topic) or Series Continuity Error (did not do the research on this work or series).

I believe Memers was responding to the idea of making Did Not Do The Research a trivia article, which would necessitate the kind of specificity they were talking about in 11 since trivia articles require the described concept to be external to the work. And Artistic License is not trivia, meaning concerns those inaccuracies that exist in the work. I already said I disagree with the idea that the inaccuracies are external to the work, so I really don't have a better way of explaining this (since I don't think it makes sense).

Look at all that shiny stuff ain't they pretty

The way I see it, the definition as it is now should be thrown out and changed to something objective, rather than a subjective appeal to "a cursory knowledge in the subject." Something like:

"A *Critical* Research Failure is an inaccurate element in a story that is so massive, or so important to the plot of a story, that its correction would de facto make the story impossible and/or unrecognizable."

Example of NOT CRF currently listed as CRF: A character in 50 Shades of Gray says he couldn't pilot a helicopter due to ashfall from a volcano five years ago, but the last eruption was more like 10 years ago, or something (?). If you change the date of the eruption (or really, pretend the eruption is fictional, or happened in a fictional volcano), the book's plot remains the exact same, because the story is not about not being able to fly a helicopter due to ashfall.

Example of actual CRF: In the Jurassic Park book, the Costa Rican Air Force bombs the fictional island with napalm. In reality, Costa Rica doesn't have an air force, and the closest equivalent (the Air Vigilance Service) is made of a dozen small transport planes and utility helicopters for civilian use that are not fit to bomb a thing. If you correct that, you are left either with a whole different ending where Costa Rica has to let the dinosaurs live for the time being, or with another different ending where Costa Rica tells a country capable of bombing the island about the dinosaurs and they agree to do it. Even changing the country is problematic because Costa Rica is basically the only country in the region that is both developed enough to support In Gen's project and small enough to be pushed aside by In Gen before it takes over the project.

tl;dr version if you prefer:

A CRF: A lost mountaineer's desperate struggle in the non-existent mountains of Illinois (because the only way to correct this is to move the story to a place with mountains).

Not a CRF: A shot of mountains in the background of a movie about a boy and his dog in an Illinois suburb (because if you delete the mountains you have the same movie).

edited 10th Dec '17 10:13:15 PM by Naram-Sin

I don't know... Your definition of CRF sounds like Artistic License.

& Costa Rica not having an air force is hardly common knowledge.

Health sure is versatile. It's possible to be both light-headed and dim-witted. At the same time, no less.

They're right: a Plot Destroying Research Failure isn't necessarily obvious or common knowledge.

Join the Five-Man Band cleanup project!

Your not-Critical Research Failure example is Artistic License – Geography. Meanwhile, your is-Critical Research Failure example is literally The Mountains of Illinois. Neither of those things are "common knowledge" and so neither of them could be CRF. Moreover, if you are from the region in question, then it could be "common knowledge" and so could be CRF if you add it as an example and explain how "everybody knows that" or how it's supposed to be "common knowledge." This is, I think, one of the reasons why this is a YMMV article.

This sounds way too similar to what people tried to make Some Anvils Need to Be Dropped for me to even humor you about it. Sorry, but I don't think that is possible to be sustainable. It's practically "Artistic License but done well." That isn't what this article is about.

edited 11th Dec '17 1:29:51 PM by WaterBlap

Look at all that shiny stuff ain't they pretty

This seems like a tricky one. I think we can all agree this trope is legitimate: I.E. a research failure so bad as to be discrediting all by itself. But applying it to outside universe examples is, as many others have stated, very subjective as to what would qualify for the trope. Anybody adding these examples is likely to have more knowledge on the subject matter than a typical person, so it's very difficult for them to be able to be objective as to what would or would not qualify. To me, something seems wrong when entire pages are devoted to the examples of specific people, since very few have the sorts of knowledge to be able to tell what facts are off.

And, much like the reasons for when the Did Not Do The Research subcategories of X does not work that way or you fail X forever were renamed to Artistic License, some of them aren't that detrimental to the experience and may even hinder it. The Costa Rica example, for instance, is not something that is immediately obvious in such a manner.

If the examples we have are to be kept, we will need to have a stricter definition of what qualifies, or consider which examples might be better off in other tropes such as Cowboy BeBop at His Computer or as some new trivia page.

Uh, Costa Rica is quite famous for it's military, or lack there of. That's like saying Switzerland's neutrality isn't common knowledge.

Maybe you didn't know it, but anyone who knows anything about the region in question did. I don't know how the government of Portugal works and haven't since Cavaco Silva left, but I'm sure if I got a work set in the "present day" published where the country was a princedom ruled by Ariel it'd cause a few eyes to widen unless I've made it obvious I'm attempting comedy(I'm going to start writing that after I click send). Maybe not your eyes but there would be entries on a wiki like ours just like the Jurassic Park example if I tried to play it straight. Not everyone can read English either, but pretty much everyone who does knows "no". So if you, an English reader, see "no" where you're used to seeing "of" or "from" go, you probably double take before you process such an incredibly obvious mistake.

That's why he wants you to have the money. Not so you can buy 14 Cadillacs but so you can help build up the wastes

^ Consider geographical myopia. Living in Europe will get me a better change of knowing about Switzerland's neutrality than Costa Rica's lack of an army.

Just to throw in my two cents, I think a work's target audience should be factored into whether or not something is considered a CRF.

For example, CriticalResearchFailure.Game Theory (Web Show) lists an example where the "Exposing Metroid's Hidden Threat" theory is outright disproven by a specific line of dialogue in Metroid Fusion. Does the average person know that an AI scanned the creatures and confirmed that they were not infected? No. However, that video is not targeted at the "average person" but is instead clearly targeted at Metroid fans, especially those who have played Fusion. Does the average Fusion player know about this line of dialogue? They certainly should, given that this dialogue is part of an unskippable story cutscene that was probably included to quell any concerns about the creatures being infected in the first place. That should qualify as a CRF, since the entire video's premise falls apart due to the failure to account for something that the video's target audience should be aware of.

So, if a work involves the Costa Rica military and is targeted at people who should be familiar with Costa Rica, then I'd say that qualifies as a CRF. If the work's target audience isn't as familiar with Costa Rica and its military (or lack thereof), then it becomes more subjective regarding whether or not it's common enough knowledge to qualify as a CRF.

... until SUDDENLY DINOSAURS.

To an better example, the recently added example about Pokemon moves is something relatively obscure(I guess that more players not memorized the entire moveset of many Pokemon).

edited 8th Jan '18 5:51:50 PM by MagBas

Do we need a crowner on what to do with this trope? This thread is going nowhere.

Join the Five-Man Band cleanup project!

I feel like we should limit this to In-Universe Examples Only partially because of the Did Not Do the Research-like Trope Decay mentioned earlier in the thread and partially to avoid arguments about what counts as an example.

Patiently awaiting the release of Paper Luigi and the Marvelous Compass.