Are focus groups used badly?

So this blog post...

Ya know, whenever I ever heard developer/author/director/show runner/whoever saying that focus groups hated something so they changed thing to be less awesome, I've always assumed that it means they picked by bad luck a non accurate focus groups, but could it be that maybe executives just use focus groups to claim that stats prove they are right and ignore everything else? ._. I've never been in the one, but I wouldn't be surprised if they do use "loaded questions", my experience with entertainment related "how I felt" quez has always been "Dang, according to my psychology class on how you should do quez, this is done completely wrong".. I wonder if this comic's situation applies to most things that seems to get ruined by following what focus groups said?

edited 4th Jun '14 7:58:02 PM by SpookyMask

The answer is almost always "YES!"

Of course, don't you know anything about ALCHEMY?!- Twin clones of Ivan the Great

How did focus groups become such a major 'thing' in the first place, anyway? I can't say I recall any time where they've succeeded in turning something bad into something good. It's always either they confirm what's already suspected or they result in making things worse.

They can be used badly, especially when certain focus groups just complain to the networks about certain shows they don't like because of the content (looking at you Invader Zim), despite most of the fans loving the shows.

I love animation, TV, movies, YOU NAME IT!

There should be no focus groups.

Like anything else, a focus group can be used effectively or poorly. A poor focus group generally occurs when the demographics of the group doesn't match the demographics of the intended market well.

And by the way, they've been around for over a hundred years in media — only there the members of the focus group pay for the privilege of being in it: in theatre it's called "out-of-town tryouts", "opening in New Haven", or "opening out of town". It's a trial run before the show opens on Broadway, and many times, the out-of-town opening has resulted in huge rewrites and changes; the play that everyone knows and loves now is not the same one that went to New Haven.

...if you don’t love you’re dead, and if you do, they’ll kill you for it.

Focus Groups just lead to Executive Meddling that ruins the overall product - GET RID OF THEM and let the artist's do their damn thing! Western Animation in general will be much better off for it, too - especially Canadian fare. Same goes for 98% of European / Australian fare as well.

Even if I had different face, I AM STILL DISGRACED.

A counterpoint: Mass Effect 3.

Focus groups hated the ending, but were ignored in favor of artists doing their own thing, resulting in a De M/Asspull mess.

Sometimes, the artists just have bad ideas.

If your focus groups make things less awesome, you're doing the focus group wrong. Problems only come if you have a badly selected samples that don't sync up well with the intended audience (like I Am Legend).

Too bad too many exist ONLY to make things less awesome, just to please audiences the thing being made clearly wasn't meant for nor was intended to ever be for.

Even if I had different face, I AM STILL DISGRACED.

Only when they're bias selected. That said, they should be screened for all audiences (though they should keep kids away from more mature fare).

"YOU FILTHY SWINE!!! I WILL KEEEEL YOU!!!

PRC 4 Eva makes a very good point. Executive Meddling is one of those Not All Tropes Are Bad tropes, because sometimes its needed. My examples tend to come more from video games, though, but you can bet your ass Nintendo wishes they had focus-tested Metroid Other M instead of giving Sakamoto total creative control (though i really don't know if Nintendo "focus-tests" anything, but its the right idea anyway).

The problem with focus groups and Western Animation is the Animation Age Ghetto, both in general (the idea that only kids will watch) and in more specific ways (like a lot of boys watching something like Winx Club for the action, a show marketed to girls). This has to do with the fact that Western Animation, especially when children are the prime market and ESPECIALLY when its an action cartoon, is very merchandise-driven, so the executives will get a very preconceived notion about who they want the show to appeal to (which is who the toy companies want to sell to), creating this perfect storm of meddling.

All i would advocate for is a more procedural marketing setup: first launch the show, figure out who is watching, then build the merch around that. If boys want a magical girl warrior doll with karate-chop action, why not make one instead of (or alongside) the one with the vanity mirror and brushable hair?

Ugh, I really don't get it. Why do these focus groups continue to exist if they're so conformist-ish?

And that tumblr post showing the focus groups for GL:TAS really makes it hard for me to see these guys as anything but totally ignorant a-holes.

In fact, I'm starting to think that the reason we don't see so much focus on the Transformers in the Transformers movies is because of these focus group a-holes.

There seems to be a lack of humans reacting to the large robots.

Of course there are! This is primarily about the Transformers! Not the-

Look, I'm saying that if there aren't enough humans for the audience to relate to, there's a good chance that will no one will see your movie.

They'll be seeing the movie for the freaking Transformers! They're not going to give two shits about some random humans! To put in simple English, "Dude, Fuck You!"

Yet we still get constantly obnoxious humans in the dang movies.

edited 5th Jun '14 1:50:49 PM by TargetmasterJoe

The human military characters were cool though. And that's less the common version of Executive Meddling, and more just Michael Bay wants full Pentagon backing so he has the US military being awesome. And Shia Le Bouf wasn't *that* bad...

The hackers subplot I could have done without, though.

Those movies. THOSE. DAMN. DAMN. MOVIES. They're a prime example(s) of why Focus Groups need to die. Transformers does not deserve such crap - and decades from now, if we're still around, America will be lambasted for generations upon generations for such sucky movies existing. Same as Twilight franchise in all its sexist/misogynistic and asinine glory...

Because after all - suckiness, due to Executive Meddling, is a stench that stays with you forever. You'll NEVER wash it off.

edited 5th Jun '14 3:05:38 PM by TheShopSoldier

Even if I had different face, I AM STILL DISGRACED.

This is why focus groups are often not effective.

what do you mean I didn't win, I ate more wet t-shirts than anyone else

Because back in the day there was only three or four major television stations so networks became invested in getting the general populace to tune into their stations over their competitors. So like every other form of marketing, they resulted to focus testing.

Focus tests in theory really aren't that bad of a concept it's just the major flaw is when the prompters start asking loaded questions.

I thought the Transformers film series literally couldn't have the Transformers on-screen for too long because there were physical limits on how much CGI the computers could process.

That Tumblr cartoon about Green Lantern The Animated Series doesn't demonstrate that focus groups are worthless, but that they're just misused. Mr Showrunner Guy watched the focus group's collective reaction to the episode and concluded that kids liked the episode. The moderators saw the reactions and concluded that the episode was too confusing, and it needed to be changed before kids would like it. The DVD sales demonstrate that Mr Showrunner Guy was much closer to the mark.

The lesson isn't to do away with focus groups. The lesson is to analyze the raw data of the groups' reaction yourself rather than blindly relying on the moderators' conclusions.

Hate focus groups so much.

Those damn Like/Dislike dials are a big part of the problem. An episode of a TV series is a story—you can't break it down moment by moment and go "This part was good, this part was bad." Ultimately, how the parts work together is the most important thing.

Say you have an action-comedy-drama show, good mix of moods, about superheroes. Two of the superheroes are dating, and they're both snarky personalities which makes for fun, not-too-mushy dialogue. Early on in the episode, they have a comical argument—funny! The kids love it! This leads to a falling-out and they spend most of the episode Not Speaking, and hence not working well together. Fights against the superhero team's enemies go badly because the baggage from the argument (with other members of the team taking sides) is adversely affecting teamwork. Older kids like the drama and danger that results; younger ones find it too scary. At the turning point of the episode, the fighting couple makes up in a tender scene. Girls love it; boys find it "sissy." In the climactic battle, the team is back together and operating at full strength, they beat the bad guys, yay! Except girls think the fight scene was a little too long.

So if you were selling to boys, would you just cut out the makeup scene? But then the final scene won't make any sense! If it was aimed at a younger audience, would you skip the drama in the middle? But then you lose the high stakes that make the final battle so much fun. It's not as granular as the dials make it out to be.

You'd be hard pressed (to say the least) to get any of the companies involved to go along with something like that. Animation, especially of the action/adventure variety can be very expensive to produce. Time spent figuring out how the show is recieved and not merchandising or optimally advertising for your target demographic is likely seen as leaking money needlessly when they could have had all there ducks in a row from the beginning, or more likely, exploit and promote the know demogrphic patterns.

Companies are intensely risk averse and seem to be only getting more risk averse as time goes on (I don't if this actually true. I speculate that it has something to do with the current economic situation), and such a "risky" endeavor as you propose will likely get you kicked out of the meeting room.

On the subject of the comic/ glimpse into the the process I find it curious (but not too surprising) that all the girls were lumped into their own group. I'm assuming this is because it was a "boys show", which is itself a questionable decision. I wonder if it's the same but opposite for "girl's show". I wouldn't be surprised if they excluded boys completely in such a case.

Depends on how you use the output from the dials. The dials could be good for saying "the cross-viewer average spiked in scenes that featured Kramer," or "Moments when Jar Jar Binks was on screen saw a drop in viewer confidence."

Which goes back to the OP's question: Yes, focus groups are used badly. They are good for tracking certain things (like identifying The Scrappy or the Ensemble Dark Horse ahead of time), but are bad when attempting to apply them to the vagaries of narrative structure.

Malcolm Gladwell gave a Ted talk that I think is somewhat relevant to the discussion. Sometimes the things people *say* they want in focus groups aren't actually what they tend to go for.

"If I asked all of you, for example, in this room, what you want in a coffee- You know what you’d say? Every one of you would say “I want a dark, rich, hearty roast.” It’s what people always say when you ask them what they want in a coffee. What do you like? Dark, rich, hearty roast! What percentage of you actually like a dark rich hearty roast? According to Howard, somewhere between 25 and 27% of you. Most of you like milky, weak coffee. Which you will never, ever say to someone who asks you what you want- that I want a milky, weak coffee. (laughter)"

If you are interested in consumer psychology: http://www.ted.com/talks/malcolm_gladwell_on_spaghetti_sauce

One thing I remember coming out of focus groups was a brief run of have sugar sodas. In focus groups it seemed like a product that would appeal to both Regular and Diet soda drinkers, in practice they both kept drinking the sodas they were already drinking.

I'm more interested in seeing exactly how kids would react to a show like Johnny Test or Breadwinners, compared to Tron Uprising and Young Justice.

I don't even like to make comparisons like that. Still, I'm curious to know if there's any visible difference other than what the focus group moderator sees. Hmmmmm...

edited 6th Jun '14 12:50:48 PM by TheLemsterPju

Even when you can break it down like that, the like/dislike dials still get misused and misinterpreted. I recall a story about the focus-group for a particular comedy. According to the like/dislike readings, all the jokes fell completely flat—hardly a "like" for any of them. But, strangely enough, there were a bunch of "likes" for the quiet bits that fell right after the punchlines. What did this mean—that the writers were unintentionally great at writing funny small talk but terrible at intentional punchlines? No, it meant that audiences were too busy laughing to even touch their like/dislike dials until several seconds after the punchlines had passed.