Why The Traditional Focus Group Is Dying

It's hard to believe Indianapolis Colts star quarterback Peyton Manning needs any introduction. He is, after all, the leader of the NFL's top-ranked team and part-time pitchman for MasterCard International. But after copy testing on the latest "Priceless" commercial revealed that a significant number of consumers weren't aware of Manning's fame, the company's advertising agency, McCann Erickson, New York, decided to tweak the spot to spell out his identity before airing it this month.MasterCard frequently turns to Millward Brown, one of the world's leading research agencies, to test copy. Testing is as routine for MasterCard as it is for packaged-goods behemoths like Procter & Gamble. Bill Cook, svp, research and standards, at the Advertising Research Foundation, estimates that more than 90 percent of the top 100 U.S. marketers use some kind of quantitative copy testing for at least some of their brands.While many creatives insist that good ideas don't need to be tested and that testing can often weaken a creative execution, most say it is hard to find a marketer willing to take an expensive gamble on their ad ideas without some early indication that their efforts will resonate with consumers and eventually impact sales. Ad agency executives are generally not averse to testing, but many, particularly those in creative roles, object to the way some marketers use test results as an on/off switch in deciding whether to produce or scrap a campaign. Agency execs and research firms say they prefer when marketers test concept ideas early to determine if strategy is sound, but some clients say they prefer to put a finished spot in front of consumers and gage reaction that way. "When it works in your favor, it can be incredibly informative and productive," says Kathy Delaney, co-president of Deutsch in New York. While acknowledging that the process is flawed because it's tough to get "the right people in a room," she says testing during conceptual development can yield interesting findings. For example, when Deutsch was developing a campaign for a pharmaceutical client that was advertising a drug for a life-threatening condition, the agency initially wanted to mobilize consumers to better understand the condition. However, when copy tests showed people were so frightened by the disease that they were immobilized, the agency revamped the ads to show the importance of the afflicted taking action against the disease. "You can gain insight into consumers that you wouldn't normally have," says Delaney. "It's a great way for clients to feel confident that their messaging is right." The great debate over how, when and whether to test at all is certain to grow louder, as the ad industry is conducting a wide-scale evaluation of current testing practices for the first time in 50 years. The assessment of the standard methodology comes at a time when agencies are increasingly pressured by their clients to better determine the effectiveness of their advertising with return-on-investment measurements. The American Association of Advertising Agencies and the ARF launched its "Emotional Response to Advertising" study last year and introduced findings at Advertising Week in September that outline the need for a new direction. It's time for the industry to follow a new course that pays much more attention to measuring consumers' emotional responses to advertising, the study's sponsors argue. There's been no significant advancement in copy testing and tracking for decades, and current methods are causing advertising to become risk-averse and predictable, they say. Final findings of the study will be presented at the ARF annual conference in March 2006 and published in The Journal of Advertising Research."There's room for innovation in this category. We should be testing at a higher level," says Alice Sylvester, svp and account plan- ning director at Foote Cone & Belding in Chicago, who is serving as the study's project manager. "Testing has been very rational, and that's led to normative work." Testing methods have to change to incorporate unconscious responses to advertising and brand communications, she argues.While Alex Bogusky of Crispin Porter & Bogusky says he would prefer to see the elimination of testing, others advocate augmenting existing practices with more emotional measures. Larry Flanagan, CMO of global marketing for MasterCard International, says copy testing is helping the company evolve its award-winning "Priceless" campaign and keep it relevant and meaningful for consumers. Millward Brown's discovery that consumers not close to the NFL were unfamiliar with Manning led to a small change that he says made a huge difference. The commercial, called "Professional Fan," features Manning in a role-reversal, acting as a fan for common workers, like mechanics, waiters and grocery clerks. Manning introduces himself to viewers by asking a stock boy at a grocery store to sign his melon. He even spells his name for the surprised clerk, who dutifully signs the fruit. Even when testing helps fine-tune creative, test scores need to be taken with a grain of salt, argues Burt Manning, chairman emeritus of JWT and an advisor to Neuro-Insight, which strives to legitimize mass market emotion-based research. The Melbourne, Australia-based company, which launched about a year ago, has developed a headset and goggles that measure brain responses and chart emotional intensity elicited while viewers watch commercials. The technology measures the extent to which material attracts or repels viewers, says Richard Silberstein, neuroscience professor and CEO of Neuro-Insight. The company has established strategic relationships with Colmar Brunton, a midsized Australian market research company. Starting in Australia, Silberstein's hope is that the technology will be widely used by the industry."I don't believe our technology renders traditional market research techniques obsolete," says Silberstein. "In fact, I think the approaches complement each other. While N-ST can give an objective insight into what is happening, in terms of the psychological response of a target audience to communication or a product, it cannot answer the question why. N-ST may indicate, for example, that males are repelled by a particular character in a film. But the only way one can tell why ... is by asking the subjects."While the ad industry generally recognizes the value of conventional testing and the knowledge base that the top research companies, like Millward Brown, Ipsos-ASI, ARS, MSW Research, Ameritest and Gallup & Robinson, have amassed, those working to develop a new scientific method contend that the current process leaves room for significant error. For instance, audiences often have difficulty articulating how they feel about a brand or spot and sometimes mimic the responses of a dominant player in a focus group.That's why marketers like Aflac and the American Legacy Foundation are very careful in how they weigh test scores. If Aflac chairman and CEO Dan Amos had heeded the response of focus group participants, the company's now-famous duck would never have made it on the air. While many participants found the duck funny, others found it "insulting." Kaplan Thaler Group execs persuaded the company to allow research firm Ipsos ASI to test the spot along with four others it created and four from another shop competing for the account. The recall score of the shop's spot was off the charts, scoring 28 percent recall, the highest score Ipsos-ASI had seen in the insurance category at the time, according to Linda Kaplan Thaler in her book Bang!. KTG won the account, and the duck made its debut in December 1999.As for Legacy, if it had heeded early results of testing of its sitcom-style campaign, "Fair Enough," it would have killed the idea right then, says Joe Martyak, evp for marketing, communication and public policy. Some teens in focus groups had trouble understanding the storyboards and the relationship between Truth and a sitcom. But, confident that enough teens would understand the core message, the client moved ahead with the campaign. Prior to its launch last February, the client ran the work through what Martyak calls its "red-flag test" to see if teens from its 12- to 17-year-old target demo liked the finished spots. "They responded very favorably," says Martyak, adding that "no Truth commercials air unless they receive a stamp of approval."The Legacy Foundation does not go by the black-and-white elements of research, says Martyak. "Sometimes great ideas are so novel and innovative that if a consumer hasn't seen messaging like that before, they could react negatively. You've got to think carefully before you flick the switch, because you could be really on the mark and may need a dimmer as opposed to an off switch." The high number of visits to www.fairenough.com, the client's Fair Enough Web site—648,896 from Feb. 1 through Sept. 7, 2005—speaks to the campaign's popularity. Bogusky applauds his client for using testing to appraise and adjust ideas and not to decide whether to kill a campaign (Arnold in Boston shares duties on the account). While some CP+B clients, such as Burger King, never test, most do. More often than not, Bogusky says testing is pretty destructive. "Copy testing will never give you an indication of the quality of work," he says. "When used poorly, you are paying people $50 an hour to tell you what to do. It can kill all things indiscriminately."Bogusky praised the auto industry, which he says once relied heavily on consumers to help design cars, for putting design back in the hands of designers. "All cars of all companies began to look alike," he says. "Everything was an egg." He suggests marketers follow the lead of companies like Dodge and put creative back in the hands of advertising professionals. Like BK, which he says uses chefs to create menu items, agencies need to have the responsibility of making ads.Bogusky's stance is clear: "As an industry, it would be smart for us to reject [testing]." That, of course, is never going to happen. "Any creative director worth his salt who really thinks a client is going to lay down $100 million bucks without a high level of confidence of success is naïve," says Joe Plummer, the chief research officer of the ARF. Plummer is leading the industrywide project to better measure emotional responses. "We know emotion is important, but we haven't tried hard to measure it," says Plummer. "A lot of current evaluation methodologies ask consumers to think too much, far more than they do when they see advertising." Why change now? For one, there's been an increased questioning of the value of advertising, says Plummer. He says the level of clutter is getting unbearable to everybody: consumers, agencies and their clients. Said Plummer, "We're at the tipping point of social science. We can't ignore the findings anymore." Researchers have been experimenting with new methodologies in recent years. Measuring emotional responses has typically involved monitoring physiological changes. Companies like AnswerStream, for instance, test skin conductance, heart rate and vascular response. Gallup & Robinson uses facial muscle changes to assess emotional responses to media.Early in the 4A's/ARF study, four popular ads in the beer category were tested using various methods to uncover conscious and unconscious emotional responses. Six research firms evaluated four campaigns. Their initial findings show that storytelling ads create more emotional response than product-centric ads. "There are robust and insightful ways to measure emotional response," says Plummer. "We didn't have this when testing developed in the '50s and '60s." Helene Paulyn, design leader of holistic communications for P&G unit Consumer Market Knowledge, is on the 4A's/ARF advisory committee, and P&G is including some of its ads in the study. "At P&G, we really believe in the mantra of consumer as boss. Our consumers guide every action we take and every decision we make," says a rep. "We continue to use copy research ... as a guide in helping us make decisions on how best to reach consumers."As the ad industry struggles to secure its place in the communications business of the future, Plummer implores agencies, marketers and research firms to work together to advance testing methods. After all, he says, "if consumers don't like the advertising, don't respect it and don't respond to it, everybody loses."

Brand Marketing

Why The Traditional Focus Group Is Dying

Jennifer Comiteau

About

Subscriptions

Events

Publications

WORK SMARTER - LEARN, GROW AND BE INSPIRED.

Subscribe today!

Jennifer Comiteau