Psst! How Do You Measure Buzz?

It's the Wednesday of Advertising Week, and industry leaders are holding court all across Manhattan. At the Time Warner Center, David Verklin of Carat is expounding on the state of media. At NYU's Kimmel Center, Michael Jeary of Della Femina Rothschild Jeary and Partners is presiding over the Account Management Conference of the American Association of Advertising Agencies. But at the Marriott Marquis in Times Square, something quite different is happening. Seth Godin, the bald, elfin author of such provocative titles as All Marketers Are Liars and Unleashing the Ideavirus, is painting a universe in which the worlds of Verklin and Jeary barely matter, if at all, any more. In an hour's time, he layers observation upon observation, telling the crowd of 250 that they "are in the same business as Giorgio Armani"; that Apple, with its talent for producing talked-about products, isn't a computer company at all; and that the Mona Lisa is priceless not because it's a great work of art but because it's a notorious one, famous mostly because it was stolen from the Louvre in 1911. In short, as the keynote speaker at the Word of Mouth Marketing Association's "Word of Mouth vs. Advertising" conference, Godin is preaching the gospel of buzz marketing. He's espousing the notion, more pervasive than ever, if still somewhat thorny, that advertisers can succeed beyond their wildest dreams without paid media and even without a flashy product to sell. (In fact, Godin reserves particular enthusiasm for LittleMissMatched, a company that sells mismatched socks, in odd numbers only, to teen girls. Wearing socks that don't match, Godin says, is buzzworthy.) If you're in marketing, the thought of provoking rampant word-of-mouth for your product may leave you drunk with possibility. But then the hangover hits. Today's marketing environment requires that you measure what you do. And how on earth do you measure a couple of teenagers gushing about socks? Indeed, the rise of word-of-mouth and buzz marketing couldn't come at a better time—or a worse one. On the one hand, it offers marketers the chance to bypass increasingly spotty communication channels and avoid the middleman known as paid media in favor of one of the oldest and most effective forms of communication. On the other hand, it is enormously difficult to quantify return on investment from this most ephemeral of media. A whole cottage industry has sprung up around the demand for word-of-mouth measurement. But so far, not even the companies that are the best at monitoring word-of-mouth feel they've cracked the code. As if measurement weren't hard enough, word-of-mouth proponents also must contend with continued attacks from consumer-advocacy groups, who equate buzz marketing with stealth marketing. Just last week, Commercial Alert levied a stinging charge: that buzz marketers perpetrate "large-scale deception" and should be investigated by the Federal Trade Commission. Despite the challenges, though, a small but devoted group is rigorously pressing forward with word-of-mouth. Its most prominent faces include Mike Nazarro and Pete Blackshaw, two former Procter & Gamble executives who are now CEO and CMO, respectively, of Cincinnati-based Intelliseek; Jonathan Carson, CEO of New York-based BuzzMetrics (which was bought late last month by Trendum, an online measurement company minority-owned by VNU, which also owns Adweek); Andy Sernovitz, a longtime Internet consultant who is now CEO of the Word of Mouth Marketing Association (WOMMA); Walter Carl, an assistant professor at Northeastern University who has conducted widely quoted research on the topic; Steve Knox, CEO of Procter & Gamble's buzz-marketing division, known as Tremor; and a smattering—but only a smattering—of ad execs, including Jamie Tedford, svp of marketing and media innovation at Havas' Arnold, and Dan Buczaczer, vp and director of Starcom MediaVest Group's Reverb unit, which was formed in March 2004 with the goal of getting consumers talking about brands. These are the sorts of people who might not be reading Phil Dusenberry's new memoir, Then We Set His Hair on Fire, but do quote freely from The Cluetrain Manifesto, a book about consumer control written by a quartet of Silicon Valley insiders. One of the latter's declarations is, "We are immune to advertising. Just forget it." The word-of-mouth cluetrain has a destination, what Knox calls "the holy grail," which is determining the ultimate impact that consumer engagement with a brand has on a brand's bottom line. Less clear is the path to get there. If there is someone out there who truly does understand what a 10-minute conversation between friends about the Swiffer might mean for the brand, he or she has yet to come forward. WOMMA is one entity that's attempting to point the way. Founded in May 2004, it began work immediately on two fundamental tasks: developing a code of ethics to take a stand against deceptive word-of-mouth practices; and writing a dictionary of terms for the medium, which would formally distance it from stealth marketing and also jump-start measurement. The code of ethics, which forbids the use of stealth marketing and calls for honesty in all aspects of word-of-mouth, has not completely fended off attacks from consumer-advocacy groups. First of all, a number of word-of-mouth marketers, such as Tremor, do not belong to WOMMA. And besides, the code is merely a set of guidelines, not legal rules. (In its letter to the FTC, Commercial Alert suggests "issuing subpoenas to executives at Procter & Gamble's Tremor and other buzz marketers that target children and teenagers, to determine whether their endorsers are disclosing that they are paid marketers." A P&G rep would not discuss the letter directly last week. The rep did say that the teens involved in its word-of-mouth campaigns are not required to disclose their involvement with Tremor, nor are they required to even talk about the products that P&G exposes them to. They can handle disclosure "whichever way they feel compelled," says the rep, adding that this is what ensures "genuine feedback.") The WOMMA dictionary, meanwhile, has helped structure what Sernovitz described as "a total linguistic disaster." Not only were people using terms interchangeably, but many companies had copyrighted certain words. "I think too many marketers use 'buzz marketing,' 'viral marketing' and 'word-of-mouth' interchangeably," says Chris Marriott, vp and general manager of the Eastern region at New York-based i-shop Digital Impact. If wrestling with definitions seems like an exercise in semantics, look back to the early days of Internet measurement, when people couldn't agree on definitions for "hits" and "page views," let alone standard sizes for ad units. People who work closely with WOMMA saw the so-called "terminology framework," drafted by its Research and Metrics Council, as a necessary step. "To not do it would be a problem," says Terry Pittman, executive director at Time Warner's America Online and a member of the council. "I thought it was actually amazing that Andy was putting so much energy into it." In other ways, though, the analogy to the Internet is limited. While metrics like hits and click-throughs are black and white, the shadings that distinguish buzz from viral from word-of-mouth will probably always contain some gray areas. Nonetheless, WOMMA's terminology begins the journey toward standardization. As Gregory Wester, vp of in-call network at VoodooVox, who helped draft the WOMMA terminology, puts it, it gives some guidance to a community that is "enthused, clueless and vulnerable." Here's a snapshot of a few WOMMA definitions: Word-of-mouth: "The act of consumers providing information to other consumers." Word-of-mouth marketing: "Giving people a reason to talk about your products and services, and making it easier for that conversation to take place." Buzz marketing: "Using high-profile entertainment or news to get people to talk about your brand." Viral marketing: "Creating entertaining or informative messages that are designed to be passed along in an exponential fashion, often electronically or by e-mail." The group even further defines "product seeding," "evangelist marketing," "influencer marketing" and the difference between "organic word-of-mouth," which refers to people expressing genuine enthusiasm for a product or service without marketer involvement, and so-called "amplified word-of-mouth," in which a marketer initiates or accelerates word-of-mouth. Of all the verbiage in WOMMA's initial framework, nothing jumps off the page quite like the so-called WOMUnit—a proposed unit of measurement for word-of-mouth, like the 30-second spot for TV. The WOMUnit is far from becoming a standard, but in theory it would represent "a single unit of marketing-relevant information shared by a consumer." Its value could be further defined by five criteria: topicality (the degree to which a consumer stays on the marketer's message), timeliness (whether the message is delivered at a relevant time in the product's marketing cycle), polarity (whether the discussion is positive or negative), clarity (whether the person receiving the WOMUnit understands the message) and depth (whether a person is passing along an e-mail vs. having a 10-minute conversation with a friend). WOMUnits could also be broken down, as Sernovitz puts it, into other "component pieces of the word-of-mouth experience," giving weight to such factors as how influential certain people are in word-of-mouth and how widely a message is distributed. While no one is suggesting that the creation of the WOMUnit will one day lead to something as concrete as a word-of-mouth upfront, those involved in the drafting process feel it's a start in helping companies learn how to budget for word-of-mouth campaigns, measure their ROI and tie them back to metrics used for offline campaigns. According to BuzzMetrics' Jonathan Carson, a co-chair of the Research and Metrics Council, WOMMA wants "to demonstrate to brand marketers that word-of-mouth is something they can reliably and dependably spend money on." Sernovitz isn't sure WOMMA will ever come out with more specific guidelines for measurement. At this point, it has contented itself with publishing a book of some 30 white papers on the subject. "It's really, really early still," Sernovitz says. "It's too early to be setting hard and fast rules." Many caution that whatever measurement models do evolve, they shouldn't adhere too closely to metrics like CPMs, reach and frequency. "If that's where we stay ... we suboptimize the thing," says Tremor's Steve Knox. While the technology framework goes to great lengths to build structure, it also acknowledges the fluidity of what it's trying to capture. It tries to avoid "bias toward word-of-mouth campaigns, as compared to 'organic' word-of-mouth" spread by consumers and bias toward online media. But it's worth asking if WOMMA, in trying be inclusive of an entire spectrum of consumer chatter about brands, has bitten off more than it can chew. Dell is an example of why that might not be the case. In September, the Austin, Texas-based computer giant initiated its first-ever word-of-mouth-only campaign to launch the Dell DJ Ditty, a tiny music player modeled after Apple's iPod shuffle. The centerpiece of the effort was a Web site called DellDitty.com, which features a paunchy, sweatsuit-wearing hipster wannabe named Mitch Ferrence who stars in mock instructional videos, replete with bad '70s-style graphics, for "skills" like lip-synching and playing air guitar. Judging by DellDitty.com, the company, a member of WOMMA, comes off as an Internet- and buzz-literate marketer. At the same time, a very different message about Dell has been spreading throughout the Internet recently, and it's a negative one. Jeff Jarvis, an influential blogger who runs a media and politics blog called BuzzMachine, posted an item in June criticizing Dell's customer service. The anti-Dell sentiment spread quickly. The initial post garnered 122 comments from visitors, and Jarvis' misadventures with Dell became episodic entertainment. Media outlets including BusinessWeek, Salon and Slate picked up the story. Jarvis even got a response from the office of Dell's chief marketing officer, although he still switched to an Apple PowerBook. (A Dell representative, when contacted for this story, said the company was "in the midst" of making changes in its approach and was looking to have greater collaboration among its marketing, PR and product departments.) So, has Dell's overall word-of-mouth experience in the past few months been positive or negative? How positive? How negative? And if WOMMA won't answer those questions going forward, who will? The answer is, a number of companies, and they fall roughly into two camps: those that believe online measurement alone is a highly effective way of quantifying overall word-of-mouth, and those who believe it is one tool among many. Most agree that the Internet is crucial to the process. "The reason that word-of-mouth marketing is becoming a tangible marketing practice is because of the Internet," says BuzzMetrics' Carson. However, everyone, including Carson, tends to quote the same statistic: that an estimated 80 percent of word-of-mouth branding communication occurs offline. To those who focus online, that's 20 percent more than was available before the Internet. And much of it is authentic and unfiltered opinion. When he worked at P&G, says Intelliseek's Blackshaw, it often wasn't until the last day of a four-day focus-group session that the company figured out what the right questions were. Plus, the quality of feedback can be unsurpassed. A marketer can actually gauge whether a comment "was just positive, or was it joy, exuberance? Was it just negative, or was it outrage?" says Blackshaw. Both Intelliseek and BuzzMetrics use a combination of technology and human analysis to measure word-of-mouth. Intelliseek claims, for instance, that it can track 6,000 online communities, 45,000 newsgroups and more than 17 million blogs. Its BlogPulse Web site serves as a window into what's here and now in the online universe: It lists the 40 posts that have been linked to most in a given day, and shows how the most prominently featured celebrities are faring compared with the day before. Take this kind of data mining and apply it to marketing, Blackshaw says, and what you have is information with "much bigger consequences across all other advertising vehicles." A pre-Super Bowl survey by Intelliseek earlier this year found that by far the most online consumers (45 percent) expected Budweiser to have the best ads on the game, with only 17 percent giving the nod to Pepsi. Those statistics may fall into the realm of fun factoids, but research into what consumers think about marketing messages can point marketers in directions they never would have considered, Blackshaw notes. Carson offers an anecdote about a study his company did late last year in the artificial-sweetener category. It showed growing negative sentiment toward Splenda, but not for the reasons that were originally assumed. The obvious potential cause was an aggressive anti-Splenda campaign from The Sugar Association, which included a Web site, truthaboutsplenda.com, which called many Splenda claims unsubstantiated. Tracking 90 online forums, the study showed that much of the negative sentiment had to do with consumer claims about side effects, not necessarily some of the core issues brought up by the sugar-industry campaign. Still, some experts question how much online word-of-mouth research reflects, at this point, the rest of the population. At the WOMMA conference, Eileen Campbell, president of global development of WPP Group market-research agency Millward Brown, said she felt online data about the youth-skewing Toyota Scion would probably be much more "projectable" than that for the Buick LeSabre. Campbell advocates an approach in which online data is integrated into a larger branding assessment. It's the 80 percent of brand conversations that take place offline that concern P&G's Tremor and other companies. Tremor, which exposes its teen members to new product ideas and marketing programs, tests the efficacy of its word-of-mouth campaigns in the most traditional of ways. It will conduct pre- and post-testing, for example, in a city where it has seeded a message and compare it to a control city (taking into account the complicating factor that even the control city can be "contaminated" by the buzz). Knox says Tremor focuses so much on offline talk because more direct contact, in person or over the phone, is much richer than online contact. "Buzz and viral are insufficient vehicles unless advocacy is attached to it," he explains. (Tremor is in the process of expanding its program beyond teens to include mothers.)Another key company engaged in offline measurement is BzzAgent in Boston, a WOMMA member that recruits unpaid volunteers to join word-of-mouth efforts and report their interactions back to the company. (There are non-cash incentives for participation.) "Our goal as a business is to activate honest, authentic world-of-mouth as it is actually happening," says BzzAgent founder and president Dave Balter. In its four-year history, BzzAgent has worked for companies like Anheuser-Busch, General Mills and Volkswagen. A recent effort called the Alpha Drivers Program gave VW loyalists the inside track on the new Jetta and input into its design, with the goal of spreading the word beyond the Alpha Drivers themselves. An opt-in e-mail inviting people to the program "blew away" direct response metrics, says Arnold's Tedford. And while the campaign also resulted in strong sales for the car among the Alpha Drivers, it also helped Arnold and VW figure out "how far out from that initial Alpha Driver we could bring the conversation." Volkswagen has since moved its account to Crispin Porter + Bogusky, an agency closely associated with buzz. But Jim Poh, director of creative content distribution at CP+B, takes word-of-mouth measurement with a grain of salt. While the end goal, he says, is always to drive sales, Poh is less interested in efforts to draw a precise road map for how word-of-mouth gets you there. In fact, he says, it might hurt. "The more you measure it, the more you find ways to manufacture buzz," he says. "And manufactured buzz starts to feel like that." In that regard, Poh may be in the minority. Most who are working in the space believe that word-of-mouth, to grow as a tool among big marketers, needs to be as quantifiable as the paid media it so assiduously avoids. As Knox says, "We just really want to get this measurement thing nailed."

Brand Marketing

Psst! How Do You Measure Buzz?

Catharine P. Taylor

About

Subscriptions

Events

Publications

WORK SMARTER - LEARN, GROW AND BE INSPIRED.

Subscribe today!

Catharine P. Taylor