Nestlé's AI-Powered Virtual Human Is Here to Answer Your Cookie-Baking Questions

Nestlé has deployed a new high-tech spokesperson to field all of your burning questions about baking its cookies. The snack giant rolled out an eerily lifelike virtual avatar this week that uses natural language AI and autonomous animation to answer basic questions about the company's Toll House chocolate chip cookie recipe. The cookie coach, named Ruth in honor of Toll House Inn founder Ruth Wakefield, can interpret and respond to a range of written or spoken queries. Orchid Bertelsen, head of digital innovation and transformation at Nestlé, said the project was inspired by the inconsistent experience that customers often reported after calling into the company's customer service line, which is staffed by employees who may not have direct baking experience themselves. The campaign is part of a bigger investment that Nestlé is making into AI and other emerging tech to improve its customer service experiences. "The creation of Ruth showcases our investment in tech innovation to ensure we’re meeting consumer needs," Bertelsen said. "We continue to monitor consumer behaviors, how they’re evolving, and look for ways to solve problems through digital innovation across our brand portfolio." How Ruth came to be Ruth has been in the works for more than two years in partnership with a New Zealand startup called Soul Machines, which specializes in making these kinds of lifelike avatars for brands. Her language capabilities are powered by sentiment analysis and dialogue creation systems from Google's natural language platform Dialogflow and Amazon Polly's text-to-speech capabilities. Bertelsen said the team added touches to make sure the bot was as lifelike and emotionally engaging as possible, such as an animation program that matches the movements of Ruth's mouth to the dialogue and support for both speech and text communication. "When designing Ruth, we knew how she made consumers feel was probably just as important—if not more important—than what she looked like," Bertelsen said. "Consumers told us she had to sound like a real person, and she had to be encouraging, empathetic and bring levity to the experience because baking is meant to be fun. We spent a lot of time getting the human interface right so people would remember how Ruth made them feel." As people begin to ply Ruth with their questions, Nestlé plans to analyze the conversations and use them to continually improve the experience that Ruth provides. The rise of branded avatars Nestlé claims to be the first major CPG company to deploy human avatars this way, but other companies are already exploring how such technology might be used for everything from customer service to sales to employee training. The burgeoning industry is driven by relatively recent advances in AI-powered language tech and lifelike animation capabilities. Last year, Soul Machines rolled out a digital brand studio that allows any company to customize its own virtual person using the startup's tools. At the time, Soul Machines chief business officer Greg Cross told Adweek the goal was to help client companies scale personalized and emotionally engaging interactions. "The objective here is not to replace people, it’s to really focus on things that are very, very difficult for organizations to deliver, like infinitely scalable customer interactions or infinitely scalable customer support at a completely different level of economics," Cross said in a previous interview.