Progressive Marketers Are Tracking a New Metric: Share of Model

Measuring how large language models perceive your brand will become essential

Mark your calendar for Mediaweek, October 29-30 in New York City. We’ll unpack the biggest shifts shaping the future of media—from tv to retail media to tech—and how marketers can prep to stay ahead. Register with early-bird rates before sale ends!

Every marketer dreads bad reviews of their brand. Now with the rise of AI-powered chat programs like ChatGPT, Meta’s Llama 2 and Microsoft’s Copilot, marketers must prepare for the worst words about their brand to be repeated as answers to search queries.

Since large language models (LLMs) will soon answer billions of search queries each day, every marketer must know what these models think about their brand.

More advanced marketers are starting to track an entirely new metric: Every good CMO knows their share of the market, but do they know their “share of model”? Measuring how each model perceives your brand, compares it to competitors and why it suggests your products to customers will become an essential responsibility for every marketing team.

Here are the three things to do now to ensure LLMs understand what makes your brand unique, and why your product is worth considering.

First impressions last

Every AI researcher will warn against expecting a model to behave like a human. But they don’t have to sell cars, coffee and credit cards for a living.

The easiest way to understand the implications of an AI-everywhere world is to look at how customers behave today. When you want to check whether a brand is reputable or relevant, you often visit its website. And, with every major language model soon connecting to the open web to answer queries, they will do that too.

Even the most iconic brands will struggle to change (or even see) the historical training sets for the foundational models that will soon power the majority of the internet, from search queries to content playlists, product recommendations and customer service. So, every single piece of content you control outside that training set becomes a vital opportunity to advocate for your brand and your products.

This is why reviewing creative assets is the first step for brands preparing to measure and optimize their share of model. Just as search engine crawlers look for keywords and cues in your metadata, every asset you’ve ever published online could be part of a model’s reasoning for the next review.

The only way to understand what the model will think is to see every single piece of content from its perspective. Humans may like your Instagram Reels, but do they teach models that your cars are reliable? Those exquisite long-form recipes on your website may help your page rank, but do they seem a tad indulgent for an LLM’s taste?

Every model has its thought process. Once you know how each model perceives those assets, you can optimize them to double down on the preferable perceptions. Think of it as the most important focus group imaginable with just four attendees: Gemini, Copilot, GPT4 and Llama.

When in doubt, ask

If you’d like to dig deeper into what the models have already learned, all you need to do is ask them.

Just as you would track brand sentiment across different audience segments, you should be prompting and comparing different models. The beauty of LLMs is that they don’t get bored in a focus group; you can prompt regularly to establish a baseline and ask progressively more complex questions about audiences, markets and proof points.

The results can be striking, as an iconic airline discovered when our team showed them that a model already responsible for over 1 billion unique conversations to date was convinced they were best suited for retirees.

With this process, together with what you already know from reviewing how each model perceives your brand, you can then match the most effective content to each model, progressively optimizing your share of model.

Optimize and observe

The good news for marketers is that these models are eager to learn. The bad news is that they never stop learning.

Nothing in an AI-everywhere world is static, and brands have to consider that a new set of metrics tracking model perception will be just as important as net promoter score (a market research metric that is based on a single survey question). It’s vital to establish share of model as a metric on par with share of market or share of voice.

As the handful of models that will dominate continue to become more unique over time, we have to constantly track their perception and optimize accordingly. Regularly reviewing their outputs will help recalibrate creative briefs to grow share of model and isolate the assets driving changes in perception.

Regardless of how often you check your share of model, it’s undeniable that models will soon shape your share of market. While many of the emerging best practices for optimizing your generative AI performance start in search, the greatest impact will come from social.

With Meta investing more in the Llama family of models and rolling out AI-enabled experiences in millions of group chats, marketers must learn the fundamentals of an AI-everywhere world and do so now, while we’re only focused on the handful of LLMs dominating the discussion today.

If you don’t embrace these models as new members of your target audience today, then you may find yourself without an audience at all tomorrow.