Upworthy Touts New Metric [Updated]

Eschews page view for 'total attention'

Upworthy, the social cause driven content upstart, says the page view is a lousy metric. It also believes that time spend doesn't cut it either when it comes to gauging success for a Web publisher.

So the company is offering up something different: attention minutes. Going forward, Upworthy says it's going to track total attention on its site and total attention per piece.

In a blog post, the company declared "If we’re trying to maximize attention for meaningful content, let’s actually solve for that."

However, just how Upworthy plans to calculate total attention is a mystery—which could further cloud Web metrics rather than providing an alternative. According to the blog post, attention minutes is a "fine-grained, conservative measure of how long people are engaging with the content on our pages." The new metrics factors in when videos are playing, mouse movements, what browsers are open and other factors. Upworthy just isn't saying which factors are tracked exactly or how they are crunched together to come up with attention minutes.

Rest assured, Upworthy has a lot of these minutes. During its last quarter, the company tracked 7 million attention minutes per day—or "13 years of attention, every day. We’re pretty proud of that."

Upworthy co-founder Peter Koechley explained that the company's new formula tries to take into account how actively engaged a user is with content—an increasingly complicated challenge as more publishers churn out text, photo and video content—while discounting open browsers that sit ignored on desktops.

"This is not out of left field," he said. "We have metrics now like time spent, and we think it works well for ecommerce but doesn't work as well for media and content. It's not terrible, it's just not really as fine grained. So we look for any signal that we think is meaningful. Mouse movement, clicks, any signal that people are still on page."

Sounds great. But who says anybody in the industry is going to follow Upworthy's lead? Well, the company plans to release the source code so that any other publisher can implement total attention, if desired. Koechley acknowledged that the Web business can move slowly, but is banking on an overarching desire to track the effectiveness on online ads better.

"If I'm an advertiser spending half a million dollars, need to find out that people aren't just letting a banner load," he said. So we just send a little ping every few seconds that essentially says, 'are you still there?'"

Here's the full post:

What Uniques And Pageviews Leave Out (And Why We're Measuring Attention Minutes Instead)

We’re big believers that you are what you measure. Our mission here at Upworthy is to draw massive amounts of attention to the most important topics. So, how do you measure that?

We dabbled with pageviews, but that’s a flimsy metric, as anyone who’s suffered through an online slideshow knows (20 pageviews! Zero user satisfaction!). Pageviews are only a great metric if you’re being paid for each pageview; we don’t run banner ads, so they’ve never meant as much to us.

Unique visitors are fine but reward breadth over depth of user experience. Shares per piece of content are quite a valuable signal, but they don’t get you all the way there. And time on site, as Google measures it, works great for e-commerce but is often confusingly broken for media companies. Google Analytics at one point had us at 21 minutes on site per visit on average; we’re good, but we know we’re not that good.

So we decided we needed a new approach. If we’re trying to maximize attention for meaningful content, let’s actually solve for that.

Introducing attention minutes, Upworthy’s new primary metric, which we’re planning to track in two forms:

Total Attention on Site (per hour, day, week, month, whatever) — that tells us (like total uniques or total pageviews) how good a job Upworthy is doing overall at drawing attention to important topics.

And Total Attention per Piece, which is a combination of how many people watch something on Upworthy and how much of it they actually watch. Pieces with higher Total Attention should be promoted more.

We love thinking this way because it rewards us for sharing content that people really enjoy and find valuable — not just stuff they click on a lot. It may mean that we don’t do quite as well on uniques or pageviews, but that’s a tradeoff we’re happy to make because this is a metric focused on real user satisfaction.

How does it work? Attention minutes is a fine-grained, conservative measure of how long people are engaging with the content on our pages. YouTube, Chartbeat, and Medium are all moving in a similar direction: They’ve all recognized the advantages of measuring whether visitors are actually engaged with their content and rolled out similar measures in recent months.

Our implementation is far more precise than “Time on Page” as it’s usually measured. Time on Page generally relies on a very sparse set of signals to figure out whether viewers are still paying attention. And especially on the last page of a visit, it can be hugely misleading. (Here’s a handy explainer about why that is. Incidentally, our Average Time on Page was 5:24 in December and January according to Google Analytics. But we just don’t think that metric is terribly reliable or useful for our purposes.)

We built attention minutes to look at a wide range of signals — everything from video player signals about whether a video is currently playing, to a user’s mouse movements, to which browser tab is currently open — to determine whether the user is still engaged. The result is a fine-grained and unforgiving metric that tells us whether people are really engaged with our content or whether they’ve moved on to the next thing.

On the charts below, we show what attention data for three popular posts looks like from the last couple of months. As you can see, attention follows very different patterns for different posts. The first one looks great if you only look at pageviews. (1 million pageviews! Woohoo!) But when you look more closely, it’s not driving much attention per pageview. The second post had far fewer pageviews but more total attention minutes. And the last post had about the same pageviews as the first but vastly more total attention minutes.

Being able to make charts like this helps us get past the focus on pageviews to really see what’s grabbing our visitors. They allow us to focus not just on how many people landed on a page but how many people watched a substantial portion. That’s how we define our biggest successes.

Last quarter, the Upworthy community spent more than 7 million attention minutes on our site per day. That’s more than 13 years of attention, every day. We’re pretty proud of that. But we’ve got a long way to go. (For reference, American adults watch 79,000 years of television per day.)

We also think this is how social networks will want to think about the content being shared on their sites — not just in terms of clicks or shares but in terms of the what’s grabbing users’ attention. A comprehensive approach that looks at clicks + watchability (attention minutes) + shares + engagement (comments and Likes) can give a strong signal about which content users find highly satisfying and rewarding, and which content they’re bored by.

In the coming months, we’ll be making the details of our implementation, including the source code, public; we’re excited to have other media companies engage with this metric. We think that adding attention minutes to the arsenal of metrics publishers look at will accelerate the drive toward quality. The media landscape is constantly changing, and how we judge success needs to evolve constantly, too.  We think attention minutes is a step in a better direction.