Via my former colleague Luigi Montanez, now of Upworthy, here’s an interesting look at how the media industry is reexamining its use of analytics. The search for more meaningful measures of media efficacy is interesting in its own right. But I think the structural incentives that surround it deserve some attention, too.
It’s worth reflecting on the metrics that have fallen out of style: most notably conversions (how often an online message leads directly to a measurable action) and impressions (how often an online message is seen, perhaps affecting the viewer unconsciously). For the past few years there has been an outsize focus on the level of social activity spurred by a message — though this kind of result is increasingly viewed as a overvalued. In the past few months, there has been growing enthusiasm, including at Sunlight, for measures of how thoroughly a message is considered by its viewer. Upworthy’s attention minutes metric is leading the charge — unsurprisingly, given the organization’s undeniable sophistication at measuring and driving traffic.
Although it would be hard to completely deny a fad dynamic to these successive waves of focus, I think efforts to find better analytics have been driven by good intentions. But I also think these efforts may be leading us to a different future than many imagine. For instance, I wonder if my colleague Eric is mistaking a feature for a bug here:
@1uigi one nice thing about pageviews/visits: it's clear how they work. Proprietary "secret sauce" replacements would be bad (and unlikely).
— Eric Mill (@konklone) March 18, 2014
I can see why Eric thinks black-box metrics would be bad. But the bespoke nature of the new, increasingly in-house analytics trends carries advantages for those creating them. And this isn’t the first time that content creators have evolved toward capturing the mechanisms by which their own success is measured. A little over three years ago, before many of the aforementioned analytics trends occurred, I wrote this:
[B]y all accounts online advertising doesn’t work very well. You can measure whether someone clicks on an ad, and often whether they buy something after that click. But it turns out they rarely do those things. So businesses aren’t willing to pay very much for ad space on websites.Is it really a coincidence that the advertising medium with the best instrumentation also appears to be the least effective? I suspect it’s not. It may be that ads never worked as well as the industry had told us; or it may be that the eyeballs/clicks/conversions funnel is a naive conceptualization of how the system works. Either way, Google has succeeded by giving advertisers what they think they want, which is analytic tools that seem to reveal that the whole enterprise is horribly ineffective.
I think the push for better tools and more efficient ads is basically a race to the bottom. In fact, less perfect instrumentation might allow the ad industry to capture a bit more revenue from business thanks to decreased efficiency.
The lure of incomparability is very strong. Forget Google AdWords for a moment. Big ad buys are still largely arranged by salespeople, working on commission, making phone calls. And how could it be otherwise, for people in the business of convincing other people? Having objective, universal measures of efficacy is not helpful to that kind of endeavor. Much better to have a measure that works for you, that people are excited about, and that you control. It could just as easily be manipulable circulation numbers as a boutique web metric.
This case can be overstated. As I’ve said, I think people are largely working on these problems in good faith — particularly at outfits like Upworthy, which focus on a social mission; or the journalists I know who have left comfortable jobs because they care about whether their work affects the world.
But the incentives for a metrics Tower of Babel are real. To some degree, they’re even admirable, insofar as they’re driven by varying conceptions of success. Is my goal to make my audience think deeply, talk loudly, or spend freely? All of these can change the world for the better; ad-buyers’ temptation to ask which is best can reasonably be resisted, even if I do a little cherry-picking to make my case.
Besides, if one is prepared, for a moment, to disregard the capacity for world-improvement that widely-viewed and ethically correct publications represent, there’s no real problem here. Advertising often has strongly positional aspects, determining who will come out on top but not the overall level of welfare (a world in which Pepsi is the number one cola may strike some as more dystopian than it does me). Not only that, advertising is in some ways a force that directly opposes human agency — it’s designed, quite explicitly, to turn dollars into altered desires and behavior. I have limited enthusiasm for the kind of improved instrumentation that might let us hone that weapon’s edge even further.
In its most benign form, advertising is a tax on industry that flows according to influences too numerous to understand. There’s an appeal to the idea of rationalizing this process, of making it measurable, quantitative and objective — making it legible, as James Scott might put it. It seems like the result would be more fair. And admittedly, the human systems that make up the alternative are not fair: they’re sexist, racist and elitist.
But I suppose I’m optimistic that those systems don’t have to stay that way. And keeping advertisers confused about what they’re buying might preserve room for some wonderful things. So three cheers for analytic innovation — even if the innovators aren’t wholly aware of what they’re doing.
