"Moneyball" and the limits of journalism's analytics revolution
Can journalists measure how much good we're doing? What if we learn something we don't want to know?
Sorry to be that guy. But for the sake of illustration:
There’s a scene in 2011’s “Moneyball” (if you’ve seen the movie, just skip ahead) where Oakland A’s manager Billy Beane, played by Brad Pitt, brings America’s modern managerial revolution to the medieval business of baseball.
Beane is surrounded by bald and aging scouts, elder statesmen of the jock class who rank prospects based on subjective criteria like looks and body types and girlfriends. It’s what they’ve historically associated with future success: players who look, act and swing the bat like Zeus. Meanwhile, the youthful and luxuriously coiffed Beane, himself a fallen Greek god who washed out of the Majors, keeps pointing to the nerdy stats guy in the room to talk about on-base percentage.
“There is an epidemic failure within the game to understand what is really happening, and this leads people who run Major League Baseball teams to misjudge their players and mismanage their teams,” the anxious Peter Brand, an economist played by Jonah Hill, explains earlier. “People who run ball clubs, they think in terms of buying players. Your goal shouldn't be to buy players, your goal should be to buy wins. And in order to buy wins, you need to buy runs. … Baseball thinking is medieval. They are asking all the wrong questions.”
The movie itself, based on Michael Lewis’ 2003 book of the same name, describes the dawn of a new empirical era in baseball, whose methods of analysis two decades later are now downright quaint. Examining walks and OBP now puts you roughly in the same stone-age analytical universe as the old scouts who tried to guess how many home runs a guy might hit based on the size of his butt. Consider this 2018 interview that the real-life Houston Astros general manager Jeff Luhnow gave to the McKinsey consulting firm after taking the club from a 51-111 record in 2013 to a World Series Championship in 2017:
“Today, it’s completely different. We now have so much technology around the ballpark and information about the trajectory of the ball, the physics of the bat swing, the physics and the biomechanics of the pitcher’s delivery - so many components now that advanced sciences have worked into our game. It’s, quite frankly, overwhelming in terms of the amount of information that we have access to and intimidating to figure out how to analyze all that information, work through it, and come up with the takeaways that will allow you to continue to do what we tried to do back in 2003, which is to make better predictions about what players are going to do in the future on the field.”
What was not included in Luhnow’s interview was a critical disclosure about how the Astros had gotten a little too observant and a little too good at predicting on-field behavior. “How did the Houston Astros’ hitters so easily pound three of the Dodgers’ hottest pitchers in two key games in Houston in the 2017 World Series?” my colleague, L.A. Times columnist Bill Plaschke, wrote in 2020. “They cheated, that’s how. They used technology at Minute Maid Park to steal the Dodgers’ signs. Their hitters knew what pitches were coming.”
Luhnow was suspended and fired not for insufficient empiricism, but for too much of it. Being the smartest guy in the room isn’t good enough, after all. The ultimate health of baseball depends on something beyond observation and analysis, on compliance with some stranger, greater moral order — with the law.
My world of journalists has its inherited pieties, its old baseball scouts, its analytical savants, its techy disruptors. Their arguments are voluminous, meaningful, and unlike in baseball, ultimately unsettleable.
The “Moneyball” shock to the modern American newsroom arrived over the past two decades in the form of web analytics tools that could tell you how many people were reading your story (not nearly enough), for how long on average (not nearly long enough), how far down the piece they got (not nearly far enough) and where they came from (probably Google). Chartbeat was my Billy Beane, telling me how often I was getting on base.
Even a passive but specific awareness of consumer behavior, like many such empirical interventions throughout human history, unleashed a hydraulic force of change on newsroom cultures in the 2000s and 2010s. Academic research over the years as shown that many individual journalists consciously evaded audience incentives to hold the line on editorial autonomy; after all, for the past century, the discipline and ethics of journalism has served as a countervailing force to the larger commercial contexts (major advertisers, owner barons, corporate chains) that journalists have usually worked within.
Nonetheless, in the crassest media companies, these analytics revolutions manifested as productivity quotas and competitive scoreboards that pit journalists against each other. Did people want to read that dry, monthslong government accountability investigation today, or did they want to hear about the Miami face-eater?
Not a proud moment in my career, but the traffic numbers didn’t lie. The executive editor and digital director of my newsroom at the time — who had hired me more as an aggregator so I could keep the website busy while other reporters did the real work — pulled me into an office one day and told me they wanted me to go for “the low-hanging fruit.” For years afterward, I retold this anecdote with horror. But if I’m being intellectually honest, looking back through my inbox today, I see several instances of me using that very same phrase before the upper management ever said it out loud to me. The traditionalist telling of this story is that for-profit newsrooms sometimes make less-than-noble editorial decisions because of the inherent commercial logic of the enterprise. Which is totally true.
But having lived through that particular (and thankfully departed) era of the clickbait internet, I’m telling you: Learning via analytics what people out there really wanted most — and what they really, truly, insistently, demonstrably and objectively, through their own consumption choices, did not want — was an enormously sobering experience that haunts me to this day. “Probably the biggest change in Internet media isn’t the immediacy of it, or the low costs, but the measurability,” Gawker founder Nick Denton told the New Yorker in 2010. “Which is actually terrifying if you’re a traditional journalist, and used to pushing what people ought to like, or what you think they ought to like.”
I don’t completely know what this conversation looks like as tech companies choke out the traffic-based internet right in front of us. There’s no such thing as clickbait on monopolistic social apps or AI-powered chatbots that don’t feature clickable links anymore. But I’ve internalized the presence of metrics to the point that I barely even register their influence over this post (as mediated by Substack), which are going to tell me exactly how many of you opened this email after I almost unconsciously decided to mention “Moneyball” in the subject line. [EDIT: The early Substack stats say the link you all are clicking on the most is about the face-eater.]
What I learned from the analytics revolution is that you can’t simply be ethical or well intentioned if you’re expecting your journalism to have an impact, whether you work for a commercial newsroom or anywhere else. If you want your most meaningful work to survive out there — competing with tweets and Netflix and Instagram and emails for people’s attention — you have to be good. The best, if possible.
But whether your best is good enough is a diagnostic question whose “Moneyball” moment hasn’t come yet. Maybe the nature of the question precludes its arrival.
The tools of journalism analytics are just a measure of attention, which is more or less a measure of consumer activity. At the enterprise level, you’re either generating enough subscribers or advertising revenue to self-sustain (or to justify continued philanthropic support or government subsidies) or you aren’t.
But there are more severe limits to using or relying on empirical observation in assessing journalism’s value to the broader project democracy — in other words, the whole reason we all exist as journalists to begin with.
Both as a working journalist and as an active combatant in the political economy of journalism on behalf of my union, for a few years I’ve been wrestling with the endgame question of “what is journalism for?” The media scholar Nikki Usher, in posted titled “The end point of better journalism: How do we know when we've made democracy better?” recently posed this conundrum at length:
“If we have concerns about democratic backsliding, what does fixing our fractured, open-to-authoritarian democracy look like? From a social science perspective, what interventions do we hope will happen to stop the slide into a country where there is no longer a shared commitment to nonviolence despite extreme disagreement? What measure, what objective, will be enough to convince us that we have reached the good/great society, assuming that there is even some baseline agreement upon which the good society even suggests?”
I love these questions. Both because I’m a fan of uncomfortable, sacred-cow-slaying inquiries, but also because of my own existential worry that there is an impassible, howling singularity of unfalsifiability awaiting thinkers of journalism on the other side of some of these questions. Certainly there are lots of studies that show that the presence of local journalism is associated with indicators of civic health and vice versa, and I cite them frequently.
But at the grand endgame stage of democratic philosophy, what we are talking about is how we’d know whether humanity has arrived at its self-evident resting state: the kind of End of History speculated by thinkers like G.W.F. Hegel, Alexandre Kojève and Francis Fukuyama, a kind of utopia of free and rational societies operating on humane principles of universal recognition, idealized in the form of a liberal democracy. (Think “Star Trek.”)
I personally find these philosophies elegant and compelling, but they are the product of speculative logic rather than quantitative analysis, and not everyone is a fan of those methods. The philosopher of science Karl Popper, the king of falsifiability, famously hated this sort of continental woo-woo, writing a whole book in 1945, “The Open Society and Its Enemies,” about how "such sweeping historical prophecies are entirely beyond the scope of the scientific method."
Some contemporary democracy scholars agree: Recent decades have demonstrated the “decisive victory of quantitative political science (with its focus on high-N statistical studies or rational-choice models) over qualitative political science, whether in the guise of political theory or of empirical work rooted in deep knowledge about particular countries and cultures.”
If such a thing as a good democracy exists, these sorts of thinkers to expect that you should be able to prove or disprove its existence with replicable statistical methods. Not only that, but if you can establish what social, economic and political features precede and correspond with democracy, then maybe you can strengthen those features to recreate democracy — as Billy Beane would say — in the aggregate. No fancy Hegelianism necessary.
Groups like Freedom House try to bring this sort of analytical power to the international democracy movement by publishing data on more than 200 countries and territories “because accurate data and rigorous analysis are essential to shining a light on the erosion of freedom.” That seems true: If I were writing a story for a newspaper tomorrow about Nikki Usher’s research questions, and trying to convey to the public some generally reliable reason to believe whether a given democracy was in trouble or not, these are the sort of analytics I would point toward as generally sturdy enough to merit discussion.
Nonetheless, the skeptic and the humanities student in me hears the sound of debris — some loose bolts or something — rattling around inside the empirically sound statement that a free press might merely be an “indicator” of democracy’s presence rather than a morally necessary feature. I get similarly anxious when I read cases for democracy based on democracy’s superior ability to deliver economic growth, win wars, or enable scientific progress; if the empirical evidence showed otherwise, then the formalism of the scientific method is quite capable of supporting a rational and vigorous case for dictatorship. Is the implication that I should support that outcome, if that’s what the evidence were to plainly show? The conscience of the freedom-lover resists.
Some human variable seems hidden in the formulation of the scientific question, here: some ought bleeding into the is. Ironically, I turn here to Allan Bloom, arch-defender of the humanities against the natural sciences and democracy’s small oppressions, who wrote in “The Closing of the American Mind”:
“Political science has always been the least attractive and the least impressive of the social sciences, spanning as it does old and new views of man and the human sciences … Economics, psychology and sociology as well as all kinds of methodological diagnosticians have been welcome guests. But there are irrepressible, putatively unscientific parts of political science. The practitioners of these parts of the discipline are unable to overcome their unexplained and unexplainable political instincts — their awareness that politics is the authoritative arena of effective good and evil. They therefore engage in policy studies whose end, whether stated or not, is action.”
It’s entirely possible that the economic conditions supporting a free press collapse before we can reach some sort of intellectual settlement on what kind of journalism is most necessary to support the best kind of democracy.
A market fundamentalist, or maybe a modernization theorist, might conclude that if the public in a democracy no longer wishes to patronize or subsidize a professional press, then maybe that’s a sign that a professional press is no longer a material precondition for (or a necessary feature of) that democracy. Perhaps independent journalists were once an empirical indicator of democracy’s presence but no longer, thanks to recent developments in the information ecosystem.
Correspondingly, amid the market collapse of conventional local news outlets, there have been lengthy conversations about reorienting the energy and funding for journalism more toward “civic information needs” or “community information needs,” which more or less advocates for swapping out the longtime political primacy of journalistic producerism for a consumer welfare standard of less overtly neoliberal branding.
And why not? Tech giants like Google and Meta have built enormously popular businesses around the principle that the consumer welfare standard is the fundamental problem to be addressed in the modern democracy’s information economy.
Maybe you don’t really need to fund the creation of content — seems like there’s plenty of that stuff out there already without Mark Zuckerberg having to pay a cent for it. Maybe all you need to do is create marketplaces that more frictionlessly connect consumers with all this free content, while happily skimming billions in advertising revenues that once funded the work of journalists in a less networked era dominated by regional newspaper advertising monopolies. This was Silicon Valley’s enormously lucrative work of the past two decades.
More recently, those companies have ventured toward the endgame of consumerist logic by funding generative AI models that cut out referral traffic to producers entirely.
After all, it’s inefficient, wandering around websites without knowing if they have the information you’re looking for. It seems more rational for users to query a one-stop, AI-assisted portal about the best way to vote in their local election or to file a complaint against their landlord. No need to go find and read the newspaper (which no longer exists for many Americans) or look for a local tenant union (which has never existed for most Americans). If you’re a grantmaker, there’s certainly no rational argument for why you should give grants for inefficient one-off civic education projects that will only ever reach a fraction of our 337 million bewildered Americans. Google, already the possessor of vast analytics on their consumer needs and the infrastructure to operationalize it, can do that job vastly more efficiently. Google is already right there on everybody’s phones; access to civic information, just a prompt away. For an effective altruist interested in satisfying civic information needs, the most efficient deployment of pro-democracy capital is buying more shares of $GOOG.
I am only being a little sardonic, here. But when you are talking about the nature of information consumption in a contemporary democratic society — whether it came from a newspaper reporter, or a community group, or a generative AI bot — the fundamental problem of 2024, whether it’s Hollywood or book publishing or or podcasting or journalism, is that there’s way too much fucking content, and relying on lower-friction, surveillance-powered, monopolistic portals to create order seems like a rational consumer response to information chaos.
In fact, strictly from a consumerist viewpoint, the public is living through a crisis of information overproduction, not journalistic underproduction. Survey data is quite clear that most Americans don’t even realize their local newsrooms are in crisis. Why would they know? They’re getting most of their information from everywhere else. For free.
Because I’m a journalist and a trade unionist — two producerist bedrocks within the democratic civil tradition — I tend to view these problems and their possible solutions through a producerist lens, albeit with more of a solidarity ethic rather than a competition ethic.
The bad news that we all already know: In any kind of saturated content environment marked by too much supply and not enough demand, there is every pressure in the world to reduce production costs for that content. And there is no greater production cost than labor: Journalists are the biggest cost to be reduced. Hedge funds and private equity firms are the best at laying us off, of course, which is why we yell at them the loudest.
But the market discipline imposed by distracted consumers doesn’t magically stop outside commercial newsrooms. You can see its force at work everywhere. Some of the biggest journalism job-cutters this year are public broadcasters. Other nonprofit newsrooms are either shrinking, contemplating folding, or initiating mergers and shared-services agreements that reproduce the cost-saving logic of the corporate chains the nonprofits rose up to replace. When you’re a journalist, the tax status and feel-goodness of your employer doesn’t really matter that much if the nonprofit industrial complex is laying you off or underpaying you just like the rest of them.
Part of the argument against policy interventions that even partially support commercial newsrooms is that the labor of doing public-interest journalism is a fundamentally unprofitable activity, and maybe policy should be goading us to pool our journalistic labor inside nonprofit firms. Does this solve enough of our problems? Panning the camera wider from my perch in Los Angeles, it seems like most content production is increasingly less profitable, with everything from Hollywood to book publishing to podcasting to the influencer economy being dominated by winner-take-all economics and fissured work arrangements. There are losers everywhere you look, making stuff for massively profitable platforms at their own loss, usually in contexts where only one side of labor-platform relationship has bargaining power to dictate terms on things like payment, distribution and (now) AI harvesting.
Not to sound like a caveman about this, but something unions innately understand at the lizard-brain level is that you’re supposed to go pick fights with exploiters wherever they’re doing their exploiting. The financialization of the local news industry has already stimulated the growth of an increasingly aggressive labor movement in opposition inside newsrooms, which has more recently matured into engaging in more work stoppages and supporting actions to check the growth of private equity and eliminate the carried-interest loophole.
While that fight continues to unfold, it’s become increasingly apparent that there are a handful of giant companies outside those newsrooms also extracting an enormous amount of wealth from newsroom workers without paying for it. The nature of American labor law doesn’t easily allow for journalists (or other creative and knowledge workers like us) to use things like strikes and the bargaining table to discipline platforms the same way we can discipline traditional employers. We can’t bargain for money that our employers never even had. So I have come to view bills like the California Journalism Preservation Act not as some tax-like funding mechanism for making Google pay for journalism labor (which is how other people often describe it), but as creating a collective bargaining table not otherwise required by law due to the novel nature of platform-labor relationships. When Google and Meta threaten to ban my journalism because they want to profit from it without paying for it, I don’t see that as any different than any employer threatening a lockout during a labor dispute.
A consumer welfare standard will never bring you within miles of formulating this sort of argument, which is based on countervailance and collective bargaining; within the broader democratic ethos, it usually appears under the label of “checks and balances.”
So in looking toward a future that analytics can’t help me predict, I borrow from older humanist traditions about the inherent good of pursuing the truth; older labor traditions about the necessity of class solidarities in confronting destructive counterparties.
The pro-democracy movement should pursue the positive freedom of supporting a free press as a check on other societal powers, not simply a negative freedom of limited constraints on expression and consumption.
We should seek to preserve journalists as a social force tasked with telling truth to power and who, in an era of hallucinatory AI content production, operate on an ethical code of pursuing accuracy and truth. We should promote a professional culture that pursues intellectual independence and values its autonomy more preciously than almost anything else. We should have workers who are trained and willing to resist Chartbeat when the analytics tell them it’s more rational to do something immoral.
This is all part of philosophical and ethical case for the existence of free press more a scientific one. In reality, in any world where news production is more heavily subsidized by taxpayers broadly rather than tech monopolies specifically, the empirical studies (and more rigorous ones like them) would become a politically necessary tool for journalists to survive the perennial budgetary contests of the democratic arena. There is no escaping the analytics revolution, in the end. Justify yourself or die.
But my final argument for a free press is that a democracy should have one as an end unto itself, and not merely a means, just as democracy itself is the proper destiny of a free people.
I sometimes wonder how people would spend their time differently if they didn't have to work such long hours to survive and weren't ensnared in the attention economy for so much of their "leisure" time. Free time + sufficient attention span to marinate in long form content have sadly become luxuries instead of human rights.
So so honored to be mentioned here! This is so so so on point with what I am thinking about - especially the humanism aspect (I love Sarah Bakewell’s new book, def recommend). And I also love how you do have these perspectives informed as you say by your work in labor and journalism - by democratic civil society. I have my qualms about Freedom House but you have made a solid argument for rechecking /rethinking. What caught me today as especially notable beyond all the other awesome thinking here is this subtle observation “ There’s no such thing as clickbait on monopolistic social apps or AI-powered chatbots that don’t feature clickable links anymore” - but yet this is the imaginary for so so many newsroom strategies still. More soon I hope !!!!!