By Professor Julian Meyrick, Professor Robert Phiddian and Dr. Tully Barnett Flinders University
On March 2, Laboratory Adelaide, a Flinders University-based research team investigating the problem of demonstrating value in arts and culture, convened a Symposium “When did value become a number?” It brought together scientists, philosophers, political analysts, economists, artists, managers, statisticians, and community and environmental activists to discuss the failure of contemporary society to meaningfully evaluate its policy options and outcomes. The following is an essay by three members of the Laboratory Adelaide team.
Pythagoras once claimed the universe was made of numbers. For the ancient Greek philosopher, numbers were the basis of all understanding, a way of plumbing the mysteries of nature, the potential of thought, and the inventions of humankind. Did he think they would facilitate the predictive algorithms of Facebook advertising, or digital monitoring of the workplace? Probably not.
These days we do more than use numbers. We worship them.
In the space of two and a half millennia, numbers have gone from object of sacred belief to tool of the practical world. They’ve become metrics. And the intensity and certainty that attached to religion now attaches to numbers, so that terms like “the average inflation rate” and “the stock market volatility index” provoke in us the same dumbstruck awe as the pyramids once provided our ancestors. These days we do more than use numbers. We worship them.
In his recent book, The Tyranny of Metrics (Princeton UP, 2018), Jerry Muller looks the gift horse of “big data” in the mouth, and asks what are the unintended consequences of taking numbers so seriously. Some things measure easily and well. Others really don’t. Even in health, where big data mining has saved many lives, and some money, metrics that become targets can have inhumane results.
For example, in the interests of transparency and competitive performance some US hospitals measure how quickly they deal with patients presenting to their emergency departments. People are often kept queued in ambulances until hospital staff are confident they can be seen in a four-hour window.
Muller shows that not everything of value can be counted and not everything that can be counted has value.
We have no idea whether such bureaucratic targets contribute to the ramping of patients in Australian hospitals (“surely not…”). But it is the sort of outcome that happens when trust in professional judgment breaks down and is replaced by ostensibly objective numbers. It’s called gaming, and no modern institution is without it.
Muller shows how a reductive and overconfident use of metrics has a corrupting influence on decision-making, for the almost proverbial reason that not everything of value can be counted and not everything that can be counted has value.
He is a historian, and knows that no systems stay perfect over time, because humans learn to adapt to them. Thus the crucial issue is how metrics get used. If those who have the contextual knowledge to discern good data from bad, use them informatively, they bring insight. If they are used to rank and reward by people without the time or capacity to meaningfully interpret them, then waste, gaming, and unjust results are inevitable.
In policy-making, the temptation is to use them as high-level bingo for hard-pressed technocrats with a dashboard of numbers in front of them and not much else. This is not a humanities scholar whinge, but a real risk to social performance in a range of sectors. Adam Creighton, economics correspondent for The Australian (not a notably left-wing publication) recently wrote: “Universities have hitched themselves even more strongly to the metric bandwagon… [H]igher education has spawned so many metrics that the cost of administrators and bureaucrats now outweighs academics” (17/2/18).
The question we began with in 2014 was “how can the value of culture be successfully measured?”
What has this to do with the problem of value in culture? Muller sketches the counterproductive use of numbers in health, policing, schools, the military, business, and foreign aid. He doesn’t mention – but how could he miss it? – the deluge of metrics washing over arts and culture. It’s a field that needs public investment but is often a terrible fit for econometric models because so much of what matters in it cannot be quantitatively measured.
This is the problem that Laboratory Adelaide, a humanities-based research project, has been grappling with for four years. The question we began with in 2014 was “how can the value of culture be successfully measured?” The answer in 2018 is “it can’t”. You can measure culture’s effects, and some of those effects provide quantifiable benefits in, say, well-being or tourism dollars. But the value of culture, like the value of love, or inner contentment, or animal companionship, is something that can only be judged.
That’s heresy for today’s devotees to the religion of metrics. The fact that, again and again, opinion polls prove false, markets unpredictable, and human life inestimable is ignored by those who cleave to the numerical credo.
Culture suffers as a consequence. But it is not alone. All aspects of life suffer, most obviously our relationship with the environment, which we love, but cannot stop destroying because balance sheets come before balance, sales before sustainability.
In this odious order of precedence, numbers are not to blame. Rather, it is our impoverished sense of value that sets them on a destructive path. It is this that needs to be checked before it is too late and there is nothing left to count. Whatever a number seems to represent, however unequivocal it may appear, it gains its persuasive force from the non-numerical framing around it, be that an essay, an image, or a Board report. A causal narrative, often hidden or implicit, always shapes our responses to a set of figures.
Measurement strategies determine who wins and who loses; who gets a grant and who gets to work in a bottle shop.
This framing is, to use a non-technical phrase, full of all sorts of shit: assumptions, presumptions, old thinking, new fads, folk wisdom, implied examples and the like. These things make up our sense of value, out of which we construct measurement strategies, our targets and tracking, like so many inlaid numerical mirrors winking back their assurances and bringing their social returns.
The last is a brutal reality. Measurement strategies determine who wins and who loses; who gets a grant and who gets to work in a bottle shop; who is celebrated, awarded and rewarded; and who is discarded, disregarded and also-ran. A whole series of social choices swings on these strategies, so if our sense of value is truncated or penurious, then the consequences are serious. Whatever is happening to culture in respect of its value is not unique, but representative. The denaturing and hyper-instrumentalism that occurs in the arts sector happens in other areas too, and frankly has taken on a demented life of its own.
As Laboratory Adelaide continued, our university, Flinders, began a major restructure. This word is in itself an implicit narrative, one with fateful overtones. It takes events that otherwise have to be explained on a case-by-case basis and packages them up in a few summative words. It relies on the implicit narrative not being questioned. The moment we do that, it’s like dropping a bag of bargain rice on a tile floor – questions are literally everywhere.
But our sense of value intercedes and boxes things up neatly. This process of storytelling that is, in fact, the operation of a complex and interlocking categorical logic, is indistinguishable from strategic policy making. It is expressed bureaucratically, which means that bureaucracy is indistinguishable from strategic policy making, as the site and facilitator of that logic.
Historically, the decay of our idea of value has been going on for some time.
Which is a long-winded way of explaining why so many of us in the arts spend so much of our time filling in forms, decanting the sweat, distress, hope, and honour of our lives into statistical fields, eligibility criteria and, inevitably, benchmarked KPIs. All evaluation is comparative evaluation, and what’s evaluated in the end, is us. Personally.
At Laboratory Adelaide we have realised that, historically, the decay of our idea of value has been going on for some time. It is possible to trace the arc of its decline in economics especially, from Adam Smith, through Ricardo and Marx, to Marshall and Samuelson. As debate unfolds over the centuries, value detaches from its moral and social ground, from values, and becomes a mathematical variable in a multi-factorial formula. It becomes price, expressed through revealed preference, leading to the stunted conclusion that free markets will fix every problem of resource scarcity the modern world faces.
In this way, the problem of value is trivialised, and handed back to the private sector even when it is clear that certain choices and decisions have collective consequences that far outstrip notions of individual consumer exchange. Talk of “public value” is an attempt to escape this intellectual constriction, but the idea hasn’t fared well in the recent neo-liberal era.
Our sense of value is thus split in two. On the one hand we have values, subjective, relative and endlessly plastic. On the other hand we have value, rigidly actuarial and invariably quantitative in its pure form. The first people dispute, the second experts calculate. The problem of value as Laboratory Adelaide first found, is it seemed very much a specialists’ game. Graphs and charts; numerical indicators with an astonishing claim to precision; methods and techniques that looked like they had been cribbed from a 16th century alchemist’s manual.
Laboratory Adelaide are not “value experts” in this sense. We know and care for our art forms, of course. But we came as amateurs to the debate, and have cleaved to that status since. For in the end value is not a problem for experts. It is a problem for everybody and anybody. Experts may have something to say about it, but as contributors, not arbiters. Evaluation is a process that involves us all.
And so it should. The problem of value outstrips every technical means to measure it because it is not, in the end, a technical problem. It is a dimension of being human, and it reaches out in all directions. The four key insights of Laboratory Adelaide’s research are:
- That there is no silver bullet methodology for conclusively proving the value of culture – metrical demonstration can never fully replace professional judgement.
- That qualitative information is crucial for evaluating what culture is and does, and that includes the very language we use to talk about it.
- That value isn’t value until someone says its value – that the evaluation of culture is as much a social process as an evidentiary one.
- That the problem of value in culture is a subset of the larger problem of meaningful evaluation the world faces today.
This will be hard. Interdisciplinary and cross-art form dialogue can easily lapse into what kindergarten teachers call “parallel playing”, where people talk about what they know, and gloss over what they don’t. It is vital to avoid this. Even if we come from different perspectives and experiences, we must strive to genuinely connect and compare. In a phrase we do not particularly like, but cannot seem to disown, we must “reach out” to cognate minds and achieve a new understanding of the common problems we all face.
The most noteworthy result of our recent Symposium for artists, managers, ecologists, policy wonks, economists, scientists and philosophers was obvious agreement that metrics are not saving culture, society or the planet. They are ascribing cost but missing value. We hope it is the start of a better conversation about evaluation in a world caught between a regressive political populism on the one hand, and a stonehearted technological vacancy on the other. Between these unworkable extremes exists a slender but real possibility space – a space of hope – wherein a more meaningful idea of value might be found.
“What Matters? Talking Value in Australian Culture Today” by Julian Meyrick, Robert Phiddian and Tully Barnett will be published by Monash University Publishing later this year.