Against fact-checking

What does it mean to check a fact? It’s a more complicated question than it may seem.

Let us take a simple example.

Primus: The sky is green.

Secunda: Let me check that fact.

The sky, as we know, is blue and not green. But what is Secunda assuming, by taking Primus at his word and checking whether the sky is in fact green? Nothing controversial: Secunda assumes that there is a sky, and that it can be attributed properties of colour. Secunda goes outside, looks up, notes that the sky is blue and not green, and strikes Primus across the forehead for being a prat.

Let’s take a more complicated example, such as the one Channel 4 FactCheck dealt with today.

Primus: Men are more likely to be raped than be falsely accused of rape.

Secunda: Hang on, let’s check that fact.

Secunda’s presuppositions are partly quite uncontroversial. There is such a thing as men, and rape, and false accusations of rape. We can all accept this, and we can broadly agree on definitions of each of those things.

The tricky presupposition comes when Primus talks about probability. Secunda may note that rape and false accusations are both, at least in theory, capable of being recorded and studied, so she might look for some data on their prevalence. However, by ‘checking the fact’ of whether men are indeed more likely to be raped than be falsely accused of rape, here are the premises Secunda must accept:

  1. It is possible to perform a valid statistical comparison on these two probabilities.
  2. The sources from which we might acquire statistics for such a comparison are capable of supporting such a comparison.

To assess the prevalence of male rape, Channel 4 uses the figures from the Crime Survey of England and Wales, which estimate a yearly figure of around 8,000 victims in the entire adult male population (about 0.05%, it adds helpfully).

To assess the prevalence of false accusations, Channel 4 has a more complex method. In its cheery mid-Atlantic dialect it notes:

First up, let’s acknowledge the elephant in the room: there is no reliable data that exists to tell us how many people are accused of rape in day-to-day life.

There are all sorts of environments — at work, at home, in public — where someone could make an allegation of rape, truthfully or otherwise, that may go unrecorded. If false, these accusations could have far-reaching and profound effects on the accused’s life, even if they never result in police or judicial action.

But we can only analyse the data we have. So, for the purposes of this article, we will look at rape allegations that make it to the courts.

This is quite a caveat. Already we have hurtled past assessing a comparable statistical population. We are no longer considering what proportion of men in the whole population are falsely accused; rather, it is what proportion of men who are prosecuted for rape are falsely accused.

Overall, in the CPS’ view, false allegations of rape are “serious but rare”. They estimate that there is one prosecution for a false rape claim out of every 161 rape cases prosecuted. That means — according to the best available data — false allegations make up 0.62 per cent of all rape cases.

I have written before about how it is possible for an instance of rape to fail to meet the criminal standard while still being a true allegation. In this instance, Channel 4 outlines the statistics – noting with considerable understatement that “the data we have is not always directly comparable” – and concludes as follows:

But from what we can tell: yes, men are more likely to be raped than be falsely accused of rape.

According to the most reliable data we have, the average adult man in England and Wales aged 16 to 59 has a 0.03 per cent chance of being raped over the course of a year (based on 2016-17 figures).

The best data we have — the number of people prosecuted for making false allegations — suggests that the average adult man in England and Wales has a 0.00021281 per cent chance of being falsely accused of rape in a year. (That’s based on 35 prosecutions for false rape allegations in 2011 compared to 16.5 million men aged 16 to 59 living in England and Wales at the time).

By this measure, a man is 230 times more likely to be raped than to be falsely accused of rape.

This is tricky, but look closely. We are being asked to compare:

  1. figures from the Crime Survey of England and Wales on the prevalence of male rape in the whole population to
  2. the proportion of prosecutions for perverting the course of justice which succeed, which are a proportion of the whole number of prosecutions for perverting the course of justice, which are a proportion of rape prosecutions which fail in court, which are a proportion of all rape prosecutions, which were themselves selected from the whole number of rapes and alleged rapes which exist in society.

Then, that proportion is taken to be exhaustive of the entire population in the way that the Crime Survey data is (statistically) a sound sample of the entire male adult population.

It is rather as if we used the number of watches successfully repaired in a given year as exhaustive of the number of watches that break down, and then compared it to the number of people who reported in a survey that their computer crashed. Many watches are not discovered when they break down. Many are not repaired at all for reasons of expense or laziness. Many are simply disposed of and replaced. Either way, the figure is simply not reflective of the underlying reality in the way that the survey on computers is.

This is why the Crime Survey of England and Wales is taken to be a better picture of the number of sexual assaults, as many go unreported. To compare like with like, we would need the Crime Survey of England and Wales to ask a question along the lines of ‘Have you ever been falsely accused of rape?’ The Crime Survey of England and Wales does not do this, for many reasons – but unless and until they do, a comparison reconstructed from prosecutorial data is inherently flawed and of virtually no use whatsoever.

Channel 4 does, to a degree, recognise this and makes an effort to steelman the case against it.

…let’s put the stats through an even stronger test.

Imagine for a second that you believe that every single one of the men prosecuted for rape in England and Wales in 2016-17 was falsely accused.

Even if that unlikely scenario were true, there would still have been more adult male victims of rape (8,000) than men prosecuted for those rapes they “didn’t commit” (5,190).

Here we are not so much comparing apples and oranges as comparing apples and lamb chops. We are still comparing crime survey data to prosecutorial data, but now we have tossed away the notion that we are comparing likelihood at all. Logically, those rape cases which succeed at court are (one imagines) the most promising cases of all and have the best evidence behind them – consequently, those are the least likely to be false accusations. A more testing statistical steelman would be not that all prosecutions are false accusations, but that all reports are false, as many more cases are reported than prosecuted. This would be closer to – though still very far away from – the statistical standard we see in the crime survey data. Instead, in purporting to use a stronger test, it actually uses a weaker and sillier one.

There is an even more serious concern. A slogan like “men are more likely to be raped than falsely accused” is not used neutrally. It is by its very nature only used when the prosecution of a rape case has inflamed passions, typically on social media. In that scenario, the relevant population for the question of false accusations is not ‘all men’, but ‘all men who are accused of rape’. It would sound ridiculous to ask “what proportion of men who are accused of rape are themselves victims of rape?”, of course. But the statistical comparison only works if the same controls are embedded in the data. Otherwise we are confounding ourselves with irrelevant statistics.

The trouble with fact checking is that by checking a ‘fact’ it endorses the epistemic framework in which the ‘fact’ is produced, e.g. that a putative comparison between the Crime Survey and prosecution records is a statistically valid idea in the first place. Fact-checking therefore is in the unenviable position of reproducing the questionable premises of the ‘fact’ it is checking. Channel 4 is unlikely to conclude by saying ‘we cannot know and any comparison purportedly based on data is meaningless’, but that is the only honest response. By assessing whether the data backs up the claim, however, it implicitly accepts that the data can back up the claim – and that apples may speak to oranges on equal terms. They cannot, and nor should we spread the idea that they can.


Leave a comment

Filed under Law

Quick update

My piece about the numbers crisis in the Church of England has been rejigged for publication at the Church Times. You can read it here. It’s shorter and better so please share that one in preference to this one.

Leave a comment

Filed under Politics

You don’t get to two billion believers without making a few enemies

It may be worth reading 1 Corinthians 12:12-26 alongside this post.

The news that General Synod had taken evangelism off the agenda this year – yet again – rather reminds me of the notorious Politico article that did the rounds after the election of Donald Trump. Confronted with deep concern on the ground about the potential for Mr Trump to pull off a surprise in key states, Mrs Clinton’s campaign preferred to stick to the strategy they had crafted from the start – and take activists away from the very states they were warning were weak spots.

“… I think it’s true that the plan was accomplished,” said a former labor leader in the state. “But the plan was not the right plan.”

Every year seems to bring a new plan for the Church of England to shore up its ailing numbers: the Thy Kingdom Come prayer drive; the Just Pray cinema ad which was unceremoniously, and predictably, dumped by distributors; countless initiatives with patchy take-up and only an occasional relevance for diminishing congregations; church plants which bloom and die without a trace, bluebells in our spiritual desert. None have worked. An 81-year-old is eight times as likely to be a churchgoer as an 18-year-old. “Terminal decline” does not quite capture the problem: we are in a tailspin. Far from expanding our numbers, it will be a miracle – in a literal sense – if we maintain them. Moreover, our potential converts know very well that we are in trouble. People can smell fear – and fear does not an appealing religion make.

In this context, the Church seems to have adopted a marketer’s view of ‘selling’ Christianity to a population which sees no real need for religion. Research indicates secular people often rather like Christians, and are glad churches exist. They may not even have an active problem with the establishment of the Church of England. Yet they are no closer to the way, the truth and the life, and a Church which cannot reach them is failing in every aspect. No marketing campaign will work on this: even a fire-and-brimstone advertising campaign warning of the eternal consequences of continued apathetic atheism is likely just to put people off, even if it does get past the Church’s significant contingent of annihilationists (among whom I count myself a reluctant member) and universalists. The general population don’t believe us when we tell them about sin, much less about Jesus, and still less about His Resurrection. They certainly won’t believe stern warnings about the fires of Hell.

The marketing approach, positive or negative, seems misguided to me. By contrast, perhaps we should look to those institutions which are successful without marketing. For instance, it does not seem wholly coincidental that social networking sites are as powerful as they are in an atomised society. Organised Christianity’s decline has been a significant part of the wider trends towards a lonelier society. Accordingly, it seems if we are to arrest the collapse – and to do so we will really need to win some converts, and fast – we ought to learn from Facebook, not from Hillary Clinton.

That doesn’t mean, I hasten to add, buying up ads or harvesting data through Cambridge Analytica. Facebook did not become one of the most powerful companies in the world through a savvy marketing campaign. That would be magical thinking: people’s brains are not simplistic electronics that can be hacked with the right combination of words and images. Facebook became one of the most powerful companies in the world because of two things:

  1. The product is well-designed for the audience to which it is aiming, i.e. everyone.
  2. Everyone else is on it (cf. Google Plus, Bebo and Myspace).

With this in mind, then, I think our best shot is as follows. First, it is critical that Christians actually go to church. The obvious problems of keeping the lights on aside, the Church needs an agglomeration effect in order to bring more people in. When people bring their friends and family to church, it must not feel like a ghost town (more like a Holy Ghost town – sorry). It is not enough for people to occasionally utter a prayer at night and sound off about how much they love Ecclesiastes on Twitter: they must go to church, every week, preferably without fail. I cannot say honestly that I am an exemplar in this, but it has become far, far too easy for Christians to write off their obligation to attend church. They cannot. It is existential. Just as ‘everyone else’ is on Facebook, ‘everyone else’ should be at church.

But, secondly, the church’s role (I use the lower case advisedly) is no less important. Church must be somewhere people feel they ought to go and don’t mind going. That is not the same as wanting to go. The people who want to go to church early on Sunday morning are a rare breed, and we should protect them (or ordain them). But most people will not want to go. It is enough, then, that church should not be an objectionable place to be. It should full to bursting with the Spirit, steeped in the traditions that tickle at the back of the secular mind and facing the world in serene knowledge that it is the living conduit to the maker and sustainer of all things.

That means well-conducted liturgy, in traditional (or, at least, formal) style; that means organ music; that means intelligent preaching, borrowed from the Church Fathers where appropriate; that means sermons that are clear, comprehensible and brief; that means the congregation properly singing the hymns; that means smooth services that start and finish at the same time every week; that means offering a newcomer a biscuit and a cup of coffee, but introducing her to the churchwarden too. It means regularity, routine, familiarity and community. It also means, as far as possible, the avoidance of the most fatal emotion anyone can feel in church: embarrassment. The Church’s obsession with gimmicks, technological and liturgical, shuts out five normal people with a normal cringe reflex for every one oddity it attracts. One should never feel embarrassed about the Gospel.

None of this can be introduced by fiat. A letter sent by the Archbishops could make influential individuals in each church sit up and listen, but the truth is that any revival in the Church is dependent upon an effort among those we already have to observe their obligations. Come. Stay. Bring a friend. Bring your children. Do what you know in your heart that you have to do. You are a member of the Body of Christ: do as He did, and keep establishing the Church, every day, every week, every month. Not just at Christmas or Easter. Strengthen your community and perhaps, one day, God willing, the bums-on-pews figures might go up year-on-year, not down.


Filed under Religion

First World Problems and waiting for Hitler

Spoilers for a recent Radio 4 drama follow.

Radio 4’s drama First World Problems, part of their ‘Dangerous Visions’ season, was a useful insight into what one might call mainstream eschatology. It envisions a future civil war in the United Kingdom, at some point in the mid-2020s following a financial crash. Scotland secedes and civil order breaks down. Republicans kick off – again – in Northern Ireland. Over the intervening months full-scale war breaks out: the drama follows the Fletcher family as they are pushed unceremoniously from their home in Manchester to Wales to Bristol to Birmingham while dealing with familial strife along the way. It’s a pretty smart piece of drama, if laden down with rather shrieky dialogue, an excruciating reference to Game of Thrones, and more than a few visits from Basil Exposition. The problems of dealing with a disabled child and an elderly grandmother are handled sensitively and without cliché, in probably the strongest character-driven parts of the drama.

The trouble is, though, that it is demonstrates a persistent habit among the creative classes in dealing with dystopia. David Runciman once referred to this tendency as “waiting for Hitler”: an excessive fondness for 20th-century models of dystopia, and a lack of imagination in how society and democracy might – will, eventually – collapse. First World Problems suffers a bit from the very thing that makes it compelling, namely the involvement of journalists from the Yugoslav wars in the writing process. The largest military power is called Greater England, without any explication of an irredentist ideology driving it (for example, it makes no effort to retake the secessionist Scotland except for the nuclear weapons at Faslane). Manchester, de facto capital of the Free English Territories, is laid under siege. Wales engages in severe ethnic cleansing of English residents (episode 4, which follows the family as they are forced to march out of Wales on foot, is the most grimly visceral and best-written episode of the series). By the time a nurse talks about an “incident” of mass war rape somewhere in the North, it begins to feel a little bit forced – not because these things don’t happen, but because the programme gave the impression of working from a Yugoslav checklist.

A more serious gap in the programme’s imagination is its failure to push the ethnic conflict as far as it could be taken. The British Civil War is a tremendously white ethnic conflict. The Serb-Bosnian model is used for the traditional South vs the traditional North, which simply is not a comparable situation. It is noticeable how much more chilling the ethnic cleansing of Wales is, partly because of the very well-handled linguistic barrier. The programme assumes that vague internal divisions resolve themselves into ethnic warfare, which pushes the model to breaking point and misunderstands the history of the Balkans.

By Episode 5, England’s substantial ethnic minorities finally get a look in, with the reappearance of the Croatian cleaner from Episode 1 and safe haven finally found – temporarily – in a substantial part of Birmingham seized by Islamic militias. Yet this idea is not handled very well: the notion that ‘Greater England’, an implicitly fascist military junta, is headquartered from London (White British population of around 45% at the last census), imagines that our civil war is taking place in the 1950s rather than the 2020s. A deep engagement with the consequences of a UK civil war might be unbroadcastable for reasons of sheer complexity, given the enormous diversity of British cities compared to their surroundings. An Islamic militia seizing large parts of Birmingham is one thing, but how would the Greater English ‘London’ cope with (for example) the secession of Tower Hamlets? What happens to the gangs in Northern cities, largely organised along ethnic lines and relatively well-armed?

It’s only a radio drama. And it’s well done. But a really ‘Dangerous Vision’ of a UK civil war would be as confusing and as all-over-the-place as the Balkan conflicts which inspired this one. As the UK becomes more diverse and ethnic relations become more complex, fiction serves an important role in elaborating the challenges society faces in simply staying together. To write out an eighth or so of the population in pursuit of a white North vs white South dynamic isn’t the science fiction we need. We cannot keep waiting for Hitler, because whatever comes in his place will completely blindside us if we do.

P.S. My favourite dystopia of recent years is Jack Womack’s tremendous 1994 novel Random Acts of Senseless Violence, which handles ethnic tension in cities (New York, in this one) very well.

Leave a comment

Filed under Culture

Three reasons why you should not say the Voltaire quote

  1. He didn’t say it.
  2. Everybody says it.
  3. No you wouldn’t.

Leave a comment

Filed under Politics

The median and the message

How does the man on the Clapham omnibus stay in shape? NHS Choices has an idea.

It’s fairly reasonable for NHS England to worry about lazy weekends. The British population is very inactive, worrying given the extraordinary physical benefits of regular exercise. The limited effect of the UK’s public health campaigns urging us to take more exercise despite years of messaging is also undeniable.

The above tweet does helpfully illustrate a pervasive style that characterises most British public health messaging. You could fruitfully compare it to the five-a-day campaign, guidelines setting out the number of alcohol units men and women should respectively drink, even the laughable ‘Go for Gold‘ effort to reduce the potential carcinogenic effect of overdone toast. The easiest way to describe it is as rational: clearly linking a defined behaviour with a defined problem, and offering a solution with a different activity.

What I mean by this is best illustrated with a simple thought experiment. Consider the sort of person whose weekend is largely sedentary: football on the TV, alcohol in the pub or at home, fried food, transport largely by car. Then, ask: is this person likely to be willing to undertake a ‘sofa workout’ at all? To ask the question is to answer it.

To expand: NHS Choices appears ignorant of selection problems. The traits which lead to couch potato syndrome are precisely opposite to the traits which would induce someone to undertake exercise on the sofa towards a defined health benefit. The problem is not the activity being recommended, the problem is that the people most in need of exercise are also the least likely to be responsive to the prescription they get. The same is true of five-a-day campaigns. If you like, the sort of person who is likely to carefully mediate what they eat, count portions, plan meals and so on is probably quite conscientious and has a little above average intelligence; she probably has a university degree, or would have if she came of age after the expansion of higher education in the 1990s. In other words, the only people on whom the messaging works are the people who will be fine anyway. The marginal benefit is almost nil.

This strikes me as a particularly serious problem in nations like the UK, where virtually all policy of import in both the private and public sectors is designed by university graduates – who, by definition, are selected for above-average conscientiousness and above-average IQ. The trouble is that, by definition, half of the population have below-average IQ and below-average conscientiousness.  It might be natural to assume that this messaging will work if everyone around you talks casually about getting their five-a-day or their half-hour of exercise; when it comes into contact with the average person, it will likely be less effective. As a result, we see the same problems again and again: public advice made for homo economicus, who calmly, rationally and dispassionately analyses his habits to eliminate the bad and promote the good. It should not need to be explained that most people simply do not (and cannot) think in this way. Public health, without coercion, at scale, remains an unsolved problem.

To even begin to address this, it is not enough to “speak broadsheet to graduates“, but we have fewer and fewer interpreters as the intellectual selection processes for powerful institutions become stronger and more rigid. Our politics is increasingly designed for a particular grade of intelligence and for a particular set of habits of mind, so it becomes hard to see where the imagination necessary to cross the rationality gap might come from – and how the gap can even be pointed out to those too bright to see it.

Leave a comment

Filed under Culture

Some book recommendations, 2017 edition

I haven’t been very hot on my reading this year – or perhaps I’ve been rather too hot (given the Unfortunate Incident in February that destroyed about half of my books, inter alia). However, from what I have read I can give you in no particular order a few recommendations and tell you one book to avoid.

Scooting around London on the Tube in the wake of the Inauspicious Thing in February, I managed to read all of Anthony King and Ivor Crewe‘s monumentally good The Blunders of Our Governments. I was introduced to this book by a friend who evangelised about it for some weeks; I borrowed a copy from another friend whom I was staying with (to replace the copy destroyed in the fire) and ate it in two days flat. London is very dull without ID, money and a door key. The book is a pretty harrowing read. It’s not something you should read if you want to be optimistic about the governance of the United Kingdom; at the same time I think it’s pretty much required reading if you’re in any way interested in how we run our society. The structure of the book is a little bit idiosyncratic and it’s a bit too long, but it benefits from its highly readable style and its encyclopaedic approach: cataloguing every single catastrophe in British policy-making in a roughly 20-year period since the poll tax and explaining why they happened doesn’t sound like a gripping read, but trust me on this.

I finally got around to reading Jane Austen‘s classic Pride and Prejudice in April, a novel I hadn’t expected to like at all. I expected a dull but worthy read, but in fact found myself cackling with laughter on public transport and close to tears at the climax. The thing Austen does particularly well is the sense of suspense, the time being built and built and built – you really notice just how long it takes for these relationships to be brought to completion, and how urgent it is for young women of this era to marry and marry early. The social commentary is all the more effective because it’s subtle, and more is said in the absences and conscious avoidances than in the fabulously witty dialogue. Again, if you haven’t read it, you’ll finish it much faster than you expected to and be a lot wiser for having given it a go. Elizabeth Bennet is also one of the most winsome protagonists I’ve ever read and I could spend days in her company. She’s great.

Speaking of wisdom, I couldn’t write a post like this without mentioning Adam Tooze‘s knock-out work The Deluge: The Great War and the Remaking of Global Order 1916-1931. It is not an easy read. Tooze is a talented writer, but the content – a dense economic history of the world with a particular eye on Europe, America and the Far East in the aftermath of the First World War – is not easy to distil into a volume as slim and portable as this. Why does it work? First, its detail is intimidating but never alienating: Tooze does not shy away from complexity but he is a skilful teacher who believes the reader is as intelligent as (but less educated than) he is, and he is a patient guide to a period which seems surprisingly alien for all its sharp cultural memory. Second, and this may be subjective and very personal, but Tooze resists the trend to write in anticipation of the future. Lesser writers would signpost everything with references to the horrors of 1939-45, but here they are only briefly and rarely alluded to. Hitler and Tojo are effectively bit parts, a genuine shock when most relatively popular history likes to treat the First as mere prologue for the Second. His theory is really grounded by this tight focus and studious refusal to hit narrative clichés. It’s gold and cheap as chips, so read it.

The fourth and final book I’d like to touch on is one which I’d counsel you to avoid: the winner of the 2016 Man Booker Prize, The Sellout by Paul Beatty. A dark satire of a black man responding to American police brutality and white-flight by instigating slavery and segregation (it makes sense in context), it was highly praised when it came out. I found it intensely frustrating. It seemed to have been written largely for shock value, and lets its joyous wallowing in all the sordid details of its narrative get somewhat in the way of its message to the point of confusion and self-indulgence. Beatty’s gift for description (particularly the sequence with the orange tree towards the end, which is beautifully realised – the reader can almost smell the sweet juice soaking into the pages) is not really well-used here. It’s a righteously angry book, but anger is quite boring unless disciplined (The Handmaid’s Tale and Animal Farm spring to mind). It bears comparison with another gallows-humour Booker winner, Aravind Adiga’s excellent The White Tiger from 2006. Adiga’s novel is a blistering condemnation of the new Indian economy and the path the country took since independence in 1947 but written with brio and panache and a wicked sense of humour; Beatty’s is tiresome and wears out its welcome rapidly, and is strikingly unfunny for a book laden with blurb quotes about the quality of its comedy. Not really worth your time. It’s no stinker, but it is quite annoying.

Happy New Year to everyone. Next year might see a better-resourced list and I’m always looking for recommendations.

Leave a comment

Filed under Culture