practical skills and applied knowledge in Security, defense, and risk management


BBC Trust releases another report whitewashing the BBC's mishandling of science

Bruce Oliver Newsome, Ph.D.

10 August 2016

On 10 August 2016, the BBC Trust released a report on the BBC's use of statistics - another BBC report that whitewashes the BBC's mishandling of science, and misrepresents the evidence that it received, despite a year on the job. 

As typical for BBC inquiries, in 2015 the BBC Trust itself formed a panel with prior vested interests, rather than outsource the job to a genuine independent reviewer or reviewers. This panel asked for submissions of evidence from many institutions, and commissioned an effective contractor to get lots of so-called audience impressions, but without (ironically) any statistical evidence. From these largely subjective impressions and opinions that it received as evidence, the report quotes selectively and inaccurately. As with many reports, much of the writing seems to have been developed mostly by staffers who lack the scientific skills of the panelists, but only the panelists are formally listed as authors.

The call for evidence was selective, and asked for evidence to vague objectives and inconsistent standards, so the submissions have no common structure. 

I became aware of the call for evidence only when the Academy of Social Sciences released to its members its submission - in itself a non-statistical and non-scientific submission, which praised the BBC without the evidence or argument to back up the praise - apparently a submission to please, not to challenge, since it failed to acknowledge my own suggestion months earlier that the Academy of Social Sciences should campaign for more BBC attention to social science. Ironically, the submission proved the Academy of Social Sciences' own separation from good scientific practice (since then, the Academy of Social Sciences has gained new leadership).

In February 2016 - six months ago - I warned the BBC Trust of the inadequacies of the Academy of Social Science's submission, as follows:

"FAO: Dame Jil Matheson Chair, BBC Trust Impartiality Review Panel
The Academy of Social Sciences (AcSS) informed its experts of its response to your call for evidence by email yesterday on 7 February 2016. I am writing to warn you that its response is misleading. 
So far as I know, as one of its registered experts, it did not survey its own experts, inform its experts of your call for evidence, or invite its experts for comment on its response before submission to you. The AcSS response has no declared authors or consultants, apart from "the wider interests of the Fellows of the AcSS".
The AcSS response asserts supposed facts about social scientists' opinions and the BBC's record without evidence or citation. Its response has no citations other than one citation of itself, one of government opinion, and a citation related to public polling.
The AcSS response suggests the superiority of the BBC over other news media, but cites no evidence. The AcSS response admits no specific dissatisfaction with the BBC, but I know I am not alone amongst academics in my concern with the decline in the BBC's presentation of the social sciences. I myself wrote to the ASS in June 2015 suggesting that the AcSS should start a campaign for the BBC to pay more attention to social science. I specifically observed the BBC's inattention to social science and its unscientific representation of the social sciences. I specifically drew attention to BBC R4's "Thinking Allowed", which is - as far as I know - the only BBC programme that markets itself as social scientific, but peddles its presenter's social and political agendas and qualitative and non-statistical research as if they are scientific or statistical. BBC R4 has one worthy programme on statistics ("More or Less") but its prescriptions are routinely violated by other BBC programmes.
You would be wrong to treat the AcSS response as a meaningful response to your call or as representative of social scientific opinion."

The BBC Trust thanked me for my submission, and promised to take it into account. Seven months went past, without any follow up from the BBC Trust, until it copied its report to me. It alerted me to the release of its report, and thanked me for my contribution in your email, but I have checked the report, and I can find no reference to my evidence, or to my contribution. Its online links included the Academy of Social Science's submission in its entirety, but ignored my correction of it. Therefore, I cannot agree with Dame Jil Matheson's statement: "I have taken account of all the valuable contributions we received in forming my conclusions, and I have based my findings firmly on the evidence we gathered."

The panel's findings are so vague and unfounded as to whitewash the BBC's failings. For instance, it repeatedly praised BBC Radio 4's "More or Less," as I praised in 2015, but does not admit, as I warned in 2015, that this programme is unrepresentative of BBC handling of statistics, and that the BBC's so-called scientific programming (such as "Thinking Allowed") routinely violates the advice given by "More or Less."

In 2015, I had independently complained to the BBC that "Thinking Allowed" was misrepresenting social science, by ignoring statistical research, and by presenting only social opinions and unreplicable observations as social science. The BBC investigated that complaint in 2015, but it does not feature in the BBC Trust's report; the report does not mention my warning direct to its panel's chairwoman; the report does not mention the other warnings that I know some of my colleagues have submitted independently.

Moreover, some of the recommendations are quite contrary to good practice, such as the recommendation (page 12) to talk about the "doubling of risk," when the doubling of probability from 50% to 100% is quite different to the doubling of probability from 0.0005% to 0.0010%. The recommendation to talk about doubling the risk is encouraging the "misleading representations" that some submissions urged the inquiry to challenge, and which the report itself recommends. Indeed, on page 82 the report appends the BBC's own training advice on the use of statistics, which notes that doubling very small risks still produces very small risks. 

As ever, the BBC serves itself, and promotes itself, and wastes great resources on reports that it can cite as evidence for self-celebration, and ironically proves the BBC's failings to engage with good scientific practices, its misrepresentations, and its spin.