Failures of Automation: Misunderstanding the Hierarchy of Evidence

Folks who are into methodological blood sports would be hard pressed to find a more engaging fight than that between the Cochraneists and the Clinicians over oseltamivir (Tamiflu). You may or may not remember the rush to stockpile this anti-viral drug after the bird flu pandemic of 2005.

The stockpiling came on the back of problematic marketing spin from the drug’s manufacturer, Roche, (which included burying negative studies) that overplayed Tamiflu’s effectiveness and underplayed side effects.

This pharmaceutical skullduggery was challenged when a group of researchers at the Cochrane Collaboration conducted a systematic review of all the available studies into the drug and determined that it was basically useless.

There was some high dudgeon in response to the methodology of this systematic review from infectious disease clinicians. Some of them derisively refer to the epistemic flaws of the Cochraneists as, 'Methodolatry'. This is an allusion to the Evidence Based Medicine movement's supposed elevation of the Hierarchy of Evidence to a divine status that elides the complexities of clinical experience and human frailties in using the Hierarchy.

To cut a long story short, the aggregate data from the Cochrane systematic review could not definitively demonstrate sufficient Tamiflu’s benefits outweighing the potential harms….

Meanwhile, the eminent gentlemen over at Science Based Medicine were adamant that on the ground, anti-flu drugs are a crucial weapon in the arsenal against a deadly disease and possibly a pandemic.

So... whom to believe?

The collators of data because data doesn't lie, right?

Or the skeptical clinicians, because skeptical clinicians are inured against various biases, right?

The source of my facetiousness should be obvious.

Clearly clinical expertise is useless without knowledge gleaned from randomised controlled trials. Meanwhile, randomised controlled trials can throw up perverse results for all sorts of reasons, and are in and of themselves useless without interpretation by people who have clinical expertise.

Indeed, this fight is a classic example of a failure to understand the epistemic mechanics of automation.

Automation may *feel like* the removal of human inputs from a process - whether that process involves manufacturing, computing, or analysing evidence - but the truth about automation is that it is always, at its source a human product. And as we well know, humans are fallible. So the processes by which automation is designed may themselves be flawed.

The clearest example of this is racist algorithms. An algorithm - an automated set of rules for the purposes of problem solving - may feel like an objective truth, but its parameters are designed by humans. And if those parameters are racist, then the way in which an algorithm will go about sorting a problem will be racist even if fully automated, with human involvement removed at the end point. To get really basic, if an input is racist, then the output will be racist too.

Automation is beloved of any individual or industry valuing standardisation and efficiency divorced from the vicissitudes of human folly, yet human folly can be inherent in any automated process.

The Hierarchy of Evidence is exactly this: it’s a tool created to attempt to counter the inherent biases in human - specifically, clinical - judgement, and it constructs a framework for evaluating evidentiary heft that is supposedly divorced from judgement calls that might be influenced by human bias.

Tying this back to Tamiflu, as much as the Evidence Based Medicine brigade like to think that their mechanisms are sufficient counter to human bias, the reality is, sets of decisions around a systematic review’s methodology still need to be made by people - what data to include versus what data to exclude, being chief among them.

The biggest failures of automation occur when automation itself is elevated as an ideal and framed as beyond human frailties - like bias, for example.

And the clinicians over at Science Based Medicine believe a fair bit of bias was involved in the unusually stringent inclusion criteria for the Tamiflu systematic review.

More broadly, a pretty obvious objection to the elevation of the Hierarchy of Evidence beyond clinician experience, is known in the trade as GIGO - Garbage in, Garbage Out. A systematic review can only be as good as the evidence it is reviewing.

If a case report is on the lowest rung of the hierarchy, and a systematic review is on the highest, the Hierarchy of Evidence by itself has no procedural mechanism for valuing a lack of conflict of interest behind a case report, and devaluing a meta-analysis comprising studies which may be riddled with conflicts of interest.

And in the specific case of Tamiflu, there is an accusation of BIBO - bias in, bias out!

But there is a further, deeper objection to over-reliance on this hierarchy.

The Hierarchy exists in an imaginary epistemic ecosystem in which all information - good and bad - is accessible. We know, however, that this is not the case - that countless trials returning 'uninteresting' results die a quiet death in the bottom of a researcher's desk drawer.

This is not to say that clinicians are always right either. Medical history is replete with doctors' fidelity to butchery and other harmful practices that were only stopped when well designed trials demonstrated the harms outweighing the benefits.

As always, in any issue dealing with complexity - and there is no more complex system known to us than the human body and brain - there is still no substitute for the emergent phenomenon that is human intelligence.

Human intelligence is needed to create the automation process and human intelligence is needed to parse automation's results.

Praise For Alex Fein's Work

'A gifted self starter'

I have known Alex Fein for 3 years. A gifted self starter, I would highly recommend her as an asset to any business, she has demonstrated an ability to handle a steep learning curve and achieve great results at the same time. I more than happy to provide this recommendation/reference.

- David Laloum IT Consulting & Project Management

'Able to consume and synthesise enormous amounts of information'

Alex is able to consume and synthesise enormous amounts of information and is obsessive about truth and accuracy. She conducts herself with responsibility and endless drive.

- Sivan Barak Bialobroda

'Strategic, creative, meticulous and bold'

Alex Fein is unique. She is strategic, meticulous, creative and bold. She was able to achieve what no one thought was possible.

- Associate Professor David Zyngier - Southern Cross University

'Her moral clarity and courage set her apart'

Alex’s is dedicated and thorough. She was wonderful to work with. Her moral clarity and courage set her apart.

- Dassi Erlich - Author: Beyond A Survivor

'A formidable intelligence... obsessive about truth and accuracy'

Alex has a formidable intelligence with a keen ability to interpret and decipher complex information rapidly and accurately. Alex has in depth knowledge of web communications and is one of the best online community builders I have ever known.

- Yoram Symons - Magic Industries

'Impressed by her intellect and ability to express complex ideas... into easily digestible content - an exceptional talent'

I’ve have always been impressed Alex Fein's intellect and ability to express complex ideas and articulate them into clever and easily digestible online content.
Alex strives to maintain her integrity over popularity which again, is a very rare commodity. Alex also manages to balance this with much respect and fairness which adds value to all the work she produces. She is an exceptional talent.

- Jennifer Weinstein - abuse survivor advocate

'She works tirelessly and passionately'

Alex works tirelessly and passionately. She is professional, organised and fair.

- Jacqui Schiff Saunders - Lonely Planet

'I highly recommend her'

Alex has shown a real strength in developing forums that often have spirited debate, ensuring a safe space in which to do so. I highly recommend her.

- David Langsam - Editor Biotech Daily Ag & Vet Weekly

'Commitment to transparent & accurate information sharing'

I have known Alex Fein and her work in the community for around 5 years. I have been impressed with Alex’s understanding of the potential of digital platforms such as social media to lead to effective social change. Her commitment to transparent and accurate information sharing, together with her strong grasp of media and digital skills, have supported the creation and growth of thriving online communities.

- Dr Miriam Faine Adjunct Lecturer, Faculty of Education, Monash University

'Knows what she's doing in the digital publishing space... completely transparent, ethical & accountable'

Alex Fein not only knows what she's doing in the digital publishing space but she also has an unwavering dedication to delivering the truth without hesitation. Alex is clever, articulate, completely transparent, ethical and accountable.

- Kathy Kaplan OAM - Founder Impact For Women

'exceptional track record of building digital communities which prioritise rigour and decency'

I have witnessed Alex Fein's exceptional track record of building thriving digital communities which prioritise rigour and decency for the past decade. She is driven by her commitment to the democratisation of good information.

- Dr Larry Stillman - Senior Research Fellow, Faculty of IT Lead Researcher, PROTIC Monash-Oxfam Project for Information Empowerment in Bangladesh Monash University

'Vast experience - I highly recommend her'

I’ve come to know Alex through a large, engaged & active twitter community - and now know her as someone who is knowledgeable, passionate & very concerned about the state of Australia’s media landscape & its impact on our democracy. She has vast experience in building & connecting online professional communities - & unique skills in examining both virtual & real life communities to determine their makeup, news consumption habits & how this motivates their actions & decisions. I think she’d provide valuable skills for your organisation.

- Denise Shrivell - media commentator and advertising executive 

'Robust and immediate engagement'

Alex has created organic media spaces that have allowed for robust and immediate engagement. It's a modern media platform that has given people a new way of engaging with current affairs and issues.

- Josh Burns MP - Member for Macnamara

'Provocative, punchy and spin free'

Meta and Galus - provocative, punchy and spin free news and debate. I didn't know what was missing until Alex created it. Thank you!

- Stephanie Hodgins May - Greens Candidate for the Seat of Macnamara

Get in touch

Please enter your name.
Please enter a subject.
Please enter a valid phone number.
Please enter a message.

We work on lands stolen from the Wurundjeri and Boonwurrung people of the Kulin Nation. Until there is redress, we ask for forgiveness and forbearance. We pay respect to their Elders, past and present and stand with them in their struggles for recognition, liberation, and justice.