Why The COVID-19 Model That Inspired UK’s Lockdown May Be “The Most Devastating Software Mistake Of All Time”

While Democrats in the US and progressives in the UK continue to push back against efforts to gradually reopen their respective economies, more evidence is emerging that calls into question the models (what the public often refers to as the “science”) which inspired governments across the world to impose crippling lockdowns on their populations.

Case in point: Since Neil Ferguson and the authors of the Imperial published its modeling for non-pharmaceutical intervention for COVID-19, a number of data scientists have taken a close look and found gaping oversights that seriously undermine the model’s credibility. Of course, this isn’t the first time we have written about Ferguson and his exploits.

Neil Ferguson

In this weekend’s Telegraph, two of these critics, David Richards, the founder and CEO of global big data leader WANdisco which is jointly headquartered in Silicon Valley and Sheffield, and Dr. Konstantin Boudnik, a pioneering big-data engineer, WANdisco’s VP of architecture and author of 17 US patents, published an editorial in which they carefully examined the model’s shortcomings. Keep in mind, the Imperial model is what ultimately inspired PM Boris Johnson to make a U-turn and adopt what has been an economically devastating lockdown – was nothing short of a catastrophe. Millions have been plunged into hardship and poverty unnecessarily, they explained. Johnson himself was infected by the virus and the public is furious with the government over its rollout of a plan to reopen.

Given the influence the model had during the early days of the outbreak, the two men argued that the software issues underpinning the model could be ‘the most devastating software mistake of all time’.

Apparently, the model’s problems are rooted in its most fundamental components. The model was written using a coding language called  Fortran which has been in use for decades.

Due to its age and inflexibility, Fortran has many inherent problems. But on top of the language itself, the code in the model was sprawling, sloppily written and extremely inefficient, the two men said, claiming it would never pass muster in the private sector.

Using straightforward, jargon-free language, the two authors explain how the model ran into a problem called “CACE”, or, ‘changing anything changes everything’ – a problem that software engineers and data scientists trying to model, well, anything, really, often encounter.

The approach ignores widely accepted computer science principles known as “separation of concerns”, which date back to the early 70s and are essential to the design and architecture of successful software systems. The principles guard against what developers call CACE: Changing Anything Changes Everything.

Without this separation, it is impossible to carry out rigorous testing of individual parts to ensure full working order of the whole. Testing allows for guarantees. It is what you do on a conveyer belt in a car factory. Each and every component is tested for integrity in order to pass strict quality controls.

It’s just the latest reminder that President Barack Obama’s advice to this year’s graduates rings true: You can’t just blindly accept what the experts and the people in charge tell you.

Read the full editorial below:

* * *

In the history of expensive software mistakes, Mariner 1 was probably the most notorious. The unmanned spacecraft was destroyed seconds after launch from Cape Canaveral in 1962 when it veered dangerously off-course due to a line of dodgy code.

But nobody died and the only hits were to Nasa’s budget and pride. Imperial College’s modelling of non-pharmaceutical interventions for Covid-19 which helped persuade the UK and other countries to bring in draconian lockdowns will supersede the failed Venus space probe and could go down in history as the most devastating software mistake of all time, in terms of economic costs and lives lost.

Since publication of Imperial’s microsimulation model, those of us with a professional and personal interest in software development have studied the code on which policymakers based their fateful decision to mothball our multi-trillion pound economy and plunge millions of people into poverty and hardship. And we were profoundly disturbed at what we discovered. The model appears to be totally unreliable and you wouldn’t stake your life on it.

First though, a few words on our credentials. I am David Richards, founder and chief executive of WANdisco, a global leader in Big Data software that is jointly headquartered in Silicon Valley and Sheffield. My co-author is Dr Konstantin ‘Cos’ Boudnik, vice-president of architecture at WANdisco, author of 17 US patents in distributed computing and a veteran developer of the Apache Hadoop framework that allows computers to solve problems using vast amounts of data.

Imperial’s model appears to be based on a programming language called Fortran, which was old news 20 years ago and, guess what, was the code used for Mariner 1. This outdated language contains inherent problems with its grammar and the way it assigns values, which can give way to multiple design flaws and numerical inaccuracies. One file alone in the Imperial model contained 15,000 lines of code.

Try unravelling that tangled, buggy mess, which looks more like a bowl of angel hair pasta than a finely tuned piece of programming. Industry best practice would have 500 separate files instead. In our commercial reality, we would fire anyone for developing code like this and any business that relied on it to produce software for sale would likely go bust.

The approach ignores widely accepted computer science principles known as “separation of concerns”, which date back to the early 70s and are essential to the design and architecture of successful software systems. The principles guard against what developers call CACE: Changing Anything Changes Everything.

Without this separation, it is impossible to carry out rigorous testing of individual parts to ensure full working order of the whole. Testing allows for guarantees. It is what you do on a conveyer belt in a car factory. Each and every component is tested for integrity in order to pass strict quality controls.

Only then is the car deemed safe to go on the road. As a result, Imperial’s model is vulnerable to producing wildly different and conflicting outputs based on the same initial set of parameters. Run it on different computers and you would likely get different results. In other words, it is non-deterministic.

As such, it is fundamentally unreliable. It screams the question as to why our Government did not get a second opinion before swallowing Imperial’s prescription.

Ultimately, this is a computer science problem and where are the computer scientists in the room? Our leaders did not have the grounding in computer science to challenge the ideas and so were susceptible to the academics. I suspect the Government saw what was happening in Italy with its overwhelmed hospitals and panicked.

It chose a blunt instrument instead of a scalpel and now there is going to be a huge strain on society. Defenders of the Imperial model argue that because the problem – a global pandemic – is dynamic, then the solution should share the same stochastic, non-deterministic quality.

We disagree. Models must be capable of passing the basic scientific test of producing the same results given the same initial set of parameters. Otherwise, there is simply no way of knowing whether they will be reliable.

Indeed, many global industries successfully use deterministic models that factor in randomness. No surgeon would put a pacemaker into a cardiac patient knowing it was based on an arguably unpredictable approach for fear of jeopardising the Hippocratic oath. Why on earth would the Government place its trust in the same when the entire wellbeing of our nation is at stake?

* * *

Source: The Telegraph