The Iowa Caucuses: What the Hell Happened, and What Didn't
It’s past midnight in the Central Time Zone and we have no precincts reporting out of the Iowa’s democratic caucuses. At this time in 2016, 97% had reported their numbers. Pundits on MSNBC, CNN, and Fox news are lamenting how such an important event could be allowed to fall into utter disarray. In jeopardy are year-long multimillion-dollar campaigns as well as the credibility and legitimacy of our democracy. Given the relative importance of Iowa’s first-in-line status in the primary election cycle, it begs the question: how the hell did the Iowa Democratic Party (IDP) and the National Democratic Party allow this to happen?
What Probably Didn’t Happen
1) The application was hacked or compromised: The Iowa caucus application didn’t fail because someone engineered a backdoor and subverted the numbers being tabulated. If you think about it for a minute, state actors would go out of their way to prevent something like that from happening so that they could continue to commit their sinister agenda. In addition, one of the teams behind the development of this application has a pretty solid background in both homeland security and cybersecurity; Defending Digital Democracy is sponsored by the Harvard Kennedy School Belfer Center for Science and International Affairs, and it aims to develop strategies, tools, and technology to protect democratic processes and systems from cyber and information attacks. No one with a background in homeland security and cybersecurity would ever allow this kind of application to be developed by anyone who did not hold some sort of clearance or private sector equivalence. At a minimum, they understand the science behind the security, and the threats that are posed by seditious elements around the world.
2) The application failed under excessive load: The application didn’t go down because the infrastructure wasn’t capable of managing peak user demand. The Iowa caucus doesn’t have a user base running in the tens or hundreds of millions – I’d be surprised if it’s more than a few hundred, if that. You could run that off of a couple of servers in your garage.
3) The application was poorly constructed: This app was not using quantum fluctuations to encrypt data. It didn’t need to estimate the angle of ascent during a midair dogfight involving five fighter jets. The app probably worked exactly like it was supposed to. It was designed by Harvard engineers, not by Elizabeth Holmes and her brother-in-law. It’s not rocket science, people. It’s just tabulating votes.
The Most Likely Scenario
If it wasn’t a sinister attempt to subvert our democracy by state actors, and if it wasn’t coded by pathologically stoned college students who got it all wrong, then what gives? If the tech is sound and bulletproof, how could this have happened?
The real scenario here is much more prosaic than an international team of hackers bent upon destroying our democracy. It goes like this: the application was deployed with a clean set of instructions that were either not read or misunderstood by just enough people to gum up the works and throw everyone into a panic. That’s it. Yes, it is really that simple.
This is something corporations deal with on a regular basis. Ten years ago, the number one reason application projects failed was resistance to change. Although that number has gone down, the private sector still invests millions of dollars every year to fight resistance to change. In every project I have been involved with over the last 10 years, there has always been a significant investment to ensure that the organization understands what the new project is going to do, how roles and responsibilities will change, and how to mitigate risks that will arise from those changes. Part of this involves training the user base on how to use the and capturing questions and problems to feed back to the developer community. Investments are also made in process and value stream mapping - mapping existing works streams to new functionality, and turning those maps into a process flow that explains to a user how their workflow will change. Finally, before the project is released into the wild, it goes through a number of mini go-lives in order to test how well the change management process has worked.
With this in mind, let’s break down what happened in Iowa a bit. The IDP introduced a few new rule changes. Nothing mind-boggling, and certainly nothing that a functioning adult could fail to understand. A new rule was that if you caucused for a candidate that ended up being “viable” (meaning they garner at least 15% of the vote in the precinct you are in), you cannot change your vote. Previously, you could change your vote irrespective of how much support your candidate received. This was designed to accelerate voting – only caucus members of non-viable candidates were allowed to change their votes in order to apportion delegates to the top vote-winners. Another change is that the IDP is publishing three numbers to describe the evening’s outcome: the vote count after the first round of the caucus; the vote count after the second and final round of the caucus; and the number of delegates for each candidate. Both of these changes have literally zero impact on application performance or viability.
But they do have an impact on how votes are captured, processed, recaptured, reprocessed, checked against physical inventory of vote cast, and finally reported. There are a hundred ways that could go sideways, and probably every one of those hundred ways found a path into the madness that ensued in Iowa tonight. Here you have a most-likely scenario: when users went live, they literally did not know how to use the app like it was supposed to be used. Maybe it was a function of the complications involved with the new rules. Maybe they didn’t know how to map the ballot paper trail into the app. Whatever it was, the users gave up when they realized that they did not know how to use it properly. They went to the “backup system” which was a telephone call, and those thousands of users quickly overloaded everything. Fear of failure paralyzed the system, and everyone involved shut down communications in order to protect the party.
On the bright side, we can probably throw out “the votes must be rigged” or “the application was compromised” opinions that have already surfaced from both TV pundits as well as political operatives. Instead, what we should demand is that our public institutions ask consultants with real-world experience in application development for advice when deploying software that will influence both our reputation around the world as well as the future of our democracy. We’ve been doing this for decades. This is a problem we could have identified before anyone said anything about what the application was supposed to do. Change management is always a problem, because yeah, people.
Jamal Khawaja