I’ve been trying to wrap my head around this incredible moment in history. In trying to wrap my head around it, I’m turning to what I know best: programs, programming, and software-intensive systems. And their flaws. This is a mini-essay on how the line between software systems and social systems is completely blurred, and how what we know of software can inform, or explain, the hyper-connected, information-rich world we live in.
Let me start with a little program:
def divide(n, d): return n/d
This happy little program takes two numbers, a numerator n and a denominator d, and returns n divided by d. Pretty straightforward. If you put it behind a service as is, let’s call it a “division service,” and the rest of the server trusts this program, it works as expected, flawlessly, time and time again anytime anyone calls it… Until one day someone passes the value zero as denominator d. At that moment, the service that is running this happy little program crashes to a halt and no one can use it anymore. “What happened?!,” we ask when we don’t get any response from the server.
If I hadn’t shown you the code above, many things could explain the failure. For example, an intruder might have broken into the server and stopped it; this would have been old-style sabotage of the kind we know from the movies, an infiltration of an attacker, and it would explain why the service isn’t responding anymore. Or it could have been a mole with insider access to the server who had been sleeping for a while and decided to spring into action at some point. Or it could have been a “man in the middle” attack, where the path between the client and the service goes through some enemy node that always returns a failure. All of these explanations fall into traditional security scenarios that our society has come to acknowledge and protect against.
But that’s not what happened. I’ve shown you the code, and I told you what happened: someone sent information that made the program crash. In programming, dividing by zero is an unrecoverable error, because the mathematicians haven’t figured out what it means to divide any number by zero. There are no intruders, moles, or subversive agents in the middle here. The program did exactly what mathematicians tell us it should: it crashed, because there is no answer to dividing by zero.
So, who do we blame for the failure? Do we blame the little program for being so fragile (it ought to check the value of d before attempting to use it as denominator!), or do we blame the person who sent a zero as denominator? What’s with this person, anyway? Doesn’t she know that dividing by zero is an invalid operation? Maybe she did it by accident. Or… maybe she suspected that the program was eagerly wanting to please by bypassing guards, becoming vulnerable to certain kinds of input information, and she took advantage of this program’s optimism to bring it to a halt.
In the programming world, we are all very familiar with the concept of failure by exploitation of a program’s optimism, reflected in lack of guards against certain kinds of input information or certain uses — be it divide-by-zero, buffer overflows, sql injections, or even denial-of-service attacks. The internals of these failures are all different, but the cause is always the same: the program trusts information that is given but that isn’t properly prepared, or engineered, to process; this leads it to an unstable, unpredictable, or unwanted state, possibly a crash.
Hackers live off this optimism. When they want to influence the behavior of a piece of software without having access to the server where it runs, they engage in what I’d like to call “software propaganda.” Wikipedia defines propaganda as “information, especially of a biased or misleading nature, used to promote a political cause or point of view. Propaganda is often associated with the psychological mechanisms of influencing and altering the attitude of a population toward a specific cause, position or political agenda in an effort to form a consensus to a standard set of belief patterns.”
The software propaganda that hackers do consists of the manipulation of input data sent to programs towards leading them to behave in a certain way, due to them trusting the input they are given, and acting on it, without being equipped to process it. Like dividing by zero. Or running a part of the program that is meant to be run only by administrators. Sending a zero on purpose to our little program up there is an act of propaganda:
The caller: “BREAKING! You can now divide by zero!”
The program: “oh wow, cool! let me do it.” *crash*
What does this have to do with what’s happening in the world?
I’m not going to talk about the break-ins into the DNC and RNC servers — that’s more or less irrelevant here. The issue I want to talk about is this: as any hacker knows, it is perfectly possible to destabilize a system without having to intrude it. Let’s call this “spooky action at a distance,” although I’m pretty sure this is not what Einstein had in mind when he used this expression. But here’s the fact we know from programs: as long as the system accepts input from anyone, chances are it has weaknesses (aka bugs) in processing some of that input that will throw it into an unstable or unwanted state.
This is true of programs, and I’m afraid this is also true of people. The real alarming situation that was part of new territory in this election was the full-on assault on people, the general public as well as specific individuals, as processors of information. The phenomenon of “fake news” and outrageous behavior in the 9 months, or so, leading up to the election was like nothing I have ever witnessed before in my life!
What happened was very similar to what hackers of programs do when they want the programs to become unstable: give them input that they may be ill-equipped to process. When the hackers don’t know exactly what input that is, they often try random input, a lot of it, until one of them slips through the guards and finds its way to a part of the program that is ill-equipped to process it. The simple process of trying random input data may be enough to throw the system into an unstable state — that’s the well-known denial-of-service attack, and that works for people too.
Some link: “BREAKING! Hillary Clinton caught in the act!”
Reader: “oh wow, cool! let me process this.” *does not check source, sways against Hillary Clinton*
Result of propaganda in programs: unstable, unwanted behavior, or even crashes.
Result of propaganda on people: psychological conditioning, possibly leading to social unrest, and, in a few cases, leading to criminal acts.
In both cases, we have spooky action at a distance: no trace of intruders breaking in, just regular information flow.
Now that we have a gigantic network of people and programs, all connected, the line between hacking programs over the Internet and hacking people over the Internet is disappearing. The methods are essentially the same. The bots that have been targeted at Internet-connected programs can easily be refurbished to target the brains of Internet-connected people, at a scale that wasn’t possible before.
The worst of it all is that it worked! The full-on disinformation assault, the propaganda, succeeded in swaying people towards believing absurdities, causing social unrest, and ultimately electing a man who is, at best, unqualified to be President and, at worst, a sock puppet of a foreign country.
So, who do we blame for this failure? Do we blame the people and institutions for being fragile and too optimistic/naive, or do we blame the producers of propaganda? Clearly, people and institutions are a lot more complex than the happy little program I showed up there, and they’re certainly a lot harder to “fix” than adding a patch with a conditional.
More importantly, it’s very clear that some players out there have mastered the art of propaganda, both for software and for people; how can we prevent brain hacking from them again, at this scale?
I wish I had answers.
This is clearly new territory for cybersecurity, although it’s been know to science fiction for a while.