Man Vs Machine

A few days ago, I was binging on Star Trek (1968) at Netflix when I watched the episode titled ‘The Ultimate Computer’. It is fascinating (sorry Mr. Spock) how relevant it is even half a century later; or may be it is more so now, than it was then.

Very briefly, a scientist has created the ultimate computer (a.k.a. M-5) which can run a Starship on its own and make all the decisions needed, making the crew and (especially) the captain redundant. And as we learn later in the episode, the Scientist has imprinted his own mind on it, so it can also learn and think.

To cut a long story short, the episode begins with a pilot installation of the M-5 in The Enterprise and ends with the machine’s destruction. Somewhere along the way, M-5 takes over completely, kills one of the crew members and also attacks other Federation ships under the mistaken belief that it is under attack. The irony is that all this while the machine thinks it is protecting human lives. Eventually, Captain Kirk uses his ingenuity to make the machine realize that it has committed murder and therefore must die and M-5 literally kills itself.

The clear message from the program was that machines are there to serve humans and should never be allowed to control them.

As I watched the program, I couldn’t help drawing a parallel to the MCAS system of the Boeing 737 MAX airplane. The MCAS was designed to prevent a plane from stalling and crashing. I guess the underlying spirit was to eliminate human error under stressful conditions. In both the incidents, the MCAS overrode the human pilots and eventually crashed the planes because it mistakenly ‘believed’ that the plane was about to stall.

The human pilots fought valiantly against the computer till the very end, but tragically, unlike in the Star Trek story, 346 humans lost in this battle of man vs machine.

It is said that to err is human. Logic then dictates (thank you, Mr. Spock) that anything built by humans should also be prone to error. But somehow, we miss that point at times. Boeing were so confident that they had built the perfect machine that they did not feel it necessary to alert airlines and pilots about the need for special training on how to deal with the MCAS. They are now building a new version of the software and I sincerely hope they remember that the final decision of what to do in a crisis should be left to the human pilots, even if that decision turns out to be a wrong one.

There’s no doubt that machines have made our life so much easier and safer but as Mr. Spock says in the episode “Computers make excellent and efficient servants but I have no wish to serve under them”.