Tracing The History Of Artificial Intelligence

Earlier this week, I found myself answering a question from a new colleague at Finning International that relates both to the research I do in the iSchool at the University of British Columbia, as well as the analytics, engineering & technology work that I lead at Finning. The questions were simple: 1) What is artificial intelligence? 2) Where does the discipline come from 3) When did it start and how did it evolve? As I sat to reflect last evening, it dawned on me that taking time to craft a clear answer to these questions might be extremely beneficial for many. Analytics, data science, and predictive intelligence are hot topics in many communities and business areas. And yet, despite this interest, few folks I have talked to have a clear understanding of the history of the discipline; one, that frames much of the work currently going on within the space. Thus, in the next few paragraphs, I will seek to trace the history of artificial intelligence, drawing and quoting from the phenomenal reference book by Dr. Michael Negnevitsky called "Artificial Intelligence". It is the first place I went for answers, and a starting point for anyone curious to understand more about the discipline of AI.

Before one can lay out a coherent history of the evolution of artificial intelligence, it is important to start by clarifying some key terms for our discussion. The first question, is the definition of intelligence. As Dr. Negnevitsky notes, intelligence is "the ability to learn and understand, to solve problems, and make decisions." Humans, arguably, are intelligent beings, though circumstance sometimes lends to seemingly unintelligent behaviors. That said, saving this debate for another time, if we accept this definition, then the next key term to define is "Artificial Intelligence", or AI. As an artificial intelligence researcher & graduate student in the University of British Columbia's iSchool with Dr. Vicki Lemieux, I would start by agreeing with Dr. Negnevitsky. In his eyes, AI is a scientific discipline that exists with the goal of making machines that can do things that would require "intelligence" if a human were to attempt it. Operationally, this means that any intelligent machine must be able to capture, organize, and then utilize knowledge gathered from many sources just a human would. As you can imagine, there are many ways to attack this task and many success stories. That said, if we accept this definition on what AI is, we can then move onto our original question regarding the history of AI research.

As many in the field of AI would agree, the birth of AI traces back to 1943, when Warren McCulloch and Walter Pitts proposed a model for artificial neural networks. In this model, neurons existed in binary on / off states. Using computer science theory, McCulloch and Pitts demonstrated that their neural network model was "equivalent to the Turing Machine", proving that "any computable function could be computed by some network of connected neurons." This foundational work is fascinating work; work, that I believe anyone in the field of AI or frankly "analytics" should review. This work, was a key complement to the 1950 seminal paper published by Alan Turing called "Computing Machinery and Intelligence". In that same period, von Neumann was leading the Electrical Numerical Integrator and Calculator project, and Shannon published his paper called "Programming a Computer for Playing Chess".

With AI birthed, the next big push came between 1955 - 65, a period that Dr. Negnevitsky refers to as the "Rise of Artificial Intelligence". During this time, there was a lot of momentum and energy focused on AI with little success. During this decade long period, John McCarthy, the creator of the term "Artificial Intelligence", defined LISP, a high level language that came two years after FORTRAN. Similarly, Newell and Simon created the GPS or General Problem Solver project at Carnegie Mellon University which was rooted in means-ends analysis. They sought to solve the AI problem using "machines" which analyzed different "states". The means-ends piece refers to an attempt to continually analyze between the current and desired end goal state. This was revolutionary work at the time but unfortunately had limited success solving large problems.

Following these failures, research in the field entered what can be categorized as the era of disillusionment. In the late 60's and early 70's, promises were made that AI machines which had the capacity to think in a manner similar to humans would exist by the mid 1980's, with machines that exceeded human intelligence existing by the year 2000. Unfortunately, there were many flaws which prevent the achievement of this, a large volume relating to computing. For example, if a problem has many steps or states to review, the tractability of the problem and time to solve it became an issue. For these challenges, Cook's work on the "Complexity of Theorem Proving Procedures" in 1971, as well as Karp's 1972 work on the "Reducibility Among Combinatorial Problems" were key.

As this work continued to emerge, the decade between 1973 and 1983 started to see the evolution and discovery of what we call expert systems. At the root, the learning that kicked off this period was the recognition that the problem space or domain of intelligent machines had to be restricted to have success solving complex problems and emulating human decision making. The creation of the DENDRAL program, developed at Stanford University by Feigenbaum, Buchanan, and Lederberg, with additional support from NASA, is a key example of this since it focused on analyzing chemicals. Following this, the development of MYCIN, a "rule-based expert system for the diagnosis of infectious diseases" by Dr. Edward Shortliffe was another. This expert system could perform better than junior doctors and on par with experienced doctors. Interestingly, it used approximately 450 if-then rules to narrow its answers. At the same time, to much fanfare, was the development of a system called PROSPECTOR, and expert system for mineral exploration led by Duda. This work, which reminds me of the important work done by many of our Finning clients in South America, Canada, Ireland, and the UK, used Bayes rules of evidence to propagate uncertainty.

Following this, the mid 80's brought us to the "rebirth of artificial neural networks". This period also overlapped with complementary work going on relating to evolutionary computation. Evolutionary computation "combines three main techniques: genetic algorithms, evolutionary strategies, and genetic programming." Evolutionary computation attempts to learn by simulating a population of individuals, evaluating their performance, creating a new population, and repeating. Seminal work by John Holland in his 1975 paper "Adaption in Natural and Artificial Systems" is a great reference. Computationally, Rumelhart, Hinton, and McClelland's 1986 works on "Parallel Distributed Processing" is also key reference. Further to this, in 1987, the first ever IEEE conference on neural networks was held. And, in 1994, Haykin released his book Neural Networks; one, that sits on my office bookshelf. This work spawned the later creation of the MATLAB Neural Network Application Toolbox by MathWorks, Inc.

The final step in the journey that we will touch on today relates to the last phase, one that Dr. Negnevitsky calls the period of "Computing with words". There is still much work to be done in the domain of neural networks (a theme for a another post). The computational components are fascinating unto themselves. That said, in this last period, which started in the late 80's, researchers focused on the use of "fuzzy logic". Fuzzy logic, also known as fuzzy set theory, was created by Dr. Lotfi Zadeh at Berkeley. Fuzzy logic models, while not fully embraced, offered many advantages to other models, including improved computational power, cognitive modelling, and representation capabilities. There has been a tremendous volume of work done here. The Japanese were some of the earliest adopters of this approach, using it so create dishwashers, washing machines, air conditioners, television sets, and other products. Even the Sdendai Subway System, built in 1986, used fuzzy logical approaches. In 1993, Kosko release a book called Fuzzy Thinking. Cox released "The Fuzzy Systems Handbook" in 1994. MATLAB also released multiple modules in this period for both fuzzy logic and neural networks.

As one looks back, it is clear that the field of artificial intelligence has a rich history. That said, understanding this history will hopefully help you now understand where AI research is heading, as well as the potential for this work when tied to robust technology. Every year, systems built using expert, neural, or fuzzy approaches are being applied to more and more problems in many disciplines - from medicine to resource sector. What's more, every year, the number of technologies that facilitate this work continues to grow, with researchers and practitioners from statistics to computer science to domain specific researchers and practitioners all working together to push the impossible forward. In fact, in the modern analytics technology stack, elements of these systems often all co-exist in a complementary fashion! This co-existence, made possible by the creation of consolidated and intelligently designed technology systems, will enable both researchers and organizations to move closer to the creation of truly intelligent machines - machines that can think as humans do, interact & converse with humans, augment our own human cognition, and potentially feel! If you have questions, or want to continue the conversation, please don't hesitate to reach out!