Expert Systems Nostalgia

Ed Feigenbaum, Joshua Lederberg, Bruce Buchanan, William Clancy, Randall Davis, .... a bunch of names more familiar to me than many of my colleagues! No, I have not met any of them. But they were the names I grew up with in my early years at NCST (now CDAC Mumbai). I was reading an article in AI Magazine 2017 volume.

It was Prof Mahabala of IIT Madras who introduced me to the field of expert systems during my B.Tech days (approx 1986-7). Umesh and I were started on a little project implementing a program to identify a bird using its features (like has feathers, has beek, etc). I think that is how expert system concept was introduced then – the notion of a generic reasoning engine and an externalised knowledge representation. It was very interesting as it could explain its reasoning, and justify why it was asking specific questions. The actual mechanism was quite simple, with a neat rule based representation with a backward chaining engine. One of the reasons for my love for AI was this observation, most of the great AI systems appeared quite simple and intuitive internally.

We went on to build India’s first expert system shell as our B.Tech project under his guidance. It came out quite well, and I was happy to note that Prof Mahabala talked about this tool (he called it IITMRule) in many forums as part of his work in this area. I think not too many B.Tech projects get such a recognition. And this triggered my interest in the field of expert systems, and I wished to join a place working in this area, post my graduation. 1987 was a time when hardly anyone knew computers, let alone things like AI or expert systems. There were very few computer labs or companies in India. It was then that I saw a notice from NCST (national centre for software technology) inviting applications for working in the area of artificial intelligence. I was tempted, and I did send an application. Interview was done and I got the job.

Expert systems was not an area at NCST then. I joined the team – not much to call a team, it was a small group just starting work under the KBCS project of the UNDP – on parallel logic programming. I had studied Prolog during B.Tech and that helped here. Dr Ramani, the then Director of NCST, was interested in expert systems too. We introduced a module on expert systems in our PGDST course. As per our convention, we make every module hands on and practical, with a balanced blend of theory and practical components. Our philosophy was that when one learns about expert systems, one must build an expert system to properly understand the subject. Given my experience of building an expert system shell (IITMRule), I put together a simple tool using the same idea using Prolog. Thus was born Vidwan!

Vidwan’s rule structure was influenced by this decision to use Prolog. I created a rule format which I can read into Prolog without having to make my own Parser. So, it looked like this:

rule(id, conc, val, cf) :- is(attr1, val1), isnot(attr2, val2).

This was a valid Prolog term, and I could read it using a ‘read’ call. This became the format of Vidwan’s rules, though we moved out of Prolog soon after. Vidwan became a useful tool for teaching. PCs was becoming common slowly, and I made a version of Vidwan for DOS using C. Squeezing the system into 640K memory was a challenge. We had a stable DOS version of Vidwan. We made this into a product with a lot of engineering, and in the 1990-1995 period, we clocked about 300 copies sold to various organisations. More about that productisation experience, I will defer to another post. I also adapted that version of Vidwan to run on Unix terminals, for wider access in the Centre.

Years later, when Windows came, a couple of team members who joined us was assigned the task of porting it to Windows GUI environment, and still later to Web environment (circa 2005). This Web version is still available, though largely dormant. I think this is the only expert system shell built in India which went to developers and users (i.e., excluding things built as BE/MCA projects).

Dr Ramani started building an application as a consultant for someone getting some chest pain, as to whether it is something serious or not. He spent a lot of time, building and cleaning the rulebase. This was our Hruday rulebase, distributed with every copy of Vidwan. A cardiologist friend of Dr Ramani also tried his hand on this and gave good feedback, and helped refine its rules.

The students doing the expert system module were required to build an application using Vidwan. We had such an enormous range of things that our students built, each running into 100-500 rules. The domains span headache diagnosis, abdominal problems, machine fault finding, agriculture advisor, financial investment advisor, and so on. Unlike today’s student projects, they were great pieces of work; I had preserved many of them in floppy disks – the portable storage devices of those times – for many years.

Eventually all this led to the idea of writing a book on expert systems. Dr Ramani, me and 3 more of my colleagues brought out a book titled “Rule based expert systems: a practical introduction”, and was published by Narosa press. It had three case studies implemented using Vidwan, including Hruday. It is now out of print; but this book in electronic form is now freely available from ResearchGate. It seems to be quite popular looking at its access statistics.

With all this activity, there was a lot of literature exploration in this area, with the limited resources we had. We had a good collection of books and journals in this area, thanks to the UNDP funded KBCS project. This included the expensive Artificial Intelligence journal where a lot of the seminal work in the field appeared. An example is a full issue devoted to William Clanceys work on building a system to learn diagnosis using MYCIN’s rules. It is still an inspiring read – the myriad challenges in exposing the hidden knowledge in a rulebase for a novice to learn from. I spent a lot of time with Lederberg’s Dendral, Buchanan’s MYCIN, Clancey’s GUIDON, DEC configuration system XCON, and many others. We had the big book of MYCIN outlining the entire evolution of MYCIN, its structure, its uncertainty reasoning framework called certainty factors, etc. It was a great resource for any expert system person. Another memorable read was the “rise of the expert company”, a little book on the success of the field of expert systems by Ed Feigenbaum. All this made these great names, household names in our little group.

We experimented with rule induction, rulebase verification, rulebase refinement, etc as a consequence of this reading. Some of these made its way into Vidwan.

As I read the article on a panel discussion in AAAI-2017 about the history of expert systems, my mind went back to those days for a while. Those were great years for AI – less hype and a lot of great work, and so many interesting problems. After a long period of slumber, it is nice to see AI waking up – but this rebirth has more hype and less work.