Machine Learning and Growing
I'll admit it. I have been an artificial intelligence (AI) skeptic for 20 years now. There were many things I didn't like about AI claims, but chief among them is that it didn't work. At least it didn't work as well as the sales people said it did. There was a lot of hand waving that ultimately led to disappointing results.
Intelligent Classifier
My "machine learning" experience was mostly informed by two tools: Verity Intelligent Classifier (VIC) and Autonomy IDOL. Both had then state-of-the-art variants of machine learning. In the case of VIC, the algorithm was a logistic regression classification engine. IDOL, on the other hand relied on Bayesian classification.
Invariably, I would dissuade customers from using either of these classifiers, and instead rely on hand-crafted rules-based classification. The reason is that the extracted features (concepts) from a given set of documents did not contain enough information to make the same decisions that a human being can make. The output from the LRC would take hours to produce, and the list of terms that it would select made no sense to a human reader.
There are problems that I labored with for months that are now a decent training set and 24 hours on a Tensor Processing Unit (TPU) cluster away from total automation. Here are a few examples.
Glaxo SmithKline
My task was to use Verity Intelligent Classifier to try to figure out how to automatically classify documents according to type. For example, clinical studies and pharmacokinetics reports. This is one of those "trivial for humans, very hard for computers" problems.
The project was so disorganized that the project sponsor could not secure a location for me to work for more than 4 hours at a time. The reason for this is that he wanted to sit and watch me work, so we needed conference rooms with projectors. I'm not making this up. He wanted to watch me work. He was neither a subject matter expert nor a programmer.
To be clear, I don't mind people watching me work. But this guy did nothing but hector me, while adding no value to the engagement.
My job consisted of manually looking at documents of each type that needed to be classified and trying to find out clues about what kind of a document it was. I would then manually tweak a classification rule to try to catch matches and reject false positives.
The process was grueling and ultimately ineffective. We never formally tested the performance of the resulting taxonomy, but it was pretty lousy. They paid a lot of money just for the babysitter, let alone the consultant (me).
Nowadays, using TensorFlow, it would be much easier and far more effective to implement a great classification neural network that would work far better.
Ziff Brothers Investments
In the CitiGroup building on the lower East Side of Manhattan, ZBI had catered lunch every day for the few dozen mathematicians, brokers, and analysts that managed the inheritance of the scions of Ziff-Davis publishing. They generously allowed consultants to stay in the building and have lunch on their dime as well. It saved us from the hoi polloi mulling around 53rd and Lex.
ZBI was looking to mine SEC filings for "occult legal proceedings". In other words, while there is a section of the 10k and 10q reports for official legal proceedings, occasionally, items that should be reported there were put in other parts of the document. One can only guess why they wanted this information - but it was non-trivial to obtain using Intelligent Classifier.
Since they had a team of mathematicians there, we were able to do some fairly cool things with the idea of "locii'. It turns out that some of the techniques we came up with are actually part of many machine learning algorithms - specifically K-Nearest-Neighbors. Either I was really clever to re-invent something that was well known to mathematicians, or those mathematicians didn't recognize the problem I posed to be solvable with KNN.
The ZBI gig was quite well run - there were a lot of smart people there. But the technology just wasn't available the way it is now. If we were doing the same gig today, you can bet your sweet bippie that we would be using modern machine learning techniques.
United Nations Monitoring, Inspection and Verification Commission (UNMOVIC)
Do you remember United Nations Resolution 1441?
I'll give you a hint: yellow cake uranium.
If you were sentient in 2002, you remember the build up to the coalition invasion of Iraq following the attacks on September 11, 2001.
UN1441 was the "hunt for weapons of mass destruction" in Iraq. Spoiler alert: there were none. I happened to work at the U.N. on that one.
Our job was to leverage hand-built chemical, biological and missile taxonomies to identify areas in Iraqi military and manufacturing documents that contained references to chemicals, germs and missiles.
It probably would have worked eventually - we were making good progress. However, shortly after I arrived, Colin Powell got in front of the Security Council and gave the pre-fab excuse to go to war. The point of our exercise was exhausted at that moment.
Machine Learning could most certainly have gotten the job done quicker, and prevented the invasion and saved the world trillions of dollars, tens of thousands of lives and led to peace in our time. So, sorry, I guess.
Brave New World
It seems that today, you can't swing a Schrodinger's cat without finding a new application for machine learning. The algorithms, hardware and infrastructure are getting better every day. It's coming sooner than people think.