Human in the Loop

... or, Why Design can never be Automated (fully)

After reading @Kaibrunner's TechCrunch article: "The Automation of Design" I was compelled to respond.  

The pull quote in his article reads:

"UX design informed by business intelligence is not beyond AI and automation’s reach."

This is not just a theory Kai is toying with, there are active teams out there working to push this agenda forward with commercial products and public-sector initiatives.  But as it stands, there are only partial solutions.  I'm here to stand firmly on grounds that true design automation and AI driven design won't ever be fully realized.  But in order to explain why, I'll need to back up to 2010.

On April 26th 2010, The DHS issued an SBIR (Small Business Innovation and Research) grant to include research and development funding for an "Automated Tool for Assessing Usability".  

(a quick note: I realize the phrase "assessing usability" doesn't naturally equate to "design" for most...  but I'd like to posit that "usability" is a core principle of design.  A design that is beautiful, but serves no utility falls more into the category of fine art, and further away from function.)

In the 2010 SBIR solicitation, it was stated that: 

"Lessons learned compilations by the military and private industry indicate that more than 50% of identified problems in software development resulted from deficiencies in the design of the UCI. "(user-computer interfaces)

... it went on to explain that:

"...a large percentage (64%) of the life cycle costs associated with software systems are due to changes required to improve the UCI due to unmet/unforeseen user requirements after the system has been deployed."

If you're paying attention to Department of Defense technology funding... we're talking about potentially 64% of $26.4 billion.  We're talking about $13B in failed technical solutions due to usability problems.  For which the DHS SBIR allocated $800k to research and develop.  

According to the Standish Group, current software development practices result in only 30% of projects being completed successfully with an almost equal amount failing completely, and the balance being challenged by overruns and incomplete solutions. - (Standish Group, reported in Forbes, 1995)

Here are some more statistics:

• 63% of all software projects run over their budgetary estimates, with the top four reasons all related to unforeseen usability problems - (Lederer, A.L. and Prasad, J. (1992)/ Nine management guidelines for better cost estimating. Communications of the AMC 35(2) (February), 51-59)
• The percentage of software code that is devoted to the interface has been rising over the years, with an average of 47-60% of the code devoted to the interface. - (MacIntyre, F. Estep, K.W., and Seiburth, J.M. (1990). Cost of User-friendly programming. Journal of Forth Application and Research 6(2), 103-115)
• 80% of maintenance is due to unmet or unforeseen user requirements; only 20% is due to bugs or reliability problems. - (Pressman, R.S. (1992). Software Engineering: A Practitioner’s Approach. McGraw-Hill, New York)
• Design changes due to usability work at IBM resulted in an average reduction of 9.6 minutes per task, with projected internal savings at IBM of $6.8 Million in 1991 alone. (Karat, 1990)
• Usability engineering has demonstrated reductions in the product-development life cycle by over 33-50%. - (Bosert, J.L. (1991) Quality Functional Deployment: A Practitioner’s Approach. ASQC Quality Press, New York)

So we're painting a picture and telling a story that the problem is real and it's expensive - hence why so many are taking up the challenge to usher in faster, automated solutions heading towards "Software on demand".  And it wouldn't be the first time...  companies like Xerox and IBM have been looking at this problem since the 80's.

Back to the original question: 

Can design be Automated?  The problem with the premise of the article is "should design be automated?"

In the vast field of design, the human-in-the-loop is a necessary factor.  The most glaring fact in the theory of automated design is that while challenges observed, strategies analyzed, and solutions implemented have certain technical and algorithmic angles, the design problems ultimately have cognitive-psychological and emotional factors that can't be computed. They require a degree of empathy and intuition to resolve.

AI and automation will get as far as design management and automated assist, but the ultimate design will require a human.  

Sorry Computer, you'll forever be the designer's sidekick. 

Phase II of the the 2010 DHS SBIR is coming to an end within this calendar year, and we all await the promise of  DesignInteractive.net's Usability Evaluation and Management System (USEMAN), my assumptions, which stem from the solicitation response my team put together, (based on considerable background research), is that there is no FULLY automated solution to be had.

The best Tony Stark could do with all his resources (albeit fiction - which makes it even more telling) is to get an automated assist from Jarvis.

Here's where MIT is on AI & Human in the Loop...  worth reading.  

Designers, IxDA's, HFE's, and Cognitive Psychologists...  tell me what you think!