Building a risk model for AI in Construction

As an IT director in the construction industry, my inbox (both email and voicemail) is full of messages about new software tools that “leverage the power of AI for construction.” This deluge is not surprising. In January 2020, Forbes magazine stated that “2018 AI investment hit an all-time high with over $9.8 Billion raised by AI companies” (https://www.forbes.com/sites/cognitiveworld/2020/01/05/is-venture-capital-investment-for-ai-companies-getting-out-of-control/#5516eba47e05).

These software vendors often use the term AI or machine learning without genuinely understanding the limitations (or, at least, not sharing those limitations with potential customers). They are relying on information asymmetry – that we in the construction industry do not know as much about machine learning as the salesperson.

Given the work that construction companies perform, we cannot accept machine learning tools without doing a risk assessment on our projects.

What are the concerns for using machine learning in construction? What do you need to know? I have gathered four significant concerns about machine learning as they relate to the construction industry. Several of these draw on the work of Janelle Shane, whose book You Look Like a Thing and I Love You is an excellent resource for understanding machine learning tools.

1. A machine learning tool cannot tell you why it made the decision it did. If you or a member of your team analyzed a data set and came to a conclusion, you could ask why they chose one option or another. A machine learning tool can give you the same output but cannot reconstruct its reasoning. It is a black box.
2. Machine learning tools can focus on the wrong outcome. In her book, Shane relates several examples of tools making choices that do not make sense to us. Sometimes, machine learning models choose not to make a choice at all. Shane cites a model that should optimize winnings in horse racing. The model decided not to bet at all.
3. The developers of a machine learning tool have a great deal of influence on how it behaves. The most important way that they affect the tool is by choosing how to “train” the tool. Machine learning tools learn by processing training data, usually a subset of what they are supposed to analyze. Selecting what data to train on is critical, as it multiplies the effect of the data. There is a famous machine learning failure where Amazon tried to use a tool to sort resumes and found that it kept selecting candidates with the same background as the developers.
4. The training set selection also has another negative effect on machine learning tools. These tools can only interpret and analyze what they know. There is a software that would display your face as a renaissance painting. It had one limitation – it could not correctly show your picture if you were smiling. The portraits that the developers used to train the software never included a smiling person.

Given these significant concerns, what should a construction executive or project manager do to determine if a machine learning tool is a good fit for a construction sector or project? I propose two specific actions to evaluate machine learning tools:

1. Unpack the tool’s assumptions. Ask the provider what training data they use. Determine who the intended user of the tool (does it match the area of construction you work in?). Understand what the tool is asking the machine learning to solve for.
2. Gauge the risk. A significant component of construction management is managing risks for a project – schedule slips, change orders, among many other factors. The use of a machine learning tool adds to the risk of the project.

I have not seen any models for assessing the risk of using a machine learning tool in a construction project. Given the lack of existing structures, I propose a model based on two variables for assessing risk:

1.  The time horizon of the analysis – retrospective, real-time, and prospective.
2. The independence of the outcome – human-assisted or human-less

 I would propose that risk rises as the time horizon nears real-time, and as the outcome becomes less human-assisted. The lack of opportunity for human intervention in the analysis (or action) is the greatest driver of risk for machine learning tools. Here is a potential model for the added risk.

 Retrospective machine learning tools

 Human assisted

Example – Financial audit data selection for further review

Risk level – Low

 Human-less

Example – Tagging of photos for safety/Personal Protective Equipment violations

Risk level – Low to Moderate

Real-time machine learning tools

Human assisted

Example – Classification of invoices and matching funds to job/phase codes

Risk level –Moderate

Human-less

Example – Autonomous construction vehicles

Risk level - High

Prospective machine learning tools

Human assisted

Example – Predictions of potential safety hazards

Risk level – Low to Moderate

Human-less

Example – Financial predictions of profitability of Requests For Proposal/Requests For Quote

Risk level – Low

In each of the cases, risk rises as human judgment is removed from the process. In addition, increasing the time horizon, both forward and backward, adds opportunities for experienced humans to intervene.

Managing a construction project requires balancing risk with potential rewards. The construction industry needs to balance the risk of adopting a machine learning tool with the benefits it can provide. As these tools become more sophisticated, their risk profile will likely shift.

Given the risks associated with adopting a machine learning tool in construction, though, I would recommend that construction firms adopt tools that enhance human decision-making – like the prediction of potential future safety hazards (those based on historic data sets). Low-risk, human-less tools – like photo tagging tools – are also a good place to start with machine learning. There is significant potential for machine learning applications to improve processes and financial outcomes in the construction industry. We just need to get past the hype.