Clash Detective Reports | BIM

Hello everyone!

I hope you had a wonderful New Year's celebration and are energized for new achievements this year! Before getting down to business, I want to wish everyone a happy new year – may it bring success in your work and joy in every day.

Usually, I don't publish original articles on my news channel, but I feel that occasionally, it might be refreshing to intersperse BIM world news with opinion columns. If you like this idea, please let me know. Previously, I used to publish articles here. I will continue to post my articles on Medium as before.

Now, onto something interesting. I'd like to share a project I've been working on over the past month. It's been kind of a hobby alongside my main job, but the outcome is definitely noteworthy.

If you recall, I previously showed you how to use CSS and JS to enhance the readability of reports from Autodesk Revit. In this article, it's time to turn our attention to Navisworks reports.

Report Generation

If you're working with Navisworks, you've likely encountered its collision reports (clash detective) – not the most user-friendly reports out there. They're understandable, but definitely not user-friendly!

You know what always bothered me? The a need to maintain a separate table for collision statistics. And where you might ask? In Excel, of course! It doesn't take a genius to see how inconvenient this is.

As you can see from this simple diagram, it's at the stage of entering data into Excel that the "human factor" comes into play, which can later backfire on us. It seems I forgot to show you the report itself and how it looks. Let me rectify that; here are the reports I attach to emails after each model check:

As evident from the table above, filling in each cell required manual work, which was extremely time-consuming. At the end of the year, having found some free time, I decided to make my work more efficient and automate this process.

Preparing Collision Data

As soon as I receive reports from Navisworks in HTML format, the question arises: how can I make them more convenient for further use without having to write a lot of code? After all, it's important that the reports remain accessible to my contractors without extra hassle. So, the task was simple: use HTML reports as a data source.

Let's look at an example of the structure of folders and their contents:

I must mention, I deliberately chose not to use any plugins or add-ons for Navisworks. The reason is simple: the solution should be as straightforward and scalable as possible.

I settled on Power BI. It's essentially the same as Excel, but more adapted for graphical reports.

One of the convenient features of Power BI is its similarity to Excel. If you, like me, have previously used Excel, you will find Power BI easy to learn. Moreover, Power BI is compatible with many extensions, making it a versatile tool. I even discussed how Speckle can be used in combination with BIM data in one of my articles. If you're interested in this topic, you can check it out here and here.

By the way, for clarity, I found a comparative table that might be useful for you in the future.

Since all my HTML reports are stored in one folder, I can easily pull them from there without worrying about updating paths. Another significant advantage is that I save the reports using their creation dates in the names. This allows us to use these names as the creation dates for checks. For example:

Here's what I ended up with. This is the table I assembled in Power BI

Here's how the final table looks. Among the advantages of this solution, I want to highlight that the table automatically interacts with the chart and has a color-coded cell system. Of course, similar things can be done using Excel, but here there is a more convenient interaction with the input data. Everything is simply organized in folders with the specified date, as I mentioned earlier. And here you have a ready table that turns the report in Navisworks into a convenient statistical representation of the progress of checks, with an intuitive interface. This approach makes the information accessible and interesting to understand.

As seen in the animation above, I can individually view each check and its statistics. This is convenient, as having a database allows you to easily create charts of your choice. The tools for creating charts are quite simple to use.

As a result, by spending some time on Power BI, I've developed a functional and effective tool for visualizing data and statistics related to model collisions. I plan to further explore and refine this tool to gain a deeper and more nuanced understanding of its capabilities.

Forecasting

I decided to enhance this article with some code, specifically a bit of specialized code. For example, many of you have encountered a situation where the Project Manager, looking at all the collision reports, asks a perfectly reasonable question:

"When, in your opinion, can we resolve all these errors?"

The question is indeed complex as it relies solely on previous experience and is heavily dependent on personal expertise and numerous unpredictable factors. However, statistics is a ruthless thing; it's based on our experience. This means that having data allows us to attempt to make predictions based on it. I started by deeply exploring what types of forecasting exist based on available data. Here are the main methods I selected at the first stage:

Jumping ahead, I'll start with method number 2, then try method 4 and also touch on method 3. I also compiled a simple diagram for myself, showing which methods I chose at each stage and their complexity in use.

Linear Regression

And then it's just a small step further. Since all the table data is stored in JSON format, I simply use Python and the scikit-learn library. This library already includes an implementation of the Linear Regression function.

# Loading data from the file DATA.JSON  
with open('DATA.JSON', 'r') as file:  
    data = json.load(file)  
  
# Creating a DataFrame  
df = pd.DataFrame(data)  
  
# Converting date strings to datetime  
df['day/month/Year'] = pd.to_datetime(df['day/month/Year'], format='%d/%m/%Y')  
  
# Calculating the number of days since the start of the period  
start_date = df['day/month/Year'].min()  
df['Days'] = (df['day/month/Year'] - start_date).dt.days  
  
# Training a linear regression model based on available data  
X = df[['Days']]  
y = df['Collisions']  
model = LinearRegression().fit(X, y)  
  
# Calculating the date when the predicted number of collisions will reach zero  
zero_collision_day = int(np.ceil(-model.intercept_ / model.coef_[0]))  
zero_collision_date = start_date + pd.Timedelta(days=zero_collision_day)  
  
# Predicting values for the next three weeks (every seventh day)  
future_days = np.array([7, 14, 21]) + df['Days'].max()  
future_days_df = pd.DataFrame(future_days, columns=['Days'])  
predictions = model.predict(future_days_df)  
  
# Adding predictions to DataFrame  
future_dates = [start_date + pd.Timedelta(days=int(day)) for day in future_days]  
predictions_df = pd.DataFrame({'Date': future_dates, 'Predicted_Collisions': predictions.round().astype(int)})  
  
# Filtering out rows with zero or negative forecast  
predictions_df = predictions_df[predictions_df['Predicted_Collisions'] > 0]  
  
# Displaying the results  
print(predictions_df)  
print("Date when the number of collisions will reach zero:", zero_collision_date.strftime('%d/%m/%Y'))        

My last check was on December 19, 2023. According to the forecast using the Linear Regression method, the nearest collision resolution date is December 27, 2023. Very promising!

ARIMA Method (Autoregressive Integrated Moving Average)

However, during my experiments, I realized that the linear approach is not suitable for forecasting. So, I decided to try the ARIMA method:

import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
import json
import numpy as np

# Loading data from DATA.JSON file
with open('DATA.JSON', 'r') as file:
    data = json.load(file)

# Creating a DataFrame and setting the date index
df = pd.DataFrame(data)
df['day/month/Year'] = pd.to_datetime(df['day/month/Year'], format='%d/%m/%Y')
df = df.set_index('day/month/Year')

# Training the ARIMA model
model = ARIMA(df['Collisions'], order=(1, 1, 1))
model_fit = model.fit()

# Forecasting for a longer period for better interpolation
long_forecast = model_fit.forecast(steps=60)  # For example, for 60 days

# Limiting the forecast output to 3 weeks (21 days)
forecast = long_forecast[:21]
forecast_dates = pd.date_range(start=df.index[-1], periods=4, freq='7D')[1:]
forecast_values = forecast[::7]  # Selecting every seventh value from the forecast
forecast_df = pd.DataFrame({'Date': forecast_dates, 'Predicted_Collisions': forecast_values.round().astype(int)})
print(forecast_df)

# Calculating the date when the number of collisions will reach zero
zero_collision_day = np.interp(0, long_forecast[::-1], np.arange(60, 0, -1))
zero_collision_date = df.index[-1] + pd.Timedelta(days=int(zero_collision_day))
print("Estimated date when the number of collisions will reach zero:", zero_collision_date.strftime('%d/%m/%Y'))        

And as a result, this method predicts February 17, 2024. The difference turns out to be almost 1.5 months. This raises the main question: Which date should I trust?

And indeed, the answer lies in the method numbered 6 - "Qualification Methods". This approach is based on the specialist's experience. In this case, considering my own experience and taking into account the two dates obtained by different methods, the second date seems more realistic.

Below, I've compiled a table for you that reflects the main differences that led to such a disparity in the dates.

Conclusion

Despite the heavy workload at the end of the year, I found time to try Power BI – a tool I've been thinking about for a long time. Now, I have a raw but extremely useful tool for displaying the current status of the project. As a bonus, I can now approximately answer the eternal question: 'When will the model be ready?' Perhaps in the future, I'll have the opportunity to delve into machine learning and train a model based on existing data to predict outcomes more accurately.

I hope you found this article interesting. I would love to hear your opinion or suggestions. It would be interesting to see or discuss methods of project management control.