The Variability of Microclimate Simulations in the AEC Industry and the Importance of Validation.

With the growth of DIY microclimate tools, it has become more important than ever to understand the quality of the results being produced. Here is what architects and building designers need to understand—and the questions to ask—before adopting a new AI or CFD microclimate tool.

As a microclimate specialist, I have been fortunate to work with some of the best architects, designers, master planners, and landscape architects in the industry today. I am always amazed at how they can juggle the many project requirements in a multidisciplinary environment. This push and pull during the design cycle results in a labor of love that reflects the project’s core principles.

When I or my colleagues at RWDI are hired for these projects, we appreciate the trust that our clients show in us. When we show them results from a series of microclimate simulations, they trust that we have done our due diligence, and are providing them with the appropriate level and quality of analysis. ?

With the growth of DIY microclimate tools, it has become more important than ever to understand the quality of the results being produced. Don’t get me wrong: I think DIY tools are critical to the health of the industry. After all, Orbital Stack was created by RWDI Ventures to bring the tools our engineers use on a day-to-day basis to the AEC market, allowing clients to explore designs directly and quickly.

Over the last two years, RWDI-Labs has undertaken a deep dive to understand the limitations of a variety of computational tools, including Reynolds-averaged Navier-Stokes equations (RANS), Large Eddy Simulation (LES), and Artificial Intelligence (AI) or Machine-learning (ML)?approaches. Part of this task included understanding what limitations various approaches have, and how we should apply them. Of course, full disclaimer, this research was motivated as much by our desire to validate the different tools as it was to ensure that our tool, Orbital Stack, was meeting expectations and was high performing. A keystone of the Orbital Stack tool was that all of the algorithms, processes and computational tools used had to be validated against wind tunnel datasets. Fortunately for us, we have access to decades of RWDI wind tunnel data, allowing us to compare computational results to this high-quality dataset.

A time and place for CFD

Computational fluid dynamics (CFD) has long been viewed in the wind engineering industry as a useful tool for early design analysis but not for accurate, late-stage design analysis. There are a variety of reasons for this:

1. High-fidelity CFD is computationally expensive. This translates into CFD historically either taking too long to execute or the need to purchase run time on expensive high-performance computing systems.
2. There is an excessive variety in CFD mathematical approaches. This might seem like an advantage but the vast number of CFD codes and approaches available reduces the reproducibility of results and creates a large collection of results for the same simulated geometry.
3. Process reproducibility is poor. The quality of CFD output is dependent on its user’s expertise and experience, more so than with wind tunnels. It is a rare occurrence that two individuals simulating the same case will arrive at the same answer. Rarer still is if the case is urban flow.?Documents such as Cost Action 732 do seek to reduce the variability in results, but there is still a wide latitude in approaches even when following this document.
4. Wind tunnel and CFD approaches are evolving at differing speeds. Wind tunnel technology is well established. There are a limited number of measurement techniques and approaches used in the commercial setting and their limitations are well understood and accepted. There is certainly the potential for a poorly designed wind tunnel but the differences between tunnels are well accepted. CFD algorithms and approaches are evolving at a rapid pace. Although this is a highly positive feature, it does mean that CFD is sometimes viewed, erroneously, as an immature analysis toolset.

Point 1 is fast becoming a non-issue. Even high-fidelity CFD approaches such as LES are becoming viable cost-wise due to access to lower-cost high-performance computing and improved algorithms.

Points 2 and 3 are related in that one must expend significant effort to create standard CFD processes that only change if there is a strong reason for that change.

The issues raised in point 4 are tackled by providing constant validation and testing of emerging toolsets. This is traditionally a weak point in the industry. New entrants often show limited evidence of the performance of their CFD approaches.

Case in point

I would like to highlight the dangers of overconfidence using a deceptively simple case. Below is a benchmark case commonly used at RWDI. It consists of two towers with a square cross-section and a 3:1 aspect ratio. There are no surrounding buildings. There are 232 Irwin sensors that measure horizontal wind speed at 1.5m above grade.

This simple geometry creates flow phenomena commonly seen in urban environments, including down-washing, corner accelerations, wakes, stagnant zones between buildings, and channeling. It is also fully exposed to approaching winds and is not sheltered by any surrounding building. This is more challenging from a flow setup perspective since accurately synthesizing/replicating the approaching boundary layer conditions is of paramount importance, particularly with LES.

The directional, mean wind speed results for flow coming from the right side of the images are shown in the image below. The background contours show the computational results for four approaches. The circles show the wind tunnel results. The principal challenge in this configuration is the deep wake created between the two buildings. The RANS model calculates vector mean and so, the out-of-the-box RANS approaches produce very low wind speeds between the buildings. In some locations, the speeds are only 10% or less of the wind tunnel results. The modified RANS engine produced by Labs and used by Orbital Stack corrects this somewhat. The modified RANS approach also tightens the wake, creating separation zones that are much more in line with the wind tunnel results. The LES approach does manage to recover the conditions between the buildings. The AI engine does remarkably well, especially since this was not one of the projects it was trained on.

The picture becomes even more complex when we combine the results with a climate file, a process that is typically used to determine wind comfort and safety. If one of the key wind directions happens to be aligned with the buildings as shown here, this would mean that the unmodified RANS would drastically under-predict comfort (and maybe even safety) conditions between the buildings. We will tackle this in the next post.

Questions to ask

Blindly trusting CFD or AI tools not produced by credible microclimate engineers is risky. Yes, the images may look cool, but it could be taking you down a design path where correcting poor choices become a costly endeavor. ?

So how do designers and architects reduce the risk? Here are a couple of ideas:

• If you are working with a new organization or tool, ask whether they have microclimate experts/engineers on staff and inquire about their experience working in the field.
• If they don’t have in-house expertise or experience, ask how they validated their tools. What kind, and how many, test cases did they use? If you are using them to calculate wind comfort, what were the statistical wind-comfort conditions when compared to wind tunnel results or real-world measurements?
• Are they trying to use rapid types of analysis in the latter stages of the design cycle? For example, it is a red flag if the project is in the detailed design stage and they are simulating a minimal number of wind directions with RANS models and providing quantitative wind comfort results.

Pretty pictures are great but understanding the required level of effort, relative to the project’s current point in the design cycle, is key.

For CFD and wind engineers working in this area, we can certainly up our game as it pertains to validating our tools. Following guidelines, such as COST 732 is important, but having a deep understanding of our toolset’s limitations and applicability is paramount. I would love to see a greater push to having validation cases out in the wild and specific applications, such as wind comfort, loading, and air quality being tackled. ???

Ultimately, with the proliferation of these tools and ease of deploying computational resources, it is incumbent on us, as the wind engineering community, to set the bar and be conscious of how much our clients depend on us to drive quality in the industry.