Vision AI in Energy Use Cases: ChatGPT & Copilot for Hydro Electric Asset Monitoring & Insights

(h/t in collaboration with Srinivasan Iyengar Alex Grubb Sridhar Srinivasan )

Disclaimer: The views expressed here and other social media are personal and meant for information/education purposes only and not representative of his employer directly or indirectly and not investment advice. Any mention of company names are for illustrative examples only.

[Note: Opinions of author only, not to be construed as that of employer. Examples are illustrative logs of sessions via ChatGPT Plus or Copilot. LLM outputs are probabilistic: different attempts of same query/prompt can lead to different outputs].

This is part of a series of LinkedIn articles/posts on ChatGPT / Copilot used for Vision AI in the Energy Industry: Copilot for Solar PV, Copilot for Cloud Typing & Local/Short term Weather Forecasting with Vision, Copilot for Wind Energy, Copilot for Geospatial Analysis (Minerology), Copilot for Well Log Analysis (Litho-Facies Classification), Copilot for Multi-Well Log + Seismic Tied interpretation, ... and more coming soon.

A silent revolution is underway. Computer Vision (images, videos, diagrams) used to be quite challenging for AI and ML to process (requiring a lot of data collection, training brittle ML models). Now with zero-shot or few shot, or with a small amount of fine tuning we can string together a number of very interesting capabilities and use cases (as Copilots).

This blog is a simplified overview of how some of these use cases are helping the Energy Transition and Energy Industry, based upon our innovation and experiences at Microsoft in the Energy Industry in Asia and beyond. Enjoy!

For simplicity, I will show these examples as simple zero-shot queries in ChatGPT. You could further make these as highly specialized Engineering, Procurement, Construction (EPC) Copilots, Operations & Maintenance (O&M) Copilots by using multimodal retrieval augmented generation (RAG), link these to timeseries AI from IoT, make these part of AI agents (with AI orchestrating AI), apply proof of thought (PoT) fact / reasoning checkers, or hook these up with digital twins, AI avatars speaking multiple languages, mixed reality or industrial metaverse experiences, field operations copilots, or with deep knowledge management systems. As you probably guessed, our teams are innovating in multimodal Generative AI (GenAI) involving all these aspects and more; and the goal of this article is to present a simplified starting point into the possibilities.

Corrosion in Penstock of Hydro-electric Power System

Corrosion is a huge challenge in hydro-electric power, especially in the penstock (which is a pipe). A penstock in hydropower is a large pipeline or conduit that carries water from a reservoir or river to the turbines in a hydroelectric power station. It operates under high pressure and is a crucial part of the hydropower system, as it directs the water flow and controls the rate at which water reaches the turbines, impacting the generation of electricity.

Impact of Corrosion in a Penstock

Corrosion in penstocks can lead to significant operational, economic, and safety challenges. Here are the primary impacts:

1. Reduced Efficiency and Energy Output: Corrosion creates rough surfaces and pitting inside the penstock, increasing frictional resistance. This reduces the flow rate and efficiency of water delivery to the turbines, leading to a lower energy output.
2. Increased Maintenance and Repair Costs: Corroded penstocks require frequent inspections, repairs, and sometimes application of protective coatings to prevent leakage and further deterioration. These maintenance activities are costly and labor-intensive, especially if the penstock is located in difficult-to-access areas.
3. Shortened Penstock Lifespan: Corrosion accelerates material degradation, reducing the penstock’s operational life. This leads to premature replacements, adding substantial capital costs over time, especially in large hydropower facilities where penstocks are costly to manufacture and install.
4. Risk of Structural Failure: Severe corrosion can weaken the penstock structure, increasing the risk of leaks or even catastrophic failure. A penstock failure can lead to uncontrolled water release, damaging downstream infrastructure and posing significant safety hazards to personnel and nearby communities.
5. Environmental Impact: A failure or leak in a corroded penstock can lead to water escaping in an uncontrolled manner, potentially eroding nearby land, harming local ecosystems, and leading to costly environmental remediation.

The Copilot session shows the impact of corrosion in different penstock pictures being detected and classified, including root cause analysis, implications and recommended actions.

Below is another session with M365 copilot on a different set of images. Note that Copilot tends to be a bit brief; and the way to handle that is via successive prompts asking for deeper and more quantitative insights. With M365 Copilot, these pictures could be part of multiple documents in sharepoint.

Coming back to ChatGPT Plus, you can also ask for a detailed quantitative engineering analysis based upon observations in the image.

In the session below, there is a quantitative engineering analysis including diameter / flow rate estimation, hydraulic efficiency estimates, and potential leak rate estimation. If you dont get exactly what you want, you can adapt the prompt to be more direct to include specific parameters or KPIs to estimate.

What is remarkable is ChatGPT's ability (also similar in Copilot since both use GPT 4o underlying models) to pick up subtle patterns as shown below, and then subsequently reason about root cause etc.

There is a summary of crack pattern, surface condition, crack depth estimation and thermal stress indications including both observations, implication and recommended actions.

Interpretation of Non-Destructive Testing / Evaluation (NDT, NDE) Data

In addition to analysis of simple RGB images, the process of non-destructive testing / evaluation (NDT or NDE) throws up complex images typically interpreted by experts. Now Copilot / ChatGPT can aid a non-expert to interpret such images / analysis, and also serve as a "buddy" for an expert to go deeper into understanding specific patterns, and not missing any artifacts.

The Copilot session below shows the interpretation of C-scan, B-scan images of a penstock (pipeline) to understand areas of pitting corrosion, localized inclusion/voids, material degradation, wall thinning, weld cracks, internal defects etc; and root causes, remediation etc.

Laser scans and laser profilometry done with a Copilot can yield very good insights as well. These not only identify and localize the corrosion hotspots, elevated patches, localized pits, but also can lead to hypothesis of the reasons (eg: non-uniform flow-induced erosion, areas of stagnation etc).

What is brilliant is the potential for quantitative assessment of material loss by studying the laser profilometry image as an expert would; and producing a draft structural integrity report for validation by a human expert.

Another example is to look at subtle crevice corrosion, in a riveted lap-joint using SR-GWT (short range guided wave testing). Here again it can understand the pattern via schematics, and perform quantitative analysis / estimation on the actual measurement/visualization. Asking for "expert interpretation" and pushing it for deeper analysis on RCA (root cause analysis), implications on risks and lifetime also yields interesting results.

Another class of NDE / NDT evaluation is to look at welding and seam quality and degradation. This is important for fabrication documentation, which in turn can feed into asset valuation analyses.

ChatGPT / Copilot can summarize specific weld defects, seam irregularities, wall thickness, material issues, and further, assess risk and propose remediations.

Assessing Structural Integrity of Hydro-electric Dam (Concrete Structures)

Understanding the structural integrity of hydro dams has significant economic value for several reasons:

1. Preventing Catastrophic Failures: Ensuring the structural integrity of dams helps prevent catastrophic failures, which can lead to massive economic losses, including damage to infrastructure, loss of life, and environmental degradation
2. Optimizing Maintenance Costs: Regular monitoring and assessment of dam integrity allow for timely maintenance and repairs, which can be more cost-effective than emergency responses to unexpected failures
3. Sustaining Energy Production: Hydro dams are crucial for renewable energy production. Maintaining their structural integrity ensures consistent energy generation, which is vital for economic stability and growth
4. Water Resource Management: Dams play a key role in water supply for agriculture, industry, and domestic use. Ensuring their integrity helps in effective water management, which is essential for economic activities
5. Insurance and Liability: Understanding and maintaining dam integrity can reduce insurance premiums and liability costs for dam operators and owners
6. Supporting Sustainable Development: Properly maintained dams contribute to sustainable development goals by providing reliable water and energy resources, which are fundamental for economic development

Investing in the structural integrity of hydro dams is not just about safety; it's a strategic economic decision that supports long-term sustainability and resilience.

ChatGPT can look at individual images of concrete structures and give an expert engineering analysis.

The analysis below looks at vertical cracks through concrete wall; water seepage, staining on masonry surface. You can imagine this kind of analysis helpful in other concrete / construction contexts as well.

Health of Hydro-Reservoirs and Sedimentation Understanding

Understanding the health and sedimentation of hydro reservoirs holds significant economic value for several reasons:

1. Prolonging Reservoir Life: Effective sediment management can extend the operational life of reservoirs, ensuring they continue to provide essential services like water supply, irrigation, and hydropower generation
2. Maintaining Storage Capacity: Sedimentation reduces the storage capacity of reservoirs, which can impact water availability for various uses. By managing sedimentation, we can maintain optimal storage levels, supporting agricultural productivity and industrial activities
3. Reducing Maintenance Costs: Regular monitoring and management of sedimentation can prevent damage to hydromechanical equipment, reducing maintenance and repair costs
4. Enhancing Energy Production: Sediment-free reservoirs operate more efficiently, ensuring consistent and reliable hydropower generation, which is crucial for economic stability and growth
5. Supporting Sustainable Development: Proper sediment management contributes to sustainable water resource management, which is vital for long-term economic development and resilience against climate change
6. Improving Water Quality: Managing sedimentation helps maintain water quality, which is essential for drinking water supplies and ecosystem health, further supporting economic activities dependent on clean water

Investing in the health and sedimentation management of hydro reservoirs is a strategic economic decision that supports sustainable development and resilience.

Specifically here are some quantitative effects:

1. Storage Capacity Loss: Sedimentation can reduce a reservoir's storage capacity by 0.5% to 1% annually
2. Economic Impact of Sediment Management: Implementing sediment management strategies can extend the life of a reservoir by 30 to 50 years
3. Hydropower Generation: Sedimentation can reduce the efficiency of hydropower plants by 10% to 20%
4. Maintenance Costs: Regular sediment management can reduce maintenance costs by up to 50%
5. Water Supply: Effective sediment management ensures reliable water supply for agriculture, industry, and domestic use. For instance, maintaining reservoir capacity can support irrigation for thousands of hectares of farmland, potentially increasing agricultural productivity by 10% to 20%

These figures highlight the substantial economic benefits of investing in the health and sedimentation management of hydro reservoirs.

ChatGPT can estimate some of these factors by analysis of geo-spatial or aerial photography. It is interesting to note that it can extrapolate conditions in a different site by analyzing analogues as attempted in the session below.

Summary

GenAI and Copilots can be used to understand various aspects of hydro electric assets: penstock corrosion, thickness, cracks; concrete dam health and anomalies; analysis of reservoir and sedimentation using geospatial data.

This zero-shot analysis by Copilot (i.e. only inference, no training) can unlock the potential of visual information picked up by any source (eg: surveillance cameras, helmet mounted go-pro cams, drone-mounted thermal cams, specialized imaging, or satellite based imaging, sky camera images).

We also looked at outputs of non-destructive evaluation and testing (NDE, NDT) and using the Copilot / ChatGPT as an Interpreter (i.e. an Interpretation Copilot) to aid personas who could range from being experts, or business folks.

The deep image understanding closer and closer to real time (< 1 min) even for rich understanding, reasoning about the images and potential linkages to visual insight histories, time series AI insights, and documents / diagrams (used in multimodal RAG patterns) can be used for a variety of workflows.

All the images and analysis were done with Copilot / ChatGPT Plus and zero-shot analysis (without any grounding/retrieval augmented generation (RAG etc). With enterprise clouds such as Azure, and private data sources, we can do far more sophisticated workflows / solutions. You could further make these as highly specialized O&M Copilots by using multimodal retrieval augmented generation (RAG), link these to timeseries AI from IoT, make these part of AI agents (with AI orchestrating AI), apply proof of thought (PoT) fact / reasoning checkers, or hook these up with digital twins, AI avatars speaking multiple languages, real time voice APIs, mixed reality or industrial metaverse experiences, field operations copilots, or with deep knowledge management systems.

As you probably guessed, our teams are innovating in multimodal GenAI involving all these aspects and more; and the goal of this article is to present a simplified starting point into this journey. Please reach out if this is of interest. Hope this was a helpful starting point!

LinkedIn: Shivkumar Kalyanaraman

Disclaimer: The views expressed here and other social media are personal and meant for information/education purposes only and not representative of his employer directly or indirectly and not investment advice. Any mention of company names are for illustrative examples only.

[Note: Opinions of author only, not to be construed as that of employer. Examples are illustrative logs of sessions via ChatGPT Plus or Copilot. LLM outputs are probabilistic: different attempts of same query/prompt can lead to different outputs].

Twitter: @shivkuma_k

All LinkedIn Articles/Posts.

If you like this article, please check out these articles: E-mobility, Cleantech, Fintech & Digital Nexus in India, Towards Scalable EV Charging: Hidden Costs of Level 2 Charging at Scale, "Towards Scalable EV Charging Infrastructure (Think Outside the Gas Station Box)", "Commercial Electric Vehicles (EV) Fleets: The Stealth Growth", "Towards Affordable, Ubiquitous, Ultra-Fast EV Charging: Part 1: Need & Battery Issues", "EV Taxi Fleets & Ride Sharing: Poised for Huge Growth", "Shared EV Transportation in India", "Understanding the Rs. 3/kWh bids in India in 2017", "Distributed / Rooftop Solar in India: A Gentle Introduction: Part 1","Rooftop Solar in India: Part 2 {Shadowing, Soiling, Diesel Offset}", "Rooftop Solar in India: Part 3: Policy Tools... Net Metering etc..." "Solar Economics 101: Introduction to LCOE and Grid Parity" , "Solar will get cheaper than coal power much faster than you think..", "Understanding Recent Solar Tariffs in India", "How Electric Scooters,... can spur adoption of Distributed Solar in India," "Solar + Ola! = Sola! ... The Coming Energy-Transportation Nexus in India", "UDAY: Quietly Disentangling India's Power Distribution Sector", "Understanding Solar Finance in India: Part 1", "Back to the Future: The Coming Internet of Energy Networks...", "Tesla Model 3: More than Yet-Another-Car: Ushering in the Energy-Transportation Nexus", "Understanding Solar Finance in India: Part 2 (Project Finance)", "Ola! e-Rickshaws: the dawn of electric mobility in India", "Understanding Solar Finance in India: Part 3 (Solar Business Models)"