AMI Disaggregation using ChatGPT

TL;DR

In this article, I detail my experience using #GPT-4, an AI model, to code a Python program that disaggregates Advanced Metering Infrastructure (AMI) data to detect electric vehicle (EV) charging. My objective wasn't solely to generate functional code; I also aimed to compare its performance against other hand-written models and a source of truth (SOT). I consciously chose not to feed GPT any pre-existing code or data, instead opting to ask questions and provide prompts based on the knowledge I'd gained from studying #ChatGPT and interacting with it. The AI model suggested various methods for the disaggregation process, anticipated potential problems, and ultimately produced a Python program. I was pleased with the resulting code, viewing it as a solid starting point that might need some refinement and additional testing against existing models and the SOT. Moving forward, my plans involve making necessary modifications, supplying the program with AMI data, and comparing its output with detected events from existing models and vehicle telematics data.

Overview

I'm going to highlight my steps coaching ChatGPT to write a Python Program to perform the process of disaggregation of AMI (Advanced Metering Infrastructure) "Smart Meter" data in order to detect EV Charging.?I've been studying ChatGPT by practicing interactions, watching videos, reading articles and taking short courses.?The goal is to not only produce the necessary code, but to validate the performance against other hand-written models and a source of truth (SOT).?I'll divulge some of my prompts but I'm being careful to not give away too much, being overly cautious to not disclose proprietary techniques.?I will not give ChatGPT any code it has not first given me.?I'll not provide it any data.?I'm using the OpenAI GPT-4

Background

Numerous businesses are interested in analyzing residential AMI data to ascertain the specific uses of the power drawn from the grid. This could include HVAC systems, pool pumps, dryers, hot water heaters, and more. A particularly popular focus is the detection of Electric Vehicle (EV) charging.

Electric utilities take a keen interest in EV Disaggregation as it allows them to report on the effectiveness of incentive programs to regulators. For instance, customers might be incentivized to charge their vehicles outside peak power demand periods, or discouraged from charging during Demand Response (DR) events. DR events occur when the power company anticipates that the demand will exceed the energy generation.

There are other ways to get this information such as vehicle telematics (getting charging events typically through an API provided by the vehicle OEM), external meters that are inserted between the customers electrical panel and EV Charger or through high-resolution power monitoring devices installed at the customer premise.??These all have costs in terms of equipment, support or requires a high degree of customer trust.?For some utility companies, there is a high penetration of these AMI meters, which can be expensive to deploy.?Regulators might insist on disaggregating data from these meters instead of incurring additional costs.

Level Set

Part of my tact here is to not "educate" GPT anymore than necessary.?The process of disaggregation is well known and the techniques are widely published.?So, I first sought to see what GPT already knows about the problem.??I will say I asked if it knew what Disaggregation is in the context of residential AMI Smart meters.?It clearly knows about the process and the data science techniques used to perform the operation.?One typical challenge for AMI meters is the sampling frequency.?It's typical to only have samples every 15-minutes.?This is primarily because of the volume of data involved in a jurisdiction and limitations of the communications network and data storage.?

After a little back-and-forth, I asked if we were limited to 15-minute data and were disaggregating for the purpose of detecting EV charging events lasting 30 minutes or longer using a Level 2 (L2) charger with a power rating of 5kw or higher could it narrow down the techniques that should be considered.?In summary, it recommended Heuristic-based methods and/or feature extraction and machine learning techniques.?GPT expounded on the guidelines for these approaches and suggested a hybrid of these two methods.?Now we're getting somewhere!

Finally, I asked: "If I provided you details on the AMI input data (data specification, data location, etc.) and the desired output specification and storage (relational database), would you be able to produce a Python program suitable for running in AWS Lambda to process the AMI data and produce the charge session data?".?It basically said yes (in a slightly wordy response) and proceeded to spit out the code outline in Python.

Skeleton

The code it kicked out is what I would have expected given my lack of detail up to this point.?It contained the following four functions:

1. ?Read_ami_data (load from CSV and return a pandas DataFrame)
2. Identify_ev_charging_sessions (feature extraction, i.e. identifying the charging events)
3. Store_charging_sessions (store the charge sessions in a relational database)
4. Lambda_handler (main routine that calls the other functions)

It politely warned me of Lambda runtime limitations and suggested I consider running in an EC2 instance or using AWS Batch.?Well, I can say it accurately predicted some headaches I've experienced.

I pointed out the obvious that the identify_ev_charging_sessions was the hard part and asked if I gave the complete specifications for the input data, could it write that code.?My prompt was: "The hard part here is obviously the identify_ev_charging_sessions function.?If you had complete specifications for the AMI DataFrame, would you be able to write that code?".??This is my first surprise.?It proceeded to ASSUME what the input columns were (complete with data type and example) and wrote the code!?

It looked too simple, and surely is.?GPT added that I should take care to take missing samples into account.?It recognized additional optimizations or refinements would be required.

Technique

The next step was to get more specific about the methods to be used.?I have a high-level understanding of how one would go about extracting the "feature" of EV Charging.?Some of this requires an understanding of what typical residential loads look like.?I asked GPT to give me a list of typical residential loads including their typical power (kw) and power factor.?(You can ask GPT what power factor (PF) is, but in summary it's a value which represents the level of inductive vs capacitive load and indicates how much power is actually put to work.?A PF of 1.0 means it's a highly efficient load and all the power is being delivered. )?The PF values in an industrial setting can get a lot lower due to the amount of motors being used.?But, what I was expecting and found was that EV charging has a very high PF (0.95-1.0). We can make use of that.?Ask GPT "Please provide a list of typical residential loads such as HVAC, EV Charging, Dryers, Ranges, Pool Pumps, etc. with their respective power factors and loads in kW."?

I drilled down a little into the PF angle based on an IEEE article called "Power factor signature analysis for disaggregation of EV charging loads from aggregated power".?I'm too cheap to buy the article but the abstract gives me some ideas.?I asked, in part " Is there a way to differentiate EV charging using these individual components that would make it stand out against other typical residential loads?".?It gave me the following ideas to explore; Active (True) Power Consumption Patterns, Time of Day Analysis, Consistency in Power Consumption, Reactive Power Patterns, Data Clustering or Classification Techniques and Non-Intrusive Load Monitoring (NILM) Techniques with a complete description of each.?

Rubber Hitting Road

After some consideration, I decided to start with a "fresh" prompt, i.e. new session and use the information I had gathered thus far from GPT and explicitly provide context based on that.?The main reason is this approach leaves no doubt as to what knowledge GPT has retained from the discussion about disaggregation.?I'm not sure if this is the correct approach for every project, but this way, I can refine the resulting prompt - ever increasing in size until I get it where I want it.?I'm sure this couldn't be used for a very large project.

The thing I've heard complaints about is GPT making stuff up, i.e. hallucinating.?Some of the examples I've seen in the coding context are likely the result of insufficient specification.?I've already seen if you ask GPT to code something up and don't give specifications, it will make up something.?It would be cool if it instead was more interactive and would ask questions - like a junior programmer would.??I might try experimenting with that as well.?I have ideas.

The Prompt

I won't get into some of the details of what I told it with regards to how to go about the charge session detection but I can say most of what I spelled out, it already knew was best practice based on my previous conversations.??This is, in summary, what I included in the 45-line long prompt:

1. ?I told GPT it was an "expert Python programmer and Data Scientist".?I reminded GPT a good programmer documents their code and opts for a clear and transparent coding approach vs code that might be difficult to understand.
2. I described the data as being 15-minute AMI data and stipulated to only consider Level 2?chargers with 5kW or greater charging between 30 minutes and 15 hours.
3. I specified that the code would run in AWS and should therefore use the boto3 library for interaction with input and output data.
4. I defined the input data source complete with schema, table and field names with data types and a description of each field and said the source data was in S3 and to use Athena to access it.
5. I specified that the meter id would be specified as an input parameter.?My thinking was this would only operate on one meter at a time and an external process would invoke the code as a Lambda.
6. I specified that the output data was to be delivered to Aurora (PostgreSQL) and detailed the database (schema), table name, field names with data types and a description of each field.?I told GPT the credentials for Aurora would be stored in Secrets Manager.
7. I had a few bullet points I categorized as "Approach".?In it I give general guidance on how to go about this, what to lookout for and assume.
8. I stipulated that if there was anything I left out by way of parameters or requirements GPT should substitute a "reasonable temporary" value and comment as such in the code.
9. Finally, I asked it to maintain any of the afore mentioned parameters in one place in the code so it could be easily maintained.

I pasted the prompt and sipped my coffee for about 5 minutes while it cranked out the code!?It did stop about half way through and showed a prompt for "Regenerate or Resume".?So, I guess it timed out.?I hit resume and it picked up right where it had stopped.?I had to do a little copy-paste to get everything in order, but that was it.

After the code, it repeated some of my specifications as confirmation, reminded me to replace a few placeholders such as endpoint specifications, credentials, etc.?It also recommended I add some logging and error handling.?It suggested I move some of the hardcoded values (which it placed at the top of the code, fully documented as I requested) to a configuration file or environment variables.

What Do I Think?

I was pleasantly surprised but not shocked based on my previous experience.?Was this the best approach vs describing the work in pieces (input, detection, store results) and bringing it together myself??I'm not sure.?I do believe I could now take these components, send it back to GPT and ask it to embellish it through a process of continuous improvement.

My first reaction to the code is that the charge event detection looks simplistic.?This is really just me thinking the process should be more difficult but I have to admit the dozen or so lines of code that achieves this looks pretty complex.?This despite my request to write "simple" code.?Perhaps this is irreducible complexity??We'll see.?The input and output looks pretty spot on and there is work to be done before it could be run.?But, not a whole lot.?Part of this was to see if I, being a novice Python programmer, would have enough material to make it work.?Will the output be a flop??Or, will it be a flop but at least as good as the existing model?

Next Steps

Now I need to make the necessary changes and supply it with some AMI data.?Then, I can compare the output to the detected events from the existing data science model and also compare with a source of truth (vehicle telematics data). That will require a little finesse and politics!?

More to come!?