Engineering the AI applications (Part 8): Connecting IoT to LLMs (Part A).

In this short post, I would like to share a few lessons that we have learnt while developing the AI applications.

A brief history and context:

At Vidcentum Technologies , we work on data and IoT integrations. We implemented a summarization feature on IoT data in our Gateway software stack. During 2021 and 2022 there was no ChatGPT and we had to find a simple way out for summarizing IoT data.

Simple way?

Not sure. But I would like to present our approach here.

1. Data collection: IoT data (sensor) data was collected using standard protocols such as Modbus/TCP, OPC-UA, and others.
2. Dataset: Derive a dataset from a good sample of the sensor data. We found this step is very crucial. Basically, preparing the data set took quite some time.
3. For each dataset entity, we had some sort of summary. For example, <X11, X12, X13, X14> is an instance of reading of 4 sensors. We added a simple summary of what it means in the real-world. Say "The system is performing within rated limits."
4. So, <X11, X12, X13, X14> instance has <S11>, a summary string.
5. In the Production: On a new (unseen) instance of the sensor data, performed K-NN. This results some 4 or 5 nearest samples from the dataset and corresponding summary strings <S>.
6. Ranking: (Re)Rank the result strings from the K-NN in the above step if required.
7. Output: Since there was no LLM (or ChatGPT), we could simple print the <S> and later we attempted a summarizing the <S>. But, we stick to providing the <S> to operator (human) inference.
8. Batch processing: The challenge of generating a summary of given "batch" of sensor data. Say operator select a window of time and ask the summary or inference on the selected data. It is a different topic on its own. I will create another post exclusively for this case; using Clusters.
9. Augmented HMI: As LLMs and RAG gaining momentum, we believe that the HMI with LLMs will find several use cases. Long way to go, but surely there is clear use case to augment HMI with LLMs.
10. Application of LLMs: In the next article (Part-B), we will discuss how we are integrating LLMs to IoT data.

Here is a screenshot of the IoT Gateway application.

Affiliations

I write posts for Vidcentum Technologies where I am founder Director. Some of the articles are published in my blogs / articles / posts as well as on the company timeline / posts.

If you are interested to know more about it, drop a message either to [email protected] or to Maruthi Pathapati or in the comments.