Evaluate your Data for AI!

Last week's edition shared a thought on "Data" and how this feeds into AI/ML or Gen AI. Data is the key and until you have your "Data" right, you won't achieve the accuracy of the models, yield the business outcome, and add considerate value to your business.

I was part of an AI readiness discussion this week where we discussed some of the critical aspects of getting Data ready, how we can get data right, what personas contribute to the data process developments, standardizations, why it is essential to get data standardized and how you can get started with evaluating your data readiness to adapt the AI journey.

Some of the key takeaways from Data implementations are -

1. Assess your existing platform before adapting the Gen AI
2. Define standards, processes, and quality measures that evaluate your data quality before you start deriving values from the data
3. Data governance is another critical aspect of data implementation
4. Identifying Data Stewards those defines policies, standards, and processes that lead to data classification, privacy, protection, and security.
5. Define finite measures to capture performance, compliance, and regulatory measures of your application as well as Data
6. Automating monitoring, alerting, and auditing cycles to run timely checks on your Data to report
7. Data cataloging and labeling - Data stewards or Domain experts can define the business relationship and value of the data feed in the form of labels. Data cataloging helps data scientists, and ML engineers to continue working on data profiling, data model design, and implementation.

Below are some of the essential data points before you begin with Gen AI use case -

1. Infrastructure and computational resources - Assess your existing applications and infra setup before adapting Gen AI use cases. Do you have the required infrapower to get started with Gen AI?
2. Data Readiness - Consider the above data pointers and evaluate whether you have data ready to be used to train foundational models for your Gen AI use case
3. Data Privacy - Do you have the right standards and classification implemented to avoid any sensitive data usage to train your models
4. Biasedness or Hallucinations - As you know, Gen AI works on context. Do you have the right data, and appropriate context to use these LLMs to train for your use cases? Do you have enough data pointers to set the context w.r.t. these foundational models?
5. Business Value - What value do you want to bring to your business by adapting to Gen AI? Do you have a business use case or just following the path of others?

There are many other parameters and areas that you need to consider while adapting Gen AI use cases.

Follow and subscribe to the newsletter to get many exciting details of Data and AI.

"Ability to say NO to temptation comes from commitment to values." - Mr. Shiv Khera