Building an AI Center of Excellence

Leading organizations are developing an AI Center of Excellence (AI CoE). This is often part of the Chief AI Officer’s responsibilities, which was covered earlier here. We look at why organizations need an AI center of excellence, the key attributes of an AI CoE, and how to build success around the AI CoE.

What is an AI Center of Excellence?

• An AI Center of Excellence provides key resources to internal organizations on best practices centered around AI. The Center of Excellence provides guidance on how to implement successful AI initiatives, build appropriate data pipelines for LLMs and agents, capture retrospectives of what has worked well across AI initiatives and across teams, and coordinate AI projects across the organization.

What are the key aspects of an AI COE Program?

There are several areas to focus on when building?a useful COE. Listed below are some of the areas:??

• Justification through ROI and Metrics -?Building a well-reasoned credible plan for why investments in AI can drive material ROI is critical for ensuring continuity across programs and taking initiatives from POCs to long term deployments. Specifically, determining which metrics to capture and track can help. For example, an AI customer service initiative may track metrics including customer satisfaction scores, a reduction in calls to a call center, the time of engagement in a chat, the percent of AI conversations that effectively answered questions, the number of tickets that required escalation from L1 to L2/L3 and how many required a human in the loop. These metrics can then be used to estimate dollar savings and revenue improvement.? ?
• AI Training –?The AI CoE can coordinate training and development plans for AI initiatives. This can involve generating and providing AI training materials to employees to help retool the workforce. It can also involve coordinating internal and external thought leadership on AI.
• AI Challenges –?The AI COE can also set challenges for organizations to embrace AI including clear measurable goals such as X% of engineering completes an AI readiness courses or releasing Y new AI first features in the next product release.
• Data & Responsible AI–?The CoE can capture and define best practices with respect to responsible AI and maintaining model cards.
• AI First Team–?The CoE can help each product unit develop an AI first solution to existing features and new products.? ?

What are some tactical ways to get started with an AI CoE??

• Appoint a Chief AI Officer and a cross functional CoE team.?
• Coordinate access to data sources and model cards that product and engineering can utilize for AI products across the organization.
• Create a general purpose email address to capture AI feedback and manage internal and external issues?[email protected]
• Define standards for AI compliance and data privacy.
• Maintain a directory of internal AI experts on specific topics such as data pipelines for LLMs, hallucinations, model testing, model evaluation, model drift, and AI hiring questions.
• Capture and build case studies around successful AI initiatives.

?

?This is a purposely short article to start discussions. Reach out to Sanjay Rao and Tau Ventures , an early stage AI focused venture fund to continue the conversation.