Considering CoPilot & Data Management

Recent conversations around AI tend to circle back to the same questions: structured versus unstructured data, Azure AI versus Copilot — you know the pattern. So when I had the opportunity to talk directly to Naveen Krishnan , an AI architect at Microsoft, I had plenty of questions to ask.

?

From the top requested AI solutions to data management to predictions for the next wave of AI tools, here’s what Krishnan shared on a recent bonus episode of The Business of Tech.

?

AI’s best fits: common solutions and use cases

?

Everyone’s buzzing about AI, but I wanted to know: Where does someone actively building Microsoft solutions see AI fitting best? How is it most effective in giving partners what they need?

?

Krishnan named retail, financial services, manufacturing, and automation the top spaces where AI can best provide new solutions or enhance existing ones. And, as we’ve all experienced by now, he also spends a decent amount of time educating customers on what their needs actually are and whether AI is the best way to help them.

?

As for specific use cases, Krishnan explained that chat capabilities remain the best, most common example of AI applications, often infused with support solutions. For example, he’s seen people use agents in their DevOps pipeline, where AI will trigger a job and attempt deployment; if it fails, a person can step in right where the code needs to be fixed.

?

Dynamic reporting is another area where AI helps out; people have been doing canned reporting for so long, they’re very interested in the idea of asking natural language questions on the fly and having AI convert it into a query.

?

Krishnan’s take on effective data management

?

For all of these solutions and use cases, there, of course, has to be a step before applying a solution: data management. Some level of structuring is required, but at the same time, AI is great at working with unstructured data. So, what does Krishnan believe is required for effective data management when working with AI?

?

He explained that while much attention has been given to parsing unstructured data into manageable components, structured data is still important for tasks like generating ad-hoc, dynamic reports (without relying on pre-defined templates).

?

Unfortunately, according to Krishnan, many legacy databases lack robust schema orientation, making it difficult for systems to interpret column names or relationships. To address this, he recommends creating specialized database views. These views filter, combine, and consolidate relevant data into a single, structured representation, which helps avoid issues like SQL injection attacks and ensures efficient querying.

?

As you’d expect, he also emphasized the importance of implementing safeguards, like restricting bots or AI systems to read-only operations and preventing them from executing destructive commands. Managing user permissions and access controls within the cloud infrastructure also boosts security, as well as fine-tuning prompts and validating system responses.

?

That’s for structured data. What about messier data, like large volumes of text?

?

Krishnan recommends preparing a data pipeline, like an ELT or ETL pipeline, to segregate your text, embed it, vectorize it, and save it on your Asure AI search. He also believes it’s essential to start with a very minimal, simple solution, then add on top of it.

?

“Once you have that, you will gain confidence in what is needed and how to handle different types of data,” he said.

?

Differentiating between Azure AI and Copilot

?

That led me to my next question: what does he think of the difference between working with Azure AI and Copilot? What factors should we consider when choosing between the two?

?

In short, think of Copilot as a prebuilt assistant that you can plug into different types of data sources (like HR payroll, Office 365, etc), and use Azure AI when you have your own idea that you want to build yourself, usually to tackle one specific, complex solution.

?

Luckily, Krishnan believes that Azure AI has enough tools to support a pretty straightforward building process. When you’re done, it’s relatively easy to embed it into your apps and keep everything up and running.

?

AI predictions for the next 24 months

?

Finally, what might the future of AI look like? It’s tough to know what awaits us longer down the road, but what about the next two years? What is Krishnan preparing for?

?

Agents are one area where he sees short-term change. He predicts a shift from AI chat systems simply providing outputs to actively executing tasks based on user input. This progression will enhance the way businesses leverage AI, making interactions more intuitive and results-driven.

?

Additionally, he predicts the emergence of multi-modal agents, where agents will specialize in specific tasks—for example, a SQL agent for database queries or a document agent for text analysis. He could also see the release of an "agent library," where businesses can access and integrate pre-built agents tailored to their needs.

?

He also thinks that token limitations will no longer be fixed with prompt caching, but with built-in solutions. And, like the rest of us, he’s curious about the next versions of ChatGPT.

?

?

Want more actionable advice from this week’s guest? Check out his Medium blog AI with Navin Krishnan for tutorials, insights, predictions, and more.

?

As always, my inbox is open for stories, questions, or whatever else is on your mind.