ApertureData Problem Of The Month: Preparing For Multimodal AI

Wishing you a very happy 2025! To get us all ready for what’s to come this year, we thought we would focus this newsletter on preparing for Multimodal AI.??

Upcoming Events

January 13, 2025 - Vishakha Gupta in Manhanttan, New York?

January 30, 2025 - Virtual Lunch & Learn - Building? A Video Semantic Search Engine With Twelve Labs & ApertureDB. I 9AM PST | Learn More and Save Your Spot

April 9 -11, 2025 -? ?ApertureData at Google Cloud Next 2025? Las Vegas, Nevada??

Tackling The Problem of The Month: Preparing For Multimodal AI?

Why Stakeholders Need to Talk Before Architecting AI Solutions??

Late last year, Raghu Banda, the host of XTRAW AI podcast asked me a question that made me articulate something I had noticed often but never actively put in words. He asked me what preparation did businesses need to do even prior to choosing their tooling and infrastructure for successfully deploying AI for their use case.?

This wasn’t a question of how we could solve their problem or what tools were necessary. It was more about who the stakeholders were, what they needed to accomplish through their AI efforts, what were the success metrics, and what they needed to agree on before getting into the architecture and implementation discussions.?

This topic resurfaced during a panel I hosted with a group of very experienced data science and AI leaders from various industry verticals (summarized as a blog here). It came up yet again during the AI Realized summit panel on data.?

Even beyond these specific events, I have so often heard or read - “garbage in, garbage out” but haven’t really read as much on how they resolve this problem before jumping head first into LLMs/RAGs/CAGs/GraphRAGs or the next best AI thing since ChatGPT!?

Why do you need to prepare for multimodal AI

Whether you're starting with text-based LLMs or RAG pipelines, tackling the challenges of multimodality thanks to awesome results from ?Gemini, Claude, Sora, or Voyage, or even building superintelligent agents (like OpenAI is doing), there are key foundational issues you must resolve to achieve your goals:

• Challenges due to lack of information:?Companies, unless they are really small startups, are traditionally siloed. It’s rare to have 60 people reporting to a CEO like with NVIDIA. Which means projects and their corresponding data often lives with the corresponding organizations. We have occasionally run into companies that strive hard to have horizontal data and platforms that are shared across applications’ teams but applications teams always have some customizations which they then customize for their use case locally, to avoid polluting the “pristine” company-wide data warehouse.

• Challenges due to misunderstanding of requirements and goals: Why are you using AI? Who is it going to benefit? Are you going to pay more to get the capabilities in house or will it pay for itself by bringing you more customers? What is the timeframe in which you can test your workflows? Do you have the in-house expertise to make it happen, or will you end up spending millions on systems integrators, only to find that the deployed pipelines fall short? Whether it's poor response accuracy, slow performance for your use case, or a lack of scalability, without a clear understanding of your requirements, the scope of data needed for customization, and a shared vision of AI's value within your organization, you risk paying far more than you get in return.

• Lack of organizational metrics to measure success: The previous point really builds the case for this one. As soon as you define your success metrics, for instance, we want to cut down customer support time from 1 day to 10 minutes, we want to ensure worker safety, or we want to increase productivity enough to arrive at 4-day work week without hurting company goals, the other pieces start to align themselves towards achieving those metrics. Suddenly it makes sense to index all your internal customer resources to be searchable by RAG pipelines or you can start using Generative AI to give you synthetic data to train classic ML models for worker safety. ?Defining the volume of relevant source data also helps determine the performance and scalability requirements of the ultimate toolchain you would need.?

• Fight between tradition and progress: Security and privacy concerns have lately been at odds with the benefits of AI models trained on vast datasets. At the same time, regulations demand concrete evidence of compliance, which can create conflicts with the full adoption of new AI techniques. ?Understanding how far you can relax constraints and defining your strict governance requirements are crucial factors in every decision. It’s also a valid reason for the C-suite to seek legal input—but they must be prepared to override a blanket "no" when necessary. With possibilities like building permissions RAG or restricted viewing of content, it is possible to achieve your goals without giving up these requirements. Similarly, it is possible to unify your data for AI reasons but not loosen lineage and compliance requirements.

• Analysis Paralysis:?Do we need RAG, is vector-based RAG enough, should we explore Graph RAG, or what about CAG? What’s the best model? Can I do a survey of all vector and graph databases? While these are all important questions, there are also great resources (AI makerspace, OSS4AI, MLOps.community, AICamp, and others) that help you answer some of these questions. Ultimately, once you are very sure of your use cases, metrics, and timeline, you can avoid going back and forth on which one of these techniques work for you. I've spoken to people who realized generative AI wasn't the right fit for them, as well as those who need cutting-edge knowledge graph techniques to fully leverage their data. The key is knowing exactly what you're looking for!

If it were easy, chat bots would quit hallucinating! But it's with a more organized and well-synchronized effort that we can really start benefiting from AI, trusting it more, more importantly, it helps you achieve your business goals faster than if you simply started by exploring the entire landscape of available AI technologies.

The DevVerse

We are trying out this new section to help our user community get inspired, share ideas, and build great AI applications/projects.

Kickstart RAG & AI Projects We have summarized a list of RAG and agentic use cases along with a collection of notebooks showing how to build RAG, question-answering systems, text search, image classification, and more and more in this blog.? ?

Cool Products We Discovered

Who knew robotics could find intriguing applications in the beauty industry but that’s exactly what Luum and Clockwork are doing, and they happen to be great examples of building multimodal AI solutions.

Some Christmas Joy With Instacart Jingle

With all the questions swirling around trustworthiness or guardrails around AI agents or AI replacing jobs (our panelists from that earlier blog didn’t think so), it was nice to see an Instacart AI agent do some good by accepting food donations!

Ready to unlock the full potential of your multimodal data workflows?

Try ApertureDB Cloud today!