Simplified Architecture to take up Generative AI in the Cloud Applications

Stay Ahead in AI with Weekly AI Roundup; read and listen on AITechCircle:

Welcome to the weekly AI Newsletter, where I provide actionable ideas and tips to assist you in your job and business.

Before we start, share this week's updates with a friend or a colleague:

Today at a Glance:

• Simplified architecture to take up Generative AI with the Cloud Applications
• Generative AI Usecase: Producing Synthetic Medical Data in Healthcare
• AI Weekly news and updates, Favorite Tip Of The Week & Things to Know
• Open Tech Talk Podcast, the latest episode on Unlocking Startup Success

Just Enough or Good Enough Architecture for Generative AI with PaaS for SaaS

I have spent over a decade working with ERP implementation for small to large enterprises. Today, I would like to talk about how to effectively and quickly integrate Generative AI into the application, which is the foundation of the organization's data.

Adding Generative AI into the existing IT landscape and organizational operations can be seen as a journey that evolves through distinct stages, each bringing benefits and challenges.

What we observe in this journey begins when employees bring productivity-enhancing tools to the organizations. These simple, off-the-shelf AI tools, such as ChatGPT, Copilot, and Grammarly, are designed to enhance productivity. They are easy to adopt and provide immediate value to knowledge workers without the complexity of the organization's security, rules, adherence, etc.

This diagram illustrates the typical organizational journey with Generative AI, highlighting the progression from adopting simple tools to building sophisticated AI models, ensuring a structured and strategic approach to AI integration. The journey reflects AI's increasing complexity and value, moving from cost-efficiency and simplicity to achieving robust business value and governance.

Generative AI in Applications:

When on-premise applications started migrating to SaaS, the fundamental questions were at the heart of managing the custom components in the SaaS applications and having the liberty of customizing the on-premise applications (e-business suite, JD Edwards, Siebel, etc.).

What do you do with the custom components unique to every organization to manage specific business processes?

We spent most of the time discussing how to address these custom extensions. At that point, the concept of PaaS for SaaS came into play, where we started having cloud services like Visual Builder/Process Cloud (OIC) and APEX, or building entirely custom on Weblogic and using the Oracle Autonomous Database or Database cloud service as a database.

Every customer, colleague, and friend is now very well versed in PaaS 4 SaaS areas, and they were able to quickly address the unique requirements from the business point of view.

Now, with the advancements in AI, specifically in Generative AI, the key question is how to integrate it into the existing ecosystem of ERP applications.

This reminds me of the concept we use in building enterprise architecture. Gartner introduced the Just Enough or Good Enough Enterprise Architecture (EA) practice to promote a pragmatic approach to enterprise architecture. The idea emphasizes creating an EA framework to meet business needs without overengineering or overinvesting in unnecessary complexities.

Here are the core principles:

• Business-driven: To secure ongoing support and investment, the EA initiative must be directly aligned with business objectives and demonstrate meaningful results within a short time frame, typically six months.
• Incremental Development: Instead of attempting to build a comprehensive architecture from the outset, start with small, manageable components that address immediate business requirements. This approach allows for flexibility and adaptation as the business evolves.
• Focus on Value: Prioritize EA activities that provide clear and measurable value to the organization. Avoid extensive documentation and processes that do not contribute to achieving business goals.

If we return to our PaaS for SaaS ecosystem, we were building 'just enough architecture' to meet the business requirements, with the addition of a PaaS layer to the SaaS applications.

Let us go through our earlier before Gen AI era, the typical architecture.

We used these additional components and SaaS to build custom extensions and integrations.

Now let's fast forward, and we are in the Generative AI era

As a business, your utmost race is to capitalize on Generative AI and how quickly you can bring/introduce it into your organization. Let's reflect on your existing applications, tech ecosystem, and what is happening.

Oracle Cloud applications (HCM, ERP, SCM, CX) have introduced built-in capabilities to utilize large language models, and several use cases are introduced as part of the fusion applications.

Generative AI is a fully managed Oracle Cloud Infrastructure service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover many use cases, including chat, text generation, summarization, and creating text embeddings.

You can follow the same principles of 'just enough architecture' to embrace Generative AI in your organization. You can extend the existing PaaS for the SaaS ecosystem with another component called Generative AI.

Oracle Cloud Applications practitioners need to think in this mindset, enabling them to have a simplified architecture.

The same rules apply now.

In this case, you are just extending the PaaS for the SaaS ecosystem with another Service: Generative AI.

You can also upgrade Oracle Autonomous Database to Autonomous Database 23ai to gain the capabilities required for Gen AI, such as AI Vector Search, etc. Review this newsletter's earlier edition on 23ai for more detailed info.

Let's try to build your to-be architecture after the addition of Generative AI.

Now, definitely, there is a 3rd phase where you want to have your own enterprise AI/Generative AI platform, as reflected at the start of this article. This is the area where you need to have your own set of chosen LLMs deployed on GPUs. This area needs more detailed analysis and thought; we will cover it in the future.

Concluding today's article, these are my views, as I am also stepping into this ride, so please, please, add your opinions and comments and extend it.

Let's lay the groundwork, and we can all benefit from it as the AI community.

Weekly News & Updates...

Last week's AI breakthroughs marked another leap forward in the tech revolution.

1. Nvidia released the Nemotron-4 340B family of language models. It includes a 340-billion parameter base model and versions fine-tuned to follow instructions and serve as a reward model in reinforcement learning from human feedback. Over 98% of data used in the model alignment process is synthetically generated, showcasing the effectiveness of these models in generating synthetic data. Link
2. Alibaba has released the Qwen2 family Large Language models, which include base and instruction-tuned models in 5 sizes: Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, and Qwen2-72B. Link
3. Claude 3.5 Sonnet from Anthropic has raised the industry bar for intelligence, outperforming competitor models and Claude 3 Opus on a wide range of evaluations, with the speed and cost of the mid-tier model Claude 3 Sonnet Link.
4. Stability AI has released the Stable Diffusion 3 Medium, the most advanced text-to-image open model. The small size allows it to run on personal computers and for organizations to run on GPUs. Link
5. Dream Machine from LumaLab, a video model for creating high-quality, realistic shots from text instructions and images using AI. Link
6. Jace from Zeta Labs AI, an AI employee, is not the only AI chatbot that can handle longer-running tasks and take action in the digital world. Link

The Cloud: the backbone of the AI revolution

• Oracle, Microsoft, and OpenAl have partnered to extend the Microsoft Azure Al platform to Oracle Cloud Infrastructure(OCI) to provide additional capacity for OpenAl. Link
• AI and the Enterprise: Oracle’s New Capabilities for Driving Business Value
• NVIDIA CEO Sees Bright Future for AI-Powered Electric Grid

Gen AI Use Case of the Week:

Generative AI use cases in the Health Care industry:

Producing Synthetic Medical Data in Healthcare with Large Language Models

Business Challenges:

Access to accurate medical data is often restricted due to privacy regulations, making it challenging to conduct research and develop new solutions.

In some areas of medical research, sufficient data is needed to train machine learning models effectively.

Real medical datasets can have inherent biases, impacting the generalizability and fairness of AI models.

Ensuring compliance with data protection laws using patient data can be complex and resource-intensive.

AI Solution Description:

Implementation with Large Language Models (LLMs):

Data Generation: LLMs can generate synthetic medical data that mirrors the statistical properties of real datasets without compromising patient privacy.

Anonymization: By creating synthetic data, LLMs ensure that no accurate patient information is used, maintaining compliance with data protection regulations

Bias Mitigation: Synthetic data can be generated to balance underrepresented groups, reducing biases in training datasets.

Scalability: LLMs can quickly produce large volumes of synthetic data, enabling extensive research and model training.

Example:

An LLM can be trained on anonymized medical data to learn patterns and relationships within the data. Once trained, the model can generate synthetic patient records that replicate the characteristics of actual patient data. These synthetic records can be used for research, training AI models, and developing new healthcare solutions without risking patient privacy.

Expected Impact/Business Outcome:

Revenue:

Access to abundant synthetic data accelerates research and development, potentially leading to faster time-to-market for new medical solutions.

User Experience: Researchers and developers gain access to high-quality, diverse datasets, enhancing their innovation ability.

Operations: Streamlined access to synthetic data reduces the administrative burden of data privacy and compliance management.

Process: Synthetic data enables more comprehensive testing and validation of AI models, improving their robustness and reliability.

Cost: Generating synthetic data is cost-effective compared to collecting and managing accurate patient data.

Required Data Sources:

• Anonymized real medical datasets
• Public health records
• Medical literature and research databases
• HMIS system

Strategic Fit and Impact Rating:

Strategic Fit: High

Impact Rating: High

Using LLMs to produce synthetic medical data addresses significant business challenges concerning data privacy, scarcity, and bias. This approach supports innovation and compliance, making it a strategically fit and high-impact solution for the healthcare industry.

Favorite Tip Of The Week:

Here's my favorite resource of the week.

• Stanford CS25: V4 I Behind the Scenes of LLM Pre-training: StarCoder Use Case from Loubna Ben Allal, ML Engineer hugging face. Video, Slides

Potential of AI

Meta FAIR has released publicly several new research artifacts:

• Meta Multi-Token Prediction Pretrained Language Models for code completion using Multi-Token Prediction.
• Meta JASCO Generative text-to-music models can accept various conditioning inputs for better controllability. A paper is available today, and a pre-trained model is coming soon.
• Meta AudioSeal: it is the first audio watermarking model explicitly designed for the localized detection of AI-generated speech, and it is available under a commercial license.
• Additional RAI artifacts, Including research, data, and code to measure and improve the representation of geographical and cultural preferences and diversity in AI systems.

Things to Know...

The Model AI Governance Framework for Generative AI (MGF for GenAI) from AI Verify Foundation outlines nine dimensions to create a trusted environment.

This framework aims to enable end-users to use Generative AI confidently and safely while fostering space for cutting-edge innovation. Recognizing that no single intervention can address all existing and emerging AI risks, the framework offers practical suggestions as initial steps, building on the existing Model AI Governance Framework for Traditional AI.

The framework seeks to facilitate international discussions among policymakers, industry, and the research community to support trusted AI development globally. This marks the first step towards developing detailed guidelines and resources for each dimension, promoting a systematic and balanced approach to AI governance.

The Opportunity...

Podcast:

• This week's Open Tech Talks episode 137 is "Unlocking Startup Success: Insights from Leopold van Oosten." The CEO of Amsterdam Standard has built 18 startups in 20 years and has won 3 significant awards through his organizations.

Apple | Spotify | Google Podcast | Youtube

Courses to attend:

• CMU Multilingual NLP 2022 course on youtube
• CMU Advanced NLP Spring 2024 course on youtube

Events:

• GITEX GLOBAL, Oct 14-18, 2024, Dubai, UAE
• EUROPEAN Conference on Artificial Intelligence, Oct 19-24, 2024 Santiago de Compostela

Tech and Tools...

• Argilla is a collaboration platform for AI engineers and domain experts who require high-quality outputs, complete data ownership, and overall efficiency.
• Amplication is a robust, open-source development platform that Instantly generates production-ready .NET and Node.js backend apps

Data Sets...

• SciRIFF, a dataset of 137K expert-written demonstrations spanning five essential task categories for literature understanding: information extraction, summarization, question answering, claim verification, and classification.
• LibriSpeech ASR is a corpus of approximately 1000 hours of 16kHz read English speech, prepared by Vassil Panayotov with the assistance of Daniel Povey. The data is derived from reading audiobooks from the LibriVox project
• Stanford Natural Language Inference (SNLI) Corpus. It is a collection of 570k human-written English sentence pairs manually labeled for balanced classification with entailment, contradiction, and neutral labels.

Other Technology News

Want to stay on the cutting edge?

Here's what else is happening in Information Technology you should know about:

• McDonald's to end AI drive-thru test due to errors reported by CNBC, one issue is understanding different accents and dialects during the taking process.
• Amazon’s new AI-powered Alexa might cost up to $10 per month, as reported by TheVerge; the new version is bringing Generative AI to Alexa.

AI First Community to Learn & Share...

Have a question or need some assistance with your AI project, or maybe you want to be part of the thriving community to learn AI together,

Click here to join the AI Tech Circle - It's your Community

That's it!

As always, thanks for reading.

Hit reply and let me know what you found most helpful this week - I'd love to hear from you!

Until next week,

Kashif Manzoor

The opinions expressed here are solely my conjecture based on experience, practice, and observation. They do not represent the thoughts, intentions, plans, or strategies of my current or previous employers or their clients/customers. The objective of this newsletter is to share and learn with the community.