Integrating ChatGPT to Your Software Stack — thinking long term!
It’s safe to assume that if you’re even remotely connected to the tech world, you’re familiar with the immense power of ChatGPT. A lot of us want to integrate ChatGPT into our software stacks, powering new features and workflows that would have been too hard to achieve previously.
If you’re thinking the same, great idea and I support you! However, ask yourself the following questions as a product owner, solution architect or software developer. I’m thinking —
Software Architecture — Think Scalability, Cost Efficiency and Agility
I believe we’re just scratching the surface when it comes to “productizing” Artificial Intelligence and bringing it to the masses. In the coming years, we would potentially see many such tools some of which would be extremely experimental, some cost-efficient, and some much like God!
As a software architect, developer, or product owner, I’m faced with the question of how to integrate ChatGPT (or any AI) while considering the following factors:
Agility and Cost Efficiency?— Can I ensure easy switching to an alternative AI solution in the future, particularly if I discover a more cost-effective, stable option? What if I wish to use a combination of ChatGPT and Bard? I would prefer to switch between these services as smoothly as changing gears on a car.
Governance— Generative AI algorithms like ChatGPT are experimental and increasingly controversial due to reports of misinformation, copyright violations, and other ethical concerns as adoption grows. Addressing AI governance would be a critical step in safeguarding my software/business.
Availability and Scalability?— As the dependency on ChatGPT grows, ensuring high availability and preventing bottlenecks become critical. Is it feasible to utilize similar AI services to ensure availability in case ChatGPT becomes unavailable? What if ChatGPT is utilized by most of the processes in my system?
Ease of Development?— How can I design this integration in a manner that allows developers to quickly build on top of available AI components without worrying about the aforementioned factors?
Putting it together! — Draft 1
This is a high level specification of how I would architect integrating 3rd party AI into my organizations network.
1. Integration Layer
The Integration layer is where we interface with 3rd party Artificial Intelligence solutions. It follows a plug-and-play model that works with the API Layer to provide a set of services to the subsequent layers.
2. API Layer
The API layer is the centerpiece of AI services available to applications and services across the organization’s network. At this layer, my focus would be on ensuring high?availability?and?scalability?as the consumption and application of these AI services continue to expand. This RPC-based API can be accessed through an array of distributed systems with internal load balancing.
This layer simplifies the developer experience by providing a single, documented API that enables them to think in terms of use cases, rather than requiring them to understand the capabilities of each 3rd party AI available in the market. Additionally, it provides a way to audit, set thresholds for, and control the services that the entire network is utilizing.
3.?Governance?Layer
If AI Governance is a concern (and it definitely should be), one of the most critical layers is the Compliance layer. Generative AI algorithms rely on massive datasets, which can potentially result in serious issues such as copyright violations, plagiarism, misinformation, hate speech, and biases in any part of the network that relies on AI services.
A large part of the governance (or compliance) layer is responsible for logging, filtering responses, and implementing advanced monitoring to improve governance and address these issues.
4. AI Capabilities Layer
At this level, I’m leveraging economies of scope by building an AI capability once and enabling multiple applications to consume it.
For example, a chatbot that’s trained on support documents for a product can be utilized by end-users on Zendesk, product teams brainstorming new features on Basecamp, or developers exploring new ideas on Slack. Another example is a proofreading bot that can be made available to everyone in the organization for tasks such as sending out emails, publishing marketing content, or sensitive conversations.
5. Application Layer
This is where your existing applications, services, user interfaces sit. I’m gonna skip this one because this is purely contextual to use case.
Conclusion
My draft design is a perspective keeping in mind:
We’re living in a time of immense potential, where AI is just beginning to reveal its vast capabilities. While some may question whether AI is making us complacent or dull, it is truly a new way of thinking that will bring about a transformation in the kinds of skills that are valued in the market.
We can create a world that is smarter, more efficient, and more interconnected than ever before. It’s important to take a moment to reflect on the potential impact of new technology, such as ChatGPT, and approach its implementation thoughtfully.
This article was original posted on my personal blog.
References and Further reading: