Observations from early generative AI engagements

Much has already been written about chatGPT and the potential impact of AI on business. Here are some observations from leading engagements across Canada on generative AI:

1. Tremendous excitement and engagement: ChatGPT inspired interest. However, it can be argued the community really started when Meta launched Llama. Less than 2 weeks later, Llama was leaked to the public, accelerating the pace of innovation. Shortly thereafter, several camelid family-name models followed. Stanford released Alpaca on March 13, followed by Vicuna, and Dromedary. Enterprise interest is palpable.
2. Cautious enterprise experimentation: In the last 3 months, I have worked with about a dozen clients with wide variety of implementations. Prompt engineering has entered general vocabulary, and Enterprise Architects become familiar with RAG and Vector Databases. Many organizations have issued policies banning use of generative AI. Italy banned and then allowed access to ChatGPT. I have seen Canadian clients embark on learning, grasping the nuances of RLHF, and experimenting with low-hanging fruit use cases such as customer care, customer segmentation, hyper-personalization, and more interesting use-cases such as AI-assisted metadata generation.
3. Debate rages on size: Clients are enamored with model parameter sizes. A whole new technical jargon is now available with GPT-3 at 175B parameters, Palm-2 with 540B, Vicuna with 13B, Llama with 7B, 13B, 33B, and 65B parameters, etc. Bigger is not always better though. Similar there are discussions about context size and perceived drawbacks. Summarization techniques and sophisticated chained prompts, maintaining vector databases to keep embeddings for custom documents and then “searching” across them by some similarity metric, fine-tuning the LLM with custom data when possible or developing custom smaller LLMs for this particular data (again, not an obvious task) are some of the techniques employed so far.
4. Implementation architectures are emerging: Reference architectures are emerge for Generative AI platform, and applications. However, early implementations are still grappling with banal questions about the location where the model is hosted for inferencing (with the model provider, or behind the enterprise firewall?), data sharing, use of vector databases, etc.
5. Concerns about HAP & Ethics: Much has been written about the dangers of stochastic parrots already. Many organizations including UNESCO have released AI Ethics frameworks. Concerns here remain about the underlying biases in the training data on gender, race, religion. Organizations in particular are also worried about Hate, Profanity and Abuse. More to come on this topic.