PS AI Labs Point of View On Generative AI and LLM's

PS AI Labs Point of View on Large Language Models (LLMs)

By Carl Norman (Lead Data Scientist) and Tony Maile (VP Global Growth)

Introduction

PS AI Labs is the Centre of Excellence for AI and Advanced Analytics within Publicis Sapient. This short paper examines the impact and main considerations for enterprises following the recent release of new Large Language Models, such as GPT4 and Bard, part of the broader fast moving generative AI field.

Although LLMs might be a leap forward for the field of AI, the extensive knowledge of PS AI Labs regarding best practices, machine learning techniques and common data science pitfalls are still key to successful project delivery and robust implementation. We can help you cut through the hype, understand what is possible for your business and how to make it a reality.

The Generative AI Revolution

What is a Large Language Model?

A Large Language Model (LLM) is a text-based AI system. Trained on a vast corpus (documents, articles, websites, books, code, etc.), LLMs are capable of generating human-like responses and excelling at a broad range of tasks (i.e. summarisation, translation, chatbots). These advanced machine learning models are based on the “Transformer” architecture, which learns patterns, concepts and relationships present in text to understand the complex structure of language at an exceptionally deep level. As a result, LLMs match or surpass human-level performance in many instances and hence they are being considered a significant breakthrough technology.

Why the current explosion?

ChatGPT when first released by OpenAI made headlines, reaching 100 million subscribers in just two months (Facebook took 4.5 years to reach the same milestone). Given LLMs were first introduced in 2017, why did ChatGPT have such a dramatic impact?

The recent GPT4 model has an incredible 1-trillion parameters (~5-years ago cutting-edge LLMs had c. 300 million parameters), beyond what people thought was previously possible. Given that observers estimate the accessible “digital universe” of data doubles in size every ~2 years, this means exponentially more information can be used for LLM training purposes.

In the past LLMs trained using traditional machine learning training methods tended to suffer from a misalignment problem. Capable of producing hyper realistic text by predicting the best next word in the sequence, but the output did not always relate sufficiently to the specified task. Now a “Reinforcement Learning” approach has been used to improve the “helpfulness”, “truthfulness” and “harmlessness” of the responses. This allows the model to tackle completely novel tasks given a clear instruction and to do so more safely.

These advancements, along with other breakthroughs in the “text-to-image” space, have resulted in a lot of attention for generative AI, boosting the funding available for further research and the training of even larger models.

Importance of new LLMs

The performance, flexibility, applicability and public availability of the new LLMs has been key to their rapid adoption. Important technical features that drive this are:

·??????Multi-Modal Nature: User can input text, images, and tabular data currently, with other modes to be added in future iterations

·??????Zero-Shot ability: Solve a task it has never seen before based on a clear instruction

·??????Longer outputs: GPT-4 can produce coherent and relevant text responses of up to ~50 pages in length

·??????Reasoning: Problems not classically considered language based, i.e. logical puzzles, coding or mathematics, are now easily solved

Where is this heading?

The world of generative AI is moving incredibly fast, it is developing at a pace beyond that seen for any other recent technological inventions. GPT-5 is scheduled for release at the end of the year, which promises to be more powerful, safer to use, and even more versatile.

Historically dominant AI players, such as Google, Amazon and Meta, have responded with new LLMs of their own and other competitors are emerging. All fighting for a share of this rapidly evolving space and innovating at breakneck speed.

Given the accessibility of the tool, an ecosystem around LLMs has sprung up almost overnight, helping to realise what is possible and attracting major investment. In turn, user feedback helps with understanding generative AI and thus breeds further advancements. Costs will reduce as systems scale up, resulting in even greater efficiencies and accessibility.

Enterprises need to position themselves now and invest wisely in generative AI technologies, ensuring they are best placed to benefit from advances in the wider field and not fall behind competitors. By putting the tools into the hands of the public, with no coding experience required and intuitive interfaces, the barriers to entry into the “AI world” have never been lower. The consequence of this is that naturally consumers and clients will expect better and smarter AI functionality from businesses.

Impact on Enterprises

OpenAI predict that “that around 80% of the U.S. workforce could be affected by the introduction of LLMs”, while “approximately 19% of workers may see at least 50% of their tasks impacted”. It is clear a lot of tasks will be augmented by AI, with the opportunity for greater productivity and higher-quality outputs.

What sort of jobs are likely to be impacted? A helpful framework is to identify and focus on tasks / workflows within your business that are more predictable (i.e. outcome is consistent with expectations) and logical (i.e. follows certain rules / patterns). Powerful AI models can learn complex behaviours and relationships from huge amounts of data, however understanding random / highly chaotic systems is still a challenge.?

Figure 1: Source "skventures"

How will generative AI change businesses? The opportunity to (i) increase revenue (e.g. improved customer interaction, product features, analysis, etc.) and (ii) reduce cost (e.g. boosting productivity, completing simple tasks, etc.) could create significant value for your company. Here are five categories to consider for potential use-cases:

·??????Efficiency: Augmentation of common / repetitive tasks (e.g. sales interactions, writing emails, summarising information, etc.)

·??????Insights: Conducting research and analysis to create clear and concise reports ?

·??????New Content: Create everything from graphics to marketing materials

·??????Interactions: Automated customer service can be more realistic, accurate and informed

·??????Problem Solving: Critical thinking, tackling logical problems and active learning are all capabilities of LLMs

See the Appendix for a broader list of impacted skills suggested by OpenAI.

What to think about for your business?

Two important questions to ask are:

??????i.????????Where will the greatest and most sustainable value creation be in the business??This usually means a well-defined existing “problem” or a known opportunity area

????ii.????????What changes need to be made to business practices / structures to take advantage of LLM models??If not setup to adopt the technologies, even if built they will remain unused

If these questions can be answered clearly and knowledgeably, you will be in an excellent position to benefit from the generative AI revolution. However, there are significant risks to implementing these technologies poorly. Security, IP defendability and safety (i.e. bias, truthfulness, regulatory compliance, etc.) are all issues that need to be carefully considered.?

?

How PS AI Labs can help

Combining Publicis Sapient’s large-scale digital transformation experience and PS AI Labs’s expertise with complex AI/ML models, we are in a unique position to help your company to strategise, build, and integrate your generative AI solutions. Although LLMs might be a leap forward for the field of AI, extensive knowledge of best practices, machine learning techniques and common data science pitfalls are still key to successful project delivery and robust implementation.

Partnerships with the likes of OpenAI and Google provides us with access to their latest models and platforms.

We can help you cut through the hype, understand what is possible and how to do it properly. Services we provide are:

• Strategy: Work with you to plan and prioritise your business transformation to benefit from generative AI and identify the high-value use-cases
• Custom Models: Fine-tune LLMs on your own data, leveraging your datasets to improve performance on key tasks beyond publicly available models, creating defendable IP
• Security: Setup a secure generative AI playground, allowing your employees to explore the tool’s possibilities, whilst keeping your data private and ideas safe
• Integration: Helping you to improve your products / business processes by swiftly deploying generative AI capabilities
• Understanding: Help to upskill employees on generative AI, explaining the benefits, risks and how to best use the technology (e.g. prompt engineering, features, etc.)
• Validation: Trust in the output of your generative models is incredibly important, we can help to test and monitor model performance and response quality
• Safety: Advising on and establishing suitable controls to minimise risk for your business
• ROI: A trillion parameter LLM is expensive to train and deploy, but we can help identify appropriate lower cost models and alternative solutions for specific business cases

Risks and Mitigations

The risks outlined below must be carefully evaluated for any generative AI solution. If implemented correctly, the mitigations detailed can minimise these risks, with PS AI Labs the ideal partner to support and guide you through this process.

• Data Recency: Despite being released in March 2023, GPT-4 is only trained on data up to September 2021, however generative AI models can be fine-tuned with new data to overcome such limitations
• Bias:?Models inherit bias present in the training data, which is driven by bias in the wider world. To combat this, companies should show great care in how data is collected and sanitised. Fine-tuning the model on clean data, thoroughly testing your system, and a “human-in-the-loop” approach to check outputs can help mitigate negative consequences
• Discrete vs. Continuous: Generative models require completion of the input (i.e. discrete task) before producing the output. This means continuous tasks (i.e. no clear beginning or end) have to be split into a series of discrete tasks to be tackled
• Lack of transparency: Due to the complexity of generative AI models, explaining the exact reason for a model’s decision is difficult. For LLMs, in some cases we can interrogate the model, by asking follow-on questions to understand the basis for its output. If model explainability is key for a particular task, there are other more interpretable solutions that can be explored
• Privacy: Private data inputted into a public model becomes part of the public domain, so a secure platform should be established first before usage, either hosting the generative AI model directly or having access to a safe API for calls
• Intellectual Property: Ownership of the materials produced or tools created using generative AI models is a complex legal situation, with current laws still being tested and new laws defined. Make sure you have suitable legal protections in place for your IP before using any commercial tools
• Output Inaccuracies: There are instances of public models such as GPT-4 and Google’s Bard providing incorrect responses with seemingly great confidence (“hallucinations”) which can be potentially very damaging. Although “truthfulness” will improve with future iterations, having a “human-in-the-loop” check the output first before it is used in any process, i.e. read a generated email before hitting send, can be one option to avoid issues whilst still increasing productivity. Other mitigations include fine-tuning models for particularly important tasks and interrogating the model based on its output
• Regulation and Compliance: Given the recentness of advancements in generative AI, regulation is still trying to catch-up as these models pose new problems that take time to understand and devise suitable rules for, especially ones that can be agreed on by all interested parties. However, these models do often touch on existing regulation, so making sure you understand the regulatory landscape and how it directly impacts your business is required to ensure compliance
• Costs: As discussed above, generative AI models are expensive to train and deploy, sometimes it is worth exploring more cost-efficient models and alternative solutions for specific business cases

?

Appendix

If we look OpenAI’s own research this helpfully breaks down the human skills that LLM’s enhance, this is could be a sensible way to explore how LLM’s can add value in an enterprise workflow or activity:

• Reading Comprehension — Understanding written sentences and paragraphs in work-related documents.
• Active Listening — Giving full attention to what other people are saying, taking time to understand the points being made, asking questions as appropriate, and not interrupting at inappropriate times.
• Writing — Communicating effectively in writing as appropriate for the needs of the audience.
• Speaking — Talking to others to convey information effectively.
• Mathematics — Using mathematics to solve problems.
• Science — Using scientific rules and methods to solve problems.
• Process - Procedures that contribute to the more rapid acquisition of knowledge and skill across a variety of domains
• Critical Thinking — Using logic and reasoning to identify the strengths and weaknesses of alternative solutions, conclusions or approaches to problems.
• Active Learning — Understanding the implications of new information for both current and future problem-solving and decision-making.
• Learning Strategies — Selecting and using training/instructional methods and procedures appropriate for the situation when learning or teaching new things.
• Monitoring — Monitoring/Assessing performance of yourself, other individuals, or organizations to make improvements or take corrective action.
• Cross-Functional Skills Note: We selected only Programming from the list of cross-functional skills because of our prior knowledge about the models’ ability to code. Programming - Writing computer programs for various purposes.

Source: “GPTs are GPTs: An Early Look at the Labor Market Impact, Potential of Large Language Models”: OpenAI, OpenResearch, University of Pennsylvania, March 27, 2023