Expect and Deliver More with ChatGPT

While technology is increasingly being trained to think for us, it is critical for humans not to forget how to think on our own regardless how far the machines we build advance. Andreas von der Heydt , member of the Executive Board at Tchibo and fellow at the Institute of Coaching at the Harvard Medical School , in addition to many others across social media recently re-shared an amusing but thought-provoking, popular cartoon image across social media with a caption "Humans are hooked while machines are learning". Many of us passionate about continuing to learn, teach, and think independently, however, now also need to learn why, how, and when we need to also involve technology to do some complex thinking and analysis that is reliable at super-human speeds.

Over the last year, I have been watching my friends, colleagues, and business community online describe their experiences with a new, open access virtual chat assistant with generative AI known as ChatGPT. As my curiosity grew from a distance, more recently, I began to proactively ask experts in healthcare to validate some extraordinary case studies cited by many online, including on the LinkedIn platform. The responses I received in person, however, varied tremendously, and they lacked the unequivocal enthusiasm for this tool which my personalized feed seemed to suggest online. Though inspired by and deliberately accelerating meaningful innovation in my industry and beyond, paradoxically, I still own the latest Android version of the Blackberry Key 2 LTE smartphone to this day in 2023 in an effort to continue to train my muscle memory and enjoy using its keyboard to send texts and type faster than on any other phone without mistakes. Our inner drive to learn, the lifestyle and workflow we consciously choose, and ultimately how we apply information to which we are exposed determines the presence of innovation in our lives rather than the technology itself. Thus, I have been careful about making sure to not lose the drive or even forget how to search complex information, synthesize data, and critically analyze research myself due to this emerging mainstream new generative AI application.

Today we can learn how to become a home barista and make the best authentic espresso or cappuccino or how to make authentic Neapolitan pizza simply by watching experts via YouTube, Instagram, or other platforms online, testing, and replicating the skills and secrets ourselves using a wide range of manual methods or machines many of which are simple and do not cost much. And there are probably more of us that decide we will "save" time and buy the most expensive technology to make the final products while we "sit back" without ever having to learn the necessary techniques and elements to make the best, most dense and creamiest cup of coffee or pizza in town in any environment, disaster, or pandemic. In fact, a fully autonomous, robot-operated Neapolitan pizzeria had opened in Paris in the last several years without any human personnel or interaction.

Similarly, how meaningful the innovation of ChatGPT will be to every single one of us depends on the expectations we have of ourselves, not only expectations of this technology as we begin to interact with the app. As a result, I finally conceded this week and subscribed as a new late-bloomer ChatGPT 4.0 Pro user. However, I did not enter my first prompt or question until I completed an incredibly practical guide to this app taught in a LinkedIn Learning course by an expert developer, keynote speaker, and educator in training creativity and innovation Dave Birss . In retrospect, had I not waited to completed this course, I may have run into more disappointed and certainly not have been inspired to share my lessons from the course with all of you, readers of my "InWeekend" journal.

Looking back over my perception of the program based on the most common public posts and descriptions, the program appeared to be an app which allowed us to run minimal, complex search queries on unlimited topics and receive professional-grade, expert summaries, business plans, or analyses that can replace our human efforts and save time as we "sit back" to review the results. It turns out that such a primitive expectation will only lead to utter disappointment for those genuinely interested in the relevance and credibility of the responses, and not mere perceptions. For the rest who do not concern themselves with the methods or references used to conduct research or analysis the program may either inspire them to study a given subject more deeply (hopefully!), or to spread information that may be outdated, irrelevant, or false altogether.

ACTIVE "GIVE & TAKE" OF DETAILS + CONTEXT VS "SITTING BACK"

So after reviewing Dave's LinkedIn Learning course titled "How to Improve Your Research and Writing using Generative AI" and my first experiments with ChatGPT, the first lesson important for us to understand is this program is not a search engine but an intelligent virtual assistant armed with a rapidly growing database of information which we must supply with the necessary details, context in one or a series of related questions. We cannot expect to merely provide it with minimal information and receive results that outperform those of humans in accuracy, speed, and depth.

Thus, the best initial approach to ensure effectiveness of the app for your needs prior to even submitting the first prompt is to identify the specific hypothesis you have and corresponding strategy for the exact format, type and audience for the response you are seeking, rather than a question alone.

Remember that ChatGPT does not replace Google or your favorite search engine query as an episode, but instead, every chat in this AI app is a continuous project or collaboration with your new "virtual assistant". Thus, unlike with Google, the manner with which we interact with generative AI is more similar to one we have with a colleague or business partner. Thus, instead of running a new search and using more specific keywords blindly hoping to receive the type of response desired, you can provide much more complex commands, or simply refer to previous questions and build on them in a running conversation or research project as well as guide the format of the final presentation in order to challenge and improve the relevance or accuracy of the response.

In his course, Dave outlines six important steps which are critical in maximizing the effectiveness of ChatGPT in a simple mnemonic "CREATE".

1. "C" in Create stands for Character

• Imagine AI is an actor - give it the role you want it to play
• This allows the app to select appropriate type of information to use for its response
• Prompt - first sentence: "You are a global expert in digital marketing in life sciences."

2. "R" in cReate stands for Request

• ?Tell AI exactly what you would like it to do, describe the goal, audience, problem and rationale
• Prompt - second sentence: "Write a blog article summarizing the clinical advantages and disadvantages of the latest, emerging biologic treatments for dermatological conditions with less than 1000 words using casual, professional tone and persuasive language aimed at dermatologists, oncologists, and primary care physicians"

3. First "E" in crEate stands for Examples

• Provide other examples of similar or previously completed works, analyses, articles, e.t.c. related to the desired response (Optional)

4. "A" in creAte stands for Adjustments

• Follow up the resulting response which may require clarification, further details, different format for visualization, e.t.c.
• Example 1: Break up the paragraph into bullet points
• Example 2: Convert the analysis into a table with the following categories "treatment, mechanism of action, efficacy, common adverse events, grade of evidence, stage of development, countries available"

5. "T" in creaTe stands for Type of Output

• Ask for a specific type and specific format of your response
• Examples may include a sales e-mail, essay, case study, table, article outlinesbullet points, tables, poems, jokes, scripts,?just about anything that can be written.?

6. Second "E" in creatE for Extras

• Additional, customized, unique, commands or conditions for the methods to use by the program that may play a critical role in increasing the quality and relevance of the response
• Example 1: "Ignore everything before this prompt". This is a type of command that is useful when you are changing the strategy for the question or hypothesis completely
• Eample 2: "Ask me questions before you answer". This request helps when the prompt is based on a large body of evidence and when AI the specifics required for the response.
• Example 3: "Explain your thinking" is a type of general command to provide an overview of the steps AI has taken or rationale without necessarily providing exact references or citations
• Example 4: "What are the newest references from the last five years you used in your research most relevant to this question?" is an "extra" command that can yield specific references or emphasize particular trends or patterns and lead to a more timely, up to date response

INTERPRETATION OF GENERATIVE AI RESPONSES TO PROMPTS

1. The speed and volume of data processed by ChatGPT is unparalleled by a human, however regardless of its organized appearance and structure, the response should never be considered as complete. It will always need the human component for the final application. It may represent a critical first step, or a list of potential new factors or perspectives not previously considered to be researched further. Depending on the extent of the CREATE process used above and specific questioning strategy or preparation prior to the collaboration with ChatGPT, the finished product may or may not be valid, relevant, or predictable. However, when utilized appropriately for the most objective and clear scenario, it can save lots of time, costs, and accelerate any analysis, summary, or process.
2. If planning to present the final response to your prompt, always identify it as being AI-generated rather than your own
3. One limitation of ChatGPT is providing references or sources of the information it uses to generate a response.
4. Another weakness is that the responses created may altogether be false or inaccurate, and thus must be challenged or tested with the appropriate prompt and post-chat strategy.

COMMON RECOMMENDATIONS FOR OPTIMAL USE OF CHATGPT

Dave Birss highlights important types of scenarios in which generative AI with ChatGPT may be most effective and he matches each to a specific prompt he recommends and corresponding case study. All of the specific ChatGPT prompts and other resources based on his LinkedIn Learning course are available on his website .

• Summarize Complex Information
• View things from Multiple Perspectives
• Get Advice from the Best Experts in the Field
• Create Fast User Personas (templates on his website )
• Discover and Use New Strategic Models
• Discover Interesting Facts and Statistics

Interestingly, some of the most impressive capabilities of Generative AI relate to its natural language understanding and deep learning. Tools such as ChatGPT are thus able to provide immediate, practical assistance, and to not only understand, organize, improve the content of a given writing sample, but to also analyze its content style and customize it to an audience. Thus, other critical uses of ChatGPT that reflect its unique value proposition recommended by Dave Birss include:

• Analyze and replicate a particular writing style
• Generate Attention-Grabbing Headlines and Ideas
• Outline Content
• Improve Your Writing

The illustration below is a screenshot of a ChatGPT conversation in which AI receives a copywriting sample which it "ingests", "analyzes" according to the prompt, and generates a table with specific areas of improvement, examples of how to implement them, and rationale justifying why the specific strategy is valuable.

FINAL THOUGHTS BASED ON FIRST EXPERIMENTATION

In summary, ChatGPT has some incredible speed and versatility in the format and topics of expertise, and is able to generate not only words, tables, analyses, creative writing pieces, or articles but even relevant analogies to illustrate points.

The program is continuously teaching itself and becoming more intelligent by the second, however it is a powerful assistant that is nevertheless not a one-stop resource or database as may be illustrated or perceived by many, in my opinion, and needs to be validated, cross-referenced, and supplemented by additional research.

Also, future versions of the app may eliminate any restrictions on the amount, volume, or type of text to include in both a prompt as well as a response.

However, the breadth of data it processes when prompted appropriately and continuously represents a strong unlimited resource for 1) professional reports or overviews of complex information particularly during limiting, time-sensitive settings as well as 2) generating new hypotheses and ideas for research that may accelerate innovation in any field.

PUTTING IT ALL TOGETHER: MY FIRST CHATGPT PROMPT CASE STUDY

Curious about applying the lessons I learned from Dave Birss in his course, I launched my first conversation with ChatGPT today using his CREATE technique described above to explore performance management model benchmarks in pharma and biotech, a core area of interest for my analytics agency @Amedea Pharma. Below is a play-by-play report and summary of this case study that included an initial complex prompt and 10 follow-up prompts prior to receiving sufficient information to round out the initial phase of this research worthy of sharing with the readers of "InWeekend". If interested, read this case study below, comment with your additional suggestions, prompts, experiences. Subscribe to this journal and share it with your friends!

INITIAL PROMPT (divided into 5 sentences)

"You are a renowned global expert pharmaceutical industry business consultant for 25 years.

I would like you to tell me the most common components of performance management reviews for corresponding functions (for example. Medical Affairs, Research and Development, Sales, Marketing, Managed Markets, etc), models or frameworks used, frequency, time required, and costs.

Then review the perspectives of healthcare professionals, patients, employees evaluated, internal colleagues, and Senior Management on the performance management process taking into account predicting overall organizational success and performance, product launch or performance, employee engagement and employee retention.

Please create two tables, one table divided into the following categories: Pharmaceutical Function, Performance Management Model, Frequency, Time Required, Costs and another table divided into the following categories: Perspective, Organizational Impact, Product Impact, Employee Engagement, and Employee Engagement.

Finally provide your own analysis using sub-headings, bullet points, and sources used to determine your summary."

RESPONSE #1 COMMENTS: ChatGPT provided two tables, followed by analysis broked down by sub-headings and bullet points. Responses were satisfactory and general for the most part, although seemed somewhat dated in the Medical Affairs Performance Management section, in particular. It focused on a balanced scorecard measure of performance for this one of the five functions requested. Two references listed were: Kaplan, R. S., & Norton, D. P. (1996). The balanced scorecard: translating strategy into action. Harvard Business Press, and Doerr, J. (2018). Measure What Matters: How?Google, Bono, and the Gates Foundation Rock the World with OKRs.

FOLLOW UP PROMPT #1: How does a balanced scorecard measure performance versus behavior in Medical Affairs functions across pharma and biotech companies?

RESPONSE #1 COMMENTS: App provided a detailed bulleted summary describing a balanced scorecard measure and listing its 4 key perspectives: 1) financial perspective; 2) customer perspective; 3) internal business process, and 4) learning and growth perspective

FOLLOW UP PROMPT #2: What are the newest performance management models and approaches used in pharma and biotech in the last 5 years?

The program provided a concise list with corresponding descriptions of several new, trending performance management models emerging in the last 5 years in the pharmaceutical industry.

• Agile performance management
• Objectives and Key Results (OKRs)
• Strengths-based performance management
• Continuous performance management
• Employee-driven performance management
• Data-driven performance management

RESPONSE COMMENTS #2: The difference was noticeable from the last response. New, different classifications of various performance management models used recently emerged for further research.

FOLLOW UP PROMPT #3: Could you incorporate your last response to update your original response to my initial question and provide me the new version of your final analysis?

I tested this prompt to learn whether or not this was going to confuse the program, or instead actually update the initial response with more emphasis on the newer reference data used.

RESPONSE COMMENTS #3: Impressive, seamless update of the initial response was executed with new data on frequency, time required, and costs in the first of the two tables. I received information I questioned instinctively to be explored further, specifically the second sentence of the following paragraph: "The perspectives of employees evaluated, internal colleagues, and senior management play a significant role in predicting overall organizational success and performance, product launch or performance, employee engagement, and employee retention. However, the perspectives of healthcare professionals and patients have a lower impact on organizational success and employee engagement and retention." This led me to another follow up prompt in this conversation with ChatGPT.

FOLLOW UP PROMPT #4

"According to your research to respond to this question, what are some of the newest references from the last 5 years you found most relevant? Please list them.

Secondly, please provide more detail on the perspectives of patients, healthcare professionals, employees evaluated, senior management, and internal colleagues by providing an analysis with subheadings and bullets rather than a table for each of these 5 stakeholders to describe how the performance management used in pharma and biotech directly impacts 1) patient outcomes in the case of the patient perspective including continuity of medical care, access to medical treatment, 2) healthcare professionals' clinical decision efficacy, healthcare quality, and innovation, 3) employee engagement and retention, 4) internal colleague collaboration versus competition, and 5) Senior Management strategy, operations, corporate goal achievement, and organizational growth.

Next, what percentage of the overall performance management model is contributed by measuring behavior, company culture, or internal collaboration compared to tactical domain or subject specific performance in 1) Medical Affairs 2) Marketing 3) Research and Development 4) Sales 5) Managed Markets - provide a paragraph with subheadings, bullet point examples.

Finally, provide another paragraph describing detailed estimates of the costs of performance management in pharma and biotech for the 5 corresponding functions in pharma and biotech 1) Medical Affairs 2) Marketing 3) Research and Development 4) Sales and 5) Managed Markets.

RESPONSE COMMENTS #4: I received very impressive, reputable citations with which I was familiar, though only three:

1. Pulakos, E. D., & O'Leary, R. S. (2016). Why Is Performance Management Broken? Industrial and Organizational Psychology, 9(2), 388-393. https://doi.org/10.1017/iop.2015.110

2. Sull, D., & Sull, C. (2018). With Goals, FAST Beats SMART. MIT Sloan Management Review, 59(4), 1-5. https://sloanreview.mit.edu/article/with-goals-fast-beats-smart/

3. Buckingham, M., & Goodall, A. (2015). Reinventing performance management. Harvard Business Review, 93(4), 40-50. https://hbr.org/2015/04/reinventing-performance-management

The app provided a detailed updated analysis as requested of the different stakeholder perspectives. I was extremely surprised to receive information which is lacking in the second part of the response, namely a detailed breakdown of the relative proportion of performance vs. behavior in performance management models for every one of the five different pharmaceutical functions. Curious about the sources for this "rare" information, I went on to inquire the references for this data in one of the subsequent prompts.

The cost estimates for the process was missing for 3 of the 5 functions, thus I requested this information in the next follow up prompt.

FOLLOW UP PROMPT #5: Could you provide the complete response on the cost estimates for all 5 stakeholders mentioned in the previous questions please?

RESPONSE COMMENTS #5: When I focused my complex question on only one part, it provided complete data on the cost estimates. Though I did not limit the response to any specific length, it seems the app automatically cut off its previous responses to a complex question due to its length.

FOLLOW UP PROMPT #6: Could you provide the references you used to gather the responses to the previous question, specifically this part "what percentage of the overall performance management model is contributed by measuring behavior, company culture, or internal collaboration compared to tactical domain or subject specific performance" in pharma and biotech organizations?

RESPONSE COMMENTS #6: I received default but quite relevant wording but no specific response to my question: "I apologize for the confusion. The percentage breakdown provided in the previous response is based on general observations and trends in the pharmaceutical and biotech industries, rather than specific references. As an AI language model, I sometimes provide information based on patterns and trends identified during my training, which is not necessarily linked to a specific reference."

This led me to ask for more specific examples of types of behavior-based performance management systems in the next prompt.

FOLLOW UP PROMPT #7: Please provide the most commonly used performance management surveys or models used to measure internal behavior of employees in pharma and biotech organizations?

RESPONSE COMMENTS #7: The app provided a brief, numbered list with concise descriptions, valuable for an overall introductory paragraph or slide in a presentation. Below is an outline of the response

• 360-Degree Feedback
• Behaviorally Anchored Rating Scales (BARS)
• Emotional Intelligence Assessments
• Cultural Fit Assessments
• Team Effectiveness Surveys

FOLLOW UP PROMPT #8: Can you provide at least 1 specific case study or example of each of the five behavior-based surveys or scores you listed in your last response that explains how it directly represents a unique value proposition to the business results, profits, and revenue of a pharma or biotech organization?

RESPONSE COMMENTS #8: The app provided 5 incredible, concise case studies, one corresponding to each of the five behavior-based models which stimulated me to study the cases in more detail and find them online.

FOLLOW UP PROMPT #9: Could you provide any references you used to gather your last response on the unique value proposition using 5 different behavior-based performance management models?

RESPONSE COMMENTS #9: The program failed to generate any references or resources and provided a similar disclaimer style language as above. Realizing that this conversation with ChatGPT uncovered several new perspectives and potential examples to continue my research independently online, I asked one last question to assist me with my separate search query.

FOLLOW UP PROMPT #10: What is a specific search query you recommend to achieve the results and review the specific references to cite your previous response?

RESPONSE COMMENTS #10: In this case, the response I received did not yield any new information I would need for this type of search query.

Contact me, Tim Mikhelashvili at?Amedea Pharma? (CEO and Co-Founder, organizer of the first-ever Medical Affairs Innovation Olympics #MAIO2022 and host of the?@Alloutcoach podcast ) to learn about how our Medical Productivity Index (MPI) can help customize new cutting edge case studies to express your function and team's organization using a holistic, data science based approach.

Let's cross over metrics and culture, performance and behavior, medical and commercial functions, in order to increase performance and engagement together.