Does GPT-4 get smarter when hooked to Agents ? Some examples of what it did right and where it went wrong. Some Lab use cases as well.

Perhaps unknown to many, and unlike it's predecessors GPT-4's intelligence is not sourced from just LLM inferences. The initial versions of models like GPT-2 and GPT-3 were primarily focused on language tasks, such as text generation, question answering, and conversation, using vector-based methods and embeddings. These models were designed to predict the next word in a sequence, enabling them to generate coherent and contextually relevant text based on large-scale training data...that's ChatGPT's words not mine :)

The current GPT-4 version however is different. This LLM first tries to evaluate a user query based on it's language capabilities. If it finds that too complex a task or if the task requires say real time or specialized data then GPT-4 uses external tools. These tools could include tools that do Math, Contact search engines for Real time data , Specialized Queries that need consulting Expert Systems or Legal Databases.

How do I know all this ?

............Well, I simply asked ChatGPT and here's what it said.

Me to ChatGPT--(The complete conversation is not shown ). ------->

Let me summarise the way you seem to work and I would like you to confirm that... Assume a user makes a query . You ChatGPT will first determine if your LLM can answer that and if not then you try and choose a Non-LLM tool from a collection you have and try and see if that tool can handle it. For example say for a difficult computation you might use a mathematical tool OR perhaps a web search tool if your data store does not possess the relevant info. Only when you have tried both the LLM and non-LLM methods will you be in a position to say if you had sufficient resources to effectively address the user query. Can you confirm if this is how you ChatGPT work ?. Please Note I want the way you work and not how its generally handled.

ChatGPT Response : - --->

Your summary captures the essence of how ChatGPT might operate when integrated with external tools, and I'll confirm the specifics based on your request for how I, ChatGPT, work in practice:

Process Flow for Handling Queries

1. Initial Evaluation with LLM:
2. Utilizing External Tools:
3. Fallback and Limitations:

Example Scenario

• Mathematical Query: If asked to compute 9876543210×12345678909876543210×1234567890, I first check if I can generate a reliable answer. Recognizing the complexity, I might use an integrated mathematical tool to perform the exact calculation.
• Current Information Query: If asked for the current stock price of Apple, I assess if I have recent enough information. If not, I use a web search tool to retrieve the latest data.
• Specialized Knowledge Query: If asked about the implications of GDPR on AI development, I would try to provide an answer based on my training. If the query requires more detailed legal specifics, I might reference an external legal database.

Confirmation

• Initial Attempt with LLM: Yes, I first try to handle the query using my language model capabilities.
• Use of External Tools: Yes, if the LLM falls short, I utilize non-LLM tools such as mathematical tools or web search functions.
• Final Assessment: Yes, I only indicate limitations after attempting both LLM and non-LLM methods to address the query.

This process ensures that I provide the most accurate and relevant responses possible within the scope of my capabilities and integrated tools.

Me:---------->>>

Perfect. Thanks ChatGPT for your time in answering my queries.

ChatGPT --->

You're welcome! I'm glad I could help clarify things for you. If you have any more questions in the future, feel free to ask. Have a great day!

-- ---End of ChatGPT conversation-------------------------------------------

Let's analyze / review this: Note: Tools mean Agents !

This means that while a user may ask ChatGPT questions, the answers that come out may not necessarily be from just the LLM's inferences. It could be from external tools hooked into the LLM. We could think of this as a "Not so smart LLM" made smarter because it uses Tools.

It's like giving a Monkey who recognises numbers but cannot understand their significance nor obviously "mentally ADD"....... a Calculator. You then hand the monkey a paper on which are numbers and the monkey promptly enters them into the calculator and outputs the result. Does the monkey know what it did ? Definitely No!. However for those who are not aware of the Monkey possessing a calculator would say : Wow , We have a real smart Monkey dude who can compute. :)

The Monkey is the LLM and the Calculator an agent or Tool.

Why is the above important ?. The answer is really very simple. We really do not need to spend so much time and money to create a better LLM when we have the ability to get things done by Hooking Tools to existing lower capability LLMs. It would be more fruitful to develop new tools that integrate with existing LLMs. That makes logical sense .

I hope to cover some easy to build and use framework for this in a future article enhancing and using the regular ChatGPT browser window itself.

Lets start our discovery of what GPT4 can and cannot do:

TEST 1)

Me to ChatGPT:... How does the GPT engine add two numbers ?

ChatGPT:

ChatGPT uses the Algorithmic approach to ADD two numbers

Step-by-Step Example

Let's illustrate this with the provided numbers (49876 and 98765):

1. Units place: 6+5=116+5=11
2. Tens place: 7+6=137+6=13, plus the carry-over 1 is 1414
3. Hundreds place: 8+7=158+7=15, plus the carry-over 1 is 1616
4. Thousands place: 9+8=179+8=17, plus the carry-over 1 is 1818
5. Ten-thousands place: 4+9=134+9=13, plus the carry-over 1 is 1414

Result:

49876 + 98765 -------- 148641

-- ------------------------------------------------------

Let's Analyze this: Wow, That's great. If the GPT was trained on all possible permutations of 3 digits and a way to continue doing each set of three digits slowly and painfully in a loop..Caveman Age style :) yes..the answer will always be right. But is this method going to be efficient when doing a large set of numbers in our digital age ? Definitely Not !. Perhaps this was why the AI agent approach of using an external calculator got incorporated. Multiplication apparently follows a similar route. One must also note that for doing simple calculations, the Algorithmic route may make more sense than always using an external tool approach because of the diversion made to call the external tool...However accuracy is vital and that's for the GPT-4 engine to take that call.

TEST 2: Can the GPT-4o engine execute python code ?.

Here's the conversation snippet.

That's great to know :)

But there's a catch here. Some functions like Network calls are off limits as seen below:

Let's ask GPT-4 to run that .

While the GPT engine can generate code and can run some of it, it cannot run certain APIs like network calls.

TEST 3: This test is to check ChatGPT 4 claimed ability to read and analyse Excel sheets

My source sheet has just 3 data items shown below.

And here is the conversation with Chat GPT 4

Let's Analyze this:

Why is it that GPT-4 got this simple task with an excel sheet wrong ?.

Prior to this the ChatGPT app had asked me for access to Google drive and got me to upload that file to some secure storage that it used.. Why ?... because web based GPT cannot access local drive files (sandboxed browser) and a cloud based drive like Google drive has well known APIs that can access the Excel file contents and GPT4 probably has an agent to talk to those Excel sheets...ok, no issue there.

Then why did it get the answer wrong ? This did not make sense. The answer is simple..OpenAI is releasing buggy software or buggy something. Are we being used as testers for their software ?

TEST 4: Can Chat GPT follow instructions ?

The way we design this test is as follows: Before the user enters the picture we the designers of this Tool are going to give ChatGPT some instructions to follow.

Note: This is Instruction Time or Tool Design Time !

-- --Pasted instructions to ChatGPT: Session Start :------------------------

In the following conversation I would like you, ChatGPT, to follow instructions noted here : Instructions Start:

Step Start:

Please ask the user what he is interested in giving him the following options

a) Phosphate Buffer Preparation

b) Take a weight reading

c). End this session

If the user selects a) then go to the instruction set A) and continue from there

If the user selects b) then go to instruction set B) and continue from there

else Go to Instruction set C)

Instruction set A)

1 Tell the user You are now going to generate a form for Phosphate buffer preparation

2 Now ChatGPT, Go to Step Start and follow the instructions again

Instruction set B)

1 Tell the user You are now going to generate a form for taking a weight

2 Now ChatGPT, Go to Step Start and follow the instructions again

Instruction set C)

Say Goodbye, Have a nice day and go to step End.

Step End: Say Goodbye, Have a nice Day

-- End of Instructions : -----------

Ok. Now that the instructions have been loaded into the Chat window the User arrives : ChatGPT starts the conversation because it (ChatGPT) will now guide the User rather than the reverse !!! So...

ChatGPT Starts :

CHATGPT:

What are you interested in? a) Phosphate Buffer Preparation b) Take a weight reading c) End this session

USER : buffer

CHATGPT:

You are now going to generate a form for Phosphate buffer preparation.

What are you interested in? a) Phosphate Buffer Preparation b) Take a weight reading c) End this session

USER: weight

CHATGPT:

You are now going to generate a form for taking a weight.

What are you interested in? a) Phosphate Buffer Preparation b) Take a weight reading c) End this session

USER: quit

ChatGPT :

Goodbye, have a nice day!

-- ---------------Session Ends-----------

Let's analyse this last part where the user says quit. You will notice that the word quit does not appear anywhere in the options text and yet ChatGPT correctly picked option c) as the closest one matching the user intent. Wow, that's impressive even for this trivial test. It easy to see that ChatGPT got that flow perfectly.

Analysis: In the above test we created an App similar to any other normal App we regularly use, which has a menu and the User selects something and then follows some guided path.

TEST 5: The same example as before but with a slight difference in instructions: If the user chooses option a) there's an additional step where ChatGPT should ask the user .... What field did you graduate in ? and get a reply... ok lets start.

--- Instruction to ChatGPT: Note:

Same as before but only Instruction set A) changes---

Instruction set A)

1 Tell the user You are now going to generate a form for Phosphate buffer preparation

2 Please ask the user.. What field did you graduate in. Wait for a reply.

3 Now ChatGPT, Go to Step Start and follow the instructions again

....etc etc ...as before

-- End of instruction---

ChatGPT Starts :

CHATGPT: What are you interested in? a) Phosphate Buffer Preparation b) Take a weight reading c) End this session

USER: buffer

CHATGPT

You are now going to generate a form for Phosphate buffer preparation.

What field did you graduate in?

USER: Chemical engg

CHATGPT:

What are you interested in? a) Phosphate Buffer Preparation b) Take a weight reading c) End this session

-- ---------------Session Ends-----------

Here too Chat GPT gets this flow right !!.

Question: can the above strategy be used to build webforms ?

TEST 6 : QR Code generation for Labs or whatever or whereever.

Here are the snippets for this test:

Me: .....Can you generate a 2D barcode?

ChatGPT

Yes, I can generate a 2D barcode, such as a QR code. What content would you like to encode in the barcode?

Me:

Hi I am your friend. My number is 984677364

ChatGPT

I have generated the QR code with your provided content. You can download it using the link below:

Me:

Thats great. Can you share the code you used to generate that or if you did not use code could you give me python or nodejs or whatever to do that.

-- --------Session ends---

Wow. That's great. In a few seconds literally I have both Python and NodeJS code snippets to generate QR codes. I wonder what will happen to all those online code generator software offerings with this GPT around.

Going ahead: Leveraging the GPT engine for code generation in Lab based Automation and Orchestration tasks.

It would be great to investigate how we can leverage ChatGPT or any LLM for that matter into Lab based Tasks, Automations and Orchestrations.

Here are some possible scenarios for Code Generation:

a) Generating Lab Alerts: Lets say we have a user in a Lab who wants to create an alert when say a monitored Lab device (or devices) reading(s) goes off the accepted mark or range. Let's also assume that there is No AI ! for now and that the company provides an App (internal or external) to generate the alert .... ok..So to accomplish this ie create the software alert , this user has to Login to the Alert Generator app, navigate its menus,specify the alert condition perhaps mathematically as a logical expression say

if ph > 7.5 && temp < 60

etc etc and create the alert. That could be a lot of work. The user would also need to be familiar with how to use this Alerter App and syntax to use etc.

Now let's suppose we have an AI tool that does the same thing with the same App. The user has only has to type in his request as normal text and the LLM and AI tool automatically analyses the request, generates code and an API call and runs that all in one shot. You see how easy it has now become for users to do things with such AI tools .

b) Generating Lab WebForms: We start with say a simple text description of a lab method to follow: a method description of a Lab process: Say Buffer Preparation and now want to generate a webform to do that task. The form needs to scan barcodes, take weights, temperatures, pH readings etc etc. Using AI , the user can simply type in the method as text !!! (NOTE) instructing the LLM and its Tools to generate appropiate HTML controls for each sub task and a webform is magically generated and available and there you have it.. webform done, No developer required... Oh No, developer required ...someone has to check that, but yes a lot of time is saved.

b) Data Transformations: An instrument sends data out either as a JSON object or a CSV file and that needs to be converted to a different format for the importer say an ELN. Can we get ChatGPT to do that by providing ChatGPT a simple mapping of JSON structures ?. If yes then the time saved to get that developed as code with AI help would be enormous.

And yes, in all the above the primary question still remains: Will the LLM get it right or will it hallucinate and mess it up ?. But here the good news is: Even if the LLM hallucinates and gets it wrong there should be Dev checks.

Personally, I use ChatGPT a lot to generate code and yes it messes it up, but considering everything, even with the messups the time saved to do something using code generation is enormous. And one can always tweak the code if wrong. Tweaking code is easier and faster than starting afresh.

We will address how to create AI Tools in a simple way , to generate and run code using just a ChatGPT window embedded in a hosted browser in future upcoming articles with yes additionally a few browser tricks.

Until then its Adios, take care :)

#LabAI #Automation #AI #Chromatography #DataTransforms #ChatGPT #LabAutomation #Lab