Migration From OpenAI to Google Vertex AI: Personal Experiences By a Healthcare Startup

In Heartery , we have just completed a successful shift from Open AI to Google’s Vertex AI. The answers are now more structured and hence clearer, everything works several times quicker, and we are perceived better by healthcare stakeholders. Hopefully, we are one step closer to Google’s Palm Med, the world’s first AI for healthcare.

What We Do

Hi, we are Leo Rosenbaum , Heartery Founder, and Vlad Antonovich , Heartery Developer. Together with a team of medical doctors and advisors, we are developing an app to manage blood pressure. Successful blood pressure management consists of lifestyle changes in food, sleep, stress and exercise; of adherence to medications and other doctor’s orders; of monitoring your blood pressure readings; and of awareness of what goes on in blood pressure related research and innovations.

Heartery Blood Pressure Pal addresses all these areas, with focus on AI-powered lifestyle assistants. We have carefully chosen the most widely recognised validated surveys, and adapted them into steps, with no edits of content. Users answer the questions step by step (there are not many), and can then obtain an assessment of how blood pressure friendly their current nutrition, sleep, stress, and physical activity are.

We trained our web tools foodforBP.com, sleepBPdeep.com, stressBPless.com, exerciseBPwise.com at ChatGPT, following the hype of February 2023. It took us quite some effort to compose and refine the prompts (Example: the definition of blood pressure friendly foods strictly without any examples, to avoid meal plans with way too much blueberries).

The lifestyle assistants assess current food habits, sleep patterns, stress levels, physical activity, in the context of blood pressure, and provide personalized, that is tailored to your individual needs and preferences, blood pressure friendly meal plans (with grocery lists and recipes), sleep well plans, three different stress management plans and three different plans for blood pressure exercise.?

So far, according to Google Analytics, several thousand users have successfully applied our web tools, to generate assessments and plans.

What We Do Not Do

1. It is important to stress right away that we are not playing doctors, our assistants are strictly limited to information about lifestyle changes, with safety tips and references to healthcare providers. If you have been diagnosed with hypertension (that is if a healthcare professional passed this judgement based on your blood pressure readings being persistently over 140/90), not just elevated blood pressure, adapting a blood pressure friendly lifestyle will only contribute to lowering your numbers down if combined with adherence to timely intake of prescribed medications and following other doctor’s orders. On the other hand, a blood pressure unfriendly lifestyle will diminish the effect of the medications, sometimes to zero. Hence, our lifestyle assistants are not substitute to medications, but work together with them.
2. We do not collect any user’s data, and do not have any database. In an anonymous and encrypted way, the user’s answers are assessed by AI via their own API, the assessment and plans are generated and presented to the user.?

The First Impression

There is no OpenAI vs Google’s AI: in terms of API, they are very similar. Having used OpenAI, Google’s Vertex seems to be a product that targets businesses, while OpenAI targets consumers.?

Launch at Google’s Vertex AI

The first thing to do was to choose the model, the way of collaboration with AI depends on it. There is a lot about it that you can google :)

Next, we started adapting our old prompts we had used with OpenAI. One of the main difficulties of working with the API is the limitation in the number of tokens for the request and response, hence we needed the most accurate and shortest versions of prompts. To improve the quality of response, we played extensively with the additional parameters that Google provides, such as Temperature, Top P, and so on (they differ for different AIs).

It took time and effort to find the right choice of values for the transmitted parameters. It involved a substantial and comprehensive assessment of responses. The same prompt with the same input parameters may return different results on different models, so when transitioning from one generative AI to another, the main lesson is that you may expect to retest the AI, and to pay special attention to differences, in order to leave the useful ones and to remove those that worsen the response.

Interesting new things and challenges

Top K is a novelty when you shift from OpenAI to Vertex AI. It regulates the randomness of the response. It is an interesting setting, worth testing in and out. Worth googling.

Also, to our surprise, when generating the response, Google's AI did not understand HTML tags without explicitly indicating in the prompt that they are present. Google's AI also has a maximum limit of 1k tokens, which requires additional program code during the transition (OpenAI's AI has 4k and 16k).

We were very pleased with the speed of Google's AI response, and there were also differences in the accuracy of the answer, if there were some things incomprehensible from the context in the prompt, Google's AI was more likely to ignore them than fantasize, which - the way we see it, it makes the answers from Google's AI safer.

The first challenge we faced when transitioning from OpenAI's API to Google's API is the difference in API connections, because OpenAI provided the API key, and Google does not have this feature available to everyone. That is why the solution to this problem was to create a Google Cloud account with the VertexAI platform connected. Google Cloud requires validation of requests using tokens for additional security.

The token is requested in a separate request to Google Cloud, this process has been further optimized, which makes the generation of the response even faster. The next challenge was adapting prompts and parameters to get valid answers from Google's AI.?

As a result of the transition, we have accelerated the process of interaction with AI, as well as improved its accuracy and safety.

Results

It is hard to say which of the two provides better content. We have tested Google’s Palm 2 extensively in the ‘manual mode’, and compared the answers. For these tests, we applied identical prompts. Contentwise, we did not observe any significant difference.

The answers differ by something else, the answers by Google’s Vertex are notably better structured. Any longer response by OpenAI is somewhat of a flow of text, that has logic and structure, but you can see the flow of thought. Vertex, with its possibility to regulate temperature (focused and deterministic vs diverse and creative) can be better adjusted to the desired output. Perhaps, it is one of the reasons for more structured results when processing users’ answers to validated surveys on sleep, stress, activity, that we employ in our lifestyle assistants to hypertension sufferers.

Another good news is that with Vertex our web tools became significantly quicker. I did not run exact tests, but can assess the speed in another way. When on ChatGPT, we experienced time delays of up to 30-45 seconds that would pass before the generated assessment or custom tailored plan appeared, this lag depended on the current time in the USA, mornings there tend to be particularly busy on a weekday. Hence, we developed progress sentences that would change every 10 seconds, and had six of them to display one after another, in the mean time. 3-4 would usually shift, before the assessment or a plan would show up on the screen. Now on Vertex, it is never more than two.

One more important, and perhaps, somewhat unexpected result. Healthcare stakeholders, from researchers to the industry players, perceive us in a more serious way. We experienced a few sceptics, when we answered ‘ChatGPT’ to the question ‘Which AI do you employ?’ We are clearly perceived much better than before now, when we answer ‘Google’s Vertex AI’.

Finally, the transfer brings us one step closer to our dream, to try our tools for hypertension sufferers with Google’s Palm Med, the world’s first AI developed for healthcare.

PS. This is for your orientation in the wild world of AI, to help navigate between OpenAI and Google’s AI.