Bending OpenAI with Traditional Programming for Unique Recipe Creation

Introduction?

In today's technological landscape, ChatGPT and other Large Language Models (LLM) have captured the attention of developers and tech enthusiasts alike. Beyond simple text interactions, these models offer innovative applications in diverse domains. One fascinating application is in the culinary arts. I used my summer learning time to develop RecipeGen, a tool that seamlessly combines the capabilities of Generative AI with structured tradiitional programming to produce personalized recipes based on user input. This article delves into the creation and mechanics of RecipeGen, highlighting its distinct features, the challenges faced during its development, and the broader implications for AI applications. I invite you to explore RecipeGen on GitHub or try it live at https://yumpop.ai.?

Why RecipeGen??

Over the past few months, I have been amazed by the recipes generated by ChatGPT. I have ventured into cooking styles I previously would never attempt, making everything from comfort foods like my mom's Chicken Paprikash to foods like Bun Cha from Vietnam, Coq au Vin from rustic France, and Doubles from the streets of Trinidad. RecipeGen wraps an engineered prompt, optimized to get the best recipes, into an intuitive user interface.??

What opportunities does an app like RecipeGen bring to the culinary world? Here are just a few:?

• Dynamic Culinary Exploration: While traditional recipe platforms often stagnate with common, oft-repeated dishes, RecipeGen is your passport to a global culinary journey. It brings the world's flavors right into your kitchen.?
• Customized Recipe Creation: Imagine having a tool that crafts recipes based on the current discounts in your local grocery store. RecipeGen has the potential to sync with online stores to use this week's coupons and promotions, ensuring your culinary explorations are both delicious and budget-friendly.?
• Equipment-Specific Dishes: Bought a new iron griddle or a fancy air fryer? RecipeGen could generate recipes tailored specifically to your newest kitchen gadget, helping you maximize its potential.?
• Local Dining Suggestions: Beyond the kitchen, if you're not in the mood to cook, RecipeGen can be further developed to pivot its suggestions towards local restaurants. Craving Doubles or Coq au Vin? Find out where you can get the best rendition in town.?
• Business Collaborations: Beyond individual users, RecipeGen has vast potential for B2B collaborations. Whether it's partnering with equipment manufacturers for digital recipe booklets or restaurants looking to introduce new dishes, the opportunities are boundless.?

Integrating a Non-deterministic LLM with Traditional Programming?

ChatGPT and LLMs have gained immense popularity this past year, with people using them for all sorts of purposes from fun to getting work done.? A substantial percentage of software engineers report productivity improvements in coding tasks. In fact, ChatGPT can generate working code in an astounding number of programming languages.? It makes sense that ChatGPT could also generate ingredients and an instruction set for a culinary Recipe. I aimed to structure these recipes for easy integration into a traditional program.?

This interest is not unique to me. Scientists and researchers in the AI field have been exploring ways to amplify the abilities of these models. A piece of research, Coupling Large Language Models with Logic Programming for Robust and General Reasoning over Text (Zhun Yang, Adam Ishay, Joohyung Lee), looks at this combination using several NLP benchmarks. Instead of solely relying on the language model, they proposed integrating it with logic-based systems. While their approach is far more scientific than my trial-and-error approach, it confirmed my thoughts about doing this type of integration.?

What if I could combine the vast knowledge and creativity of a model like ChatGPT with structured traditional programming? By doing this, I could create a system that understands the nuances of human prompts and then uses traditional programming to provide more tailored responses. In the context of RecipeGen, this would mean not just suggesting recipes based on a database, but genuinely understanding user preferences and constraints to curate unique dishes.?

My journey was less about scientific rigor and more about iterative experimentation with some observable results to adjust the functionality. Each modification brought me closer to a tool that could blend the intuitive understanding of ChatGPT with the precision of traditional programming. Along the way, I made sure to emit metrics and logs that would surface issues. That data exposed several challenges with the integration of LLM with traditional programming:?

Non-Deterministic Responses:?

• The model's non-deterministic nature can yield varying answers and diversions from the structured response format, even for identical prompts.?
• Achieving a 95% success rate in garnering parsable responses required filtering common, non-parsable elements the model might include. In the YumPop version, I’m at about 95% success for responses and for the others the app makes the best attempt to fail gracefully.??
• Interestingly, certain types of behaviors come in clusters. For example, I might get two or three responses in a row that say “As an AI model I can’t create recipes.” Then it will go back to usual and useable responses.??

Model Version Updates:?

• As model versions get updated, the nature of their responses will change.?
• Maintaining a service based on these models necessitates constant vigilance to monitor, detect, and adapt prompts and code to shifts introduced by new model versions.?

Language Variabilities:?

• Responses differ significantly between written languages, impacting both the structure and performance metrics.?
• For instance, Japanese responses demand more tokens than English (approximately 15% more tokens for Japanese in my experience), affecting response times and other metrics.?

Service Misuse and Safety Concerns:?

• There is a risk of generating potentially harmful content, as observed with a recipe generator by a New Zealand supermarket (as reported by The Guardian).?
• The eagerness of models to provide answers to all queries makes prompt engineering and response filtering essential. Current mechanisms in RecipeGen (and further in the YumPop version) either reject certain requests outright or alert users to potential hazards in a recipe depending on how the model responds.?

Observability in RecipeGen?

The integration of LLMs with traditional programming, like RecipeGen, presents some significant challenges. These challenges are not just technical but also pertain to ensuring a consistently good user experience. Enter observability.??

Observability is not merely about monitoring or logging. It is about gaining a holistic, real-time understanding of the system's inner workings and the quality of its outputs. Observability ensures that when anomalies arise, they can be swiftly diagnosed and rectified.?

• Consistency and Quality: RecipeGen's purpose is to deliver recipes tailored to user preferences. But how do we know it is meeting this promise? Collecting feedback metrics like `recipegen_likedrecipes` and `recipegen_dislikedrecipes` offer a direct reflection of user satisfaction. We can use this sentiment to fine-tune the model over time.?
• Performance: While AI's capabilities are astounding, user patience is finite. On average, the model is taking about 15 seconds to fully respond. Although it may be worth the wait, users may question whether the website is working or has stopped responding. Tracking `recipegen_recipetime` (the time it takes for a full response from the OpenAI service) helps to monitor this over time. Implementing streaming responses like the experience in ChatGPT is also very helpful to deal with the wait.?
• Model Behavior Understanding: Parameters like `frequencypenalty`, `presencepenalty`, and `temperature` influence the model's creativity. Monitoring these attributes alongside output metrics can offer insights into their ideal values. For instance, structured outputs might benefit from a lower temperature, but we will not know unless we capture this information.?
• Localization: For a global tool, understanding performance variations across languages is essential. The differences between English and Japanese can be significant, including the response time and the number of completion tokens used. So language is an important attribute to attach to every metric.??

In addition to metrics, logging and/or tracing remain important tools to understand how the application is working. In RecipeGen, when a parsing error occurs, it is often a result of the model deviating from the expected structure in its response. Relying on a debugger in such scenarios would be futile given the model's unpredictable nature. Logs, on the other hand, capture the exact prompt and the model's response, allowing for an in-depth post-mortem analysis. This information aids in refining the system—be it tweaking the prompts, adjusting model parameters, or enhancing the parsing logic.?

I will cover more observability in RecipeGen in my Observability newsletter.?

More to Learn?

The integration of Generative AI and structured programming in RecipeGen represents a glimpse into the possibility for future AI applications. While I have learned a lot through this experiment, there is still more to learn. I’d like to add follow-on interactions that iterate on the recipe, which is exactly what I do in the ChatGPT interface when I’ve been creating recipes. For example, I often cannot acquire one of the ingredients in my small-town grocery store in Japan so I ask ChatGPT for a replacement.??

There is a literal explosion of tools available to assist with these types of applications. I have yet to explore Langchain and its extractions tools that parse output from an AI model. I currently just ask for YAML structured data back and use a general purpose YAML parsing library. I am interested in how services like Microsoft’s Semantic Kernel or Azure Machine Learning Prompt Flow will help if I move beyond a single shot prompt. I currently just use a text file and update the name/version of the file to associate metrics with specific versions of the prompt.??

As we push the boundaries of AI integration in our daily lives, RecipeGen offers a unique intersection of tech and taste. Try https://yumpop.ai and explore a new dimension in recipe creation. Your usage and feedback will be invaluable in refining and expanding its capabilities. If you like a recipe, or find some interesting response, feel free to copy and paste to a comment in this article. Try it out, and let's shape the future of culinary AI together.?

?