Do You Even System Prompt?
I think that too often we don't use the power of a well crafted System Prompt when using GenAI. I'm not sure why, but I suspect it's because the value isn't well understood - we know it's a thing, but not how it can be applied.
To show the difference a well crafted System Prompt makes, I built a small experiment using this code from my GitHub repo for extracting text with Azure OCR. And this System Prompt I built for assisting me with Python coding projects: https://github.com/pohagan72/Prompt_Library/blob/main/Deep_Thinking_System_Prompts/Python_Helper.md
My hypothesis was simple - by using my System Prompt I can copy/paste the source code from my project and ask for a prompt that when used will create a new functional version of the original project. My goal being to make the end user experience as simple as possible, and with the richest, most robust generated response.
Here’s what happened:
I used this user prompt for the tests with and without my System Prompt. In both cases I used OpenAI 4o-mini and included the code from my OCR project with proper tick marks.
Can you create a single prompt that, when executed, will result in this code being generated?
Without my System Prompt:
The generated prompt was vague and high-level. Clearly I would end up with python of some kind but much of the nuance of what's required to have a properly functioning app would be missing.
Create a Flask web application that performs Optical Character Recognition (OCR) on images in a specified folder using the Azure Computer Vision API. The application should have a simple web interface where users can input the folder path, start the processing, and download the results as an Excel file. Implement retry logic for Azure API calls, convert unsupported image formats to a supported format, and display the processing progress in real-time. Include logging for errors and processing steps, and ensure thread safety when updating results and progress.
With my System Prompt
The generated prompt is detailed, comprehensive and actionable. By using my System Prompt it's clear that I'll be given a much higher value output if I use the prompt that's generated. Likely still with tweaking and edits made, but it's obvious that the result will be far more complete and robust.
?"Create a Flask application that processes images in a folder using Azure Computer Vision OCR. The application must meet the following detailed requirements:
Core Functionality:
Accept a folder path containing images via a user interface.
Process all images in the folder concurrently using Azure Computer Vision OCR.
Extract text from each image and store the results in an Excel file.
Provide real-time progress updates to the user via a progress bar and status messages.
Allow the user to download the Excel file containing the OCR results.
Technical Requirements:
Use the flask framework for the web application.
Use the azure-cognitiveservices-vision-computervision library for OCR processing.Implement concurrent processing using ThreadPoolExecutor to handle multiple images simultaneously.
Use a global list with a Lock to safely store OCR results across threads.
Include retry logic for OCR operations using the tenacity library.
Handle unsupported image formats by converting them to a supported format (e.g., JPG) using the PIL library.
Generate an Excel file using the openpyxl library to store the OCR results.
Provide detailed logging for debugging and monitoring.
User Interface:
Create an HTML page with a form for entering the folder path.
Include a progress bar and status messages to show real-time progress.
Use JavaScript to periodically fetch progress updates from the server and update the UI.
Allow the user to download the Excel file once processing is complete.
Error Handling and Robustness:
Validate the folder path input and handle cases where the folder is empty or invalid.
Handle errors during OCR processing, including unsupported image formats and API failures.
Ensure the application gracefully handles large numbers of images and high concurrency.
Use defensive programming principles to validate inputs and preemptively handle edge cases.
Code Structure:
Organize the Flask app into routes for the main page, processing, and progress updates.
Use global variables to track the total number of images, processed images, and processing state.
Implement a function to check if a file is an image and another to convert unsupported formats.
Implement a function to perform OCR with retry logic and store results in a thread-safe manner.
Implement a function to process all images in a folder and generate the Excel file.Include detailed comments and docstrings for all functions and routes.
HTML and JavaScript:
Create a clean, responsive HTML page with a form, progress bar, and status messages.
Use JavaScript to handle form submission, fetch progress updates, and trigger file downloads.
Ensure the progress bar updates dynamically based on the number of images processed.
Logging and Monitoring:
Configure logging to record processing events, errors, and progress.
Log the start and end times of processing, as well as any errors encountered.
Security Considerations:
Use secure_filename to sanitize file paths.
Avoid hardcoding sensitive information like Azure credentials (use placeholders).
Ensure the application does not expose sensitive data or internal errors to the user.
Performance and Optimization:
Use concurrent processing to maximize performance.
Optimize memory usage by processing images in chunks if necessary.
Ensure the application can handle large datasets efficiently.
Testing and Validation:
Include test cases for empty folders, unsupported formats, and large datasets.
Validate the Excel file generation and ensure it contains the correct extracted text.
Test the UI for responsiveness and correctness.
Provide the complete Python and HTML code for the application, including all necessary imports, configurations, and comments. Ensure the code adheres to PEP8 standards and includes type hints where appropriate."
Why This Matters
This example is intentionally extreme to highlight the difference a well-designed System Prompt makes. I'm imagining a future where I will have a library of well defined use-case specific System Prompts that I can use based on the intent of the user prompt.
For example, if the user prompt is about Python then I can first understand the intent (it's about Python), pull in the right System Prompt and only then send the user prompt to the GenAI model with the right framing and context.
But the value of a well crafted System Prompt is obvious. By using one we get both:
And like so many other things, this is one of my journeys - building a library of System Prompts will aid my journey this year as I become an AI-First Product Manager.
?
?
?