Random Check for ChatGPT Prompt Quality

How do you know that a prompt produces the high quality results?

You may think the human-in-the-loop iterative approach.

Well, that helps. But what if this is heavy duty task?

Let us examine below prompt and see how without human-in-the-loop we can check quality using some algorithmic approach.

i want to listen to an inspirational speech that is from American 
in recent years that is not political and is given by female        

We can randomly pick one Specifier, say, female, and negate it to "non female", like below

i want to listen to an inspirational speech that is from American 
in recent years that is not political and is given by non-female        

Although the results are mostly correct, I do see some transient errors that one item occurred on results from both prompts.

Therefore, random testing can be useful here. By randomly picking a Specifier, and negating it, we can check whether the results contain overlapping items. Those items are problematic.

Takeaway

Random checking the prompts result with negated specifiers will help on issue detection.