Key challenges in prompt engineering

??? Token Size Limit: LLMs have a maximum token limit for inputs, restricting the amount of context that can be provided, which may affect the accuracy of completions. For calculating total token size of context window for a LLM call to generate the response, "prompt size(instructions) + context(retrieved chunks) size" will be calculated. I would suggest to use the context(chunks) size as less as the total context size of LLM model token size limit. It might consume only the limited amount of prompt info for generating response. It might also truncate the LLM model response data. Because, LLM Model token size always contains both the prompt size + LLM model output size. It is AI Architect role to consider before implementing for the particular use case.

?? Context Limitations: LLMs may struggle to maintain context over longer prompts or conversations, leading to inconsistencies or loss of relevant information.

?? Data Availability: Finding suitable data for prompt engineering can be challenging, especially for domain-specific or specialized language, affecting the quality of the prompts. For example, getting the adequate context data from the VectorDB or the external knowledge base is difficult.

?? Data Quality: The quality of data used for prompt engineering directly influences the quality of the generated outputs. Poor data quality leads to suboptimal results. What we feed as context, what we get as LLM response if we want to get specific answers. The context(retrieved chunks) should be accurate and relevant.

??Evaluation Complexity: As the number of prompts increases, managing and evaluating their effectiveness becomes complex, making it hard to track experiments and draw meaningful conclusions. There are ways to evaluate the LLM accuracy like including citations(references, page-number), DeepEval & RAGAS framework, Azure PromptFlow and human in the loop feedback

?? Latency and Costs: Complex prompts increase processing time, resulting in higher latency and costs due to the larger number of tokens involved in each model call.

?? Impact of Small Changes: Minor modifications to prompts can lead to significant and often unpredictable changes in the model’s output, complicating the task of maintaining accuracy and consistency. Whenever the prompt changes, we need to do the testing with our benchmark-golden dataset(question(user-input), answer(LLM response), context(retrieved chunks), ground-truths(correct answer to the question, human-annotated).

?? Unpredictability: The behavior of LLMs with new or altered prompts can be unpredictable, especially in cases where slight changes can lead to drastically different results. This is really a nightmare for AI Engineers.

?? Scalability Issues: Scaling prompt engineering for large projects or across multiple teams can be difficult due to the manual and iterative nature of the process. This is a project level issue. The AI Architect or Lead-in-charge of the project has to take ownership of the prompt to resolve the issue. Any changes or approvals should be vetted by the Lead.

?? Bias and Fairness: Prompts can inadvertently introduce or amplify biases in the model’s output, leading to fairness issues, especially in sensitive applications. Nowadays, this is hugely taken care in the LLM models

?? Security Concerns: Crafting prompts that avoid generating harmful, misleading, or biased content is challenging, requiring careful consideration and testing.

?? Reproducibility: Ensuring that prompts yield consistent results across different runs or environments can be difficult, affecting the reliability of the output. This is important and we need to configure some parameters of LLM models like temperature and seed.