Summarization and Prompting

I recently came across a preprint from Griffin Adams et al that covered a new approach called Chain of Density for providing summarization from GPT-4 systems that improved on the baseline, and allowed tuning of the output to fit human preferences.? The approach itself is very understandable - essentially, start with a relatively loose summary, and continue to make it iteratively more dense by including more entities.

The approach itself is interesting, but, as someone who primarily follows the biology / bioinformatics space, the publication caught my imagination for another reason.? I’d personally laughed a bit at the concept of “prompt engineering” of LLMs as a serious pursuit. Of course, learning to prompt well is useful, but in the past, this seemed at first like trial and error, and later, extremely specific to the system at hand.? Someone capable of creating a good prompt for Midjourney couldn’t always take the same one to DALL-E and generate a consistent result.??

This preprint, on the other hand, shares the prompt that they used to create this result and - it’s essentially a string of logical processes, written in English.? I’m sure you could throw it at an earlier LLM and it would fail, but the reasoning is clear enough that I don’t think you’d have to worry about taking it to whatever GPT-5 is going to be and having it break.? This particular case had me thinking that the future of prompt engineering is going to look a lot less like machine programming, and a lot more like traditional logic.