GPT Poem Iteration

This article is inspired by a question Jaime Bishop posted to my last article:

What would happen if you push the bot to improve the cat drawing further? Life is rarely binary and seldom without subjectivity even if some like to peddle absolutes.

Can AI Critique AI?

This got me thinking. The Cat drawing problem probably isn't the best example to use to answer this question since, getting ChatGPT to draw stuff is well outside its core area of expertise. Assuming the underlying question is "Can AI critique the output of AI", then my instinctive answer is: Yes. For one thing this is the fundamental principle behind a whole family of neural networks called GAN's (Generative Adversarial Models). A GAN involves creating 2 neural networks where one network is trained to do its best to create "good" output whilst another is trained to assess whether the output from the first is "good" or not - then the critical network cajoles the generating network into producing better and better output.

ChatGPT is not a GAN (it is an autoregressive machine learning model) but Jamie's question got me to thinking whether ChatGPT could be used in a GAN-like way as both the generator and critic.

To investigate this I started by asking ChatGPT to "write a poem about sun shining on a lake on a breezy day". Here is the original output:

ChatGPT as Critic

Next I experimented with how to make ChatGPT critique the poem by asking "Can you suggest any improvements to the poem below..."

With a bit of tweaking I was able to craft the query such that ChatGPT would; not only phrase the improvements as more definite, actionable instructions; but also actually apply these to the poem. The finished query was:

Below is a poem. Can you suggest any improvements? Please provide the suggestions in the form of a set of definite instructions to the author that can be followed, not suggestions. Avoid words like "consider" or "experiment". Please also apply these instructions to the poem to create a revised version.

The next step was to quickly knock up some code to call GPT3 using its API. This meant I could extract the revised poem from the output and iterate the process again and again. Here is the code:

const { Configuration, OpenAIApi } = require('openai')

const configuration = new Configuration({
    organization: '<ORG ID HERE>',
    apiKey: '<API KEY HERE>',
});

const openai = new OpenAIApi(configuration);

async function enhance(poem) {
    let query = `
Below is a poem. Can you suggest any improvements? Please provide the suggestions in the form of a set of definite instructions to the author that can be followed, not suggestions. Avoid words like "consider" or "experiment". Please also apply these instructions to the poem to create a revised version.

The output should look like this
===================
Improvements:
- Do this
- Do that
- Do the other

Revised poem:
blah blah blah
blah blah blah
blah blah blah
...
===================

Here is the poem
===================
${poem}
`;

    const completion = await openai.createChatCompletion({
        model: "gpt-3.5-turbo",
        messages: [
            {role: "system", content: "You are a skilled poetry editor designed to help improve the quality of poems."},
            {role: "user", content: query}
        ],
    });

    if (completion.data.error) {
      throw completion.data.error.message;
      return '';
    }
    return completion.data.choices[0].message.content;
}

async function iterate(poem) {
  var count=50;
  while(count--) {
    var response = await enhance(poem);
    console.log("\n"+response);
    let parts = response.split(/Revised poem:/);
    if (typeof(parts[1])=='undefined') break;
    poem = parts[1];
  }
}

const poem = `
The sun descends upon the sparkling lake,
Its radiance reflected in every wake,
As water waltzes with the gentle breeze,
And leaves rustle in the trees with practiced ease.

The sky is dipped in a palette of blue,
As wispy clouds form shapes both old and new.
Aroma of pine trees and fresh-cut grass,
Mix with the lake's scent, the air crisp and vast.

A metaphor, this lake, a mirror to the soul,
Reflecting back our deepest truth, our ultimate goal,
A fleeting moment of beauty, a glimpse of eternity,
A reminder that life is but a journey, a precious opportunity.

With every passing moment, the lake reflects,
Memories of the past, as well as new prospects,
As time moves forward, and we follow along,
Journeying towards eternity, to which we belong.
`
iterate(poem);        

Running this code then provided me with a poem that, in theory, should get better and better. Here is the result:

Poem:

The sun descends upon the sparkling lake,
Its radiance reflected in every wake,
As water waltzes with the gentle breeze,
And leaves rustle in the trees with practiced ease.

The sky is dipped in a palette of blue,
As wispy clouds form shapes both old and new.
Aroma of pine trees and fresh-cut grass,
Mix with the lake's scent, the air crisp and vast.

A metaphor, this lake, a mirror to the soul,
Reflecting back our deepest truth, our ultimate goal,
A fleeting moment of beauty, a glimpse of eternity,
A reminder that life is but a journey, a precious opportunity.

With every passing moment, the lake reflects,
Memories of the past, as well as new prospects,
As time moves forward, and we follow along,
Journeying towards eternity, to which we belong.

Improvements:
- Avoid cliches, such as "mirror to the soul" and "precious opportunity"
- Use stronger and more specific verbs to increase the poem's vividness and clarity
- Vary the sentence structure and length to keep the reader engaged

Revised poem:

The sun descends upon the glittering lake,
Its light glinting off every rolling wave,
As water dances with an effortless grace,
And leaves sway in the trees with tranquil pace.

The sky is awash in a rich shade of blue,
With clouds floating in forms both old and new.
Pines faintly scent the breeze, and grass freshly shorn,
In the crisp air, a mix of scents reborn.

A symbol of life, this lake, forever flowing,
Mirroring our thoughts as they ebb and flowing,
Capturing our memories and hopes so bold,
A fleeting moment of beauty, a tale to be told.

Each passing moment, the lake reflects,
Of tales of the past and the future's prospects,
And as we journey through life's rugged terrain,
We're reminded that eternity's gain is worth the pain.

Improvements:
- Use enjambment to convey the flow and movement of water.
- Use figurative language to convey a greater sense of the beauty of the lake.
- Remove cliches and focus on fresh imagery to convey the depth of the lake's symbolism.

Revised poem:

The sun descends upon the glittering lake,
Its amber glow resplendent on each crest,
As water ebbs and flows in fluid grace,
A symphony of light on liquid breast.

The sky is awash in a deepening hue,
A canvas for the dreams of clouds to bloom.
Pines whisper secrets to the restless breeze,
As shadows lengthen, night begins to tease.

A symbol of life, this lake, forever flowing,
A mirror for our souls, with depths unknowing,
Its ripples tell of tales beyond our ken,
Of joys and sorrows, love and loss, and then

Each passing moment, the lake reflects,
A story vast, with secrets to detect,
And as we journey through life's rugged terrain,
We learn that beauty's worth both joy and pain.

Improvements:
- Avoid unnecessary descriptions that may come off as redundant.
- Use punctuation more effectively to control the rhythm of the poem.
- Make sure that the poem flows logically from one line to the next.

Revised poem:

The sun descends, a glowing amber light,
On rippling waves that dance with fluid grace,
A symphony of shining, glittering white,
On liquid breast that holds no trace, no face.

The deepening sky, a canvas for the dreams
Of drifting clouds that turn to blooms and fade,
As pines whisper secrets to the breeze that streams,
And shadows lengthen, night begins to bade.

Forever flowing, the lake reflects our fate,
A mirror for our souls, with depths unknown,
Its ripples tell of tales both dark and great,
Of joys and sorrows, and seeds that we've sown.

Each passing moment, the lake ensnares,
A story vast, with secrets to uncover,
And as we journey through life's rugged affairs,
We learn that beauty's worth, both joy and suffer.

Improvements:
- Use concrete language to describe the scenery and emotions.
- Avoid using cliched phrases such as "liquid breast" or "canvas for the dreams".
- Ensure that the rhythm of the lines is consistent.
- Consider using punctuation for clarity.

Revised poem:

The setting sun casts an amber glow,
On waves that undulate with fluid flow,
A symphony of glittering white,
On a watery stage with no face in sight.

The sky deepens, a canvas of dreams,
Where drifting clouds turn to blooms and streams,
As whispers from pines chorus with the breeze,
Shadows stretch, night prepares to seize.

The lake flows endlessly, it captures fate,
Reflects our souls with depth no one can rate,
Its ripples tell of tales both light and grim,
In joy and sorrow, our path gets dim.

Each passing moment, memories enmesh,
A tale vast, with secrets many to thresh,
As life undulates through rugged affairs,
Beauty's treasure is worth all pains and scares.

Improvements:
- Eliminate clichéd phrases e.g. "canvas of dreams"
- Use active verbs
- Vary sentence and line lengths
- Avoid forced rhyme

Revised poem:

Amber sun on waves below,
Undulating fluid, hey nonny no,
A sea of sparkle, a dazzling sight,
Yet over its vastness looms the night.

The sky in hues that true dreams awaken,
Where drifting clouds their shapes are taken,
Wind-whipped pines a whisper raise,
As shadows stretch in the fading blaze.

Lake that flows and seems to know,
Mirrors souls in depth and rainbow,
Tales both sweet and dark it tells,
As we tread a labyrinthine quell.

Moments that wrap in enmeshing mist,
Secrets to all but the lake, exist,
As life drags on in many a despair,
Jewels of beauty are worth the wear.

Improvements:
- Use more varied and specific language, avoid cliches.
- Remove unnecessary words and phrases.
- Use punctuation to create a more coherent structure.

Revised Poem:

Amber sun dips below the waves,
Undulating fluid, hey nonny nonny,
A sea of sparkle, a dazzling sight,
Yet over its vastness looms the night.

The sky in hues that dreams awaken,
Clouds drift, shapeless: then shape-taking,
Wind-whipped pines whisper their prayer,
As shadows lengthen in the fading glare.

Lake flows, it seems to know,
Reflecting depths, kaleidoscopes of soul,
Its tales are sweet or dark, it tells,
As we tread its labyrinthine quell.

Moments enmeshed in misty shrouds,
Secrets to all but the lake, are vowed,
Life drags on in many a despair,
But Beauty's worth the wear and care.        

Conclusion

Well... hey nonny nonny! Did it work? You decide:

I'm no English Literature scholar, but the final output does look better to me. I particularly like the last line "But Beauty's worth the wear and care"

And Finally...

Just for fun I fed the entire text of the final iteration of the poem verbatim into DALL-E. Here is the result: