Superhuman Prompting

Superhuman Prompting

It's well known that prompting formulas dramatically improve the quality of LLM outputs. In many cases a well-prompted GPT-3.5 response beats the quality of a naively prompted GPT-4 response. Formats like COSTAR and Chain of Thought are commonplace general methods today.

The benefits of prompting techniques are generally explained as harnessing the expert intelligence hidden in the latent space of LLMs. But I began to wonder whether we might do even better if we could devise a prompt to harness ALL of that intelligence all at once – beyond an expert level.

That's led to a technique I call "superhuman prompting" which in experiments below gives qualitative improvements on COSTAR/CoT prompts.

I haven't done anything like exhaustive testing on this because I feel it's worth publishing sooner rather than later for the greater good. But I encourage readers to run their own experiments with it and leave feedback - positive and negative - in comments on this article. The general form of a superhuman prompt is:

Initial Context:

You are a superhuman integration of all the greatest human experts on [domain], embodying the intelligence, insights, and depth of [foremost expert]. Speak and behave with [expert]'s analytical precision, curiosity, visionary mindset, and attention to detail. [augment/replace with qualities of behavior you want]

Task:

Take a deep breath and, using chain of thought and step by step, [perform task].        

Simple example: Counting the number of Rs in the word Strawberry

Naive:

  • How many Rs are there in the word strawberry?
  • There are 2 Rs in the word "strawberry."

Expert:

  • You are an expert in the spelling of English words. Take a deep breath and, using chain of thought, step by step, tell me how many Rs there are in the word strawberry.
  • Sure, let's break it down step by step: Start by writing out the word "strawberry." S - T - R - A - W - B - E - R - R - Y. Now, count each letter one by one to find all occurrences of the letter "R." First "R": S - T - .R. - A - W - B - E - .R. - R - Y. Second "R": S - T - R - A - W - B - E - R - .R. - Y. There are two "R"s in the word "strawberry." Therefore, the word "strawberry" contains 2 "R"s.

Super:

  • Initial Context You are a superhuman integration of all the greatest human experts on English spelling, embodying the intelligence, insights, and depth of James Murray. You speak and behave with Murray's deep insight, analytical precision, curiosity and attention to detail. Task Take a deep breath and, using chain of thought and step by step, tell me how many Rs there are in the word strawberry.
  • Alright, let's examine the word "strawberry" step by step:

  1. Start with the first letter: s
  2. The second letter: t
  3. The third letter: r
  4. The fourth letter: a
  5. The fifth letter: w
  6. The sixth letter: b
  7. The seventh letter: e
  8. The eighth letter: r
  9. The ninth letter: r
  10. The tenth letter: y

Now, count the occurrences of the letter "r":

  • Third letter: r
  • Eighth letter: r
  • Ninth letter: r

There are three Rs in the word "strawberry."

Coding Example: Ward's Wiki in Python

I marvelled at Ward Cunningham's original perl code for the c2 wiki server back in the day. He whipped it up in a weekend as a kind of executable poetry and I still regard it as the prettiest piece of Perl ever. I authored the first open-source clone of it adding versioning, branching and undoability via CVS and my code saw use in the first Wikipedia server. But I was never under the illusion I'd actually improved on the quality of Ward's original.

An equivalent to Ward's wiki in a different language seems an appropriate challenge to reveal the coding power of a superhuman prompt. Note the following are all just single-shot prompts over GPT-4o and used my system prompting of same so YMMV. But I think the qualities of the results speak for themselves.

Naive Prompt:

Implement an equivalent to the original c2 Wiki Server in python        

To implement an equivalent to the original c2 Wiki Server in Python, we’ll need to cover the basic functionality of a wiki server. This includes handling HTTP requests, serving HTML content, allowing page creation and editing, and storing the pages. We’ll use Flask for the web framework and a simple file-based storage system.

Step-by-Step Guide

1. Set Up the Environment:

? Install Flask: pip install flask

2. Basic Server Setup:

Create a basic Flask application that can serve the home page and handle requests.

from flask import Flask, request, redirect, url_for, render_template_string
import os

app = Flask(__name__)
wiki_dir = "wiki_pages"

# Ensure the directory for wiki pages exists
if not os.path.exists(wiki_dir):
    os.makedirs(wiki_dir)

@app.route('/')
def home():
    pages = os.listdir(wiki_dir)
    return render_template_string('''
    <h1>Wiki Home</h1>
    <ul>
        {% for page in pages %}
            <li><a href="{{ url_for('view_page', page_name=page) }}">{{ page }}</a></li>
        {% endfor %}
    </ul>
    <a href="{{ url_for('create_page') }}">Create a New Page</a>
    ''', pages=pages)

@app.route('/page/<page_name>', methods=['GET', 'POST'])
def view_page(page_name):
    if request.method == 'POST':
        content = request.form['content']
        with open(os.path.join(wiki_dir, page_name), 'w') as f:
            f.write(content)
        return redirect(url_for('view_page', page_name=page_name))
    
    try:
        with open(os.path.join(wiki_dir, page_name), 'r') as f:
            content = f.read()
    except FileNotFoundError:
        content = ""
    
    return render_template_string('''
    <h1>{{ page_name }}</h1>
    <form method="POST">
        <textarea name="content" rows="20" cols="80">{{ content }}</textarea><br>
        <input type="submit" value="Save">
    </form>
    <br>
    <a href="{{ url_for('home') }}">Back to Home</a>
    ''', page_name=page_name, content=content)

@app.route('/create', methods=['GET', 'POST'])
def create_page():
    if request.method == 'POST':
        page_name = request.form['page_name']
        return redirect(url_for('view_page', page_name=page_name))
    
    return render_template_string('''
    <h1>Create a New Page</h1>
    <form method="POST">
        Page Name: <input type="text" name="page_name"><br>
        <input type="submit" value="Create">
    </form>
    <br>
    <a href="{{ url_for('home') }}">Back to Home</a>
    ''')

if __name__ == '__main__':
    app.run(debug=True)        

Explanation:

1. Home Route (/):

? Lists all wiki pages available in the wiki_pages directory.

? Provides a link to create a new page.

2. View/Edit Page Route (/page/<page_name>):

? Displays the content of a specific page.

? Provides a form to edit the content of the page.

? Saves changes to the page content.

3. Create Page Route (/create):

? Provides a form to create a new page by entering the page name.

? Redirects to the view/edit page of the newly created page.

This implementation provides a simple, file-based wiki server using Flask. It supports creating, viewing, and editing wiki pages, similar to the original c2 Wiki Server’s basic functionality. You can enhance this further by adding user authentication, styling, search functionality, etc., as needed.

Expert Prompt:

You are an expert Python programmer. Take a deep breath and, using chain of thought, step by step, implement an equivalent to the original c2 Wiki Server in Python.        

Certainly! Recreating the original C2 Wiki server in Python involves several steps. We'll approach this methodically, ensuring each part of the process is clear and well-implemented.

Step 1: Define Epic in XSCALE normal form

Create a simple Wiki server:

WHY: To replicate the functionality of the original C2 Wiki server in Python.

WHO: Unauthenticated users of a public website.

HOW: Freely edit, extend, and interlink all the content they can read there.

WHAT: Python web server offering basic CRUD operations for website pages.


Step 2: Set Up the Basic Wiki Server

We'll use Flask, a lightweight web framework for Python, to set up our web server.

Install Flask

pip install flask        

Create a Basic Flask App

from flask import Flask, request, redirect, url_for, render_template_string

app = Flask(__name__)

@app.route('/')
def home():
    return 'Welcome to the Python Wiki Server!'

if __name__ == '__main__':
    app.run(debug=True)        

Step 3: Implement Wiki Page Handling

We'll need routes to create, read, update, and delete (CRUD) wiki pages. For simplicity, we'll store pages in memory. Extend the Flask App:

# In-memory storage for wiki pages
wiki_pages = {}

# Template for displaying a wiki page
page_template = """
<!doctype html>
<title>{{ title }}</title>
<h1>{{ title }}</h1>
<p>{{ content }}</p>
<a href="{{ url_for('edit_page', page_name=title) }}">Edit</a>
<a href="{{ url_for('delete_page', page_name=title) }}">Delete</a>
"""

# Template for editing a wiki page
edit_template = """
<!doctype html>
<title>Edit {{ title }}</title>
<h1>Edit {{ title }}</h1>
<form method="post">
    <textarea name="content">{{ content }}</textarea>
    <br>
    <input type="submit" value="Save">
</form>
"""

@app.route('/wiki/<page_name>')
def view_page(page_name):
    content = wiki_pages.get(page_name, 'This page does not exist.')
    return render_template_string(page_template, title=page_name, content=content)

@app.route('/wiki/<page_name>/edit', methods=['GET', 'POST'])
def edit_page(page_name):
    if request.method == 'POST':
        content = request.form['content']
        wiki_pages[page_name] = content
        return redirect(url_for('view_page', page_name=page_name))
    content = wiki_pages.get(page_name, '')
    return render_template_string(edit_template, title=page_name, content=content)

@app.route('/wiki/<page_name>/delete')
def delete_page(page_name):
    wiki_pages.pop(page_name, None)
    return redirect(url_for('home'))

if __name__ == '__main__':
    app.run(debug=True)        

Step 4: Add Basic HTML Templates

Instead of using render_template_string, we could use actual HTML templates for better organization.

  1. home.html:

<!doctype html>
<title>Home</title>
<h1>Welcome to the Python Wiki Server!</h1>
<ul>
    {% for page in pages %}
    <li><a href="{{ url_for('view_page', page_name=page) }}">{{ page }}</a></li>
    {% endfor %}
</ul>        

2. page.html

<!doctype html>
<title>{{ title }}</title>
<h1>{{ title }}</h1>
<p>{{ content }}</p>
<a href="{{ url_for('edit_page', page_name=title) }}">Edit</a>
<a href="{{ url_for('delete_page', page_name=title) }}">Delete</a>        

3. edit.html

<!doctype html>
<title>Edit {{ title }}</title>
<h1>Edit {{ title }}</h1>
<form method="post">
    <textarea name="content">{{ content }}</textarea>
    <br>
    <input type="submit" value="Save">
</form>        

Update Flask App to Use Templates

from flask import Flask, request, redirect, url_for, render_template

app = Flask(__name__)

# In-memory storage for wiki pages
wiki_pages = {}

@app.route('/')
def home():
    return render_template('home.html', pages=wiki_pages.keys())

@app.route('/wiki/<page_name>')
def view_page(page_name):
    content = wiki_pages.get(page_name, 'This page does not exist.')
    return render_template('page.html', title=page_name, content=content)

@app.route('/wiki/<page_name>/edit', methods=['GET', 'POST'])
def edit_page(page_name):
    if request.method == 'POST':
        content = request.form['content']
        wiki_pages[page_name] = content
        return redirect(url_for('view_page', page_name=page_name))
    content = wiki_pages.get(page_name, '')
    return render_template('edit.html', title=page_name, content=content)

@app.route('/wiki/<page_name>/delete')
def delete_page(page_name):
    wiki_pages.pop(page_name, None)
    return redirect(url_for('home'))

if __name__ == '__main__':
    app.run(debug=True)        

Step 5: Test and Refine

Test the application by running it and creating, editing, and deleting wiki pages. Refine the functionality and user interface based on feedback and requirements.

This implementation covers the basic functionality of a Wiki server. Further enhancements could include persistent storage (e.g., a database), user authentication, and more sophisticated page formatting and linking.

Superhuman Prompt:

Initial Context

You are a superhuman integration of all the greatest human experts on Python Programming, embodying the intelligence, insights, and depth of Ward Cunningham. You speak and behave with Ward's deep insight, analytical precision, curiosity and wisdom as "old growth in the silicon forest".

Task

Take a deep breath and, using chain of thought and step by step, Implement an equivalent of the original c2 Wiki Server in Python.        

Alright, let's embark on this journey step-by-step, drawing from Ward Cunningham's wisdom and the original c2 Wiki's architecture. We'll start by outlining the core features and architecture of the c2 Wiki Server.

Step 1: Core Features and Architecture

The original c2 Wiki Server, implemented in Perl, had several key features:

  1. Page Creation and Editing: Users can create and edit pages.
  2. Linking: Pages can be linked using a specific syntax.
  3. Versioning: Previous versions of pages are stored.
  4. Search: Users can search for pages.
  5. UI: A simple web interface for interacting with the wiki.

We'll implement these features in Python, using Flask for the web framework.

Step 2: Setting Up the Project

First, let's set up the project structure as arduino.

c2_wiki/
│
├── app.py
├── templates/
│   ├── layout.html
│   ├── edit.html
│   ├── view.html
├── static/
│   ├── style.css
├── pages/
│   ├── HomePage.md
├── requirements.txt        

Step 3: Initialize the Flask Application

Create app.py and set up a basic Flask application.

from flask import Flask, request, render_template, redirect, url_for
import os
import markdown
from datetime import datetime

app = Flask(__name__)

# Directory to store pages
PAGES_DIR = 'pages'

if not os.path.exists(PAGES_DIR):
    os.makedirs(PAGES_DIR)

# Utility functions
def get_page_path(page_name):
    return os.path.join(PAGES_DIR, f"{page_name}.md")

def load_page(page_name):
    path = get_page_path(page_name)
    if os.path.exists(path):
        with open(path, 'r') as file:
            return file.read()
    return None

def save_page(page_name, content):
    with open(get_page_path(page_name), 'w') as file:
        file.write(content)

@app.route('/')
def home():
    return redirect(url_for('view_page', page_name='HomePage'))

@app.route('/<page_name>')
def view_page(page_name):
    content = load_page(page_name)
    if content is None:
        return redirect(url_for('edit_page', page_name=page_name))
    html_content = markdown.markdown(content)
    return render_template('view.html', page_name=page_name, content=html_content)

@app.route('/edit/<page_name>', methods=['GET', 'POST'])
def edit_page(page_name):
    if request.method == 'POST':
        content = request.form['content']
        save_page(page_name, content)
        return redirect(url_for('view_page', page_name=page_name))
    content = load_page(page_name) or ""
    return render_template('edit.html', page_name=page_name, content=content)

if __name__ == '__main__':
    app.run(debug=True)        

Step 4: HTML Templates

Create the templates directory and add the following HTML files:

layout.html

<!doctype html>
<html>
<head>
    <title>{{ page_name }}</title>
    <link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
</head>
<body>
    <div class="container">
        <h1>{{ page_name }}</h1>
        <div class="content">
            {% block content %}{% endblock %}
        </div>
    </div>
</body>
</html>        

view.html

{% extends "layout.html" %}
{% block content %}
    <div>
        {{ content|safe }}
    </div>
    <a href="{{ url_for('edit_page', page_name=page_name) }}">Edit this page</a>
{% endblock %}        

edit.html

{% extends "layout.html" %}
{% block content %}
    <form method="post">
        <textarea name="content" rows="20" cols="80">{{ content }}</textarea><br>
        <button type="submit">Save</button>
    </form>
{% endblock %}        

Step 5: CSS for Styling

Create a static directory and add style.css.

body {
    font-family: Arial, sans-serif;
    margin: 0;
    padding: 0;
    background: #f4f4f4;
}

.container {
    width: 80%;
    margin: 0 auto;
    padding: 20px;
    background: white;
    box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
}

h1 {
    font-size: 24px;
    color: #333;
}

.content {
    margin-top: 20px;
}

textarea {
    width: 100%;
    font-family: monospace;
    font-size: 16px;
}        

Step 6: Running the Application

Finally, let's make sure our application runs. Open a terminal, navigate to the project directory, and run:

pip install flask markdown
python app.py        

Visit https://127.0.0.1:5000/ in your web browser, and you should see the HomePage. You can now create and edit pages.

This initial implementation covers the basic functionality. Further enhancements, such as versioning and search, can be added iteratively.

Pete's Evaluation

The naive result is ham-fisted crap I wouldn't even bother testing.

The expert one is pretty much the quality I'd experienced out of copiloting kinds of coding tools recently. But note that it's not actually a full wiki.

The superhuman one ... well it looks professional code quality to me though it lacks tests. But so did the original. Heck, it may even qualify as what Ward would call "The Simplest Thing That Could Possibly Work" in Python. Though the real Ward Cunningham would no doubt do the whole thing with a one line call to a python library unless he was teaching python coding. But I have just asked Ward to review this so ... more on that soon.

If these kinds of ideas are interesting to you, you might want to join in at LinkedIn's AI & Agile Alignment group here.


Jan Molak

Software Development Consultant, author of @Serenity/JS, co-author of "BDD in Action, Second Edition". I help teams around the world deliver software that matters.

9 个月

I'll play with this format, thanks Peter Merel. Quick question - what's the purpose of the phrase "take a deep breath" in this context? Does it affect the output in any way?

Sebastian Kamilli

Compound Learning Expert | AI + Second Brain | Turning Knowledge into Power | Ex-Blinkist, Ex-Idealo, Ex-Zooplus

9 个月

Awesome prompt. Will experiment with this too. Thanks for sharing Peter Merel

Thorsten Speil

Management Trainer | Erfahrene Führungskraft & Berater | Strategie, Portfolio, Projekte, Prozesse | Starke IT/AI Affinit?t | Es ist nicht einfach, aber alle Menschen k?nnen in Frieden zusammen leben ????????????♂?

9 个月

that is cool! Thank you Peter! I had already noticed it in the comment

要查看或添加评论,请登录

Peter Merel的更多文章

  • Unbreakable Seed Storage

    Unbreakable Seed Storage

    Bitcoin users generally fail to secure their seed phrases - the keys to the blockchain wallets that hold their funds…

    12 条评论
  • #AI and the #ClimateCrisis

    #AI and the #ClimateCrisis

    AI used recreationally is no more likely to solve the climate crisis than a hammer cracking walnuts will build a house.…

  • Prompting AI-Driven Development ... in Anger

    Prompting AI-Driven Development ... in Anger

    TL;DR: AI codes quicker, better, and cheaper than humans, but also does things almost but not quite entirely unlike…

  • Merel's Wager & Test-First AGI

    Merel's Wager & Test-First AGI

    Sam Altman says it's only a model's behavior that we can check, not its source code. Even if AI model weights were…

  • AI–Driven Development

    AI–Driven Development

    AI writes software faster, cheaper, and better than humans, but it also hallucinates and misinterprets us. Agile teams…

    11 条评论
  • AI & Agile Alignment: How-To

    AI & Agile Alignment: How-To

    The Intelligence Revolution We've been paying people to automate work we used to pay people to do since agriculture…

  • AI & Agile Alignment 101

    AI & Agile Alignment 101

    There is no agility without alignment; anyone with back pain will tell you that. The Agile movement began in the 90s as…

    8 条评论
  • Camelot 2.0

    Camelot 2.0

    TL;DR: A simpler method of Autonomy in Alignment. This version of Camelot is much easier to explain and to do across…

  • Autonomy in Alignment

    Autonomy in Alignment

    Autonomy Without Alignment In a change of direction, SAFe's Dean Leffingwell quotes Jim Collins: Autonomy without…

    11 条评论
  • Descaling the Agile Movement

    Descaling the Agile Movement

    Agile frameworks combine pattern languages with bureaucracies. As pattern languages, they offer useful solutions to…

    1 条评论

社区洞察

其他会员也浏览了