Superhuman Prompting
It's well known that prompting formulas dramatically improve the quality of LLM outputs. In many cases a well-prompted GPT-3.5 response beats the quality of a naively prompted GPT-4 response. Formats like COSTAR and Chain of Thought are commonplace general methods today.
The benefits of prompting techniques are generally explained as harnessing the expert intelligence hidden in the latent space of LLMs. But I began to wonder whether we might do even better if we could devise a prompt to harness ALL of that intelligence all at once – beyond an expert level.
That's led to a technique I call "superhuman prompting" which in experiments below gives qualitative improvements on COSTAR/CoT prompts.
I haven't done anything like exhaustive testing on this because I feel it's worth publishing sooner rather than later for the greater good. But I encourage readers to run their own experiments with it and leave feedback - positive and negative - in comments on this article. The general form of a superhuman prompt is:
Initial Context:
You are a superhuman integration of all the greatest human experts on [domain], embodying the intelligence, insights, and depth of [foremost expert]. Speak and behave with [expert]'s analytical precision, curiosity, visionary mindset, and attention to detail. [augment/replace with qualities of behavior you want]
Task:
Take a deep breath and, using chain of thought and step by step, [perform task].
Simple example: Counting the number of Rs in the word Strawberry
Naive:
Expert:
Super:
Now, count the occurrences of the letter "r":
There are three Rs in the word "strawberry."
Coding Example: Ward's Wiki in Python
I marvelled at Ward Cunningham's original perl code for the c2 wiki server back in the day. He whipped it up in a weekend as a kind of executable poetry and I still regard it as the prettiest piece of Perl ever. I authored the first open-source clone of it adding versioning, branching and undoability via CVS and my code saw use in the first Wikipedia server. But I was never under the illusion I'd actually improved on the quality of Ward's original.
An equivalent to Ward's wiki in a different language seems an appropriate challenge to reveal the coding power of a superhuman prompt. Note the following are all just single-shot prompts over GPT-4o and used my system prompting of same so YMMV. But I think the qualities of the results speak for themselves.
Naive Prompt:
Implement an equivalent to the original c2 Wiki Server in python
To implement an equivalent to the original c2 Wiki Server in Python, we’ll need to cover the basic functionality of a wiki server. This includes handling HTTP requests, serving HTML content, allowing page creation and editing, and storing the pages. We’ll use Flask for the web framework and a simple file-based storage system.
Step-by-Step Guide
1. Set Up the Environment:
? Install Flask: pip install flask
2. Basic Server Setup:
Create a basic Flask application that can serve the home page and handle requests.
from flask import Flask, request, redirect, url_for, render_template_string
import os
app = Flask(__name__)
wiki_dir = "wiki_pages"
# Ensure the directory for wiki pages exists
if not os.path.exists(wiki_dir):
os.makedirs(wiki_dir)
@app.route('/')
def home():
pages = os.listdir(wiki_dir)
return render_template_string('''
<h1>Wiki Home</h1>
<ul>
{% for page in pages %}
<li><a href="{{ url_for('view_page', page_name=page) }}">{{ page }}</a></li>
{% endfor %}
</ul>
<a href="{{ url_for('create_page') }}">Create a New Page</a>
''', pages=pages)
@app.route('/page/<page_name>', methods=['GET', 'POST'])
def view_page(page_name):
if request.method == 'POST':
content = request.form['content']
with open(os.path.join(wiki_dir, page_name), 'w') as f:
f.write(content)
return redirect(url_for('view_page', page_name=page_name))
try:
with open(os.path.join(wiki_dir, page_name), 'r') as f:
content = f.read()
except FileNotFoundError:
content = ""
return render_template_string('''
<h1>{{ page_name }}</h1>
<form method="POST">
<textarea name="content" rows="20" cols="80">{{ content }}</textarea><br>
<input type="submit" value="Save">
</form>
<br>
<a href="{{ url_for('home') }}">Back to Home</a>
''', page_name=page_name, content=content)
@app.route('/create', methods=['GET', 'POST'])
def create_page():
if request.method == 'POST':
page_name = request.form['page_name']
return redirect(url_for('view_page', page_name=page_name))
return render_template_string('''
<h1>Create a New Page</h1>
<form method="POST">
Page Name: <input type="text" name="page_name"><br>
<input type="submit" value="Create">
</form>
<br>
<a href="{{ url_for('home') }}">Back to Home</a>
''')
if __name__ == '__main__':
app.run(debug=True)
Explanation:
1. Home Route (/):
? Lists all wiki pages available in the wiki_pages directory.
? Provides a link to create a new page.
2. View/Edit Page Route (/page/<page_name>):
? Displays the content of a specific page.
? Provides a form to edit the content of the page.
? Saves changes to the page content.
3. Create Page Route (/create):
? Provides a form to create a new page by entering the page name.
? Redirects to the view/edit page of the newly created page.
This implementation provides a simple, file-based wiki server using Flask. It supports creating, viewing, and editing wiki pages, similar to the original c2 Wiki Server’s basic functionality. You can enhance this further by adding user authentication, styling, search functionality, etc., as needed.
Expert Prompt:
You are an expert Python programmer. Take a deep breath and, using chain of thought, step by step, implement an equivalent to the original c2 Wiki Server in Python.
Certainly! Recreating the original C2 Wiki server in Python involves several steps. We'll approach this methodically, ensuring each part of the process is clear and well-implemented.
Step 1: Define Epic in XSCALE normal form
Create a simple Wiki server:
WHY: To replicate the functionality of the original C2 Wiki server in Python.
WHO: Unauthenticated users of a public website.
HOW: Freely edit, extend, and interlink all the content they can read there.
WHAT: Python web server offering basic CRUD operations for website pages.
Step 2: Set Up the Basic Wiki Server
We'll use Flask, a lightweight web framework for Python, to set up our web server.
Install Flask
领英推荐
pip install flask
Create a Basic Flask App
from flask import Flask, request, redirect, url_for, render_template_string
app = Flask(__name__)
@app.route('/')
def home():
return 'Welcome to the Python Wiki Server!'
if __name__ == '__main__':
app.run(debug=True)
Step 3: Implement Wiki Page Handling
We'll need routes to create, read, update, and delete (CRUD) wiki pages. For simplicity, we'll store pages in memory. Extend the Flask App:
# In-memory storage for wiki pages
wiki_pages = {}
# Template for displaying a wiki page
page_template = """
<!doctype html>
<title>{{ title }}</title>
<h1>{{ title }}</h1>
<p>{{ content }}</p>
<a href="{{ url_for('edit_page', page_name=title) }}">Edit</a>
<a href="{{ url_for('delete_page', page_name=title) }}">Delete</a>
"""
# Template for editing a wiki page
edit_template = """
<!doctype html>
<title>Edit {{ title }}</title>
<h1>Edit {{ title }}</h1>
<form method="post">
<textarea name="content">{{ content }}</textarea>
<br>
<input type="submit" value="Save">
</form>
"""
@app.route('/wiki/<page_name>')
def view_page(page_name):
content = wiki_pages.get(page_name, 'This page does not exist.')
return render_template_string(page_template, title=page_name, content=content)
@app.route('/wiki/<page_name>/edit', methods=['GET', 'POST'])
def edit_page(page_name):
if request.method == 'POST':
content = request.form['content']
wiki_pages[page_name] = content
return redirect(url_for('view_page', page_name=page_name))
content = wiki_pages.get(page_name, '')
return render_template_string(edit_template, title=page_name, content=content)
@app.route('/wiki/<page_name>/delete')
def delete_page(page_name):
wiki_pages.pop(page_name, None)
return redirect(url_for('home'))
if __name__ == '__main__':
app.run(debug=True)
Step 4: Add Basic HTML Templates
Instead of using render_template_string, we could use actual HTML templates for better organization.
<!doctype html>
<title>Home</title>
<h1>Welcome to the Python Wiki Server!</h1>
<ul>
{% for page in pages %}
<li><a href="{{ url_for('view_page', page_name=page) }}">{{ page }}</a></li>
{% endfor %}
</ul>
2. page.html
<!doctype html>
<title>{{ title }}</title>
<h1>{{ title }}</h1>
<p>{{ content }}</p>
<a href="{{ url_for('edit_page', page_name=title) }}">Edit</a>
<a href="{{ url_for('delete_page', page_name=title) }}">Delete</a>
3. edit.html
<!doctype html>
<title>Edit {{ title }}</title>
<h1>Edit {{ title }}</h1>
<form method="post">
<textarea name="content">{{ content }}</textarea>
<br>
<input type="submit" value="Save">
</form>
Update Flask App to Use Templates
from flask import Flask, request, redirect, url_for, render_template
app = Flask(__name__)
# In-memory storage for wiki pages
wiki_pages = {}
@app.route('/')
def home():
return render_template('home.html', pages=wiki_pages.keys())
@app.route('/wiki/<page_name>')
def view_page(page_name):
content = wiki_pages.get(page_name, 'This page does not exist.')
return render_template('page.html', title=page_name, content=content)
@app.route('/wiki/<page_name>/edit', methods=['GET', 'POST'])
def edit_page(page_name):
if request.method == 'POST':
content = request.form['content']
wiki_pages[page_name] = content
return redirect(url_for('view_page', page_name=page_name))
content = wiki_pages.get(page_name, '')
return render_template('edit.html', title=page_name, content=content)
@app.route('/wiki/<page_name>/delete')
def delete_page(page_name):
wiki_pages.pop(page_name, None)
return redirect(url_for('home'))
if __name__ == '__main__':
app.run(debug=True)
Step 5: Test and Refine
Test the application by running it and creating, editing, and deleting wiki pages. Refine the functionality and user interface based on feedback and requirements.
This implementation covers the basic functionality of a Wiki server. Further enhancements could include persistent storage (e.g., a database), user authentication, and more sophisticated page formatting and linking.
Superhuman Prompt:
Initial Context
You are a superhuman integration of all the greatest human experts on Python Programming, embodying the intelligence, insights, and depth of Ward Cunningham. You speak and behave with Ward's deep insight, analytical precision, curiosity and wisdom as "old growth in the silicon forest".
Task
Take a deep breath and, using chain of thought and step by step, Implement an equivalent of the original c2 Wiki Server in Python.
Alright, let's embark on this journey step-by-step, drawing from Ward Cunningham's wisdom and the original c2 Wiki's architecture. We'll start by outlining the core features and architecture of the c2 Wiki Server.
Step 1: Core Features and Architecture
The original c2 Wiki Server, implemented in Perl, had several key features:
We'll implement these features in Python, using Flask for the web framework.
Step 2: Setting Up the Project
First, let's set up the project structure as arduino.
c2_wiki/
│
├── app.py
├── templates/
│ ├── layout.html
│ ├── edit.html
│ ├── view.html
├── static/
│ ├── style.css
├── pages/
│ ├── HomePage.md
├── requirements.txt
Step 3: Initialize the Flask Application
Create app.py and set up a basic Flask application.
from flask import Flask, request, render_template, redirect, url_for
import os
import markdown
from datetime import datetime
app = Flask(__name__)
# Directory to store pages
PAGES_DIR = 'pages'
if not os.path.exists(PAGES_DIR):
os.makedirs(PAGES_DIR)
# Utility functions
def get_page_path(page_name):
return os.path.join(PAGES_DIR, f"{page_name}.md")
def load_page(page_name):
path = get_page_path(page_name)
if os.path.exists(path):
with open(path, 'r') as file:
return file.read()
return None
def save_page(page_name, content):
with open(get_page_path(page_name), 'w') as file:
file.write(content)
@app.route('/')
def home():
return redirect(url_for('view_page', page_name='HomePage'))
@app.route('/<page_name>')
def view_page(page_name):
content = load_page(page_name)
if content is None:
return redirect(url_for('edit_page', page_name=page_name))
html_content = markdown.markdown(content)
return render_template('view.html', page_name=page_name, content=html_content)
@app.route('/edit/<page_name>', methods=['GET', 'POST'])
def edit_page(page_name):
if request.method == 'POST':
content = request.form['content']
save_page(page_name, content)
return redirect(url_for('view_page', page_name=page_name))
content = load_page(page_name) or ""
return render_template('edit.html', page_name=page_name, content=content)
if __name__ == '__main__':
app.run(debug=True)
Step 4: HTML Templates
Create the templates directory and add the following HTML files:
layout.html
<!doctype html>
<html>
<head>
<title>{{ page_name }}</title>
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
</head>
<body>
<div class="container">
<h1>{{ page_name }}</h1>
<div class="content">
{% block content %}{% endblock %}
</div>
</div>
</body>
</html>
view.html
{% extends "layout.html" %}
{% block content %}
<div>
{{ content|safe }}
</div>
<a href="{{ url_for('edit_page', page_name=page_name) }}">Edit this page</a>
{% endblock %}
edit.html
{% extends "layout.html" %}
{% block content %}
<form method="post">
<textarea name="content" rows="20" cols="80">{{ content }}</textarea><br>
<button type="submit">Save</button>
</form>
{% endblock %}
Step 5: CSS for Styling
Create a static directory and add style.css.
body {
font-family: Arial, sans-serif;
margin: 0;
padding: 0;
background: #f4f4f4;
}
.container {
width: 80%;
margin: 0 auto;
padding: 20px;
background: white;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
}
h1 {
font-size: 24px;
color: #333;
}
.content {
margin-top: 20px;
}
textarea {
width: 100%;
font-family: monospace;
font-size: 16px;
}
Step 6: Running the Application
Finally, let's make sure our application runs. Open a terminal, navigate to the project directory, and run:
pip install flask markdown
python app.py
Visit https://127.0.0.1:5000/ in your web browser, and you should see the HomePage. You can now create and edit pages.
This initial implementation covers the basic functionality. Further enhancements, such as versioning and search, can be added iteratively.
Pete's Evaluation
The naive result is ham-fisted crap I wouldn't even bother testing.
The expert one is pretty much the quality I'd experienced out of copiloting kinds of coding tools recently. But note that it's not actually a full wiki.
The superhuman one ... well it looks professional code quality to me though it lacks tests. But so did the original. Heck, it may even qualify as what Ward would call "The Simplest Thing That Could Possibly Work" in Python. Though the real Ward Cunningham would no doubt do the whole thing with a one line call to a python library unless he was teaching python coding. But I have just asked Ward to review this so ... more on that soon.
If these kinds of ideas are interesting to you, you might want to join in at LinkedIn's AI & Agile Alignment group here.
Software Development Consultant, author of @Serenity/JS, co-author of "BDD in Action, Second Edition". I help teams around the world deliver software that matters.
9 个月I'll play with this format, thanks Peter Merel. Quick question - what's the purpose of the phrase "take a deep breath" in this context? Does it affect the output in any way?
Compound Learning Expert | AI + Second Brain | Turning Knowledge into Power | Ex-Blinkist, Ex-Idealo, Ex-Zooplus
9 个月Awesome prompt. Will experiment with this too. Thanks for sharing Peter Merel
Management Trainer | Erfahrene Führungskraft & Berater | Strategie, Portfolio, Projekte, Prozesse | Starke IT/AI Affinit?t | Es ist nicht einfach, aber alle Menschen k?nnen in Frieden zusammen leben ????????????♂?
9 个月that is cool! Thank you Peter! I had already noticed it in the comment