AI Conductors: We Need a Better Term than 'Prompt Engineers'

Engineering is a specific kind of expertise. The way we're using AI is more like making music.

For a hot minute in college, I was a music major. I was a saxophone player in high school band and bass in chorus and thought my talents would best put to work on the concert stage.

At my university, I signed up for everything I could—symphonic band, jazz band, concert chorus, chamber chorus, private saxophone, small ensembles, music theory, and music history. Then, my curiosity got the best of me, and I also started taking classes in literature, linguistics, creative writing, broadcasting, and anything else that fascinated me. I ended up with two degrees in English and a minor in music.

One of my favorite classes was conducting, taught by a happy, exacting band director who showed us that every gesture and facial expression conveyed meaning to the ensemble we led. We had to nonverbally convey the intent of the composer to musicians trained to watch you like a hawk.

Of course, most of this work happens in rehearsal, where the conductor gets to tell stories, ask for improvements, and repeat passages of the musical work as many times as it takes to get it right. Your audience feels your impact as a conductor based on the hundreds of hours you spend building these relationships with the musicians.

Here are the basics of conducting as I understand them:

1. Conducting is the opposite of dance. Instead of responding to the sound of the music, you demonstrate what you’d like the music to sound like a millisecond before the musicians produce it.
2. Imagine a plane in front of your chest. Your hands bounce off the plane in time with the music. The count of one is vertical, two is inward, three is outward, and four is a curling bounce that sets up one again. (There are many variations on these patterns.)
3. If you need the ensemble to be quieter or more precise, the plane rises closer to your face. If you need the ensemble to be louder and more expressive, the plane lowers to waist level.
4. You keep track of everyone’s entrances, make eye contact with those musicians, and offer clear cues.
5. You visually express how notes should start and when notes should end.

There is so much more nuance than this, which is why I was impressed by the conducting skills of both Bradley Cooper as Leonard Bernstein in Maestro (2023) and Cate Blanchett as a fictional conductor in Tár (2022). They showed the absolute command and artistry of that elaborate choreography and the pact each musician must make with in person on the podium.

In each movie, the characters abuse the trust of those who follow them, relationships based on absolute power. I don’t admire these self-destructive behaviors, though I appreciate the drama they produce.

Let’s Call Ourselves AI ‘Conductors’

The term we use for typing instructions into AI chatbots is “prompt engineering,” and I can understand why. The first people to have access to these sophisticated large language models were software engineers. The term stuck as OpenAI, Anthropic, and other companies opened their models to the public.

But thinking of myself as an “engineer” feels disingenuous. I get the same feeling when someone compliments my cooking by saying “You’re a really talented chef.”

Engineers and chefs gain specialized degrees in their fields, and what they do involves heavy documentation, testing, scaled production, and commercial intent. Of course, the same can be said about “conducting,” but it’s possible to imagine enthusiastic amateurs learning the basics. They would be no less “conductors” than someone in a community painting class is an “artist.”

When I work with an AI model, I feel very much like I’m directing an orchestra. Here are some processes I regularly use that take me back to my sophomore year conducting class:

1. Warm ups — Just like musical ensembles need to exercise their muscles and activate the mechanics of their instruments, an AI needs to orient itself to the task through three or four prompts that show me it understands the assignment.
2. Playback — I ask the AI to play back parts of what its writing to make sure it achieves the right tone, rhythm, and level of detail. I might ask it to write the introduction of a blog several times then follow that model for the rest of the piece.
3. Playthrough — I almost always ask the AI to attempt to write an entire piece without stopping. The reason is that LLMs tend to perceive each response as a complete piece and write intros and conclusions, even if you carefully explain that it’s only one section. I find I can avoid repetitive phrasing, redundant examples, and extra filler sentences if I have it do the whole thing.
4. Feedback — I give detailed feedback whenever I am not happy with a response and ask it to let me know if it understands why I’m not happy and give me a version that will make me happy.
5. Practice — I’m constantly practicing the fundamentals and documenting what I learn. These resource guides help me find a new plateau when I begin my next prompt session.

I think the discipline I developed as a musician and wannabe conductor has helped me be a better AI user. I don’t expect perfection on my first try, and I understand that the final performance is a negotiation between a human driver and billions of examples of human work.

My goal to create a performance that feels fresh, alive, and in the moment. I understand that my audience has hundreds of other sources for similar information. To stretch the music metaphor, why attend a concert when I can hear the same song on Spotify?

In my many years as a performer, I’ve seen people pulled from audiences at concerts to conduct orchestras, which perform beautifully on their own while the guest baton-waver does her best not to embarrass herself.

I don’t want to be that kind of AI user.

I want to be a conductor. That means being prepared, expressive, dedicated, in control, and always one millisecond in the future.

This post is 100% human written.