ClaudePS: A Prompting Tool for Claude Sonnet

If you are, like me, an extensive user of Claude Sonnet 3.5, you create multiple projects, each having dozens of conversations, which contain numerous messages and generated artifacts.

So it's no surprise that, days later, I sometimes finding myself look for that specific crafty prompt, enlightening response, or that important artifact that I need following a sudden mental flash, but after rooting around for twenty minutes, give it up. That's when I think: "Why does this brilliant LLM have such a *** UI that there isn't even a search function!"

Another thing that happens is, that when I've had a productive session and possibly generated 50 artifacts or more, I now have to go download them item by item, as if Anthropic thought I would just create one or two of them.

It's clear someone forgot some essential functions!

Introducing ClaudePS

Since I started this series to explore creating code with AI, I got tired of the drudgery of Claude Projects, yet I was condemned to use them because Claude Sonnet 3.5 is just that good.

Finally, one fine day (October 24, 2024 at 1:22 PM to be precise; how do I know? ClaudePS!), I started down an interesting rabbit hole with a simple prompt:

A few weeks later, working off again, on again, with the help of my faithful Coding assistant, I had a tool that had a number of interesting features for my work, and which I have made available as open source for those that want to download it and try it / extend it. The code being entirely generated by AI, this is also illustrative of what that might look like, although there are a few idiosyncrasies particular to writing a Chrome extension.

The code is available here:

PromptScript

I originally started this project in order to implement PromptScript, a language dedicated to the idea that english is the new scripting language and the LLM a kind of operating system. Having a tool to automate tasks such as typing "please continue" whenever the LLM is waiting for permission in a long response seemed a natural thing to do, and a bit of a challenge. So basically this was a language that did one thing!

This required interacting with the UI hence the Chrome extension approach. I had never been interested in creating a Chrome Extension, so I knew nothing about it. Claude's knowledge of Chrome Extension helped me get the basics going without reading up on anything or studying any code. But as expected, I could not live in the bliss of ignorance when it came time to doing downloads and other useful functionality, as on its own the project started getting too big and Claude took it in unwanted i.e. non-functional directions.

Claude generated a document for the PromptScript language that you can read here:

Here's an example of a simple PromptScript script in which I ask Claude to count to three and tell me when it's done. This is a trivial example but in theory Claude could be generating a series of documents or doing other task, with just a simple approval needed between each. PromptScript gives you the opportunity to go get a coffee while waiting for Claude to tell you it's done.

"Simple" Mode

From the one-command "joke" language, I started adding additional functions as the Chrome Extension's access to make my life easier, so don't be started to see a bit more than one command in the document...

In fact, I figured casual users (i.e. anyone other than computer language nerds like me!??) would probably not want to learn PromptScript so I created a "simple" mode to give direct access to the main functions without having to drop down into the low-level prompt engineering of PromptScript!

In fact you could live your entire life without ever seeing, or even hearing about, PromptScript! But if you really must, click on the "Command Mode" icon.

From there you can create scripts and actually access all of the commands in "simple" mode, plus more...

Trying it out yourself

Because this is just an experiment, I was too lazy to publish this through the Google Chrome store, but you can get it from GitHub pre-built:

Installing from GitHub Release

1. Go to the Releases page
2. Download the latest claudeps-v*.zip file
3. Unzip the downloaded file
4. Open Chrome and navigate to chrome://extensions/
5. Enable "Developer mode" in the top right corner
6. Click "Load unpacked" and select the unzipped directory

Note: When installing from GitHub, Chrome may warn about developer mode extensions. This is normal for extensions not distributed through the Chrome Web Store.

Exploring a Project

The Project Chat Function

When you're in a project you can use the Project Chat function to generate... a list of chats! Which you already have. But once you have them, you can use a few interesting functions not available in the Vanilla Claude web app.

Sort Your Chats

Brain the size of a planet but unable to sort chats, or even tell you when then happened!

Luckily ClaudePS has these advanced features!

Yes, it's 2024 and any self-respecting app would have the dates and the sorting - but Claude doesn't, but ClaudePS does - so there!

Download Project Chats

Sometimes you want to take your chats and work on them with other tools. Too bad! Unless you have Claude PS: then you can use the "Project Chats" function to select one or more conversation and download them as markdown files. (And from there I use Obsidian but there are plenty of Markdown editors you can use).

You can download each file individually, or bundle them up in a single file. And if the default name provided by Claude isn't the best - you can change it! or include a folder path.

Project Knowledge

The Knowledge function generates a list of the files in the Project Knowledge. Now a lot of times these are files you uploaded - but I find it convenient to promote generate artifacts to the Project Knowledge in case there are multiple versions of the same artifact and you want to identify the one you want to keep.

Also this function lets you download all of the files as a bundle if so desired.

Preview

If you're not quite sure what's in a particular Chat, you can use click on the "preview" icon (shaped like an eye) and load up the messages without having to navigate to a new page, discover it's the wrong conversation, then navigate back, with a few seconds to load each page... This is a zippy interpretation that even includes the markdown formatting!

The heading of the preview window gives you interesting stats and allows you to filter on the messages with artifacts in case you're trying to find one amidst the lighthearted banter that clutters up the conversation.

Favorites

You can start a particular message you want to remember and assign a custom label:

You can access the favorites from the top menu, and display either all starred or just the ones with a particular label:

You can use the Summarize to generate a prompt that will summarize the displayed messages and create an artifact with the summary in your current chat, which can be useful if you're trying to import information from previous chats. You can change the prompt to focus on what it most important in the current conversation:

The summary gets generated in your current chat, or you can start a new one:

Project Search

As mentioned at the start of this discussion, a typical project has numerous chats, messages and artifacts. It's quite clear early on in your use of Claude that you NEED a search!! Browsing through various chats and message looking for the insightful.

So of course ClaudePS has an AI search, powered by Claude itself! When you click on Seach, a prompting window opens allowing you to provide a description of what you're interested in. Clicking on the hourglass kicks off a chat-by-chat search, the results of which progressively appear in the conversations list:

There's a quick summary of what it found directly in the table including number of matched messages, but you can click on "view details" to see the matches in more detail:

And if that's not enough, you can drill down to the actual message:

Message Exhaustion / Automatic Retry

Depending on how many busy the server is, how many tokens you've used in your prompts and repsonses, how may messages, the position of the moon in the zodiac, sooner or later you will see the "Message limit reached" message and you will be blocked.

If you were in the middle of running a script, ClaudePS will check periodically to see if Claude is back and continue where it left off. This allows you to take advantage of messages becoming available at inconvenient times such as 3 AM!

Technical Design

All of the code and the major decisions on project structure and module patterns were in fact mostly determined by Claude. Some of it is a bit quirky, but overall it's pretty easy to maintain and enhance.

Unit tests were created, but they are not very effective because a significant amount of mocking would need to be done to truly have the tests be isolated from API calls and the UI environment.

Claude generated the following technical description:

# Claude Extension Architecture

## Project Structure

The project consists of 54 source files (excluding tests) organized into the following main directories:

- `src/`: Main source code (54 files)
  - `services/` (13 files): Core business logic and API services
    - AliasService.ts
    - AnthropicService.ts
    - ChromeSettingsService.ts
    - ClaudeCache.ts
    - CommandExecutor.ts
    - ConversationRetrieval.ts
    - DocumentDownload.ts
    - DocumentRetrieval.ts
    - ProjectRetrieval.ts
    - ProjectSearchService.ts
    - ScriptExecutionManager.ts
    - ScriptRunner.ts
    - SettingsService.ts
    - StarService.ts
    - StorageService.ts
    - WindowStateService.ts
  - `ui/components/` (11 files): UI-related components
    - ContentPreview.ts
    - ConversationPreview.ts
    - DownloadTable.ts
    - DownloadWidgets.ts
    - DraggableManager.ts
    - EditableCell.ts
    - EditableLabel.ts
    - FloatingWindow.ts
    - HelpManager.ts
    - SearchResultPreview.ts
    - StarredItemsPreview.ts
    - StatusManager.ts
    - UIStateManager.ts
  - `utils/` (21 files): Utility functions and command handling
    - Commands/ (10 files)
      - aliasCommands.ts
      - BaseCommandInfo.ts
      - CommandMap.ts
      - contentCommands.ts
      - knowledgeCommand.ts
      - projectCommands.ts
      - repeatCommand.ts
      - rootCommand.ts
      - settingsCommand.ts
      - simpleCommands.ts
      - stopConditionCommands.ts
    - createConversation.ts
    - deleteConversation.ts
    - ElementWaiter.ts
    - EventStreamParser.ts
    - getClaudeIds.ts
    - getConversation.ts
    - getHeaders.ts
    - parseUtils.ts
    - PathExtractor.ts
    - pathUtils.ts
    - requestCompletion.ts
    - ScriptParser.ts
    - splitText.ts
    - trace.ts
  - `types/` (2 files): TypeScript type definitions
    - index.ts
    - storage.ts
  - Root files (7 files):
    - ClaudeExtension.ts
    - analyzeConversations.ts
    - background.js
    - content.ts
    - manifest.json
    - options.html
    - options.js
    - styles.css

## Organization Principles

The codebase follows these key organizational principles:

1. **Service-Based Architecture**: Core functionality is organized into services (e.g., AnthropicService, StorageService)
2. **Component-Based UI**: UI elements are built as reusable components
3. **Command Pattern**: Extensive use of command pattern for handling user interactions
4. **Type Safety**: Comprehensive TypeScript types defined in `types/` directory
5. **Test Coverage**: Parallel test structure matching source files

## UI Implementation

Due to Chrome extension security restrictions preventing direct CDN downloads, the UI is implemented with:

1. **Vanilla TypeScript**: No external UI frameworks
2. **Internal CSS**: Styles defined in `styles.css`
3. **Custom Components**:
   - `FloatingWindow.ts`: Base window component
   - `DraggableManager.ts`: Window drag functionality
   - `EditableLabel.ts`/`EditableCell.ts`: Inline editing components
   - Various preview components for different content types

## API Integration

The extension handles several types of API calls:

1. **Anthropic API Calls** (`AnthropicService.ts`):

   - Conversation creation/deletion
   - Completion requests
   - Event stream parsing for responses

2. **Chrome Extension APIs**:

   - Storage access
   - Tab management
   - Message passing between components

3. **Internal Service Calls**:
   - Document retrieval
   - Project search
   - Settings management

## Data Persistence

Data persistence is handled through multiple layers:

1. **Chrome Storage** (`StorageService.ts`):

   - Conversation history
   - User settings
   - Project data
   - Command aliases

2. **Window State** (`WindowStateService.ts`):

   - UI positions
   - Visibility states
   - Current selections

3. **Cache Layer** (`ClaudeCache.ts`):
   - Temporary conversation data
   - Performance optimizations

## Command System

The command system is built around several key components:

1. **Command Parsing** (`splitText.ts`, `ScriptParser.ts`):

   - Handles quoted and unquoted text
   - Parses command arguments
   - Supports nested commands

2. **Command Types**:

   - Simple commands (`simpleCommands.ts`)
   - Content commands (`contentCommands.ts`)
   - Project commands (`projectCommands.ts`)
   - Knowledge commands (`knowledgeCommand.ts`)
   - Settings commands (`settingsCommand.ts`)
   - Alias commands (`aliasCommands.ts`)

3. **Command Execution** (`CommandExecutor.ts`, `ScriptExecutionManager.ts`):

   - Command validation
   - Argument processing
   - Error handling
   - Script execution flow

4. **Command Mapping** (`CommandMap.ts`):
   - Command registration
   - Command lookup
   - Help text generation

The command system supports:

- Nested command execution
- Variable substitution
- Error handling and recovery
- Command aliasing
- Script execution
- Stop conditions

This architecture enables powerful command chaining while maintaining readability and maintainability through clear separation of concerns and comprehensive testing.
        

Code Review

In general, I found the names were fine, the directory structure workable, although some things were misnomers. Also some of the descriptions in the "architecture document" are a bit of hyperbole - for example the so-called "services" are just functional modules, though most of them use web services. There's also a line vaunting the power of typescript, which is really tablestakes in this day and age.

UI

The general appearance is minimalist but kind of cool!

I'm not too crazy about the UI coding approach, we would end up with code like:

  private createDialog(): void {
    this.dialog.style.cssText = `
        position: fixed;
        top: 50%;
        left: 50%;
        transform: translate(-50%, -50%);
        width: 800px;
        max-width: 90%;
        max-height: 80vh;
        z-index: ${ContentPreview.baseZIndex};
        display: none;
      `;

    this.content.style.cssText = `
        background: ${this.colors.background};
        border-radius: ${borderRadius};
        width: 100%;
        height: 100%;
        display: flex;
        flex-direction: column;
        box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15);
        position: relative;
      `;

    this.header.style.cssText = `
        padding: ${spacing.md};
        border-bottom: 1px solid ${this.colors.border};
        cursor: move;
      `;

    this.dialog.appendChild(this.content);
    document.body.appendChild(this.dialog);

    document.addEventListener("keydown", (e) => {
      if (e.key === "Escape" && this.isOpen) {
        this.close();
      }
    });        

Crude. But effective!

I did struggle trying to load the CDN version of tailwind, and later some of the other packages. I didn't want to setup the whole postCSS build process used to support tailwind directly and anyway I figured the whole thing was just going to be a text box for the script and a text box for the output- I didn't think I would add the download table, preview windows etc. On the whole though I hardly ever had to fiddle with that code so it looks like Claude has a very good handle on it.

Prompting Claude

In order to prompt Claude, I decided to just find the text box where humans type and push the prompt text into its content, then send an "Enter" key. Crude but it effective!

Then there was a process to wait until the full response was received by periodically checking for the absence of an element with the style indicating it was in the process of being generated. I fully expect this to break in the coming weeks or months!

Claude API calls

There is no way to access Chats and Artifacts through regular APIs, however the Claude web UI uses RESTful APIs which I reverse engineered using LLM help. The interesting thing about this approach is that it allows me to bypass the entire problem of authentication because the extension is already running in a logged-in Claude app, so it will naturally send all of the cookies including the authentication stuff to Claude.

The only exception is the optional use of the official Claude completion API for searching, because I found that the number of messages it required to search through multiple chats would use up all the messages. So if you're willing to spend $025 per search, it might be worth it! I save the API key via a custom property page in the Chrome Extension settings.

From the browser Dev Tools, I found the APIs of interest by looking at what they returned, then turned to ChatGPT to transform the headers and payload into a fetch call I could try out immediately in the javascript console. This allowed me to experiment a bit. I used ChatGPT because this type of code generation is very standard and I saved a few messages from the paltry allowance we get in Claude!

API Cache

Once messages are exchanged with Claude, they never change so I decided to cache them in Browser local storage. At first I tried using the Google Chrome Extension property storage, but frequent updates to this tended to cause failures - I guess it's a bit flaky and better to use for saving values that don't change frequently. Also there's a much smaller size limit than Browser local storage.

I invalidate the conversations based on a timestamp retrieved from the project information, in case new messages are added, in which case the conversation is fetched again. This significantly reduced the number of API calls the add-in was making, which did have me worried at first - I didn't want to be booted off of Claude because of abusing their APIs!

Command Processing

Claude suggested the Command Map pattern, with a split between Parsing code (for the scripting!) and execution code. To be honest, I had to push it a lot to improve the structure of the code and make it easy to add new commands. Currently there's a one-stop-shop class structure that encapsulates command concerns including command-specific parsing and execution in the same file, making it easy to correlate the two.

Nevertheless the requirements were probably a bit unusual compared to the typical web app or extension, so I give it kudos for the attempt which was a good starting point, and for being shapeable into an easy-to-maintain organization that also allows me to supply a dynamic list of buttons in "Simple" mode that vary according to the context i.e. show only project buttons at the project level, add some chat buttons at the chat level, and hide all buttons outside of any project i.e. at the Claude welcome page.

Tests

A number of unit tests were created by Claude. It was somewhat helpful but I think this is one area that needs improvement, compared to some of the other projects I am working on. I think the particularity of working in the Chrome Extension constraints, and of reverse engineering UI behavior, make it less obvious/worthwhile to develop a lot of tests. Basically we'd need to create some sophisticated mocks of the Claude APIs and DOM - a big project unto itself. Considering the shelf life of this extension, it didn't seem worth it to drill deep on this.

Documentation

The code has consistent if minimal comments. The PromptScript manual was written by Claude with my edits. It was great to be able to update the document once in a while based on the latest state of the code. I also had it reverse engineer the "architecture" description. This is truly an obvious win for developers as it's tedious and time consuming so it tends to be avoided or not maintained.

Final Impression

Pretty, pretty good as Larry David might say. Crude. But Effective.

The end result is a half-decent app built part-time in a few weeks, easy to extend and maintain for a human or for AI, and packs a fair amount of functionality in an architecture constrained by Google Chrome Extensions restrictions. I was able to use to download code, find obscure messages, and even avoir having to type "Please continue" over and over. Not bad for a whim! I might use it to provide some of the missing MCP client features that Claude doesn't have yet! We shall see...

Building it had moments of fast-paced firehose code generation, alternating with soul crushing Chrome Extension limitations followed by elation when Perplexity or stack overflow can to the rescue. The fact that the Chrome Extension API is a cousin of the Android API makes things worse because an LLM can get confused between the two and suggests things that work for Android but not Chrome.

Another issue is when the LLM decides to mangle functionality to do "Improvements" - working with source control is essential here in order to quickly restore large chunks of code randomly missing. The AI giveth, and the AI taketh away... so you have to be vigilant. Some of my frequently used phrases: "What's this??" "WHY are you doing this?" "NO!!! Bad AI! Do not remove half the code! Do not change the code to pass the failing tests!"

So there's a mix of joy and frustration involved, although I learned to structure my prompts to avoid creating openings that led to going off the rails. This also confirmed the importance of Test-Driven Development because the strange behavior of the AI can strike at any time, so having the tests suddenly fail because half a file is gone is helpful. Unless the AI changes the test too! So vigilance is key.

Of course the whole thing will be obsolete in six months if Anthropic gives us a much improved UI with some of these basic functions... but we'll always have PromptScript!

But this demonstration pushed me to using it for even more substantial development efforts, including a Dev Tools MCP server that completely changed my AI Coding workflow by unleashing Claude into the code tractor's seat, coming up soon in another instalment of "Me and my AI Coding buddy!" A few months ago I had my doubts - but now I truly am a believer that AI Coding is becoming as essential as debuggers, source control, profilers, linters, and other software engineering tools...

As someone who uses Claude a lot and hates its UI limitations, Martin Béchard had great satisfaction in using it to create a tool that deals with his pet peeves. If you need help using ClaudePS or want to contribute some AI-generated code to ClaudePS, please reach out!