Beyond Basic Chats: Optimizing ChatGPT and Claude for Serious Document Work

I want to share how to better utilize documents and files with ChatGPT or Claude. This guide is designed for helping non-coders who are trying to make sense of their documents without building complex systems or investing a lot of time.

One of the reasons I'm writing this post is that I've seen many people simply put documents or files into a chat, expecting great insight and outputs. In my experience, putting files and documents into a knowledge base with a custom prompt is much more efficient and provides much better results.

This approach does require using a paid ChatGPT/Claude pro account, which costs about $20 a month.

Time to implement: < 20 mins

Open Source vs. Commercial Models and Privacy Considerations

As much as I love open-source models, and despite their progress in reaching the capabilities of frontier models (and being able to run them locally), commercial LLM's like ChatGPT and Claude still dominate in certain tasks.

One of the main advantages of open-source and locally run models is how they ensure privacy and handle sensitive data in your files and documents.

Before getting started with sending anything over the web, I strongly suggest redacting or removing all personal information from files or documents that you're uploading.

As someone who worked in privacy, I think the best approach is to always keep your private and sensitive information to yourself and not rely on anyone else "to do the right thing", and therefor I prefer the "better safe than sorry" approach.

There are many ways to redact or simply manually remove sensitive information (name, ID number, SSN, phone number, address, medical data, financial data, etc.). I won't go through this in this tutorial.

By the way, one of the projects I've been working on is a locally run program that removes sensitive data before sending files to third parties. I might share a GitHub link for this in the future if there's enough demand.

Claude or ChatGPT?

Both are very similar, lately I've found Claude to be a better product in terms of UI, output, and functionality. That said, Claude is limited with things like access to the internet and running code (ChatGPT can do these things), so I find myself alternating between the two depending on the task.

What Problem Are We Trying to Solve?

As you've likely noticed, the longer the conversation with ChatGPT or Claude, the less efficient it becomes. When sharing documents via chat, sometimes it misses crucial information, and often you have to keep reminding it of the purpose of the chat. Additionally, they make things up (known as hallucinations).

Definitions:

Hallucinations happen for many reasons, from limited training data to poor user prompts without enough information, and long, complicated conversations (due to a small context window).

A context window is essentially how much focus the LLM has before it loses attention (this is constantly improving). It relates to tokens, but we won't delve into that here.

The bottom line is that the longer the chat and the more files you put into that conversation, the context window shrinks, resulting in hallucinations and poor output.

When uploading a document to a regular chat, that document becomes part of the limited context window. Creating a custom GPT or Claude project circumvents this as the files/documents are stored separately.

How Does Creating a Custom GPT or a Claude Project Solve This Problem?

The benefit of making a custom GPT or Claude project (they are essentially the same thing) is that you can give them custom instructions and upload the documents before the chat even begins - meaning it's not part of that same limited context window.

They can also reference the added files with better understanding, providing much better results and outputs.

You can always change the custom instructions and knowledge base (uploaded files), and most importantly, you can start unlimited new chats with them.

Now, I'll briefly walk you through how to create either a custom GPT or Claude project (depending on which service you're subscribed to).

ChatGPT: How to Create a Custom GPT

To create a custom GPT, you must have a paid account. Here are the steps:

Go to the "explore GPT" tab and then click "create".

Once you click create, you have two options:

1. Use a chat interface to set up your GPT and create your initial prompt (recommended to start)
2. Set it up completely manually on the config page

Start with the chat interface since it can properly summarize your wants and needs into a better prompt

Then switch back to the manual config to then upload relevant files, change the prompt, and turn on code interpreter

Finish by clicking "Create" on the top right, then choose to make it public or private. In this use case, I recommend making it private as we will use it to evaluate and summarize our sensitive data.

Success!

Claude: How to Create a Claude Project

To create a Claude project, you must have a paid account. Here are the steps:

Click "projects", then click create project.

Claude Projects only have manual configuration, to improve efficiency, I recommend starting a separate regular chat with Claude to describe the functionality of the Claude project you are building in detail, then ask it to create a custom prompt for a Claude project with your input. (Not pictured).

Now add files to your knowledge base via the "Add Content" button and then enter either the prompt you had claude create for you, or one you create yourself into the "set custom instructions" setting.

Note the space available in your project for uploading files changes after each upload (about a gig per project, but don't quote me).

Once you create your project, you can start chatting and your done.

Additional Tips and Pointers

• For ChatGPT, I recommend always turning on "code interpreter and data analysis" as it helps read the documents.
• For both ChatGPT and Claude, after building either a custom GPT or Claude project, keep in mind that you can have unlimited conversations with that tool, and each conversation will start with the custom prompt and uploaded documents.
• You don't need to have super long conversations to maintain context, and I still recommend keeping your conversations limited and starting new ones, as you will still hit a context window (albeit a bit later then before), and ChatGPT and Claude will start to be less effective with time.
• One helpful workaround ive discovered for maintaining conversation context between chats:

1. Ask the LLM to summarize the current conversation before ending it (and starting a new one).
2. Have it extract important points, ideas, future tasks, discarded options, and learnings.
3. Save this summary to the respective knowledge base, usually with a title including the date and "conversation summary".
4. In your new fresh chat, ask the LLM to review the latest updates in the knowledge base, so you're not starting from scratch.

Lastly, I want to mention that a more advanced method of optimizing AI for document processing is to set up RAG (Retrieval-Augmented Generation). RAG enhances the AI's ability to access and use relevant information from a large corpus of documents. However, setting up RAG is beyond the scope of this post, which focuses on simpler, quicker solutions.

Wrapping things up

By creating custom GPTs or Claude projects and following these tips, you can significantly improve how you work with documents and files using AI. This approach helps maintain context, reduces hallucinations, and allows for more efficient and accurate interactions with your data.