5 Rules For Mastering AI Assistants With CustomGPT.ai

We love CustomGPT.ai,? it’s enabled us to create a “lab” for testing AI models, with high accuracy results. ?Whenever we demo, people ask, “how did you do it?” ?

?

Leveraging insights from testing over 30 models with hundreds of iterations using CustomGPT.ai to test AI job assistants, here are five rules you can adopt to create accurate, efficient, and effective bots for AI assistant applications, including sales assistants, customer service, marketing experts, or even for specific jobs like ER nurse, solar installer, or first responder.

?

Rule Number 1. Questions are More Important Than Answers

80% of your upfront effort should focus on the understanding the questions your AI must answer. ?The emphasis on problem definition underscores the foundational aspect of any AI model's purpose: solving specific problems or answering particular questions for users.? Before your users ask your model questions, your model needs to know the examples of questions it might be asked.

?

Dedicate a significant portion of your time to meticulously defining the questions you want your LLM to tackle.? This involves understanding the users' needs profoundly and anticipating the variety of ways they might express their inquiries. Straightforward, well-defined questions guide the model in delivering precise, relevant answers, enhancing user satisfaction and engagement.

?

Pro Tips:

• Bring together an “editorial board” of experts to help See Rule 4.
• State your questions in multiple ways to ensure you understand the user intent. Design Thinking “How might we” and Agile user stories (As a.. I Need.. So That..) are good frameworks to use.
• Use the questions as a baseline to test the model as it is being built
• For example, bellow is one problem as described by one of our editors for the Disability Genius model described in two different ways.

• ?

Rule Number 2. AI Without a Board Doesn’t Know Where to Play The Game

This rule speaks to the necessity of setting clear boundaries and definitions around the problem space your AI model operates within. You ensure your model remains focused by understanding and constantly reassessing the boundaries.? Start with the questions from Rule 1, and define the dimensions around the question, such as locations, tools, people, roles, and any other dimension that accurately describes your questions space.

?

Pro Tips:

• Define the dimensional scope as early as possible
• First, decide on big dimensional buckets such as locations, companies, people, roles, products, regulations, etc.
• Instead of describing each dimension, tag each one like a social media post.
• When assembling the model, add these dimensions as part of the “Bill of Materials”, see Rule Number 3.
• Example based on the question in Rule 1

• ?

Rule Number 3. Think Like a Manufacturer, Assemble Your Parts Carefully

Bad raw materials, or too much inventory can sink a manufacturing company and can sink your model too by making it less likely to provide the right answers. ?Each piece of data added to the model needs careful planning, selection, and cleaning.? Only add data at the rate-of-demand;? start slowly and focus on answering your key questions first before adding site maps and large data sets. This approach allows for greater control, transparency, and adaptability, enabling you to refine specific parts of your model without overhauling the entire system.

?

Pro Tips:

• Define a “Bill of Materials” for the components that go into your model. Keep a list of those “parts.”
• Give each data set a part number and add each part to the model one at a time
• Treat data as a set of standardized parts,? for example, Part #001 in all our test models is a Glossary of Terms, which is made up definitions of the tags generated in Rule 2, enabling us to control the vocabulary of the AI.
• Select and vet each component or data source for quality and relevance.

?

Rule Number 4. AI Training is a Multi-Player Game

Embedding human wisdom into the model goes beyond initial programming; it requires a continuous, human-led editorial process to weave the fabric of real-world understanding into the digital realm. The human touch in AI development is indispensable, acting as the compass that directs its growth, ensuring the technology is a trustworthy and effective companion to human users. For example, with Disability Genius, the AI persona has been adapted over 30 times to date with the goal of understanding the nuances of emotions and challenges.

?

Pro Tips:

• Establish an editorial board that continuously adds new questions to the model based on experienced needs or wants, including customers, employees, vendors, or industry experts.
• Use AI Editorial board meetings as an opportunity to expand problem understanding.
• Set a calendar for Editorial board meetings early.

?

Rule Number 5. Your Model is Only as Good as Your Commitment to Improvement

If your model was launched last week, it might already be getting stale.? We see this with our own tests.? Arguably, this is the most challenging rule; continuous improvement is critical to maintaining the relevance and effectiveness of your AI model.

?

This involves technical refinements and staying attuned to your users' evolving needs and problems. Regularly assess the quality of your model’s responses and proactively curate and adjust your model to address new challenges and opportunities.

?

Pro Tips:

• Test your model with questions from Rule Number 1.
• Implement a feedback loop with users to gather insights on your model's performance and areas for improvement.
• For example, set up a Slack channel with the model conversations for the editorial team to review and use the CustomGPT.ai API to send the data over.

?

By following these rules, you can ensure that you not only meet the current needs of your users but are also poised to evolve and improve over time. Interested in learning more, need a template or help getting started?? please reach out and connect.

?