Building AI Saas: 7 key product decisions

Sharing my notes from a recent session on the key decision points when building a LLM SaaS (Software-as-a-service) offering?—?a cloud-based service that delivers Large Language Model-powered applications and tools over the internet.

• Total read time should be 7 mins.
• Commentary through the Product lens (technical decisions not covered).
• Decisions 1–3 are AI related. My discussion was for conversational/agentic flow, but the points should be valid for non-conversational products as well.
• Decisions 4–7 are applicable for any SaaS offering.

AI-Related Product Decisions

1. Prompt Tune Fine-Tuning or Model Fine-Tuning? If you are pre-PMF (Product Market Fit) and use cases are evolving/data is changing a lot, it is best to take an off-the-shelf model and fine-tune the prompt. This approach is more agile, has lower setup costs, and even running costs have been decreasing (GPT-4 omni is priced at 1/6th that of GPT-4, for instance). However, if the use case is known and proven, using a fine-tuned model would be ideal for further reducing running costs and improving performance. A fine-tuned model trained on proprietary data generates the moat that most enterprises seek, though the initial setup costs are higher.
2. Monolithic Prompt or Multi-Agent/Multi-Tool? A monolithic/single prompt is fine for low complexity scenarios and if you are just starting off/chasing PMF. However, single prompts tend to have high regression issues, and the model’s adherence to instructions decreases as the prompt size increases, leading to higher token costs. A multi-agent setup allows for rapid expansion of scope coverage, reduces regression issues, and improves adherence to instructions since each agent has a focused prompt. While token costs may decrease, the number of calls to LLMs and potential latency due to hand-offs might increase.
3. Which Model to Use? With new models emerging frequently and specialized models (for summarization, translation, answering questions, voice, image generation, etc.) becoming available, it can be challenging to decide which model to use. It is advisable to build a setup where models can be swapped out with minimal regression issues and to break down the offering into modular tasks. A multi-agent or multi-tool setup, where each agent/tool can be powered by a different model, allows changes with limited overall impact.

SaaS-Related Product Decisions

1. Build Narrow or Build Wide? The decision could be to build a Horizontal SaaS (servicing multiple industry verticals, like Salesforce, Slack) vs. a Vertical SaaS (servicing a particular industry vertical like Clio (legal), Shopify (online retailers)). Another consideration is whether to focus on a particular user segment or build wide. In my opinion, understanding the ‘who’ you are building the product for is crucial. If you are at a pre-PMF stage, it is best to build wide and maybe start as a horizontal SaaS, refining towards the user base and offering where you get the most traction. Post-PMF, it could evolve into an offering for a particular segment or vertical.
2. Single Tenant or Multi-Tenant? Single-tenant SaaS is where each customer has a dedicated instance of the software, including its own database and infrastructure, providing greater customization, enhanced security, and dedicated resources, but at higher costs. Multi-tenant SaaS involves multiple customers sharing a single instance of the software, offering cost efficiency, easier maintenance, and scalability, but with limited customization and potential security concerns. For startups, it makes sense to start with single-tenant setups and, after 4–6 implementations, evolve into a multi-tenant setup once a common set of features is established.
3. On-Premise or Cloud Hosted? Some industries (e.g., finance, healthcare) may have strict data security and compliance requirements favoring on-premise solutions. Large enterprises with existing infrastructure may prefer on-premise for control, while SMEs may favor the flexibility of cloud-based solutions. On-premise solutions typically require higher initial investments in hardware, infrastructure, and ongoing maintenance but offer higher customization. Cloud-based solutions are easier to scale, deploy more quickly, and update seamlessly, making them attractive for many industries due to their flexibility and lower total cost of ownership. Consider the competitive landscape and accordingly position your startup in the market.
4. Which Business Model to Choose?

• Subscription-Based Model: Customers pay a recurring fee (monthly or annually) for access to the software. Preferred by Salesforce and Microsoft 365 for predictable revenue and ongoing customer relationships.
• Freemium Model: Basic features are free, with advanced features available through a paid subscription, as seen in Slack and Dropbox. This model attracts a large user base with potential conversions to paying customers.
• Pay-As-You-Go Model: Customers pay based on usage, such as the number of users or data processed, as with AWS, GCP, and Azure. This model offers flexibility and cost-effectiveness.
• Tiered Pricing Model: Multiple pricing tiers with different features and services, targeting various customer segments, used by HubSpot and Mailchimp.
• Per-User Pricing Model: Charges are based on the number of users or seats, making it simple and scalable, as seen with Zoom and Asana.
• Flat-Rate Pricing Model: A single price for all users, preferred for its simplicity and predictability, exemplified by Basecamp.
• Feature-Based Pricing Model: Customers pay based on features used, allowing highly customized plans, as with Zoho CRM and Zendesk.
• Usage-Based Pricing Model: Charges are based on actual software usage, aligning costs with usage, attractive to startups and growing companies like Twilio and Snowflake.

By understanding and leveraging these business models, SaaS companies can tailor their offerings to meet market demands, optimize revenue, and build sustainable relationships with their customers.

Other Key Decisions

There are several other key decisions a SaaS product team needs to consider, such as the right mix of self-service vs. paid/free first and second line support, whether first-line customer support should be outsourced or kept within the team (initially best kept within for valuable feedback), user authentication (SSO vs. custom), mobile strategy (responsive web design vs. native apps), and other technical decisions (data storage, frontend frameworks, versioning, deployment strategies, vertical scaling vs. horizontal scaling, etc.).

In conclusion, building a successful LLM SaaS offering involves a myriad of strategic decisions, from fine-tuning AI models to choosing the right business model and deployment method. Each choice must align with the startup’s goals, customer needs, and industry standards to ensure scalability, security, and user satisfaction. By carefully evaluating these factors, SaaS startups can position themselves effectively in the market and drive sustainable growth.

Hope you found it useful.

If you are also building LLM enterprise Saas applications then it would be great to connect up and compare notes, so do please feel free to shout out or leave your comments.?