What are the most effective ways to test a product's scalability?

A simple question I ask myself is "How many people can relate to this problem?" The more people can find this a problem and searching for value out of my product, then the scalability takes a crucial place, expanding cross-borders and then comes the diversity of the product. Different people from different backgrounds find different value in your product. Your product might not always be used for the intended purpose, it's up to a Product Manager and their team to find that scalable-value.

When developing a product, we always need to make the product generalised, and thus scaleable. This is CS 101 simply put. While learning coding we were taught that our piece of code should work on a million outcomes otherwise it's not successful. The same applies to product development too. If my product breaks after a certain threshold, it becomes a limiting factor for the company's growth. So always test the scenarios during beta, if the servers will handle the load, or the cloud will store the data well, and all the edge cases. This will ensure proper scalability, and make the product efficient.

One thing I have found helpful in defining scalability is formulating extensive end-to-end scenarios for all the success metrics that were defined in the product requirements exploration. For example, if one of the success metrics was a button click percentage, analyzing scenarios leading up to that click will provide metrics for scalability. If the button click was related to a form submission, for instance, we can decide the scalability metrics as the following: 1. Load time of the form 2. Duration to render all form components such as dropdowns, option lists etc. 3. Response time to load dynamic form components (if any) 4. Availability of service handling form submissions 5. Concurrent form submissions success rate ...and so on

I like to view scalability from a resourcing perspective. Can the product cater for an increase in system usage without the need to increase human resources in support of this increase? Human resourcing could be in the form of additional support staff, additional operations staff or more engineering staff.

While load testing tools provide valuable insights, they cannot fully replicate real-world usage patterns or unpredictable user behavior. Sometimes, real-life deployments and user feedback can offer more accurate information about a product's scalability.

Totally. One of the story was with Snapchat. As the number of users grew, Snapchat started to slow down and crash. The team at SnapChat realized that they needed to make some improvements to the app's scalability in order to keep up with the demand. They started by doing all of the above and found the things that were causing the most problems. The story of SnapChat shows that it is important to proactively plan for scalability when developing a product. By identifying and fixing problems of scalibility early on, you can avoid having to deal with major performance problems down the road which could impact your chances.

Repeat and iterate the process One thing I’ve found helpful... Maintain a scalability log. Every time you test and iterate, note down the challenges faced, solutions tried, and results obtained. This provides a historical perspective and aids in future scalability challenges.

Identifying Optimization Opportunities Every system, however advanced, can be refined. Using the discussed methods can boost efficiency. A central focus is cost optimization: as the system load grows, how do costs shift? In fields like AI, as data increases, how do we maintain model accuracy? Metrics are pivotal. They're not one-size-fits-all and should align with the application and its business goals. By selecting and closely watching these tailored metrics, organizations can spot and action necessary improvements effectively.

Usually, companies with a strong internal technical focus will have the scalability focus largely on "technical scalability" instead of "product scalability" early on. This results in way too many resources spent on "future" stuff that usually does not help validate your Product-Market fit early on. The best product companies I've seen work is when there is a concept thats being validated with simple no-code tools that barely hang together, but focus hard on delivering value. After the PMF validation, technical scalability can follow suit.