How can you scale software engineering projects for growing user bases?

This calls for multi pronged approach: - Suitable application arch/technologies (micro/serverless/distributed) and deployment models (IaaS/PaaS/Containers/Functions) - Adopting agile and DevOps principles in the SDLC - Shift left practices shall assist delivering with agility/efficiency and performance: Integrate best principles in Arch and design phase itself like: o Use of checklist based on the 15 factor app Design to o hardens and improve security posture. o be resilient (log and monitor capabilities by integrating SDK/API into the app components. o comp that can be developed in a polyglot fashion - Bringing right mix of resources – technical & management - Make use of accelerating technologies (like the DNS based LB, Cache and CDN

To scale a new SaaS project, consider AWS's Serverless Architecture. Use CloudFront for the front end, Cognito for user authentication, Lambda for the back end, Serverless Aurora RDS or DynamoDB, and S3 for storage. These services scale automatically and are cost-effective. For existing projects, scaling tactics vary based on the team's expertise. Options include adding servers behind a load balancer or modularizing the codebase. Often, scaling focuses on specific features, not the entire product, allowing those features to scale as independent services.

In my architectural design process, I find it crucial to establish early estimates for user numbers and system traffic over the next two years. Effective communication of these projections among stakeholders is vital for goal alignment. Once everyone comprehends the objectives, resource planning and feature selection become more precise, aiding in the appropriate tool selection. This proactive approach ensures that your architecture is well-equipped to meet future demands and aligns all team members toward shared goals.

The best goal is to manage the resources via high quality pattern code. The artificial intelligence based projects needs to be assessed in all phases of development regarding resource consumption and auto scale capabilities

Think big! (yes, Amazon cliché) but its real, imagine your service explodes and all of sudden it's been used by thousands of users all over the world, will you be ready to attend the demand? Think maintainability before functionality, think logging even before monitoring. No airplane will carry just enough fuel to reach its destination, what if weather sends it to another airport miles away? Set your goals with a clear objective in mind, but whatever you do always be ready for what will happen after that, can you carry on with your project?

In my experience, clear goals should shape your architecture. It should be simple for easy maintenance yet adaptable and scalable without massive overhauls. Skillful use of open-source tools accelerates implementation. The landscape is evolving, with generative AI becoming a vital element in architectural planning. This addition ensures your system remains dynamic, aligning with AI advancements, enhancing efficiency, and adapting to future challenges seamlessly.

Working for start-ups, many times I've experienced excess development efforts that were invested just to keep the prototype-turned-product going another month. As user base grows (and this sometimes multiplies overnight), the existing technology stack simply can't sustain the load and change rate, and a brave decision needs to be made to shift to a new one.

Stateless scalability is the key, write portable software that you can run anywhere. Your service must be able to scale over night to attend whatever demand comes up. If your solutions sounds too simple they are probably just right, if you start implementing very complex and delicate mechanisms right from the beginning then you might end up reverting a lot of changes in the long run to maintain compatibility. And remember, your solution will just be a part in the users journey, everything must be integrated.

Working on mutiple digital transformations, architecture is very under estimated and overlooked. This might end up in a bigger price by you visioning another transformation right at the end of one.

While DevOps was born over 10 years ago as a set of best practices the Platform Engineering is focused on measurable outcomes like release frequency, time to market, mean time to recover from outage and change-failure ratio.

A crucial lesson I've learned is the importance of not overloading your CI/CD toolset. In a previous company, I encountered a situation where they had over 10 monitoring and logging tools for their SaaS platform. A majority of these tools consumed resources but remained unused. The key is to establish a standard, implement it, and thoroughly document the processes. Furthermore, it's essential to offer regular training to your team to ensure efficient utilization of these tools. This approach streamlines operations, eliminates resource waste, and optimizes the functionality of your CI/CD infrastructure.

In the agile space, do not limit the option for a team in just frameworks. Improve your lean thinking process. Talk daily with the business part of your product in business language. As per practices, I'd suggest pair/mob programming instead of code review, as per my point of view, it's a process that generates waste by design.

What was simple and manageable for when the number of users and interactions was small, no longer holds when the load increases and the QoS demands have to meet a mature solution provider. DevOps systems (when properly maintained and adapted) provide early warning and help in mitigating animalities in the development and delivery chain; without them, the organization might be crippled by its own success.

In my expereience, Agile has helped in identifying problems early and assist business in envisioning the change for better. Show and tell works better than plain old theory based use case creation.

Hiring and on-boarding new team members takes time. Add that to the inherent late response in adapting the diversity to the needs (e.g. adding more customer-facing personnel over product-centric enthusiasts), and you might end up with a silo culture of non-interacting departments and poor customer experience. One of the Agile Movement key mottos is having highly integrated and balanced project teams. I've seen cases where such a team was able to reach solutions to complex issues with just a few "clicks of a mouth" - conversing freely and openly, suggesting alternatives, and committing to a decision once reached.

In my experience it is very important to understand that not all the persons think on being top performers. That was the nature of human beings 20 years ago. Nowadays there are plenty of people that are not that commited to work, and even when it creates unbalanced teams in the motivation perspective, we also need those persons on our teams.

Scaling the projects from this perspective involves not only technical considerations but also building a strong and diverse team. Organized cross-functional teams with members possessing diverse skills (development, testing, design, etc.) will foster collaboration and quicker decision-making.

While building a strong and diverse team, make sure the team is aligned to the same goal. Work closely with the team, check their pulse and strive to build a nurturing environment for them to excel.

Research shows that the most successful organizations in software engineering are best at quick software modifications and the highest quality. The indicator of good software architecture is how quickly it can be modified. How quickly an organization can react to user requirements or competition actions. The key to software engineering scalability is reducing cognitive load related to the process and technology, and allowing engineers to focus on the business problem instead. To achieve that organizations build software development platforms as an internal product. The platform's goal is to improve software engineering efficiency at a high scale treating engineers as its users.

Automate anything, automate everything. But here are a few rules: -Your automation must help save 100 times the time it took you to implement it. Otherwise it will not be worth it, find something more productive to do. -Run your process manually at least once before you even try to automate it. If you can't run it you can't automate it. -Fix and simplify, any problems found during the manual execution, otherwise those problems will become 100x worse once automated. -Don't underestimate automation scripting, this is just like any other piece of code, it must contain validation, edge case and error handling, logging and notifications. Robots can only do what they were programmed to do.

If you include also infrastructure/base services provisioning in your automation technique, you would also be creating part of your disaster recovery strategy

In my experience, priortizing automation and optimization plays a vital role as well. While there may be many areas to automate and optimize, address the low hanging fruits with better ROI.

In 2023 it's time to realize that software engineering is engineering indeed. Neither a business process nor a construction process. Engineering requires prototyping, experiment, measurement, modification and repeating this cycle. In software engineering measurement is testing. The faster the tests are, the more features can be tried and verified.

Software engineering involves resolving many tradeoffs. For example, how much time should be spent on finding and fixing all the bugs vs. shipping early to get feedback from real customers. To resolve this particular question, I have often times implemented the following approach: 1. Make sure there are zero bugs in mainline flows that we expect many users will exercise. 2. Make a judgment call on which additional cases might be important. For example, if consequences for the user are significant (e.g. data loss), those cases need to be fixed as well. 3. Record in telemetry every and each unexpected case that hasn't been fully handled. 4. Ship. After shipping, analyze telemetry and see which of the unexpected cases need fixing. Repeat.

Generative AI tools like GitHub's Copilot and AWS Codewhisperer are redefining software development. Developers, once burdened by repetitive tasks, can now get real-time code suggestions and debugging, boosting productivity. For instance, CodeWhisperer users were 57% faster in a challenge. Moreover, it ensures quality with integrated security scans, flags vulnerabilities, and promotes responsible coding by filtering biased content. Such tools led to a 46% productivity boost in a WestPac experiment. Together, these platforms promise efficiency, security, and cost-effectiveness in the ever-evolving world of coding.

Test automation and production release controls that constrain regressions are key to building trust. It also helps your team focus on features instead of production support. It's not easy but it is efficient

A key driver of efficiency in Software Development Lifecycles is the automation and rapid delivery of products. It is crucial to focus on automating all manual steps involved in building, testing, and deployment. Being able to ship fast gives the development team true agility, ie. the ability to react fast to new requirements, bugs and other challenges. Being able to ship fast and frequently enables the team to deliver value to steakholders on a regular basis, thereby building trust and accountability.

It is vital to collect the data necessary to inform scaling decisions. A few years ago, I led a project that had a sudden uptick in utilization, from a few simultaneous users to several thousand users using the system at once. The thing that saved us was putting enough metrics in place to identify the parts of the system that were preventing scale: seeing the actions users were performing (different than the ones we expected) and where the weak links were (latency and I/O issues are the hardest to measure and troubleshoot), and having enough automation to iterate quickly.

Some of the most common problems I've seen start with a lack of understanding of the user journeys and in some cases how they differ across channels (consumer web vs consumer mobile vs call center/support vs partner APIs). A key insight on this is being able to break down the journey and the internal processes into small independently consumable stories/tasks. Another key aspect that is often an after thought is non-functional requirements such as response times, security standards, compliance requirements (PII/PCI-DSS/WCAG 2/etc), horizontal scalability and resiliency. My final thought on this is understanding what architecture patterns are appropriate where. A simple CRUD service certainly doesn't need to follow a DDD CQRS pattern.