What are the most successful continuous delivery patterns in software development?

A good pattern to use here is blue-green deployments if you can. Where you have 2 environments available behind a load balancer, one with users accessing production and the other with no outside access. You leave the "live" site alone and deploy to the restricted site. You can now test the restricted site safely. Once testing is satisfied you release by flipping the switch to turn the restricted site to production and the production site to restricted using the load balancer.

Automated Testing Trunk-Based Development Feature Toggles Canary Releases Blue/Green Deployment Rollback Capabilities Infrastructure as Code (IaC) Monitoring and Logging Continuous Feedback Loops Environment Parity Artifact Repository Management Decoupled Deployments

Deployment Pipelines, Feature Toggles, Automated Testing and Monitoring Feedback loops these patterns collectively contribute to a mature continuous delivery process, fostering agility, reliability and a faster time-to-market for software releases.

Continuous Integration and Pipeline approach is required this is I think a backbone for most successful continuous delivery patterns which I thought.

In my experience this pattern adds complexity, which have to be handled. Better: 1 Transform tests to automated tests where possible, and include them into regression tests. 2 Try all changes on branches. Combine branches only with what is desired for next release. Test everything on the next release snapshot.

This should also affect how you test. To prevent finding issues when deploying to production, your pre-release testing (usually user acceptance testing) should be done on a deployment from Trunk, not a particular release. This ensures that what you're releasing to production is the accepted solution and no "merge problems" come up after the fact. This means your testing in production after release is now just "Did everything release correctly?" and not feature testing in production.

With a good base of experienced developers always available, with relatively small projects, this can boost up the whole team. However, as the time goes by and projects gain complexity, the stress become higher and higher. I do not recommend this point usually. Exception: where all changes are documented and coverage is 100% per release. Yes, only if 100% coverage can be achieved by the team. 99% is not good enough with this approach.

Each change should have its branch (change request) and must be merged into the production only after the right level of quality assurance: code review, UT/IT, end-to-end test, performance test, security tests, etc. There must be a mechanism to revert the change at any stage of the development, including reverting after deployment to production.

I know that trunk based development is an often used way to maintain code. Anyway I see some drawbacks and is should be weighted if a team should choose it. Daily commits e.g. could put the main codebase in an undeliverable state that could prevent emergency releases e.g. in case of CVEs. In regulated environments it could be required to do reviews of code changes before they reach delivery. The mentioned drawbacks of feature based development in branches could be tackled in other ways as well.

To be able to do that it is important that all the development teams have the same up-to-date local copy of the trunk to work all the changes out without changing the trunk. That source control branch should be part of the packet to deploy?to the buffer I mentioned?in bullet number 1. That way the test team will do all their job deciding if it is acceptable. If it is accepted for promotion, that branch will be sent back to the development team to update the trunk with all the accepted changes in order to only keep one branch. The copy used during development of the new features/fixes will be destroyed.

In my experience while Micro services are great however, we should be careful about why we want to go with micro services. If you are trying to modernize your application and thinking about moving to microservices then do the right analysis. Assess how it will impact your development, security patching, operation cost etc. Evaluate if there is an outage, what it takes for you to deploy all your micro services to bring up the service. You need to plan for monitoring, logging and other infrastructure as needed.

The Microservice architecture with a consistent way of defining the REST APIs (or SOAP, etc) is the best way to progress. However, the architecture must consider a tradeoff between the many microservices and its deployment cost. A good architecture should ensure you use the minimum hardware required to provide the required response time and scalability to ensure the cost is within control. It would help if you considered the ability to load multiple services and move services from one compute machine to another while designing the large system.

Separation for microservices is the key for resilient and scalable system, as each component can be scaled according to it's demand and it also reduces CoGS (by avoiding scaling hybrid component). One "litmus paper" I use for good separation of microservices is testability. Same as we apply for classes in code, we aim for high cohesion and loose coupling, adhering to that makes each microservice testing easier.

I would not agree that scalability and evolvability of a software is directly connected to CD. Hence I would also not directly connect a decision for microservice architecture to the decision doing CD. Both have their needs and both have their drawbacks and not all could be tacked with technology. But entering microservices could be a heavy driver for infrastructure complexity and before a decision one should enlighten all sides of the card.

Microservices is the new trendy way to implement multilayer architectures. In the way nobody know the future, it is important to make the microservices layer absolutly agnostic fo the used technlogy. To do that, is very important to add a "data intercange" layer between layers, for instance using JSON or XML,

Designing automated pipelines that orchestrate the entire delivery process, including building, testing, and deployment. Ensures consistency, repeatability, and traceability in the software delivery process.

Using containers and orchestration in CD ensures consistency and reliability, but the key is integrating them into automated pipelines. This ensures scalability and resilience. Practices like standard images, declarative configurations, and recovery strategies are crucial. While complex, platforms like AWS provide detailed documentation to ease implementation.

Unless there is a reason, the organization must consider developing only containers. Refer to 12factor.net for more about microservice containers. It is assumed that the multi-tenant cloud service must require deployment orchestration, scaling up with a minimum number of initial deployments, and stateless architecture. The organization should also create a disaster recovery and multiple instances in the geographies to ensure high availability. It all sounds tricky; however, most of these are well-documented architecture on the AWS.

Same way to use as many independent layers as possible is very good for the long lasting of the architecture, containers are a good choice. I'm now doing some testing with Event Driven Architectures where the container could be one of the deployed items to one of the "producers-consumers queues" of the EDI (Event Driven Infrastructure). Again, just trying to be "obsessed" getting minimum interdependency between all the involved teams.

These are the ground bricks! How one can even try CD without these? Where quality and user satisfaction are important for each release, these points are like talking and writing, no one needs to think hard about, just do it by reflex (almost) flawlessly.

Establishing a deployment pipeline is crucial in CD, acting as the backbone for delivering quality software. Automating each step, from integration to delivery, is not just a best practice but a necessity to ensure quality and user satisfaction with each release. These pipelines not only enhance agility and reliability but also enable rapid adaptation to changes, key elements for success in continuous delivery.

I have mentioned it before, for big projects/programs it is more important to define promotion buffers. To do not have a well defined pipeline/process is equal to do not be able to delivery anything. Again, I'm running some tests and experiments using Apache Kafka as EDI to implement CI/CD mechanism based on Events and not based on dates/availability.

1. Good observability strategy a. SRE team with the right kind of dashboards to monitor 24*7 b. Strategy for tracing, logging and alerts and strict adherence to it c. The runbook for each likely failure with action to recover 2. Avoid too much dependency on third-party software 3. 100% automation. 4. Create many customer environments and automate customer scenarios in those environments. 5. Ensure an excellent quality “Engine”, just like a plane. 6. Create multiple production pipelines for relatively loosely coupled microservices. The best way to describe it is using the roundabout architecture in the UK and other countries. 7. Dev/QA must be one team reporting to one manager. Everyone owns the Quality.

I like to see every new feature release as an "experiment". This experiment is coded, tested and then made available to a small set of consumers first. If there are no issues found, then we gradually increase the traffic exposure. If issues are observed them we dial-down to zero (not roll back the deployment) and fixes are added.. whole process is repeated again. Key thing is to keep the experiment duration short and automate dial-up/dial-down process as much as possible. Adding human review is not necessary - but sometimes needed when metrics are giving conflicting results. Needless to say - this requires investment in building right instrumentation (metrics) and monitoring to "observe" how experiment is doing.

Selection of the right combination of tools to construct the CICD Pipeline Clearly mark the combination for CI and CD distinctly as this assembly is the most crucial part of laying down any CI and CD pipelines. Both areas demand different set of security, monitoring and management tools that should offer a cohesive array of connectivity between the various stages. One of the reasons of the CICD pipeline breakage is either the tools are not compatible for the array or the sequence of the pipeline is not in the right order and not quite iterative.

Psychologically it’s important to build an environment for the tech team so that they can do fearless deployments without worrying much about adverse impact to customers. This can only be done if the pipeline has all the quality and security measures covered comprehensively.

... @LinkedIn: Where are my responses? I wanted to add few important words to one response, but all my contributions to this suggest are not visible here !?