登录查看更多内容

How do you design systems to handle unpredictable workloads?

由人工智能和领英社区提供技术支持

How do you design systems to handle unpredictable workloads? This is a common challenge for many developers and engineers who need to build scalable and reliable applications that can cope with changing demands and traffic patterns. In this article, you will learn some key principles and techniques for designing systems that can adapt to unpredictable workloads without compromising performance, availability, or quality.

此文章中的业界达人

由社区从 11 条内容中精选。了解更多

1 Understand the workload

The first step in designing systems for unpredictable workloads is to understand the nature and characteristics of the workload itself. What are the sources and drivers of the workload? How often and how much does it vary? What are the peak and average values? What are the expected and worst-case scenarios? How does the workload affect the system's resources, such as CPU, memory, disk, network, and database? Understanding the workload will help you define the system's requirements, constraints, and goals, as well as identify the potential bottlenecks and risks.

添加您的观点

Garen Y?ndem

Lead Backend Engineer @ Afiniti
举报内容
It might be a time of day or rush-hour when your system will crash most. If you are operating in a limited zone it is easy to handle such scenarios. You can just place a couple triggers and viola! Your system is scaled to n number of nodes. However the tricky part comes when unpredictable loads happen like weather changes. To be prepared for such scenarios, you can make use of machine learning models. You can provide historical weather or traffic data to the model itself and the model can make scalability decisions for you.

已翻译

赞

加载更多内容

2 Design for scalability

The second step in designing systems for unpredictable workloads is to design for scalability. Scalability is the ability of a system to handle increasing or decreasing workload without losing performance or quality. There are two main types of scalability: horizontal and vertical. Horizontal scalability means adding more nodes or instances to a system to distribute the workload. Vertical scalability means increasing the capacity or power of a single node or instance to handle more workload. Both types of scalability have advantages and disadvantages, depending on the system's architecture, cost, complexity, and availability. You should choose the appropriate type of scalability based on your system's needs and trade-offs.

添加您的观点

Saikat Chakrabortty

Infra | SRE | Kubernetes | CKA | Node.js - Fullstack
举报内容
Generally, we see the need for adaptations for horizontal scaling around the market. and when we talk about scalability at this moment pretty much the momentum is towards the Kubernetes. in any case for scalability, it's better to have the warmed instances take up the load or have capacity that way. Have the applications and systems with well-defined metrics which you could use to increase let's say the replicas if there are needs. Having the auto-scaling nodes then adds to the advantage.

已翻译

赞
Rajdeep Saha

Principal Solutions Architect @AWS | Bestselling Author | YouTuber (100K+) | Public Speaker | Tech Advisor
(已编辑)
举报内容
This is a good starting point but real-world systems are more nuanced. When it comes to scaling, there are two main factors - scaling limit (i.e. peak scaling limit. For e.g. how many EC2s we need to scale to horizontally), and scaling rate. The former is well discussed, but the later causes bad user experience if not handled. For example, configuring horizontal scaling is not enough during an exciting product release, where millions of users might try to buy the product at once. You must also ensure the scaling rate (burst) keeps up with user demand. Implement pre-warmed resources (e.g. scheduled scaling, cluster overprovisioner for K8s, provisioned concurrency for AWS Lambda etc.).

已翻译

赞

加载更多内容

3 Implement elasticity

The third step in designing systems for unpredictable workloads is to implement elasticity. Elasticity is the ability of a system to automatically adjust the amount of resources allocated to the workload based on the current demand. Elasticity can help you optimize the system's performance, availability, and cost by adding or removing resources as needed. Elasticity can be implemented at different levels of the system, such as the application, the platform, or the infrastructure. For example, you can use load balancers, auto-scaling groups, or cloud services to dynamically scale your system up or down based on the workload.

添加您的观点

Rajdeep Saha

Principal Solutions Architect @AWS | Bestselling Author | YouTuber (100K+) | Public Speaker | Tech Advisor
举报内容
While scaling is critical, choosing the underlying infra (i.e. what you scale) properly is equally important. For example, you may run a CPU-intensive application in a general purpose VM and keep scaling it. But to achieve the best cost performance, utilize CPU-intensive VMs (i.e. what you scale) for that workload. Find out the ideal metric for scaling (pro tip - it's not CPU all the time!), and choose and scale underlying infrastructure based on that metric.

已翻译

赞

加载更多内容

4 Monitor and measure

The fourth step in designing systems for unpredictable workloads is to monitor and measure the system's behavior and performance under different workload conditions. Monitoring and measuring can help you validate your system's design, identify and resolve issues, and improve your system's efficiency and quality. You should monitor and measure various metrics and indicators, such as the workload volume, the system's throughput, latency, error rate, availability, utilization, and cost. You should also use tools and methods, such as dashboards, alerts, logs, traces, benchmarks, and tests, to collect and analyze the data and feedback.

添加您的观点

加载更多内容

5 Apply feedback loops

The fifth step in designing systems for unpredictable workloads is to apply feedback loops. Feedback loops are processes that enable you to learn from the system's performance and behavior and use the insights to improve the system's design and operation. Feedback loops can help you detect and correct errors, optimize and automate decisions, and innovate and experiment with new features and solutions. You should apply feedback loops at different stages of the system's lifecycle, such as the planning, development, deployment, operation, and maintenance.

添加您的观点

加载更多内容

6 Follow best practices

The sixth and final step in designing systems for unpredictable workloads is to follow best practices. Best practices are proven and recommended guidelines and principles that can help you design and implement systems that are scalable, reliable, and efficient. To achieve this, you should use modular and decoupled components that communicate asynchronously, caching and queueing techniques to reduce latency and improve throughput, stateless and idempotent services that can be easily replicated and retried, fault-tolerance and resilience patterns such as retries, timeouts, circuit breakers, and fallbacks to handle failures and errors, as well as load testing and chaos engineering techniques to simulate and test the system's behavior and performance under different workload scenarios.

添加您的观点

加载更多内容

7 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

加载更多内容

Systems Design

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

How do you design systems to handle unpredictable workloads?

1

2

3

4

5

6

7

1 Understand the workload

2 Design for scalability

3 Implement elasticity

4 Monitor and measure

5 Apply feedback loops

6 Follow best practices

7 Here’s what else to consider

Systems Design

给文章评分

感谢您的反馈

更多Systems Design相关文章

更多相关阅读内容

How do you design systems to handle unpredictable workloads?

1

2

3

4

5

6

7

1 Understand the workload

2 Design for scalability

3 Implement elasticity

4 Monitor and measure

5 Apply feedback loops

6 Follow best practices

7 Here’s what else to consider

Systems Design

给文章评分

感谢您的反馈

查看其他技能