Cloud costs - areas, problems and solutions

Part 2

In the Part 1 of the series I focused on defining what is a cost effective cloud system and explained how to match supply and demand for cloud resources and how to use cost-effective resources. Let's continue to use the problem-and-its-solution approach to explore the topic further.

Expenditure Awareness

Building an expenditure awareness is often a first step to successful cloud cost management. For many IT professionals it may be something they have never done before, so let's start with very simple solutions: setting a budget, creating notifications, cost exploring and reporting. More advanced steps are cost classification, enforcing tagging, testing for cloud costs and using cost KPIs.

Problem: Notifications about potential exceeding planned cloud costs

We have a limited budget for our cloud resources and we would like to be notified when there is a risk of exceeding the budget

Solution:

Use AWS Budgets to create cost & usage budgets, then set alarms and e-mail notifications when actual or forecasted costs and usage exceed defined thresholds. Budgets can be set per account, service or services, cost allocation tags or combinations of them.

Problem: Cost exploring

We would like to have a simple visibility into costs of AWS resources, be able to group costs by defined dimensions or focus on exploring only interesting categories.

Solution:

Use Cost Explorer to examine costs from many angles. There are many dimentions and filtering options that you may use, for example:

• Group or filter costs per service, AWS region or AWS account
• Filter out tax, refund or discounts so you see the full cost of used resources
• Use different granularity: daily or monthly
• Explore costs for the last day, month, year or a custom period
• Explore costs of resources tagged with a particular set of tags

Cost Explorer reports can be saved and reviewed later on.

Problem: Regular insights about costs in a structured form

Our teams would like to receive cost reports in a structured and defined format on a regular basis.

Solution:

• Define a regular data export to an S3 bucket in a desired format (CSV, parquet)
• Use tools like Amazon Athena to query cost data. Use Amazon QuickSight or any other Business Intelligence Tools to create sophisticated cost reports.
• Use spreadsheet - For some organizations it is enough for cost reporting.

Note: cost and usage data include many information like AWS accounts, usage of security services etc. Be careful when sharing raw cost and usage reports with people within your organization.

Problem: Allocation of costs to business units, teams, projects & activities

We run many applications in the AWS cloud that are used by different business units, teams, projects. We would like to allocate costs to these units, teams & projects and our activities (development, testing, production usage)

Solution: Use cost allocation tags

Examples of tags:

• Business unit
• Project owner
• Project name
• Team name
• Stage: dev, test, uat, pre-prod, prod etc.
• Function: application, monitoring, cicd, security etc. - to group resources serving a particular purpose

Then use AWS Cost Explorer to present costs grouped by tags.

Note: tags used for cost allocation needs to be activated in the Billing and Cost Management section of the AWS Management Console.

?

Problem: Enforcing cost allocation tagging

We understand that resource tagging helps to manage costs & environments, we implemented resource tagging policies, but some team members do not comply to these policies

Solution:

• Include check for proper resource tagging in your CI/CD processes
• Enforce tagging in terraform or CloudFormation templates
• Use tag policies in AWS Organization or enforce tagging via Service Control Policies
• Use AWS Config rules to detect untagged resources
• Trigger notifications about untagged resources
• Remove resources that do not have a required set of tags?

Problem: Justification of increased cloud costs

Our cloud costs constantly increase and our financial department is preoccupied with cloud costs “getting out of control”. At the same time we observe growth in usage of our applications and we have recently hired more developers. How can we justify growth of cloud costs?

Solution:

Design business KPI relevant to your business. Examples: cloud costs per application user, cloud costs per developer. Track KPIs values over time. If cloud costs per application user are stable or decrease, there is a good chance that your financial department will accept it.

Problem: Testing cloud applications for cloud costs

We are aware that cloud costs is a complex issue and we would like to assess the real costs of resources as accurately as possible

Solution:

Launch load tests that will resemble production load and gather data about cost & usage using AWS Cost Explorer API. Compare the tests results with your assumptions.

Security and costs

When using cloud, we pay what we use - even if we did not intend to do so. Some cloud cost are effect of cloud security incidents or misconfiguration.

Problem: Protection against incurring cloud costs due to security issues

How can we protect ourselves from incurring cloud costs that result from security issues or misuse of our AWS accounts?

Solutions:

• Follow cloud security principles, including the least privilege principle (i.e. granting your users, roles & AWS services only necessary permissions) or multiple security layers
• Use security tools and services - like AWS Guard Duty - for detection of suspicious behaviour on your accounts (like mining cryptocurrencies)
• Use AWS Shield to prevent DDoS attacks that would result in scaling out your resources due to increased traffic
• Protect your public APIs and their backend services against excessive usage by setting usage plans
• Avoid committing access keys to code repositories, especially public ones to prevent taking over your AWS account, resulting in spinning up resources you did not plan to use.

Problem: Dealing with unwanted cloud costs after a security incident

One of our developers committed AWS access keys to a public repository. These keys were used to launch EC2 instances in many regions and our company has to pay for AWS resources

Solution:

Contact AWS support, describe the situation and ask for a refund. There is a chance that your request will be accepted.

Cost Optimization

Increased cost awareness makes sense only when it finally leads to optimize cloud systems.

Problem:

How to ensure that cost optimization will actually take place?

Solution:

Perform regular cost reviews, create backlog of cost optimizations and implement them - ideally as a part of your cloud infrastructure development

The goal of the cost review is to look for possible cost optimizations - having in mind your specific requirements and other pillars of well architected cloud solutions. Ideally data for the review should be gathered automatically and then analyzed together with a Cloud Solutions Architect.?

The review may include:

• Report summarizing cloud costs all AWS accounts in your organization and per account using AWS Cost Explorer - for the last 12, 6 or 3 months
• Report summarizing monthly costs per service and a list of top N services where costs are the highest, services where costs exceeded a given amount or cost percentage in the last 12, 6 or 3 months and services with sudden or unexpected cost increase.
• Detailed reports summarizing monthly cost usage per service and usage type for the selected services
• Optionally: more granular / filtered reports: daily costs and usage - per service or reports of costs and usage per tag
• For the selected services: analysis of resources metrics, review of pricing models and possibilities to exchange for newer solutions or managed services
• Identification of unused or underused resources - using AWS Trusted Advisor, CloudWatch metrics (identification of such resources can be done automatically using Lambda functions or scripts)
• Recommendations generated by AWS Trusted Advisor
• Getting Reserved Instances and Saving Plans recommendations
• Review of new and improved AWS services: analysis if existing services or resources could be replaced with more efficient and cheaper ones (for example: new types of EC2 instances are more efficient and often cheaper)

?The review should result in a list of decisions to make and then - a backlog of tasks to do in order to optimize costs of your environments.

Hope you enjoyed the series and find it insightful and useful. Stay tuned for in-depth articles about specific aspects of cloud cost management. If you are particularly interested in any cloud-cost related topic - let me know - I will gladly share my knowledge and experience.

?