Cost management on AWS

Abstract

Businesses are in a digital transformation race to move faster, innovate more, and remain competitive. This document shares Amazon Web Services (AWS) perspectives on why digital transformation requires a shift and evolution in financial management processes, how organizations can adapt, and what AWS solutions can help you succeed and attain financial savings on Amazon web services.

The technological shift towards multi account logically segregated architecture makes cost conscious culture essential in an organization’s IT life cycle.

Business Problem

As per FLEXERA STATE OF THE CLOUD REPORT, organizations waste approximately 30% of public cloud spend.

1.??????Transparency and visibility – With multi account AWS structure spanning across 100+ accounts for enterprise customers, resource visibility based on BU, custom tags, etc. often pose to be a big challenge.

2.?????Cost Management– Division of cost into various business functions and strategies is difficult when chargeback and showback are involved.

3.?????Optimization strategy – Individual billing accounts get sidelined and are neglected in the centralized optimization wave. A Single roof optimization console is not available for enterprises with diverse billing organizations.

4.?????Reliability and accountability – Dedicated role for FinOps is neglected which leads to almost 30% of cloud resource wastage spend.

5.??????Governance and Control – Centralized audit capabilities from IAM, Security Group, Bucket, policies and permissions, and downloadable reports and pie charts for business folks.

6.?????Resource Optimization – Wastage tracker is missing for many organizations like old snapshots, AMI, non-utilized load balancers, etc.

7. Inaccurate forecasting?-?Traditional methods of budgeting and financial variance analysis do not translate well to the cloud. Technology and Finance teams may lack effective, centralized budgeting and monitoring mechanisms that allow them to accurately forecast usage, keep costs in check and effectively align cloud investments with business objectives.

Introduction: FinOps – Cost Conscious Culture

As per Flexera report,?

·??????61 percent of organizations plan to optimize cloud costs in 2021, making it the top initiative for the fifth year in a row.?

·??????59 percent of organizations plan to focus on cloud migration.?

·??????76 percent of organizations use cost efficiency and savings to measure cloud progress.?

The number of cloud services and options available to developers has exploded. Why? Because enterprises want to develop applications that are differentiated to add value to their customers and businesses. They want new services that give them the greater capability and perform more of the undifferentiated heavy lifting so they can produce applications that add business value. Far from a simple power supply, enterprises are seeking both breadth and depth of capability.

But with increasing sophistication, it’s natural for enterprises to lose sight of which services they are using and how much these services cost.

Enter?cloud financial management (CFM)/FinOps.?FinOps?is a discipline that combines tools, processes, and practices to manage and optimize cloud costs while letting enterprises innovate and scale to take advantage of new business opportunities and deliver additional business value. The aim of CFM?isn’t?to squeeze costs but, rather, to let business objectives be met without unnecessary expense.?

Impact of diverse AWS architectural considerations on Cost

With the evolution of deployment strategies and the ever-increasing agility that Amazon web services provide, it becomes difficult to keep billing in check.

Modern Services impacting cost trend?

Containerization platforms,?cloud-based configuration?management tools, Data warehousing?are amidst wide adoption in SMB’s as well as enterprises.?

Surprise cost of smaller services like CloudTrail, KMS, events

Usual cloud spends of surrounding management services ranges from 5 – 10% of the total consumption which is always put in the back seat. To effectively manage the configuration there must be a mechanism to check on them regularly.

Multi Account Strategy:

AWS Landing Zone and control tower define a secure multi account enterprise backbone necessary for sustainable architecture, but with a large number of accounts comes the responsibility of reducing the wastage spend done in multiple business unit accounts.

Shift of IAAS to PAAS

Technology shift from infrastructure to platform as a service solves a ton of purposes and lets businesses concentrate on application rather than maintenance. Specific focus on financial implications with this new age swiftness is very important or else business units can run out of budget soon.

There are numerous other factors that impact cost effectiveness on public cloud and need to be paid heat to, for an organization to be cost efficient on cloud.

AWS Well Architected Lens for Financial Management

Implementing Cloud Financial Management enables organizations to realize business value and financial success as they optimize their cost and usage and scale on AWS.

A successful cloud journey is embarked on with the best financial practices in place. AWS ensures that the customers get all possible best practices in the form of AWS Well Architected review.

The pillars of cost-conscious culture are depicted below,

Implementing Cost Allocation and Cost visibility

For implementing all the best practices, we discussed above, we need tools and controls.

Amazon provides a variety of services that help us in estimating the spend and getting visibility into the service wise costs, tag-based cost, business unit cost, and other factors. This requires a deliberate structure for your accounts and resources, which enables Finance to track spending flows and ensure that teams are accountable for their portion of the bottom line.

Cost Allocation Tags

Create a granular view of your organization’s consumption patterns by using cost allocations tags and form a structured resource tagging strategy. For Cost Reporting, tag every AWS Service used with information, such as owner, stack type, and associated application.

Tags can be AWS generated tags or user defined tags.

Tag Policies and AWS Resource Groups

Use Tag policies to maintain consistent tags, including the preferred case treatment of tag keys and key values. You can also use AWS Resource Groups to manage and organize your resources.

Tag policies?are a type of policy that can help you standardize tags across resources in your organization's accounts. In a tag policy, you specify tagging rules applicable to resources when they are tagged. A tag policy can also specify that non compliant tagging operations on specified resource types are?enforced.

AWS Control Tower and AWS Organizations

Create and provision new AWS accounts with AWS Control Tower for preconfigured blueprints (e.g., AWS Single Sign-On for directory and access) and easily set up and govern a new secure, multi-account AWS environment.

Create a holistic view of your organization’s AWS accounts by using AWS Organizations, reflecting your business needs, and simplifying billing via one single payment for all of your AWS accounts.

AWS Cost Explorer

AWS Cost Explorer lets you visualize, understand, and manage your AWS cost and usage over a set period of time. You can quickly get started by exploring data at a high level (in the chart and tabular formats), and then look for additional detail by creating more specific views through filters and groupings. For example, you can examine the top cost driver by service first, then further investigation to understand who used that service via the team tag or cost categories.

AWS Cost and Usage Report

AWS Cost and Usage Report (CUR) provides the most comprehensive set of AWS cost and usage data. It includes additional metadata about AWS services, pricing, details around your pricing models (such as Reserved Instances, Savings Plans), and the tags and cost categories that you created. AWS delivers the CUR multiple times a day to the Amazon S3 bucket you specify; from there the report can be integrated with services such as Amazon Athena, Amazon Redshift, and Amazon Quick Sight.

AWS commercial Plans ?

Reserved Instances and Savings plan and Spot Instances

AWS services can be procured through multiple pricing models. You can purchase cloud resources with on-demand, pay-as-you-go prices or receive discounts when you select one- or three-year term-based commitments. You can even purchase spare compute capacity at a much lower price.

If your financial situation allows you to make a longer-term commitment, and you have predictable workloads for instances like Amazon EC2 or Amazon RDS (to name a couple), Reserved Instances (RIs) enable you to reduce costs (up to 72% off OnDemand pricing).

You can leverage Spot instances and get a benefit of approximately 80% for non critical workloads.

If you want to commit to a dollar amount per hour rather than to a specific instance, Savings Plans is another commitment-based purchase option and offers up to 72% off On-Demand pricing.

Advanced FinOps – Evolving with time

With a lot of enterprise client drifting towards containerization of applications, it is very important to utilize the services at their best.

Kubernetes optimization (KOPS)

·??????Container resource optimization and pod level utilization metric analysis

·??????Running Kubernetes worker nodes on AWS Spot Instances

AWS Reserved Instance marketplace

·??????A lot of times customers get stuck with wrong commitment or are struck by sudden conditions in which they want to get rid of the commitment, in that case customers can leverage AWS reserved instance marketplace to sell the underutilized/waste RI’s.

Third party tools for Cloud financial management

Few enterprise resellers want to hide their credit, refund, taxes etc. and hence cannot share AWS cost explorer access. In some cases, the finance team is not comfortable with using AWS console, or the customer might have a multi cloud environment and would want to manage the spend under a single roof, in that case third party tools come in handy.

Inference

Cloud financial management/FinOps is a continuous process and needs to be driven in parallel with technology decisions with experience consulting partners.

FinOps as a practice reduces business risk and enhances the handshake between technology and finance, which helps the organization in the long run to maximize cloud benefits and in turn invest in future endeavours.