Understanding Cloud Cost & Usage Reports - AWS CUR, part 1

Introduction

There are currently a great many tools in the market to help you maximize the benefits of FinOps, but in the first stages it might be good to understand how the different cloud providers cost and usage reports work.

If you are anything like me, you like to know how things work. Understand them to be able to visualize how to put them to use.

Also, since at the start of the FinOps journey visibility of your costs and usage is paramount to know where to go from there for your particular case, understanding the ins an outs of these reports can help you build some basic dashboards before committing to a third party tool.

This article aims to be the first of a series covering the three top Cloud Service Providers (Amazon Web Services, Microsoft Azure & Google Cloud Platform).

AWS CUR - The basics

Since AWS was first, they have been evolving their reports for a longer time, and thus they have been going through several cycles of improvement.

Currently, these reports are know as "Cost & Usage Reports", or CUR for short.

The CUR is a huge file by now and, if you have a sizable consumption with many accounts under an organization, it will be a very large dataset.

Basic structure

You can think of the file as a huge spreadsheet, with millions of rows (and more than 300 columns).

If you just wish to know how much money you spent in a certain period, you just need to sum the column line_item_unblended_cost (for the rows in that period, that is).

However, for each "usage" of a resource there are several lines with different types. A few examples:

For the use of some S3 storage you could have two different lines:

• One for the raw amount
• One for the applicable taxes

For the purchase of a partial upfront reserved instance purchase (a large one):

• A line for the upfront part
• A line for the first month recurring fee
• A volume discount for the purchase (that one would have a negative unblended_cost)
• A line for the applicable taxes

Much of the same calculations that result from summing up the unblended cost across lines are also detailed in the main "usage" line using specific fields. You have specific fields for discounts, for example, and you have another field called line_item_net_unblended_cost that is the result of applying any discounts to the line_item_unblended_cost field.

This gives you a lot of flexibility to perform queries for many different purposes.

And then, to put those 300 fields to work, depending on the type of line some will be populated to show you the details specific for that type of usage (for example, the usage of RIs or Savings Plans). So you can check how you are making use of the reservations, for example.

Remember this is not just cost, is also usage.

Field groups

Fields are grouped in sets related to different types of activity:

• identity_*: these simply allow to identify each line in the file.
• bill_*: information about billing (invoice number, billing period, ...)
• line_item_*: usage of specific resources (quantity, cost, basic description, ...)
• product_*: useful information about the resource used (such as the instance type for EC2, for example)
• pricing_*: specific pricing information (unit of measure, pricing, term, ....)
• reservation_*: information about the usage (or lack thereof) of a purchased Reserved Instance.
• discount_*: discounts applied
• savings_plan_*: like reservation but for Savings Plans usage.
• resource_tags_*: specific cost allocation tags (will vary depending of the cost allocation tags you define).

Basic usage information

The basic usage information can be obtained from the line_item_* group of fields, the most relevant ones that will allow you to understand and run some queries on the files are:

• line_item_usage_account_id: the account the usage is associated with.
• line_item_line_item_type: this determines the type of information we will find in the rest of fields, this is a key field to understand and query the information of the CUR.
• line_item_usage_start_date and line_item_usage_end_date: date and time of the period the usage refers to.
• line_item_product_code: product code that identifies the product or service the usage refers to (AWSLambda or AmazonEC2 for example).
• line_item_usage_amount: amount used, the units used are in the field pricing_unit (hours, GB or GB/month for example).
• line_item_unblended_cost: the actual cost (without any applicable discount, or taxes).

Types of line

The different types of lines can be discriminated using the line_item_line_item_type field. These are the most common types of lines (some types and some fields even, will appear depending on your type of contract and migth be under an NDA, those I will not cover here):

• Usage: What you are consuming at on-demand prices, that is, not covered by Savings Plans or Reserved Instances.
• Credit: Credits appplied to your consumption. These are negative amounts inside a given invoice.
• Tax: Taxes applied to a given usage.
• Fee: Up front payments for Reserved Instances, and for some other upfront payments.
• Refund: A refund, usually part of a Refund invoice, to correct a pricing error for example.
• DiscountedUsage: Usage that was covered by a RI. line_item_unblended_cost will be 0, but all the reservation_* (and product_*) fields will contain juicy details.
• RIFee: You will get one a month of these for each RI, with the recurring cost of the RI (might be 0 if it was all-upfront). It also gives you details on what you used and what you didn't use of the RI.
• SavingsPlanCoveredUsage: Similar to DiscountedUsage but for SavingsPlans. This time the savings_plan_* fields will be populated with details and the line_item_unblended_cost will have the effective cost paid.
• SavingsPlanNegation: As the SavingsPlanCoveredUsage lines have a value for the line_item_unblended_cost, this line is just the same amount but negative so that when you sum all lines without discrimination one cancels the other.
• BundledDiscount: The use of some services might come with a discount for the use of a different service, and this line will show you the details of the discount you got for that second service.
• RiVolumeDiscount: Sometimes, when you make a large RI purchase you receive a volume discount.
• SavingsPlanRecurringFee: Just like the RIFee but for Savings Plans, this time however you get one line per day instead of one per month (at least with daily CUR).
• SavingsPlanUpfrontFee: Like the Fee type represents any upfront fee, but only for Savings Plans.

This is just barely scraping the surface of what you can do with the CUR, but it should help you extract some useful information from them.

#cloud #finops #costoptimization #cloudfinancialmanagement