Configure Cost and Usage Report inputs for the Splunk Add-on for AWS¶

Complete the steps to configure Cost and Usage Report inputs for the Splunk Add-on for Amazon Web Services (AWS):

You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
Configure AWS services for the Cost and Usage Report input.
Configure AWS permissions for the Cost and Usage Report input.
(Optional) Configure VPC Interface Endpoints for STS and S3 services from your AWS Console if you want to use private endpoints for data collection and authentication. For more information, see the Interface VPC endpoints (AWS PrivateLink) topic in the Amazon Virtual Private Cloud documentation.
Configure Cost and Usage Report inputs either through Splunk Web or configuration files.

Enable prefixes so that AWS delivers the reports into a folder with the name of the prefix. Timestamps and report names can be used to filter results if you do not want to ingest all the reports.

After you configure your Cost and Usage Report inputs, see Access billing data for the Splunk Add-on for AWS for more information about data collection behavior and how to access the preconfigured reports included in the add-on.

See the Cost and Usage Report section of the AWS documentation for more information on AWS-side configuration steps.

Configure AWS permissions for the Cost and Usage Report input¶

You need these required permissions for the S3 bucket to collect your Cost and Usage Reports:

Get*
List*

In the Resource section of the policy, specify the Amazon Resource Names (ARNs) of the S3 buckets that contain billing reports for your accounts. ListAllMyBuckets is required when you use an asterisk (*) character.

See the following sample inline policy to configure Billing input permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "s3:Get*",
            "s3:List*",
        ],
        "Resource": ""arn:aws:s3:::<your bucket name>"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:ListAllMyBuckets"
        ],
        "Resource": "*"
    }
  ]
}

For more information and sample policies, see http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-permissions-ref.html.

Configure a Cost and Usage Report input using Splunk Web¶

To configure inputs using Splunk Web:

Click Splunk Add-on for AWS in the navigation bar on Splunk Web home.
Click Create New Input > Billing > Billing (Cost and Usage Report).

Fill out the fields as described in the following table: -

Argument in configuration file	Field in Splunk Web	Description
AWS input configuration
`aws_account`	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your Billing data. In Splunk Web, select an account from the drop-down list. In inputs.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
`aws_iam_role`	Assume Role	The IAM role to assume. Verify that your IAMAssume role has enough permission to access your S3 buckets. For more information, see Add and manage IAM roles in the Manage accounts for the Splunk Add-on for AWS topic.
`aws_s3_region`	AWS Region (Optional)	The AWS region that contains your bucket. In inputs.conf, enter the region ID. Provide an AWS Region only if you want to use specific regional endpoints instead of public endpoints for data collection. See the AWS service endpoints topic in the AWS General Reference manual for more information.
`s3_private_endpoint_url`	Private Endpoint (S3)	Private Endpoint (Interface VPC Endpoint) of your S3 service, which can be configured from your AWS console. Supported Formats: `<http/https>://bucket.vpce-<endpoint_id>-<unique_id>.s3.<region_id>.vpce.amazonaws.com` `<http/https>://bucket.vpce-<endpoint_id>-<unique_id>-<availability_zone>.s3.<region_id>.vpce.amazonaws.com`
`sts_private_endpoint_url`	Private Endpoint (STS)	Private Endpoint (Interface VPC Endpoint) of your STS service, which can be configured from your AWS console. Supported Formats: `<http/https>://vpce-<endpoint_id>-<unique_id>.sts.<region_id>.vpce.amazonaws.com` `<http/https>://vpce-<endpoint_id>-<unique_id>-<availability_zone>.sts.<region_id>.vpce.amazonaws.com`
`bucket_name`	S3 Bucket	The S3 bucket that is configured to hold Billing Reports.
`private_endpoint_enabled`	Use Private Endpoints	Check the checkbox to use private endpoints of AWS Security Token Service (STS) and AWS Simple Cloud Storage (S3) services for authentication and data collection. In inputs.conf, enter `0` or `1` to respectively disable or enable use of private endpoints.
`report_prefix`	Report Prefix	Prefixes used to allow AWS to deliver the reports into a specified folder.
`report_names`	Report Name Pattern	A regular expression used to filter reports by name.
Splunk-related configuration
`start_date`	Start Date	This add-on starts to collect data later than this time. If youleave this field empty, the default value is 90 days before the input isconfigured.Once the input is created, you cannot change its value.
`sourcetype`	Source type	A source type for the events. Specify a value if you want to override the default of `aws:billing`. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
`index`	Index	The index name where the Splunk platform puts the billing data. The default is main.
Advanced settings
`interval`	Interval	Enter the number of seconds to wait before the Splunk platform runs the command again, or enter a valid cron schedule. The default is86,400 seconds (one day). This interval applies differently for Monthly report types and Detailed report types. For Monthly report types, the interval indicates how often to run the data collection for the current month’s monthly report and how often to check the previous month’s monthly report’s etag to determine if changes were made. If the etag does not match an already-downloaded version of the monthly report, it downloads that report to get the latest data. For Detailed report types, the interval indicates how often to check the previous month’s detailed report etag to determine if changes were made. If the etag does not match a report already downloaded, it downloads that report to get the latest data. The present month is never collected until the month has ended. Because AWS Billing Reports are usually not finalized until several days after the last day of the month, you can use the cron expression `0 0 8-31 * *` to skip data collection for the first seven days of every month to avoid collecting multiple copies of not-yet-finalized reports for the just-finished month.
`temp_folder`	Temp folder	Full path to a non-default folder with sufficient space for temporarily storing downloaded detailed billing report .zip files. Take into account the estimated size of uncompressed detailed billing report files, which can be much larger than that of zipped files. If you do not specify a temp folder, the add-on will use the system temp folder by default.

Configure a Cost and Usage Report input using configuration files¶

To configure inputs in inputs.conf, create a stanza using the following template and add it to $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.

[aws_billing_cur://<name>]
start_by_shell = true
aws_account = <value>
aws_iam_role = <value>
aws_s3_region = <value>
bucket_name = <value>
bucket_region = <value>
private_endpoint_enabled = <value>
report_names = <value>
report_prefix = <value>
s3_private_endpoint_url = <value>
start_date = <value>
sts_private_endpoint_url = <value>
temp_folder = <value>
host_name = s3.amazonaws.com

Some of these settings have default values that can be found in $SPLUNK_HOME/etc/apps/Splunk_TA_aws/default/inputs.conf:

[aws_billing_cur]
start_by_shell = false
aws_account = <value>
aws_iam_role = <value>
bucket_name = <value>
bucket_region = <value>
report_names = <value>
report_prefix = <value>
start_date = <value>
temp_folder = <value>

The previous values correspond to the default values in Splunk Web. If you choose to copy this stanza to /local and use it as a starting point to configure your inputs.conf manually, change the stanza title from aws_billing:cur to aws_billing:cur://<name>.