Configure CloudWatch Log inputs for the Splunk Add-on for AWS¶

Prerequisite

You must manage accounts for the add-on. See Manage accounts for the Splunk Add-on for AWS.

Complete these steps to configure CloudWatch Log inputs for the Splunk Add-on for Amazon Web Services (AWS):

Configure AWS services for the CloudWatch Log input.
Configure AWS permissions for the CloudWatch Log input.
(Optional) Configure VPC Interface Endpoints for STS and logs services from your AWS Console if you want to use private endpoints for data collection and authentication. For more information, see https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html.
Configure CloudWatch Log inputs either through Splunk Web or configuration files.

Note

Due to rate limitations, don’t use pull-based (API) input configurations to collect CloudWatch Log data which has the source type aws:cloudwatchlogs:*. Instead, use push-based (Amazon Kinesis Firehose) input configurations to collect CloudWatch Log and VPC Flow Logs. The push-based (Amazon Kinesis Firehose) input configurations for the Splunk Add-on for AWS include index-time logic to perform the correct knowledge extraction for these events through the Kinesis input as well.

Forward OCSF-transformed CloudWatch Logs¶

Version 8.3.0 and later supports OCSF-transformed CloudWatch Logs sent to Splunk through Amazon Kinesis Data Firehose and Splunk HEC. Use this push-based path instead of the CloudWatch Logs pull input for OCSF events.

The supported log types and source types are:

Log type	Source type
VPC Flow Logs	`ocsf:aws:vpcflow`
CloudTrail	`ocsf:aws:cloudtrail`
Route 53 Resolver	`ocsf:aws:route53resolver`
AWS WAF	`ocsf:aws:waf`
EKS Audit	`ocsf:aws:eks:audit`

Complete the following steps for each log type:

Configure the AWS service to deliver logs to a CloudWatch Logs log group that uses the Standard log class.
Configure a CloudWatch Logs transformation using the AWS parseToOCSF processor. For more information, see the CloudWatch Logs transformation overview.
Configure an HEC token with the source type that matches the log type in the table above. Use this token when you configure the Firehose destination.
Configure an Amazon Kinesis Data Firehose delivery stream to send data to Splunk HEC. Under Transform records, turn on both Decompress source records from Amazon CloudWatch Logs and message extraction. Message extraction removes the CloudWatch Logs envelope and sends the transformed OCSF document in each logEvents[].message field to Splunk. For more information, see Configure Amazon Kinesis Firehose to send data to the Splunk Platform and Extract message after decompression of CloudWatch Logs.
Create a subscription filter for the log group and select the Firehose delivery stream. Enable Apply on transformed logs. If you use the AWS API or CLI, set applyOnTransformedLogs to true. This setting is required for CloudWatch Logs to forward the transformed OCSF events instead of the original ingested events.
Search Splunk for the selected source type to verify that the transformed events are indexed.

Create a separate Firehose stream and HEC token for each log type when you need to assign different source types.

Configure AWS permissions for the CloudWatch Log input¶

Required permissions for Logs:

AWS service	Permissions
Logs	`DescribeLogGroups` `DescribeLogStreams` `GetLogEvents`
S3	`GetBucketLocation`

See the following sample inline policy to configure CloudWatch Log input permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:GetLogEvents",
        "s3:GetBucketLocation"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

You must also ensure that your role has a trust relationship that allows the flow logs service to assume the role. While viewing the IAM role, choose Edit Trust Relationship and replace that policy with this one:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Configure a CloudWatch Logs input using Splunk Web¶

To configure inputs using Splunk Web, select Splunk Add-on for AWS in the navigation bar on Splunk Web home, and choose one of the following menu paths depending on the data type you want to collect:

Create New Input > VPC Flow Logs > CloudWatch Logs
Create New Input > Custom Data Type > CloudWatch Logs

Fill out the fields as described in the table:

Argument in configuration file	Field in Splunk Web	Description
`account`	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your CloudWatch Logs data. In Splunk Web, select an account from the drop-down list. In `aws_cloudwatch_logs_tasks.conf`, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
`aws_iam_role`	Assume Role	The IAM role to assume. Verify that your IAM Assume role has enough permission to access your Log Group. For more information, see Manage AWS IAM Roles for Splunk Add-on for Amazon Web Services
`region`	AWS Region	The AWS region that contains the data. In `aws_cloudwatch_logs_tasks.conf`, enter the region ID.
`private_endpoint_enabled`	Use Private Endpoints	Check the checkbox to use private endpoints of AWS Security Token Service (STS) and AWS Simple Cloud Storage (S3) services for authentication and data collection. In inputs.conf, enter `0` or `1` to respectively turn off or enable use of private endpoints.
`logs_private_endpoint_url`	Private Endpoint (Logs)	Private Endpoint (Interface VPC Endpoint) of your logs service, which you can configure from your AWS console. Supported formats: `://vpce- - .logs. .vpce.amazonaws.com` `://vpce- - -. logs. .vpce.amazonaws.com`
`sts_private_endpoint_url`	Private Endpoint (STS)	Private Endpoint (Interface VPC Endpoint) of your STS service, which you can configure from your AWS console. Supported formats: `://vpce- - .sts. .vpce.amazonaws.com` `://vpce- - - .sts. .vpce.amazonaws.com`
`groups`	Log group	A comma-separated list of log group names. Do not use wildcards.
`stream_matcher`	Stream Matching Regex	REGEX to strictly match stream names. Defaults to `.*`
`only_after`	Only After	GMT time string in `%Y-%m-%dT%H:%M:%S` format. If set, only events after this time are queried and indexed. Defaults to 1970-01-01T00:00:00.
`sourcetype`	Source type	A source type for the events. If you are indexing VPC Flow Log data through CloudWatch Logs: If you are using event index, the sourcetype value is `aws:cloudwatchlogs:vpcflow`. If you are using metric index, the sourcetype value is `aws:cloudwatchlogs:vpcflow:metric`. Enter `aws:cloudwatchlogs` if you are collecting any other types of CloudWatch Logs data.
`index`	Index	The index name where the Splunk platform puts the CloudWatch Logs data. The default is main.
`query_window_size`	Query Window Size (minutes)	Specify the interval of data to be collected in each request (in minutes). Default=10, Min=1 & Max=43200 (30days). For example, if the calculated start date is 2024-01-01T00:00:00 (midnight on January 1, 2024) and query window size is 60 minutes, then end date for the request is 2024-01-01T00:01:00 (one hour after midnight). The time period continues sliding by 60 minutes until no more recent logs are available.
`interval`	Interval	The number of seconds to wait before the Splunk platform runs the command again. The default is 600 seconds.
`metric_index_flag`	Use Metric Index?	Whether to use metric index or event index. The default value is No (use event index). This field is only visible when creating VPC Flow Logs -> CloudWatch Logs inputs.

Configure a CloudWatch Logs input using configuration files¶

To configure the input using configuration files, create $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/aws_cloudwatch_logs_tasks.conf using the following template:

[<name>]
account = <value>
groups = <value>
index = <value>
interval = <value>
only_after = <value>
region = <value>
private_endpoint_enabled = <value>
logs_private_endpoint_url = <value>
sts_private_endpoint_url = <value>
sourcetype = <value>
stream_matcher = <value>
metric_index_flag = <value>
query_window_size = <value>

The following example shows stanza of VPC Flow Log data from 2 log groups:

[cloudwatch_logs_data]
account = splunkapp2
groups = SomeName/DefaultLogGroup, SomeOtherName/SomeOtherLogGroup
index = default
interval = 600
only_after = 1970-01-01T00:00:00
region = us-west-2
sourcetype = aws:cloudwatchlogs:vpcflow
stream_matcher = eni.*
metric_index_flag = 0
query_window_size = 10