Skip to content

Configure CloudTrail Lake Inputs for the Splunk Add-on for AWS

Complete the steps to configure CloudTrail Lake inputs for the Splunk Add-on for Amazon Web Services (AWS):

  1. You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
  2. Configure AWS services for the CloudTrail Lake input.
  3. Configure AWS permissions for the CloudTrail Lake input.
  4. (Optional) Configure VPC Interface Endpoints for STS and Cloudtrail services from your AWS Console if you want to use private endpoints for data collection and authentication. For more information, see the Interface VPC endpoints (AWS PrivateLink) topic in the Amazon Virtual Private Cloud documentation.
  5. Configure CloudTrail Lake inputs either through Splunk Web or configuration files.

Configure AWS services for the CloudTrail Lake Input

The Splunk Add-on for AWS collects JSON events from an Cloudtrail Lake event data store using SQL based query

AWS configuration process

To collect the data using Cloudtrail Lake input an event data store is required to be configured on AWS. There are various types of event data stores which can be created.

  1. To create an event data store, see the following topics in the AWS documentation:
  2. To stop or start ingestion for event data stores see the Stop and start event ingestion topic in the AWS CloudTrail User Guide.

Configure Cloudtrail Lake permissions

AWS Service Permissions
CloudTrail GetQueryResults
StartQuery
ListEventDataStores
DescribeQuery

See the following sample inline policy to collect the data from CloudTrail Lake event data store using CloudTrail Lake modular input

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "cloudtrail:GetQueryResults",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "cloudtrail:StartQuery",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "cloudtrail:ListEventDataStores",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": "cloudtrail:DescribeQuery",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        }
    ]
}

Configure a CloudTrail Lake Input using Splunk Web

To configure inputs in Splunk Web:

  1. Click on Splunk Add-on for AWS in the navigation bar on Splunk Web home.
  2. Click Create New Input > CloudTrail Lake.
  3. Use the following table to complete the fields for the new input in the .conf file or in Splunk Web:
Argument in configuration file Field in Splunk Web Description
aws_account AWS Account The AWS account or EC2 IAM role the Splunk platform uses to access data present in CloudTrail Lake event data store. In Splunk Web, select an account from the drop-down list. In inputs.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
aws_iam_role Assume Role The IAM role to assume, see Manage AWS IAM Roles for Splunk Add-on for Amazon Web Services topic.
aws_region AWS Region The region in which CloudTrail Lake event data store is present.
private_endpoint_enabled Use Private Endpoints Check the checkbox to use private endpoints of AWS Security Token Service (STS) and AWS CloudTrail services for authentication and data collection. In inputs.conf, enter 0 or 1 to respectively disable or enable use of private endpoints.
cloudtrail_private_endpoint_url Private Endpoint (CloudTrail) Private Endpoint (Interface VPC Endpoint) of your CloudTrail service, which you can configure from your AWS console.
Supported Formats :
://vpce--.cloudtrail..vpce.amazonaws.com://vpce---.cloudtrail..vpce.amazonaws.com
sts_private_endpoint_url Private Endpoint (STS) Private Endpoint (Interface VPC Endpoint) of your STS service, which you can configure from your AWS console.
Supported Formats :
://vpce--.sts..vpce.amazonaws.com<br>://vpce---.sts..vpce.amazonaws.com
input_mode Input Mode Two types of input modes Index Once and Continuously Monitor.
  • Index Once input mode only ingests the data once.
  • Continuously Monitor input mode collects the data at every interval.
event_data_store Event Data Store The Cloudtrail Lake event data store from which the data will be collected.
start_date_time Start Date/Time Select a start date and time to specify how far back the initial data collection should go. If not specified, data collection defaults to starting 7 days in the past. Start date/time should be in ‘YYYY-MM-DDTHH:MM:SS’(UTC) format.
end_date_time End date/time This parameter is required if you selected Index Once input mode.
  • End date/time should be in ‘YYYY-MM-DDTHH:MM:SS’(UTC) format.
sourcetype Sourcetype A source type for the events. Specify a value if you want to override the default of aws:cloudtrail:lake. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
index Index The index name where the Splunk platform puts the event data store data. The default is main.
query_window_size Query Window Size (minutes) This parameter is used to control the chunk size. For example, if the calculated start date is 2022-01-01T00:00:00 (midnight on January 1, 2022), the end date for the SQL query will be 2022-01-01T00:01:00 (one hour after midnight) if the query window size is 60 minutes.
delay_throttle Delay Throttle (minutes) CloudTrail typically delivers events within an average of about 5 minutes of an API call. This time is not guaranteed. This parameter specifies how close to “now” the end date for a query may be (where “now” is the time that the input runs). For Continuously Monitor input mode at every interval invocation the input will collect the data from checkpointed start_date_time till current UTC time - delay_throttle
interval Interval (in seconds) Data collection interval. The value is only applicable for Continuously Monitor input mode. For Index Once input mode this value would always be -1.

Configure a Cloudtrail Lake Input using a configuration file

To configure inputs in inputs.conf, create a stanza using the following template and add it to $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.

Below is the example of input stanza for Index Once input mode

[aws_cloudtrail_lake://<name>]
aws_account = <value>
aws_region = <value>
end_date_time = 2023-12-20T10:07:25
event_data_store = <value>
index = <value>
input_mode = index_once
interval = -1
private_endpoint_enabled = 0
query_window_size = <value>
sourcetype = <value>
start_date_time = 2023-12-15T10:07:25

Below is the example of input stanza for Continuously Monitor input mode

[aws_cloudtrail_lake://<name>]
aws_account = <value>
aws_region = <value>
delay_throttle = <value>
event_data_store = <value>
index = <value>
input_mode = continuously_monitor
interval = 3600
private_endpoint_enabled = 0
query_window_size = <value>
sourcetype = <value>
start_date_time = 2023-12-15T10:07:25