Skip to content

Configure CloudTrail Lake inputs for the Splunk Add-on for AWS

Complete the steps to configure CloudTrail Lake inputs for the Splunk Add-on for Amazon Web Services (AWS):

  1. You must manage accounts for the add-on as a prerequisite. See Manage accounts for the Splunk Add-on for AWS.
  2. Configure AWS services for the CloudTrail lake input.
  3. Configure AWS permissions for the CloudTrail lake input.
  4. (Optional) Configure VPC Interface Endpoints for STS and cloudtrail services from your AWS Console if you want to use private endpoints for data collection and authentication. For more information, see the Interface VPC endpoints (AWS PrivateLink) topic in the Amazon Virtual Private Cloud documentation.
  5. Configure CloudTrail lake inputs either through Splunk Web or configuration files.

Configure AWS services for the CloudTrail lake input

The Splunk Add-on for AWS collects JSON events from an cloudtrail lake event data store using SQL based query

To collect the data using cloudtrail lake input an event data store is required to be configured on AWS. There are various types of event data stores which can be created.

  1. To create an event data store, see the following topics in the AWS documentation:
    • Create an event data store for CloudTrail events topic in the AWS CloudTrail User Guide
    • Create an event data store for CloudTrail Insights events topic in the AWS CloudTrail User Guide
    • Create an event data store for events outside of AWS topic in the AWS CloudTrail User Guide
  2. To stop or start ingestion for event data stores see the Stop and start event ingestion topic in the AWS CloudTrail User Guide.

Configure AWS permissions for the CloudTrail lake input

Required permissions for the cloudtrail to collect the data from an event data store using CloudTrail lake modular input

  • GetQueryResults
  • StartQuery
  • ListEventDataStores
  • DescribeQuery

See the following sample inline policy to collect the data from CloudTrail Lake event data store using CloudTrail lake modular input

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "cloudtrail:GetQueryResults",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "cloudtrail:StartQuery",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "cloudtrail:ListEventDataStores",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor3",
            "Effect": "Allow",
            "Action": "cloudtrail:DescribeQuery",
            "Resource": "arn:aws:cloudtrail:*:<account>:eventdatastore/*"
        }
    ]
}

Configure a CloudTrail Lake input using Splunk Web

To configure inputs in Splunk Web:

  1. Click on Splunk Add-on for AWS in the navigation bar on Splunk Web home.
  2. Click Create New Input > CloudTrail Lake.
  3. Use the following table to complete the fields for the new input in the .conf file or in Splunk Web:
Argument in configuration file Field in Splunk Web Description
AWS input configuration
aws_account AWS Account The AWS account or EC2 IAM role the Splunk platform uses to access data present in CloudTrail lake event data store. In Splunk Web, select an account from the drop-down list. In inputs.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
aws_iam_role Assume Role The IAM role to assume, see “Add and manage IAM roles” in the Manage accounts for the Splunk Add-on for AWS topic.
aws_region AWS Region The region in which CloudTrail lake event data store is present.
private_endpoint_enabled Use Private Endpoints Check the checkbox to use private endpoints of AWS Security Token Service (STS) and AWS CloudTrail services for authentication and data collection. In inputs.conf, enter 0 or 1 to respectively disable or enable use of private endpoints.
cloudtrail_private_endpoint_url Private Endpoint (CloudTrail) Private Endpoint (Interface VPC Endpoint) of your CloudTrail service, which can be configured from your AWS console.
Supported Formats :
://vpce--.cloudtrail..vpce.amazonaws.com://vpce---.cloudtrail..vpce.amazonaws.com
sts_private_endpoint_url Private Endpoint (STS) Private Endpoint (Interface VPC Endpoint) of your STS service, which can be configured from your AWS console.
Supported Formats :
://vpce--.sts..vpce.amazonaws.com<br>://vpce---.sts..vpce.amazonaws.com
input_mode Input Mode Two types of input modes Index Once and Continuously Monitor.
  • Index Once input mode only ingests the data once.
  • Continuously Monitor input mode collects the data at every interval.
event_data_store Event Data Store The cloudtrail lake event data store from which the data will be collected.
start_date_time Start Date/Time Select a Start date/time to specify how far back to go when initially collecting data. If no date/time is given, the input will start 7 days in the past.
end_date_time End date/time This is only required in case of Index Once input mode.
Splunk-related configuration
sourcetype Sourcetype A source type for the events. Specify a value if you want to override the default of aws:cloudtrail:lake. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
index Index The index name where the Splunk platform puts the event data store data. The default is main.
Advanced settings
query_window_size Query Window Size (minutes) This parameter is used to control the chunk size. For example, if the calculated start date is 2022-01-01T00:00:00 (midnight on January 1, 2022), the end date for the SQL query will be 2022-01-01T00:01:00 (one hour after midnight) if the query window size is 60 minutes.
delay_throttle Delay Throttle (minutes) CloudTrail typically delivers events within an average of about 5 minutes of an API call. This time is not guaranteed. This parameter specifies how close to “now” the end date for a query may be (where “now” is the time that the input runs). For Continuously Monitor input mode at every interval invocation the input will collect the data from checkpointed start_date_time till current UTC time - delay_throttle
interval Interval (in seconds) Data collection interval. The value is only applicable for Continuously Monitor input mode. For Index Once input mode this value would always be -1.

Configure a Cloudtrail Lake input using a configuration file

To configure inputs in inputs.conf, create a stanza using the following template and add it to $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/inputs.conf. If the file or path does not exist, create it.

Below is the example of input stanza for Index Once input mode

[aws_cloudtrail_lake://<name>]
aws_account = <value>
aws_region = <value>
end_date_time = 2023-12-20T10:07:25
event_data_store = <value>
index = <value>
input_mode = index_once
interval = -1
private_endpoint_enabled = 0
query_window_size = <value>
sourcetype = <value>
start_date_time = 2023-12-15T10:07:25

Below is the example of input stanza for Continuously Monitor input mode

[aws_cloudtrail_lake://<name>]
aws_account = <value>
aws_region = <value>
delay_throttle = <value>
event_data_store = <value>
index = <value>
input_mode = continuously_monitor
interval = 3600
private_endpoint_enabled = 0
query_window_size = <value>
sourcetype = <value>
start_date_time = 2023-12-15T10:07:25