Configure SQS inputs for the Splunk Add-on for AWS¶

Complete the steps to configure SQS inputs for the Splunk Add-on for Amazon Web Services (AWS):

Prerequisites You must manage accounts for the add-on. See Manage accounts for the Splunk Add-on for AWS.

Configure AWS services for the SQS input.
Configure AWS permissions for the SQS input.
Configure SQS inputs either through Splunk Web or configuration files.

Configure AWS services for the SQS input¶

If you plan to use the SQS input, you must perform the following steps:

Set up a dead-letter queue for the SQS queue to be used for the input for storing invalid messages. For information about SQS dead-letter queues and how to configure it, see https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html.
Configure the SQS visibility timeout to prevent multiple inputs from receiving and processing messages in a queue more than once. Set your SQS visibility timeout to 5 minutes or longer. If the visibility timeout for a message is reached before the message is fully processed by the SQS input, the message reappears in the queue and is retrieved and processed again, resulting in duplicate data. For information about SQS visibility timeout and how to configure it, see https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html.

Note

The SQS input supports up to four queues collecting data simultaneously. If more than four queues are configured, only four will begin data collection, while the others wait for a running queue to complete. Consequently, if a queue takes a long time to process messages, it may delay the start of data collection for other queues.

Configure SQS permissions¶

AWS Service	Permissions
SQS	`GetQueueAttributes` `ListQueues` `ReceiveMessage` `GetQueueUrl` `SendMessage` `DeleteMessage`

See the following sample inline policy to configure SQS input permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:GetQueueAttributes",
                "sqs:ListQueues",
                "sqs:ReceiveMessage",
                "sqs:GetQueueUrl",
                "sqs:SendMessage",
                "sqs:DeleteMessage"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Configure an SQS input using Splunk Web¶

To configure inputs using Splunk Web:

Click Splunk Add-on for AWS in the navigation bar on Splunk Web home.
Click Create New Input > Custom Data Type > SQS.
Use the following table to complete the fields for the new input in the .conf file or in Splunk Web:

Argument in configuration file	Field in Splunk Web	Description
`aws_account`	AWS Account	The AWS account or EC2 IAM role the Splunk platform uses to access your SQS data. In Splunk Web, select an account from the drop-down list. In aws_sqs_tasks.conf, enter the friendly name of one of the AWS accounts that you configured on the Configuration page or the name of the automatically discovered EC2 IAM role.
`aws_iam_role`	Assume Role	The IAM role to assume, see Manage AWS IAM Roles for Splunk Add-on for AWS.
`aws_region`	AWS Region	The AWS region that contains the log notification SQS queue. In aws_sqs_tasks.conf, enter the region code. For example, the region code for the US East region is us-east-2. See https://docs.aws.amazon.com/general/latest/gr/rande.html.
`sqs_queues`	SQS queues	The name of the queue to which AWS sends new SQS log notifications. In Splunk Web, you can select a queue from the drop-down list, if your account permissions allow you to list queues, or enter the queue name manually. The queue name is the final segment of the full queue URL. For example, if your SQS queue URL is `http://sqs.us-east-1.amazonaws.com/123456789012/testQueue`, then your SQS queue name is `testQueue`. You can add multiple queues separated by commas.
`sourcetype`	Source type	A source type for the events. Enter a value only if you want to override the default of `aws:sqs`. Event extraction relies on the default value of source type. If you change the default value, you must update props.conf as well.
`index`	Index	The index name where the Splunk platform puts the SQS data. The default is main.
`interval`	Interval	The number of seconds to wait before the Splunk platform runs the command again. The default is 30 seconds.

Configure an SQS input using configuration files¶

To configure the input using configuration files, create $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/aws_sqs_tasks.conf using the following template:

[<input_name>]
aws_account = <value>
aws_iam_role = <value>
aws_region = <value>
sqs_queues = <value>
index = <value>
sourcetype = <value>
interval = <value>

The following example shows input stanza for SQS

[sqs_data]
aws_account = splunkapp2
aws_region = ap-south-1
index = default
interval = 30
sourcetype = aws:sqs
sqs_queues = queue-1,queue-2