Configure Cloud Pub/Sub Lite inputs for Splunk Add-on for Google Cloud Platform

Configure Cloud Pub/Sub Lite inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Scaling

• To scale the data ingestion, It is recommended to configure the number of Pub/Sub Lite inputs which should be equal to or less than the number of partitions that you have in the Pub/Sub Lite Topic.
• You might observe data duplication less than 1%

Configure Cloud Pub/Sub Lite inputs using the Splunk Web

Follow these steps to configure Cloud Pub/Sub Lite inputs.

1. Click Create New Input in the Inputs tab, and then choose Cloud Pub/Sub Lite.
2. Enter the Name, Credentials, Projects, Location Type, Region/Zone, Pub/Sub Lite Subscription, Index, Sourcetype, Number of Threads, Messages Outstanding, and Bytes Outstanding using the information in the inputs parameter table.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Pub/Sub Lite inputs using the configuration file

See the following steps to configure Cloud Pub/Sub Lite inputs:

1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.
2. Create a file named inputs.conf, if it does not already exist.
3. Add the following stanza for Cloud Pub/Sub Lite input:

   1. Input configuration for Regional Location Type

      [google_cloud_pubsub_lite://<input_stanza_name>]
      bytes_outstanding = <value>
      google_credentials_name = <value>
      google_project = <value>
      location = regional
      messages_outstanding = <value>
      number_of_threads = <value>
      pubsublite_regions = <value>
      pubsublite_subscriptions = <value>
      sourcetype = google:gcp:pubsublite:message
      

      2. Input configuration for Zonal Location Type

      [google_cloud_pubsub_lite://<input_stanza_name>]
      bytes_outstanding = <value>
      google_credentials_name = <value>
      google_project = <value>
      location = zonal
      messages_outstanding = <value>
      number_of_threads = <value>
      pubsublite_subscriptions = <value>
      pubsublite_zones = <value>
      sourcetype = google:gcp:pubsublite:message
      

      4. Save and return to your Splunk instance.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web:

Attribute	Corresponding field in Splunk Web	Description
input_stanza_name	Name	Enter a unique name of the Google Cloud Pub/Sub Lite input.
google_credentials_name	Credentials	Stanza name defined in google_cloud_credentials.conf.
google_project	Projects	google project ID.
location	Location Type	Select the Regional Topic or Zonal Topic. If Regional topic is selected then Region dropdown will be available and if Zonal topic is selected then Zone dropdown will be available. The default is Regional.
pubsublite_regions	Region	Select the supported region for Pub/Sub Lite service.
pubsublite_zones	Zone	Select the supported zone for Pub/Sub Lite service.
pubsublite_subscriptions	Pub/Sub Lite Subscription	Google Cloud Pub/Sub Lite subscriptions list based on the selected region/zone.
index	Index	The index in which to store Google Cloud Pub/Sub Lite data.
sourcetype	Sourcetype	The sourcetype to use for this input.
number_of_threads	Number of Threads	The number of threads used to collect Cloud Pub/Sub Lite data in parallel. The default value is 10.
messages_outstanding	Messages Outstanding	Count of after how many max messages, TA will pause receiving messages if we have not acknowledged 1st message. The default value is 1000.
bytes_outstanding	Bytes Outstanding	Count of after how much Megabyte, TA will pause receiving messages if we have not acknowledged 1st message. The default value is 10.