Overview

Splunk Add-on for Google Cloud Platform

Version	4.7.1
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata Compute Engine Resource Metadata Cloud Storage Resource Metadata Kubernetes Resource Metadata VPC Access
Add-on has a web UI	Yes. This add-on contains views for configuration.

The Splunk Add-on for Google Cloud Platform allows a Splunk administrator to collect Google Cloud Platform events, logs, and performance metrics data using Google Cloud Platform APIs. You can then analyze the data or use it as a contextual data feed to correlate with other Google Cloud data in the Splunk platform.

Download the Splunk Add-on for Google Cloud Platform from Splunkbase.

For a summary of new features, fixed issues, and known issues, see Release Notes for the Splunk Add-on for Google Cloud Platform.

For information about installing and configuring the Splunk Add-on for Google Cloud Platform, see Installation and configuration overview for the Splunk Add-on for Google Cloud Platform.

See Questions related to Splunk Add-on for Google Cloud Platform on the Splunk Community page.

Source types for the Splunk Add-on for Google Cloud Platform

The Splunk Add-on for Google Cloud Platform (GCP) provides the index-time and search-time knowledge for Google Cloud Platform logs and billing data in the following formats:

To better align with the Google Cloud Platform, and to provide a better understanding of the data coming from the cloud, the 4.0.0 release of the Splunk Add-on for Google Cloud Platform contains improvements to sourcetyping that affect the google:gcp:pubsub:audit:auth, google:gcp:pubsub:audit:change, and google:gcp:pubsub:message source types. These improvements provide more granular sourcetyping on incoming data from your GCP deployment, enhancing your ability to investigate and simplifying the development of dashboards in Splunk that use GCP data. Upgrading to version 4.0.0 or higher will cause any inline searches, pivots, or reports that use these source types to not work for the GCP data that is being ingested after upgrading to version 4.0.0 of this add-on. To ensure continuity of searches and reports on GCP data coming in after the upgrade to version 4.0.0 or later, review and perform the steps contained in the Upgrade the Splunk Add-on for Google Cloud Platform topic in this manual.

Source type	Description	CIM data models
google:gcp:pubsub:audit:data_access	Data from Pub/Sub (GCP Authentication Audit Logs)	Authentication
google:gcp:pubsub:audit:admin_activity	Data from Pub/Sub	Change
google:gcp:pubsub:audit:system_event	Data from Pub/Sub	Change
google:gcp:pubsub:audit:policy_denied	Data from Pub/Sub
google:gcp:pubsub:access_transparency	Data from Pub/Sub
google:gcp:pubsub:audit:auth	Data from Pub/Sub (GCP Authentication Audit Logs)	Authentication
google:gcp:pubsub:message	Data from Pub/Sub	Authentication
google:gcp:pubsub:platform	Data from Pub/Sub	None
google:gcp:pubsublite:message	Data from Pub/Sub Lite	None
google:gcp:monitoring	Data from Cloud Monitor service	None
google:gcp:billing:standard_usage_cost	Data from Standard Usage Cost reports	None
google:gcp:billing:detailed_usage_cost	Data from Detailed Usage Cost reports	None
google:gcp:billing:pricing	Data from Pricing Table reports	None
google:gcp:buckets:accesslogs	Cloud Storage Bucket server access logs for a storage account	Change
google:gcp:buckets:csvdata	CSV contents of objects present in the Cloud Storage Bucket	None
google:gcp:buckets:data	Generic source type for the contents of other file extensions. Depending upon file extension (.csv, .xml and .json), the sourcetypes would be categorised in google:gcp:buckets:csvdata, google:gcp:buckets:xmldata, and google:gcp:buckets:jsondata, respectively	None
google:gcp:buckets:jsondata	JSON contents of objects present in the Cloud Storage Bucket	None
google:gcp:buckets:metadata	Cloud Storage Bucket metadata	None
google:gcp:resource:metadata	Resource Metadata of Compute Engine, Cloud Storage, Kubernetes and VPC Access	None
google:gcp:buckets:xmldata	XML contents of objects present in the Cloud Storage Bucket	None
User defined	Modular input. See the REST API reference page for more information.	None
google:billing:json	Data from billing that is in JSON value.	None
google:billing:csv	Data from billing that is in CSV value.	None
google:gcp:billing:report	Data from billing reports.	None
google:gcp:gsuite:admin:directory:users	Data from G Suite users.	None
google:gcp:compute:instance	Data from Compute Engine virtual machine instances.	None
google:gcp:compute:vpc_flows	Data from Compute Engine VPC flow logs.	None
google:gcp:security:alerts	Data from security alerts.	None

Release notes for the Splunk Add-on for Google Cloud Platform

To upgrade the version 4.7.1 of this add-on, see the Upgrade the Splunk Add-on for Google Cloud Platform topic in this manual. Upgrade the Splunk Add-on for Google Cloud Platform

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform was released on December 11, 2024.

About this release

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	9.1.x, 9.2.x, 9.3.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Upgraded the Splunk SDK to the latest version, ensuring compatibility with future cloud-based deployments.

Fixed issues

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.7.1 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Release history for the Splunk Add-on for Google Cloud Platform

The latest version of the Splunk Add-on for Google Cloud Platform is version 4.7.1. See Release notes for the Splunk Add-on for Google Cloud Platform for the release notes of this latest version.

Version 4.7.0

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform was released on September 30, 2024.

About this release

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	9.1.x, 9.2.x, 9.3.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Provided compatibility for IPv6.
• Provided custom sourcetype support for Cloud Storage Bucket and Pub/Sub inputs.
• Security Fixes.
• Cloud Monitoring Input Enhancement:
  • Added “Host” field extraction.
  • Fixed data duplication issues.

Fixed issues

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.7.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.6.0

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform was released on June 26, 2024.

About this release

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	9.0.x, 9.1.x, 9.2.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Compatibility with Python 3.9.

Fixed issues

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.6.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.5.0

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform is not backward compatible. Downgrading to version 4.4.0 will lead to complete data duplication because of significant data collection changes regarding the Cloud Storage Bucket Input. Additionally, users will need to reconfigure their Cloud Storage Bucket inputs.

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform was released on April 18, 2024.

About this release

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	9.0.x, 9.1.x, 9.2.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

Improved performance for the Cloud Monitoring and Cloud Storage bucket inputs.

• Resolved a higher memory utilization issue with the Cloud Monitoring input.
• Resolved the following issues with the Cloud Storage Bucket input:
  • Fixed data collection issues for the bucket size of more than 50,000 files.
  • Migrated inputs from google_cloud_storage_buckets.conf file to inputs.conf under the stanza google_cloud_bucket_metadata.
  • Resolved parallel data collection issue for more than four inputs.
  • Each Cloud Storage Bucket input will have its separate .log file for logging and the naming convention for that would be splunk_ta_google_cloud_platform_google_cloud_bucket_metadata_<input_name>.log

Fixed issues

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.5.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.4.0

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform was released on Jan 23, 2024.

About this release

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	9.0.x, 9.1.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Provided support for GCP Workload Identity federation feature (which is currently supported on AWS-EC2 and Azure-VM Workload).
• Migrated from file-based checkpointing to the KV Store checkpointing mechanism on the Bigquery Billing input.

Fixed issues

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.4.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.3.0

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform was released on November 1, 2023.

About this release

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.2.x, 9.0.x, 9.1.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Enhanced support for data ingestion from the Cloud Monitoring service.
  • Added support for filtering monitored projects that are contained within scoping projects. For more information, see the Metrics scopes overview topic in the Google Cloud documentation.
  • Migrated from file-based checkpointing to the KV Store checkpointing mechanism for the Cloud Monitoring input.

Fixed issues

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.3.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.2.0

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform was released on Jul 26, 2023.

About this release

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.2.x, 9.0.x, 9.1.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud Pub/Sub Lite Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Added support for two new types of input data collection:
  • Cloud Pub/Sub Lite
  • Cloud Pub/Sub Based Bucket

Fixed issues

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.2.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.1.1

Version 4.1.1 of the Splunk Add-on for Google Cloud Platform was released on January 16th 2023.

About this release

Version 4.1.1 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.2.x, 9.0.x
CIM	5.0.2
Platforms	Windows,Linux
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

Fixed issues

Version 4.1.1 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.1.1 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.1.1 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.1.0

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform was released on January 16th 2023.

About this release

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.1.x, 8.2.x, 9.0.x
CIM	5.0.2
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Resource Metadata (Compute Engine, Cloud Storage, Kubernetes, VPC Access)

New features

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Added support for three new types of Resource Metadata input data collection:
  • Cloud Storage data
  • Kubernetes data
  • VPC access data
• Renamed Resource metadata input to Resource Metadata (Compute Engine).
• Added support to collect Firewall related metadata in the existing Compute Engine input.
• Enhanced UI experience.
• Fixed proxy data collection issue.
• Fixed compatibility issue of Billing input in Windows and MacOS.
• Minor bug fixes.

Fixed issues

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.1.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 4.0.0

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform was released on September 22, 2022.

About this release

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.0.x, 8.1.x, 8.2.x
CIM	4.20
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Compute Engine

New features

To better align with the Google Cloud Platform, and to provide a better understanding of the data coming from the cloud, the 4.0.0 release of the Splunk Add-on for Google Cloud Platform contains improvements to sourcetyping that affect the google:gcp:pubsub:audit:auth, google:gcp:pubsub:audit:change, and google:gcp:pubsub:message source types.

These improvements provide more granular sourcetyping on incoming data from your GCP deployment, enhancing your ability to investigate and simplifying the development of dashboards in Splunk that use GCP data. Upgrading to version 4.0.0 or higher will cause any inline searches, pivots, or reports that use these source types to not work for the GCP data that is being ingested after upgrading to version 4.0.0 of this add-on.

To ensure continuity of searches and reports on GCP data coming in after the upgrade to version 4.0.0 or later, review and perform the steps contained in the Upgrade the Splunk Add-on for Google Cloud Platform topic in this manual.

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Added sourcetype support for the Data Manager Google Cloud Platform input.

Fixed issues

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 3.2.0

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform was released on April 5, 2022.

About this release

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.0.x, 8.1.x, 8.2.x
CIM	4.20
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud BigQuery Billing Google Cloud Storage Compute Engine

New features

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Added support for the Google Cloud BigQuery Billing input through the following sourcetypes:
  • google:gcp:billing:standard_usage_cost
  • google:gcp:billing:detailed_usage_cost
  • google:gcp:billing:pricing

Previously billing data was ingested in the google:gcp:billing:report sourcetype.

Google has deprecated regular file export of your Cloud Billing data to CSV and JSON. To export your Cloud Billing data for analysis, use Cloud Billing export to BigQuery.

To ingest Cloud BigQuery Billing data, you must delete your existing billing inputs before you upgrade to versions 3.2.0 and later of this add-on. After upgrading, you can then recreate your billing inputs. See the Configure Cloud BigQuery Billing inputs for the Splunk Add-on for Google Cloud Service topic in this manual.

Fixed issues

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 3.2.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

Third-party software attributions for the Splunk Add-on for Google Cloud Platform

Version 3.1.1

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform was released on September 8, 2021.

About this release

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	8.0.x, 8.1.x, 8.2.x
CIM	4.18
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud BigQuery Billing (Deprecated) Google Cloud Storage Compute Engine

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Added native support for Cloud Storage - Usage log format.

  Previously this data was ingested in the generic google:gcp:buckets:*data sourcetype.

• UI component upgrades for compatibility with future versions of the Splunk software (jQuery removal).

• The billing API used to fetch billing data is now deprecated. Users won’t be able to ingest billing data using the billing input.
• Self-Service restart fix. Rolling restarts have been eliminated.
• The sourcetype: google:gcp:buckets:accesslogs, which has been configured to work with the Change data model in the Common Information Model (CIM).
• Improved CIM support for storage access events.

See Choose your Splunk Enterprise upgrade path for the Python 3 migration to learn more about migrating your deployment to Python3.

Fixed issues

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 3.1.1 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

• configparser
• future
• GoogleAPIpythonclient
• Google Auth Library Python
• oauth2client
• requests
• u-msgpack-python
• remotepdb
• SortedContainers
• httplib2
• python-dateutil
• rsa
• Select2
• urllib3

Version 3.0.2

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform was released on July 22, 2020.

About this release

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	7.0.x ,7.1.x, 7.2.x, 7.3.x, 8.0.x
CIM	4.16
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud Billing Google Cloud Storage Compute Engine

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

New features

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Increased Assets and Identities CIM data model compatibility
• Increased Network Traffic CIM data model compatibility

See Choose your Splunk Enterprise upgrade path for the Python 3 migration to learn more about migrating your deployment to Python3.

Fixed issues

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 3.0.2 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

• configparser
• future
• GoogleAPIpythonclient
• Google Auth Library Python
• oauth2client
• requests
• u-msgpack-python
• remotepdb
• SortedContainers
• httplib2
• python-dateutil
• rsa
• Select2
• urllib3

Version 3.0.0

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform was released on March 4, 2020.

About this release

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	7.0.x ,7.1.x, 7.2.x, 7.3.x, 8.0.x
CIM	None
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud Billing Google Cloud Storage Compute Engine

New features

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Support for GCP resource metadata collection. Collect resource metadata to track configurations across deployments and compare them, and administer large deployments in GCP securely.
  • Users can assign sourcetype while configuring an input from the Inputs page.
  • See the REST API reference page to see the Cloud Resource Metadata APIs.
• Support for the GCP Google Cloud Storage input.
  • Support to ingest data directly from GCS buckets and bucket metadata of the selected buckets.
  • Support for the following file formats: XML, CSV, JSON, TEXT
• Support for the following sourcetypes, introduced as part of the Cloud Storage Bucket input:
  • google:gcp:buckets:metadata
  • google:gcp:buckets:csvdata
  • google:gcp:buckets:jsondata
  • google:gcp:buckets:xmldata
  • google:gcp:buckets:data

See Choose your Splunk Enterprise upgrade path for the Python 3 migration to learn more about migrating your deployment to Python3.

Fixed issues

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 3.0.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

• configparser
• future
• GoogleAPIpythonclient
• Google Auth Library Python
• oauth2client
• requests
• u-msgpack-python
• remotepdb
• SortedContainers
• httplib2
• python-dateutil
• rsa
• Select2
• urllib3

Version 2.0.0

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform was released on October 21, 2019.

About this release

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	7.0.x ,7.1.x, 7.2.x, 7.3.x, 8.0.x
CIM	None
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud Billing

New features

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform contains the following new features.

• Support for Python 3

See Choose your Splunk Enterprise upgrade path for the Python 3 migration to learn more about migrating your deployment to Python3.

Fixed issues

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 2.0.0 of the Splunk Add-on for Google Cloud Platform incorporates the following third-party software or libraries:

• configparser
• future
• GoogleAPIpythonclient
• Google Auth
• Requests
• u-msgpack-python
• remotepdb
• Python sortedcontainer
• httplib2
• python-dateutil
• Select2
• urllib3
• httplib2shim

Version 1.2.0

Version 1.2.0 of the Splunk Add-on for Google Cloud Platform is compatible with the following software, CIM versions, and platforms:

Splunk platform versions	6.6 or later
CIM	None
Platforms	Platform Independent
Vendor Products	Google Cloud Pub/Sub Google Cloud CloudMonitor service Google Cloud Billing

New features

Version 1.2.0 of the Splunk Add-on for Google Cloud Platform contains the following new and changed features.

• Optimized Cloud pub/sub inputs to ingest metadata of events
  • The following is an example of the new event format.

{
    publish_time: 1510876651
    message: {
        id: 172600725985922
        data: "foo,bar"
        attributes: {
            key1: xxx,
            key2: xxx
        }
    }
}

• Migration tool for upgrading versions
• Renamed sourcetypes and sources
  • Changed google:pubsub to google:gcp:pubsub:message
  • Changed google:billing:csv to google:gcp:billing:report
  • Changed google:billing:json to google:gcp:billing:report
  • Changed google:cloudmonitor to google:gcp:monitoring
• Source change of billing
  • Change from {file_name} to URI https://storage.cloud.google.com/{bucket_name}/{file_name}
• Source change of pubsub
  • Change from {project_name}:{subscription_name} to projects/{project_name}/subscriptions/{subscription_name}
• Proxy type http_no_tunnel is no longer supported

Fixed issues

Version 1.2.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 1.2.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 1.2.0 of the Splunk Add-on for Google Cloud Platform does not incorporate any third-party software or libraries.

Version 1.1.0

Version 1.1.0 of the Splunk Add-on for Google Cloud Platform has the same compatibility specifications as version 1.1.0.

Fixed issues

Version 1.1.0 of the Splunk Add-on for Google Cloud Platform fixes the following issues:

Known issues

Version 1.1.0 of the Splunk Add-on for Google Cloud Platform contains the following known issues:

Third-party software attributions

Version 1.1.0 of the Splunk Add-on for Google Cloud Platform does not incorporate any third-party software or libraries.

Hardware and software requirements for the Splunk Add-on for Google Cloud Platform

Splunk admin requirements

To install and configure the Splunk Add-on for google cloud platform, you must be a member of the admin or sc_admin role.

Google Cloud Platform requirements

To use this add-on, you must have a valid Google Cloud Platform account with sufficient permissions to configure the Google Cloud Platform services from which you want to collect data. You must also have permission to create IAM roles or Compute Engine IAM roles and users so that you can set up accounts with the appropriate permissions that the add-on can use to collect data from your Google Cloud Platform services. Enable the following administrative privileges:

• Storage Object Viewer or Storage Object Admin
• Compute Admin

To get data from Google Cloud Platform, enable related APIs in Google Cloud Platform.

Splunk platform requirements

Because this add-on runs on the Splunk platform, all of the system requirements apply to the Splunk software you use to run this add-on.

• For Splunk Enterprise system requirements, see System Requirements in the Splunk Enterprise Installation Manual.
• If you are managing on-premises forwarders to get data into Splunk Cloud, see System Requirements in the Splunk Enterprise Installation Manual, which includes information about forwarders.

The field alias functionality is compatible with the current version of this add-on. The current version of this add-on does not support older field alias configurations.

For more information about the field alias configuration change, refer to the Splunk Enterprise Release Notes.

Installation overview for the Splunk Add-on for Google Cloud Platform

To install and configure this add-on on your supported platform, follow these steps.

1. Install the add-on.
2. Configure the Google Cloud account, or confirm your existing configurations.
3. Configure the Google Cloud Service permissions to match those required by the add-on.
4. Configure your inputs to get your Google Cloud Platform data into the Splunk platform.

Ended: Overview

Installation

Install the Splunk Add-on for Google Cloud Platform

1. Get the Splunk Add-on for Google Cloud Platform by downloading it from Splunkbase or browsing to it using the app browser within Splunk Web.
2. Determine where and how to install this add-on in your deployment, using the tables on this page.
3. Perform any prerequisite steps before installing, if required and specified in the tables below.
4. Complete your installation.

If you need step-by-step instructions on how to install an add-on in your specific deployment environment, see the installation walkthroughs section at the bottom of this page for links to installation instructions specific to a single-instance deployment, distributed deployment, or Splunk Cloud.

Distributed deployment

Use the tables below to determine where and how to install this add-on in a distributed deployment of Splunk Enterprise or any deployment for which you are using forwarders to get your data in. Depending on your environment, your preferences, and the requirements of the add-on, you may need to install the add-on in multiple places.

Where to install this add-on

Unless otherwise noted, all supported add-ons can be safely installed to all tiers of a distributed Splunk platform deployment. See Where to install Splunk add-ons in Splunk Add-ons for more information.

This table provides a reference for installing this specific add-on to a distributed deployment of the Splunk platform.

Splunk platform component	Supported	Required	Action Required/Comments
Search Heads	Yes	Yes	Install this add-on to all search heads where Google Cloud Platform knowledge management is required. As a best practice, turn visibility off on your search heads to prevent data duplication errors. These errors can result from running inputs on your search heads instead of, or in addition to, on your data collection node.
Indexers	Yes	Conditional	Not required, because the parsing operations occur on the heavy forwarders. But while using the HEC token, indexer build installation is required on indexers.
Heavy Forwarders	Yes	Yes	This add-on requires heavy forwarders to perform data collection via modular inputs and to perform the setup and authentication with Google Cloud Platform in Splunk Web.
Universal Forwarders	No	No	This add-on requires heavy forwarders.

Distributed deployment compatibility

This table provides a quick reference for the compatibility of this add-on with Splunk distributed deployment features.

Distributed deployment feature	Supported	Comments
Search Head Clusters	Yes	Disable add-on visibility on search heads. You can install this add-on on a search head cluster for all search-time functionality, but configure inputs on forwarders to avoid duplicate data collection. Before installing this add-on to a cluster, remove the inputs.conf file.
Indexer Clusters	Yes	Before installing this add-on to a cluster, remove the inputs.conf file.
Deployment Server	No	Supported for deploying unconfigured add-ons only. Using a deployment server to deploy the configured add-on to multiple forwarders acting as data collectors causes duplication of data. The add-on uses the credential vault to secure your credentials, and this credential management solution is incompatible with the deployment server.

Installation walkthrough

Refer to Installing add-ons in Splunk Add-ons for detailed instructions describing how to install a Splunk add-on in the following deployment scenarios:

• Single-instance Splunk Enterprise
• Distributed Splunk Enterprise
• Splunk Cloud

Upgrade the Splunk Add-on for Google Cloud Platform

After upgrading the Splunk Add-on for Google Cloud Platform from 4.3.0 to version 4.4.0 or higher, your Splunk platform deployment might receive duplicate events for the BigQuery Billing input(s) in the first invocation after the upgrade.

Standard Upgrade Guide

1. Disable all your inputs before you upgrade the add-on. Otherwise, you might see errors in the log files, resulting in data loss against your already configured inputs.
2. Upgrade the Splunk Add-on for Google Cloud Platform to the required version and follow the version-specific upgrade guide.
3. Enable each desired input to start data collection. Enable Storage Blob inputs in small batches.

Upgrade to versions 4.x or later

To upgrade this add-on from versions 3.2.0 and earlier to versions 4.0.0 and later, perform the following steps.

1. Verify that you are running version 8.0.0 or later of the Splunk platform.
2. Disable all running inputs.
3. Upgrade to the latest version directly from Splunk web UI or upgrade using the downloaded add-on package.
   1. Upgrade to latest version directly from Splunk web UI
      • From the Splunk web home screen, click the gear icon (Manage Apps) next to Apps.
      • Check for Splunk Add-on for Google Cloud Platform in the list of Apps/Add-ons and click “Update to “.
      • Accept the license agreement, enter Splunkbase credentials and download/install the add-on.
   2. Upgrade the add-on using the downloaded add-on package
      • Download latest version of the add-on from Splunkbase
      • Install the Splunk Add-on for Google Cloud Platform
4. Restart your Splunk platform, if you are using a Splunk Enterprise deployment.
5. Enable all inputs.

6. If you have constructed searches or reports that reference the google:gcp:pubsub:audit:auth, google:gcp:pubsub:audit:change, or google:gcp:pubsub:message source types, you must update those queries, so that you add, in addition to the aforementioned source types, the following new source types:

   • google:gcp:pubsub:audit:admin_activity
   • google:gcp:pubsub:audit:data_access
   • google:gcp:pubsub:audit:system_event
   • google:gcp:pubsub:audit:policy_denied
   • google:gcp:pubsub:access_transparency
   • google:gcp:pubsub:platform

   In order to search on GCP data that was ingested into your Splunk platform deployment through this add-on before your upgrade to version 4.0.0 and later, you need the old source types in your query.

   To search on new GCP data that comes in after the upgrade, you need to add the new source types to your queries. Adding source types to your existing search queries and reports, instead of replacing source types, lets you search both your old data and your new data, and materialize results for both of them in the same query.

   For example, the following query lets you search on both the old and new source types:

   Query with old source types before upgrade to 4.0.0 or later:

   Search

   index="main" sourcetype="google:gcp:pubsub:audit:auth" OR sourcetype="google:gcp:pubsub:audit:change" OR sourcetype="google:gcp:pubsub:message"

   Updated query with both old and new source types after upgrade to 4.0.0 or later:

   Search

   index="main" (sourcetype="google:gcp:pubsub:audit:auth" OR sourcetype="google:gcp:pubsub:audit:data_access") OR (sourcetype="google:gcp:pubsub:audit:change" OR sourcetype="google:gcp:pubsub:admin_activity" OR sourcetype="google:gcp:pubsub:system_event") OR (sourcetype="google:gcp:pubsub:message" OR sourcetype="google:gcp:pubsub:audit:policy_denied" OR sourcetype="google:gcp:pubsub:access_transparency" OR sourcetype="google:gcp:pubsub:platform")

   For more information, see the Mapping table for version 4.0.0 sourcetype enhancements section of this topic.

   Event types have not been affected by the version 4.0.0 feature improvements. Searching on event types will stay the same as in previous versions. So if your search queries are based on event types and not source types, skip this step.

7. Save your changes.

   Google has deprecated regular file export of your Cloud Billing data to CSV and JSON. To export your Cloud Billing data for analysis, use Cloud Billing export to BigQuery.

   To ingest Cloud BigQuery Billing data, you must delete your existing billing inputs before you upgrade to versions 3.2.0 and later of this add-on. After upgrading, you can then recreate your billing inputs. See the Configure Cloud BigQuery Billing inputs for the Splunk Add-on for Google Cloud Service topic in this manual.

Mapping table for version 4.0.0 source type enhancements

To better align with the Google Cloud Platform, and to provide a better understanding of the data coming from the cloud, the 4.0.0 release of the Splunk Add-on for Google Cloud Platform contains improvements to sourcetyping that affect the google:gcp:pubsub:audit:auth, google:gcp:pubsub:audit:change, and google:gcp:pubsub:message source types.

These improvements provide more granular sourcetyping on incoming data from your GCP deployment, enhancing your ability to investigate and simplifying the development of dashboards in Splunk that use GCP data. Upgrading to version 4.0.0 or higher will cause any inline searches, pivots, or reports that use these source types to not work for the GCP data that is being ingested after upgrading to version 4.0.0 of this add-on.

To ensure continuity of searches and reports on GCP data coming in after the upgrade to version 4.0.0 or later, review and perform steps in the Upgrade steps section of this topic.

See the following table for information on which source type should be used when updating your search queries after upgrading to versions 4.0.0 and later.

Source type on versions 3.2.0 and earlier	Source type on versions 4.0.0 and later
google:gcp:pubsub:audit:change	google:gcp:pubsub:audit:admin_activity
google:gcp:pubsub:audit:auth	google:gcp:pubsub:audit:data_access
google:gcp:pubsub:audit:change	google:gcp:pubsub:audit:system_event
google:gcp:pubsub:message	google:gcp:pubsub:platform
google:gcp:pubsub:message	google:gcp:pubsub:audit:policy_denied
google:gcp:pubsub:message	google:gcp:pubsub:access_transparency
google:gcp:pubsub:message	google:gcp:pubsub:message

Ended: Installation

Configuration

Configure the Google Cloud account

In order to gather data from the Google Pub/Sub, Google Cloud Monitoring, and Google Cloud Billing using this add-on, create a Google Cloud Service or External account for each project from the Google Cloud Console. For more information, see the Create Service Accounts, Create and delete service account keys, and Configure workload identity federation with AWS or Azure topics in the Google Cloud documentation. A service account’s credentials, obtained from the Google API Console, include a uniquely generated email address, a client ID, and at least one public/private key pair. You might also need to add a predefined role to your service account. After you have your new public/private key pair, save it to your machine.

An external account’s credentials obtained from the Google API Console, includes credential source info, impersonated service account url, token url, audience. After you have your credential, save it to your machine. These credentials are only supported for the AWS and Azure virtual machines.

Your new public/private key pair is generated and downloaded to your machine, and it serves as the only copy of this key. You are responsible for storing it securely.

To get data from Google Cloud Platform, you need to enable related APIs in Google Cloud Platform. For more information, see the Google Cloud storage APIs & Reference and Getting Endpoints Quickstart documentation.

Configure the Google Cloud Platform service permissions

To gather data from buckets via Storage you must have the Viewer or Admin IAM roles in the project to create, delete, or modify a bucket. The following table shows details of the IAM roles.

Input Name	Role Title	Logging Permissions	Resource Type
Cloud Monitoring	Monitoring Viewer roles/monitoring.viewer	cloudnotifications.activities.list monitoring.alertPolicies.get monitoring.alertPolicies.list monitoring.dashboards.get monitoring.dashboards.list monitoring.groups.get monitoring.groups.list monitoring.metricDescriptors.get monitoring.metricDescriptors.list monitoring.monitoredResourceDescriptors.get monitoring.monitoredResourceDescriptors.list monitoring.notificationChannelDescriptors.get monitoring.notificationChannelDescriptors.list monitoring.notificationChannels.get monitoring.notificationChannels.list monitoring.publicWidgets.get monitoring.publicWidgets.list monitoring.services.get monitoring.services.list monitoring.slos.get monitoring.slos.list monitoring.snoozes.get monitoring.snoozes.list monitoring.timeSeries.list monitoring.uptimeCheckConfigs.get monitoring.uptimeCheckConfigs.list opsconfigmonitoring.resourceMetadata.list resourcemanager.projects.get resourcemanager.projects.list stackdriver.projects.get stackdriver.resourceMetadata.list	Read-only access to get and list information about all monitoring data and configuration.
Storage bucket	Storage Admin roles/storage.admin	firebase.projects.get orgpolicy.policy.get recommender.iamPolicyInsights.get recommender.iamPolicyInsights.list recommender.iamPolicyInsights.update recommender.iamPolicyRecommendations.get recommender.iamPolicyRecommendations.list recommender.iamPolicyRecommendations.update resourcemanager.projects.get resourcemanager.projects.list storage.buckets.create storage.buckets.createTagBinding storage.buckets.delete storage.buckets.deleteTagBinding storage.buckets.enableObjectRetention storage.buckets.get storage.buckets.getIamPolicy storage.buckets.getObjectInsights storage.buckets.list storage.buckets.listEffectiveTags storage.buckets.listTagBindings storage.buckets.setIamPolicy storage.buckets.update storage.managedFolders.create storage.managedFolders.delete storage.managedFolders.get storage.managedFolders.getIamPolicy storage.managedFolders.list storage.managedFolders.setIamPolicy storage.multipartUploads.abort storage.multipartUploads.create storage.multipartUploads.list storage.multipartUploads.listParts storage.objects.create storage.objects.delete storage.objects.get storage.objects.getIamPolicy storage.objects.list storage.objects.overrideUnlockedRetention storage.objects.setIamPolicy storage.objects.setRetention storage.objects.update	Grants full control of buckets and objects.
Cloud Pub/Sub Based Bucket	Pub/Sub Viewer roles/pubsub.viewer Pub/Sub Subscriber roles/pubsub.subscribe Storage Object Viewer roles/storage.objectViewer	pubsub.schemas.get pubsub.schemas.list pubsub.schemas.listRevisions pubsub.schemas.validate pubsub.snapshots.get pubsub.snapshots.list pubsub.subscriptions.get pubsub.subscriptions.list pubsub.topics.get pubsub.topics.list resourcemanager.projects.get serviceusage.quotas.get serviceusage.services.get Serviceusage.services.list pubsub.snapshots.seek pubsub.subscriptions.consume pubsub.topics.attachSubscription storage.objects.get storage.objects.list	View topics, subscriptions, and snapshots. Consume messages from a subscription, attach subscriptions to a topic, and seek to a snapshot. Cannot write or create GCS resources.
Metadata/Compute Engine	Compute Viewer roles/compute.viewer	compute.acceleratorTypes.get compute.acceleratorTypes.list compute.addresses.get compute.addresses.list compute.autoscalers.get compute.autoscalers.list compute.backendBuckets.get compute.backendBuckets.getIamPolicy compute.backendBuckets.list compute.backendBuckets.listEffectiveTags compute.backendBuckets.listTagBindings compute.backendServices.get compute.backendServices.getIamPolicy compute.backendServices.list compute.backendServices.listEffectiveTags compute.backendServices.listTagBindings compute.commitments.get compute.commitments.list compute.diskTypes.get compute.diskTypes.list compute.disks.get compute.disks.getIamPolicy compute.disks.list compute.disks.listEffectiveTags compute.disks.listTagBindings compute.externalVpnGateways.get compute.externalVpnGateways.list compute.firewallPolicies.get compute.firewallPolicies.getIamPolicy compute.firewallPolicies.list compute.firewallPolicies.listEffectiveTags compute.firewallPolicies.listTagBindings compute.firewalls.get compute.firewalls.list compute.firewalls.listEffectiveTags compute.firewalls.listTagBindings compute.forwardingRules.get compute.forwardingRules.list compute.forwardingRules.listEffectiveTags compute.forwardingRules.listTagBindings compute.futureReservations.get compute.futureReservations.getIamPolicy compute.futureReservations.list compute.globalAddresses.get compute.globalAddresses.list compute.globalForwardingRules.get compute.globalForwardingRules.list compute.globalForwardingRules.listEffectiveTags compute.globalForwardingRules.listTagBindings compute.globalForwardingRules.pscGet compute.globalNetworkEndpointGroups.get compute.globalNetworkEndpointGroups.list compute.globalNetworkEndpointGroups.listEffectiveTags compute.globalNetworkEndpointGroups.listTagBindings compute.globalOperations.get compute.globalOperations.getIamPolicy compute.globalOperations.list compute.globalPublicDelegatedPrefixes.get compute.globalPublicDelegatedPrefixes.list compute.healthChecks.get compute.healthChecks.list compute.healthChecks.listEffectiveTags compute.healthChecks.listTagBindings compute.httpHealthChecks.get compute.httpHealthChecks.list compute.httpHealthChecks.listEffectiveTags compute.httpHealthChecks.listTagBindings compute.httpsHealthChecks.get compute.httpsHealthChecks.list compute.httpsHealthChecks.listEffectiveTags compute.httpsHealthChecks.listTagBindings compute.images.get compute.images.getFromFamily compute.images.getIamPolicy compute.images.list compute.images.listEffectiveTags compute.images.listTagBindings compute.instanceGroupManagers.get compute.instanceGroupManagers.list compute.instanceGroupManagers.listEffectiveTags compute.instanceGroupManagers.listTagBindings compute.instanceGroups.get compute.instanceGroups.list compute.instanceSettings.get compute.instanceTemplates.get compute.instanceTemplates.getIamPolicy compute.instanceTemplates.list compute.instances.get compute.instances.getEffectiveFirewalls compute.instances.getGuestAttributes compute.instances.getIamPolicy compute.instances.getScreenshot compute.instances.getSerialPortOutput compute.instances.getShieldedInstanceIdentity compute.instances.getShieldedVmIdentity compute.instances.list compute.instances.listEffectiveTags compute.instances.listReferrers compute.instances.listTagBindings compute.instantSnapshots.get compute.instantSnapshots.getIamPolicy compute.instantSnapshots.list compute.interconnectAttachments.get compute.interconnectAttachments.list compute.interconnectLocations.get compute.interconnectLocations.list compute.interconnectRemoteLocations.get compute.interconnectRemoteLocations.list compute.interconnects.get compute.interconnects.list compute.licenseCodes.get compute.licenseCodes.getIamPolicy compute.licenseCodes.list compute.licenses.get compute.licenses.getIamPolicy compute.licenses.list compute.machineImages.get compute.machineImages.getIamPolicy compute.machineImages.list compute.machineTypes.get compute.machineTypes.list compute.maintenancePolicies.get compute.maintenancePolicies.getIamPolicy compute.maintenancePolicies.list compute.networkAttachments.get compute.networkAttachments.getIamPolicy compute.networkAttachments.list compute.networkEdgeSecurityServices.get compute.networkEdgeSecurityServices.list compute.networkEndpointGroups.get compute.networkEndpointGroups.getIamPolicy compute.networkEndpointGroups.list compute.networkEndpointGroups.listEffectiveTags compute.networkEndpointGroups.listTagBindings compute.networks.get compute.networks.getEffectiveFirewalls compute.networks.getRegionEffectiveFirewalls compute.networks.list compute.networks.listEffectiveTags compute.networks.listPeeringRoutes compute.networks.listTagBindings compute.nodeGroups.get compute.nodeGroups.getIamPolicy compute.nodeGroups.list compute.nodeTemplates.get compute.nodeTemplates.getIamPolicy compute.nodeTemplates.list compute.nodeTypes.get compute.nodeTypes.list compute.organizations.listAssociations compute.packetMirrorings.get compute.packetMirrorings.list compute.projects.get compute.publicAdvertisedPrefixes.get compute.publicAdvertisedPrefixes.list compute.publicDelegatedPrefixes.get compute.publicDelegatedPrefixes.list compute.regionBackendServices.get compute.regionBackendServices.getIamPolicy compute.regionBackendServices.list compute.regionBackendServices.listEffectiveTags compute.regionBackendServices.listTagBindings compute.regionFirewallPolicies.get compute.regionFirewallPolicies.getIamPolicy compute.regionFirewallPolicies.list compute.regionFirewallPolicies.listEffectiveTags compute.regionFirewallPolicies.listTagBindings compute.regionHealthCheckServices.get compute.regionHealthCheckServices.list compute.regionHealthChecks.get compute.regionHealthChecks.list compute.regionHealthChecks.listEffectiveTags compute.regionHealthChecks.listTagBindings compute.regionNetworkEndpointGroups.get compute.regionNetworkEndpointGroups.list compute.regionNetworkEndpointGroups.listEffectiveTags compute.regionNetworkEndpointGroups.listTagBindings compute.regionNotificationEndpoints.get compute.regionNotificationEndpoints.list compute.regionOperations.get compute.regionOperations.getIamPolicy compute.regionOperations.list compute.regionSecurityPolicies.get compute.regionSecurityPolicies.list compute.regionSecurityPolicies.listEffectiveTags compute.regionSecurityPolicies.listTagBindings compute.regionSslCertificates.get compute.regionSslCertificates.list compute.regionSslCertificates.listEffectiveTags compute.regionSslCertificates.listTagBindings compute.regionSslPolicies.get compute.regionSslPolicies.list compute.regionSslPolicies.listAvailableFeatures compute.regionTargetHttpProxies.get compute.regionTargetHttpProxies.list compute.regionTargetHttpProxies.listEffectiveTags compute.regionTargetHttpProxies.listTagBindings compute.regionTargetHttpsProxies.get compute.regionTargetHttpsProxies.list compute.regionTargetHttpsProxies.listEffectiveTags compute.regionTargetHttpsProxies.listTagBindings compute.regionTargetTcpProxies.get compute.regionTargetTcpProxies.list compute.regionUrlMaps.get compute.regionUrlMaps.list compute.regionUrlMaps.listEffectiveTags compute.regionUrlMaps.listTagBindings compute.regionUrlMaps.validate compute.regions.get compute.regions.list compute.reservations.get compute.reservations.list compute.resourcePolicies.get compute.resourcePolicies.getIamPolicy compute.resourcePolicies.list compute.routers.get compute.routers.getRoutePolicy compute.routers.list compute.routers.listBgpRoutes compute.routers.listRoutePolicies compute.routes.get compute.routes.list compute.routes.listEffectiveTags compute.routes.listTagBindings compute.securityPolicies.get compute.securityPolicies.getIamPolicy compute.securityPolicies.list compute.securityPolicies.listEffectiveTags compute.securityPolicies.listTagBindings compute.serviceAttachments.get compute.serviceAttachments.getIamPolicy compute.serviceAttachments.list compute.snapshotSettings.get compute.snapshots.get compute.snapshots.getIamPolicy compute.snapshots.list compute.snapshots.listEffectiveTags compute.snapshots.listTagBindings compute.sslCertificates.get compute.sslCertificates.list compute.sslCertificates.listEffectiveTags compute.sslCertificates.listTagBindings compute.sslPolicies.get compute.sslPolicies.list compute.sslPolicies.listAvailableFeatures compute.sslPolicies.listEffectiveTags compute.sslPolicies.listTagBindings compute.storagePools.get compute.storagePools.getIamPolicy compute.storagePools.list compute.subnetworks.get compute.subnetworks.getIamPolicy compute.subnetworks.list compute.subnetworks.listEffectiveTags compute.subnetworks.listTagBindings compute.targetGrpcProxies.get compute.targetGrpcProxies.list compute.targetHttpProxies.get compute.targetHttpProxies.list compute.targetHttpProxies.listEffectiveTags compute.targetHttpProxies.listTagBindings compute.targetHttpsProxies.get compute.targetHttpsProxies.list compute.targetHttpsProxies.listEffectiveTags compute.targetHttpsProxies.listTagBindings compute.targetInstances.get compute.targetInstances.list compute.targetInstances.listEffectiveTags compute.targetInstances.listTagBindings compute.targetPools.get compute.targetPools.list compute.targetPools.listEffectiveTags compute.targetPools.listTagBindings compute.targetSslProxies.get compute.targetSslProxies.list compute.targetSslProxies.listEffectiveTags compute.targetSslProxies.listTagBindings compute.targetTcpProxies.get compute.targetTcpProxies.list compute.targetTcpProxies.listEffectiveTags compute.targetTcpProxies.listTagBindings compute.targetVpnGateways.get compute.targetVpnGateways.list compute.urlMaps.get compute.urlMaps.list compute.urlMaps.listEffectiveTags compute.urlMaps.listTagBindings compute.urlMaps.validate compute.vpnGateways.get compute.vpnGateways.list compute.vpnTunnels.get compute.vpnTunnels.list compute.zoneOperations.get compute.zoneOperations.getIamPolicy compute.zoneOperations.list compute.zones.get compute.zones.list resourcemanager.projects.get resourcemanager.projects.list serviceusage.quotas.get serviceusage.services.get serviceusage.services.list	Read-only access to get and list information about all Compute Engine resources, including instances, disks, and firewalls. Allows getting and listing information about disks, images, and snapshots, but does not allow reading the data stored on them.
Metadata/Kubernetes	Kubernetes Engine Viewer roles/container.viewer Kubernetes Engine Service Agent roles/container.serviceAgent	container.apiServices.get container.apiServices.getStatus container.apiServices.list container.auditSinks.get container.auditSinks.list container.backendConfigs.get container.backendConfigs.list container.bindings.get container.bindings.list container.certificateSigningRequests.get container.certificateSigningRequests.getStatus container.certificateSigningRequests.list container.clusterRoleBindings.get container.clusterRoleBindings.list container.clusterRoles.get container.clusterRoles.list container.clusters.connect container.clusters.get container.clusters.list container.componentStatuses.get container.componentStatuses.list container.configMaps.get container.configMaps.list container.controllerRevisions.get container.controllerRevisions.list container.cronJobs.get container.cronJobs.getStatus container.cronJobs.list container.csiDrivers.get container.csiDrivers.list container.csiNodeInfos.get container.csiNodeInfos.list container.csiNodes.get container.csiNodes.list container.customResourceDefinitions.get container.customResourceDefinitions.getStatus container.customResourceDefinitions.list container.daemonSets.get container.daemonSets.getStatus container.daemonSets.list container.deployments.get container.deployments.getScale container.deployments.getStatus container.deployments.list container.endpointSlices.get container.endpointSlices.list container.endpoints.get container.endpoints.list container.events.get container.events.list container.frontendConfigs.get container.frontendConfigs.list container.horizontalPodAutoscalers.get container.horizontalPodAutoscalers.getStatus container.horizontalPodAutoscalers.list container.ingresses.get container.ingresses.getStatus container.ingresses.list container.initializerConfigurations.get container.initializerConfigurations.list container.jobs.get container.jobs.getStatus container.jobs.list container.leases.get container.leases.list container.limitRanges.get container.limitRanges.list container.managedCertificates.get container.managedCertificates.list container.mutatingWebhookConfigurations.get container.mutatingWebhookConfigurations.list container.namespaces.get container.namespaces.getStatus container.namespaces.list container.networkPolicies.get container.networkPolicies.list container.nodes.get container.nodes.getStatus container.nodes.list container.operations.get container.operations.list container.persistentVolumeClaims.get container.persistentVolumeClaims.getStatus container.persistentVolumeClaims.list container.persistentVolumes.get container.persistentVolumes.getStatus container.persistentVolumes.list container.petSets.get container.petSets.list container.podDisruptionBudgets.get container.podDisruptionBudgets.getStatus container.podDisruptionBudgets.list container.podPresets.get container.podPresets.list container.podSecurityPolicies.get container.podSecurityPolicies.list container.podTemplates.get container.podTemplates.list container.pods.get container.pods.getStatus container.pods.list container.priorityClasses.get container.priorityClasses.list container.replicaSets.get container.replicaSets.getScale container.replicaSets.getStatus container.replicaSets.list container.replicationControllers.get container.replicationControllers.getScale container.replicationControllers.getStatus container.replicationControllers.list container.resourceQuotas.get container.resourceQuotas.getStatus container.resourceQuotas.list container.roleBindings.get container.roleBindings.list container.roles.get container.roles.list container.runtimeClasses.get container.runtimeClasses.list container.scheduledJobs.get container.scheduledJobs.list container.serviceAccounts.get container.serviceAccounts.list container.services.get container.services.getStatus container.services.list container.statefulSets.get container.statefulSets.getScale container.statefulSets.getStatus container.statefulSets.list container.storageClasses.get container.storageClasses.list container.storageStates.get container.storageStates.getStatus container.storageStates.list container.storageVersionMigrations.get container.storageVersionMigrations.getStatus container.storageVersionMigrations.list container.thirdPartyObjects.get container.thirdPartyObjects.list container.thirdPartyResources.get container.thirdPartyResources.list container.tokenReviews.create container.updateInfos.get container.updateInfos.list container.validatingWebhookConfigurations.get container.validatingWebhookConfigurations.list container.volumeAttachments.get container.volumeAttachments.getStatus container.volumeAttachments.list container.volumeSnapshotClasses.get container.volumeSnapshotClasses.list container.volumeSnapshotContents.get container.volumeSnapshotContents.getStatus container.volumeSnapshotContents.list container.volumeSnapshots.get container.volumeSnapshots.list recommender.containerDiagnosisInsights.get recommender.containerDiagnosisInsights.list recommender.containerDiagnosisRecommendations.get recommender.containerDiagnosisRecommendations.list recommender.locations.get recommender.locations.list recommender.networkAnalyzerGkeConnectivityInsights.get recommender.networkAnalyzerGkeConnectivityInsights.list recommender.networkAnalyzerGkeIpAddressInsights.get recommender.networkAnalyzerGkeIpAddressInsights.list resourcemanager.projects.get resourcemanager.projects.list	Read-only access to Kubernetes Engine resources.
Metadata/VPC	Serverless VPC Access Viewer roles/vpcaccess.viewer	resourcemanager.projects.get resourcemanager.projects.list vpcaccess.connectors.get vpcaccess.connectors.list vpcaccess.locations.list vpcaccess.operations.get vpcaccess.operations.list	Viewer of all Serverless VPC Access resources
Metadata/Cloud Storage	Storage Admin roles/storage.admin	firebase.projects.get orgpolicy.policy.get recommender.iamPolicyInsights.get recommender.iamPolicyInsights.list recommender.iamPolicyInsights.update recommender.iamPolicyRecommendations.get recommender.iamPolicyRecommendations.list recommender.iamPolicyRecommendations.update resourcemanager.projects.get resourcemanager.projects.list storage.anywhereCaches.create storage.anywhereCaches.disable storage.anywhereCaches.get storage.anywhereCaches.list storage.anywhereCaches.pause storage.anywhereCaches.resume storage.anywhereCaches.update storage.bucketOperations.cancel storage.bucketOperations.get storage.bucketOperations.list storage.buckets.create storage.buckets.createTagBinding storage.buckets.delete storage.buckets.deleteTagBinding storage.buckets.enableObjectRetention storage.buckets.get storage.buckets.getIamPolicy storage.buckets.getObjectInsights storage.buckets.list storage.buckets.listEffectiveTags storage.buckets.listTagBindings storage.buckets.restore storage.buckets.setIamPolicy storage.buckets.update storage.managedFolders.create storage.managedFolders.delete storage.managedFolders.get storage.managedFolders.getIamPolicy storage.managedFolders.list storage.managedFolders.setIamPolicy storage.multipartUploads.abort storage.multipartUploads.create storage.multipartUploads.list storage.multipartUploads.listParts storage.objects.create storage.objects.delete storage.objects.get storage.objects.getIamPolicy storage.objects.list storage.objects.overrideUnlockedRetention storage.objects.restore storage.objects.setIamPolicy storage.objects.setRetention storage.objects.update	Grants access to view objects and their metadata, excluding ACLs. Can also list the objects in a bucket.
BigQuery	BigQuery Data Viewer roles/bigquery.dataViewer	bigquery.datasets.get bigquery.datasets.getIamPolicy bigquery.models.export bigquery.models.getData bigquery.models.getMetadata bigquery.models.list bigquery.routines.get bigquery.routines.list bigquery.tables.createSnapshot bigquery.tables.export bigquery.tables.get bigquery.tables.getData bigquery.tables.getIamPolicy bigquery.tables.list bigquery.tables.replicateData resourcemanager.projects.get resourcemanager.projects.list	Access to view datasets and all of their contents When applied to a project, access to run queries, create datasets, read dataset metadata, and list tables. When applied to a dataset, access to read dataset metadata and list tables within the dataset.
BigQuery	BigQuery User roles/bigquery.user	bigquery.bireservations.get bigquery.capacityCommitments.get bigquery.capacityCommitments.list bigquery.config.get bigquery.datasets.create bigquery.datasets.get bigquery.datasets.getIamPolicy bigquery.jobs.create bigquery.jobs.list bigquery.models.list bigquery.readsessions.create bigquery.readsessions.getData bigquery.readsessions.update bigquery.reservationAssignments.list bigquery.reservationAssignments.search bigquery.reservations.get bigquery.reservations.list bigquery.routines.list bigquery.savedqueries.get bigquery.savedqueries.list bigquery.tables.list bigquery.transfers.get bigquerymigration.translation.translate resourcemanager.projects.get resourcemanager.projects.list	Access to view datasets and all of their contents When applied to a project, access to run queries, create datasets, read dataset metadata, and list tables. When applied to a dataset, access to read dataset metadata and list tables within the dataset.
Cloud pub/sub lite	Pub/Sub Lite Viewer roles/pubsublite.viewer	pubsublite.operations.get pubsublite.operations.list pubsublite.reservations.get pubsublite.reservations.list pubsublite.reservations.listTopics pubsublite.subscriptions.get pubsublite.subscriptions.getCursor pubsublite.subscriptions.list pubsublite.topics.get pubsublite.topics.getPartitions pubsublite.topics.list pubsublite.topics.listSubscriptions	View topics, subscriptions and reservations Consume messages from a subscription, attach subscriptions to a topic, and seek a snapshot. Access to view all resources/projects
Cloud pub/sub lite	Pub/Sub Lite Subscriber roles/pubsublite.subscriber	pubsublite.locations.openKafkaStream pubsublite.operations.get pubsublite.subscriptions.getCursor pubsublite.subscriptions.seek pubsublite.subscriptions.setCursor pubsublite.subscriptions.subscribe pubsublite.topics.computeHeadCursor pubsublite.topics.computeMessageStats pubsublite.topics.computeTimeCursor pubsublite.topics.getPartitions pubsublite.topics.subscribe	View topics, subscriptions and reservations Consume messages from a subscription, attach subscriptions to a topic, and seek a snapshot. Access to view all resources/projects
Cloud pub/sub lite	Serverless VPC Access Viewer roles/vpcaccess.viewer	Resourcemanager.projects.get vpcaccess.connectors.get vpcaccess.connectors.list vpcaccess.locations.list vpcaccess.operations.get vpcaccess.operations.list	View topics, subscriptions and reservations Consume messages from a subscription, attach subscriptions to a topic, and seek a snapshot. Access to view all resources/projects
Cloud pub/sub	Pub/Sub Viewer roles/pubsub.viewer	pubsub.schemas.get pubsub.schemas.list pubsub.schemas.listRevisions pubsub.schemas.validate pubsub.snapshots.get pubsub.snapshots.list pubsub.subscriptions.get pubsub.subscriptions.list pubsub.topics.get pubsub.topics.list resourcemanager.projects.get serviceusage.quotas.get serviceusage.services.get serviceusage.services.list	View topics, subscriptions, and snapshots. Consume messages from a subscription, attach subscriptions to a topic, and seek to a snapshot.
Cloud pub/sub	Pub/Sub Subscriber roles/pubsub.subscribe	pubsub.snapshots.seek pubsub.subscriptions.consume pubsub.topics.attachSubscription	View topics, subscriptions, and snapshots. Consume messages from a subscription, attach subscriptions to a topic, and seek to a snapshot.

Configure billing export to Google Cloud Platform

To get your daily usage and cost estimates data in Splunk Add-on for Google Cloud Platform, you must enable billing export data in your Google Cloud Platform instance using your Google login credentials. For more details, see the Export Billing Data to a File topic in the Google Cloud documentation.

Configure log export to Google Cloud Pub/Sub

To gather data from activity logs via the Pub/Sub API, use your Google credentials to configure log export to Cloud Pub/Sub in your Google Developers Console. You must also have the Owner or the Logging/Logs Configuration Writer IAM roles in the project to create, delete, or modify a sink. See the following details of IAM roles:

Role Name	Role Title	Logging Permissions	Resource Type
roles/owner	Owner	roles/editor logging permissions logging.privateLogEntries.list logging.sinks.{create, delete, update}	project
roles/logging.configWriter	Logs Configuration Writer	logging.exclusions.{list, create, get, update, delete} logging.logMetrics.{list, create, get, update, delete} logging.logs.list logging.logServiceIndexes.list logging.logServices.list logging.sinks.{list, create, get, update, delete} resourcemanager.projects.get	project, organization, folder, billing account

For more information, see the Configure and manage sinks topic in the Google Cloud documentation.

Set up the Splunk Add-on for Google Cloud Platform

You can configure the add-on either through Splunk Web or by making changes directly in the configuration files. Due to the complexity of the setup, configure the add-on in Splunk Web.

Configure the Splunk Add-on for Google Cloud Platform using Splunk Web

To configure the Splunk Add-on for Google Cloud Platform using Splunk Web, complete the following steps:

1. Go to the Splunk Add-on for Google Cloud Platform configuration page, either by clicking the name of the add-on on the left navigation banner on the home page, or by going to Manage Apps, and then clicking Launch App in the row of Splunk Add-on for Google Cloud Platform.
2. Click Configuration tab to set up Google credentials, proxy, and logging level.
3. On the Google Credentials tab, click Add.
4. Then enter a name, select Account Type and paste the GoogleAccount JSON object you created in Create account in the Google Account Credentials field. Then click Add.
5. If you are using a proxy, click on the Proxy tab, check the Enable Proxy checkbox and fill in the fields to specify the Host, Port, Username, and Password and then click Save.
   • (Optional) If you checked Enable Proxy, check the DNS resolution box if you want to perform DNS resolution through your proxy.
   • (Optional) If you checked Enable Proxy, select the type of proxy to use in the Proxy Type field. Supported proxy types are http and socks5
6. (Optional) If you want to change the Logging level, click on the Logging tab, select a new level from the drop down menu, and click Save.

Set up the add-on using configuration files

Configure credentials of the Splunk Add-on for Google Cloud Platform by completing the following steps:

1. Create a file named google_cloud_credentials.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create a stanza in google_cloud_credentials.conf using the following template:

   [<name>]
   google_credentials = <value> # Google account key that is in json format and can be downloaded from Google admin console.
   account_type = <value> # Google credential type
   

   • You can add multiple Google credentials in google_cloud_credentials.conf. You need to remove all the line breaks in the JSON file to make it in one line, and then paste it to google_cloud_credentials.conf.

   For example, remove the line breaks in the following JSON file:

   {
     "type": "service_account",
     "project_id": "my-project",
     "private_key_id": "32a3be8f2f0dcfe967ea558e486deaereacfas0c2497e",
   }
   

   Then, paste the following into the google_cloud_credentials.conf file:

     google_credentials={"type": "service_account","project_id": "my-project","private_key_id": "32a3be8f2f0dcfe967ea558e486deaereacfas0c2497e",}
   

Configure proxy and logging levels of the Splunk Add-on for Google Cloud Platform

Configure proxy and logging levels of the Splunk Add-on for Google Cloud Platform by completing the following steps:

1. Copy the google_global_settings.conf file from $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/default to $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.
2. Open the local version of the file in a text editor.
3. Provide the necessary values and change default values as you see fit (See the google_cloud_global_settings.conf.spec, contained in the $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README directory for reference).
   • Enable the proxy by setting proxy_enabled to 1.
   • Change the proxy_type to http, or socks5 if necessary.
   • Change the proxy_rdns to 1 if you want the DNS lookup to go through the proxy. Leave it at 0 if you want to use the local machine to do a DNS lookup.
   • Change loglevel to DEBUG or ERROR if desired.
   • If you want the Splunk platform to index only the events when the scan is completed successfully, skipping those that were aborted or are still running, change index_events_for_unsuccessful_scans to 0. This parameter is not exposed in Splunk Web.

After updating google_global_settings.conf, restart the Splunk platform in order to make the changes and encrypt the proxy username and password.

Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform

Version 4.0.0 of the Splunk Add-on for Google Cloud Platform introduced sourcetype changes to the Pub/Sub input to support Data Manager compatibility. Check with your Splunk platform administrator to verify your prebuilt Splunk searches.

Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Configure Cloud Pub/Sub inputs using the Splunk Web

Follow these steps to configure Cloud Pub/Sub inputs.

1. Click Create New Input in the Inputs tab, and then choose Cloud Pub/Sub.
2. Enter the Name, Credentials, Projects, Pub/Sub Subscriptions and Index using the information in the inputs parameter table.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings, and then Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Pub/Sub inputs using the configuration file

Follow these steps to configure Cloud Pub/Sub inputs.

1. Create a file named google_pubsub_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template (See the google_cloud_pubsub_inputs.conf.spec, contained in the $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README directory for reference).

   [<name>]
   google_credentials_name = <value>
   google_project = <value>
   google_subscriptions = <value>
   index = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to your configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web or in a configuration file:

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Google Cloud Pub/Sub input.
google_credentials_name	Credentials	Stanza name defined in google_cloud_credentials.conf.
google_project	Project	Google pubsub project ID.
google_subscriptions	Pub/Sub Subscriptions	Google pubsub subscription names. You can add several subscriptions separated by “,”.
index	Index	The index in which to store Google Cloud Pub/Sub data.

Configure Cloud Monitoring inputs for the Splunk Add-on for Google Cloud Platform

Configure Cloud Monitoring inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file. Use the information in the inputs parameters table below in order to send your events and metrics to a Splunk platform event index.

Configure Cloud Monitoring inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Cloud Monitoring.
2. Enter the Name, Credentials, Project, Monitored Projects, Cloud Monitor Metrics, Interval, Start Time and Index using the information in the inputs parameter table.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Monitoring inputs using configuration file

1. Create a file named google_cloud_monitor_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template (Use the google_cloud_monitor_inputs.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference).

   [<name>]
   google_credentials_name = <value>
   google_project = <value>
   google_monitored_projects = <value>
   google_metrics = <value>
   polling_interval = <value>
   oldest = <value>
   index = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Google Cloud Monitoring input
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google project ID
google_metrics	Cloud Monitor Metrics	Google cloud monitor metrics. You can add several google metrics separated by “,”. You can set it to .* to add all the metrics.
google_monitored_projects	Monitored Projects	Google Cloud Platform monitored projects. You can add several Google Cloud monitored projects, separated by commas (,), and you can set it to All, in order to add all the monitored projects.
polling_interval	Interval	Data collection interval in seconds. The default is 300 seconds.
oldest	Start Date Time	The add-on starts collecting data with a date later than this time. The format is YYYY-DD-MMThh:mm:ss. For example 2016-07-15T09:00:00.
index	Index	The index in which to store Google Cloud Monitoring data.

Configure Cloud BigQuery Billing inputs for the Splunk Add-on for Google Cloud Platform

Google has deprecated regular file export of your Cloud Billing data to CSV and JSON. To export your Cloud Billing data for analysis, use Cloud Billing export to BigQuery.

Configure Cloud BigQuery Billing inputs for the Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below. For ease-of-use, configure your inputs for Cloud Billing via Splunk Web.

Configure Cloud BigQuery Billing inputs using Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Google Cloud BigQuery Billing.
2. Enter the Name, Credentials, Project, BigQuery Dataset, BigQuery Table, Start Date, Interval and Index using the information in the inputs parameter table.

Google Cloud BigQuery Billing generates billing report at 00:00 every day at UTC−07:00 timezone.

For more information, see the Export Cloud Billing data to BigQuery topic in the Google Cloud documentation.

Configure Cloud BigQuery Billing inputs using the configuration file

1. Create a file named google_cloud_billing_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. Use the google_cloud_billing_inputs.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README as a reference).

   [<name>]
   google_credentials_name = <value>
   google_project = <value>
   google_bq_dataset = <value>
   google_bq_table = <value>
   ingestion_start = <value>
   index = <value>
   polling_interval = <value>
   

   3. Save the file and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Google Cloud BigQuery Billing input
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google project ID
google_bq_dataset	BigQuery Dataset	Google BigQuery dataset name
google_bq_table	BigQuery Table	Google BigQuery table name
polling_interval	Interval	Data collection interval in seconds. The default is 3600 seconds.
ingestion_start	Start Date	The add-on starts collecting data with a date later than this. Note: Google Cloud Billing generates a billing report at 00:00 every day at UTC−07:00 timezone.
index	Index	The index in which to store Google Cloud billing data.

Configure Cloud Storage Bucket inputs for Splunk Add-on for Google Cloud Platform

Configure Cloud Storage Bucket inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Configure Cloud Storage Bucket inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Cloud Storage Bucket > Cloud Storage Bucket.
2. Enter the Name, Credentials, Project, Bucket, Interval, Number of Threads and Index, using the information in the inputs parameter table.
3. Save your changes.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Storage Bucket inputs using configuration file

Follow these steps to configure Cloud Storage Bucket inputs.

1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.
2. Create a file named inputs.conf, if it does not already exist.

3. Add the following stanza for Cloud Storage Bucket input:

   [google_cloud_bucket_metadata://<input_stanza_name>]
   bucket_name = <value>
   conf_version = v1
   google_credentials_name = <value>
   google_project = <value>
   index = <value>
   interval = <value>
   number_of_threads = <value>
   

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Google Cloud Storage Bucket input.
bucket_name	Bucket	Google Bucket Name
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google Project ID
index	Index	The index in which to store Google Cloud Storage Bucket.
interval	Interval	Data collection interval in seconds. The default is 3600 seconds.

Advanced settings

Attribute	Corresponding field in Splunk Web	Description
number_of_threads	Number of Threads	Specify the count to determine the number of concurrent file downloads to be ingested into Splunk.

Configure Resource Metadata inputs for Splunk Add-on for Google Cloud Platform

Configure Resource Metadata inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Restart your Splunk instance or toggle the enable input after you make any configuration changes in the configuration files or in Splunk Web.

Configure Resource Metadata inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata.
2. Enter the Name, Credentials, Projects, Zones, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.
3. Save your changes.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Resource Metadata inputs using configuration file

1. Create a file named google_cloud_resource_metadata_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template.

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   google_zones = <value>
   index = <value>
   sourcetype = <value>
   

   3. Save and return to your Splunk instance.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Resource Metadata input.
google_credentials_name	Credentials	The stanza name defined in google_credentials.conf
google_project	Projects	Google resource metadata project ID
google_zones	Zones	A deployment area for Google Cloud resources within a region.
google_apis	APIs	Resources of Compute Engine.
index	Index	The index in which to store Google Cloud Storage Bucket.
sourcetype	Sourcetype	Name of the Sourcetype.

Configure Google Workspace audit logs for the Splunk Add-on for Google Cloud Platform

Configure the HTTP Event Collector (HEC) to ingest Google Workspace (GWS) audit logs.

To Configure Google GWS audit logs for the Splunk Add-on for Google Cloud Platform, perform the following steps.

1. Configure, view, and route audit logs for Google Workspace to Google Cloud. See the View and manage audit logs for Google Workspace topic in the Google Cloud documentation.
2. Share data from your Google Workspace account with services in your organization’s Google Cloud Platform (GCP) account. See the Share data with Google Cloud Platform services topic in the Google Workspace Admin Help documentation.
3. Export your audit logs to your Google Cloud Pub/Sub. See the Configure and manage sinks topic in the Operations Suite manual in the Google Cloud documentation.
4. Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform. See the Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform topic in this manual.

Export your Google Workspace directory user list to the Splunk Add-on for Google Cloud Platform

1. Download the users list and export it to their pubsub topic. See the Download a list of users topic in the Advanced user management section of the Google Workspace Admin Help documentation.
2. Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform. See the Configure Cloud Pub/Sub inputs for Splunk Add-on for Google Cloud Platform topic in this manual.

For information on Assets and Identity extractions and configurations, see the Collect and extract asset and identity data in Splunk Enterprise Security topic in the Administer Splunk Enterprise Security chapter in the Splunk Enterprise Security manual.

Configure Pub/Sub topics in Google Cloud

Configure Pub/Sub topics in Google Cloud to ingest data into your Splunk platform deployment.

1. Navigate to the Google Cloud project you’ve configured to be used for the log aggregation across your organization.
2. Create the Pub/Sub Topics. Navigate to Pub/Sub in your project and create two topics:
   1. A primary topic to hold messages to be delivered.
   2. A secondary, dead-letter topic, to store undeliverable messages when Dataflow cannot stream to the HTTP Event Collector (HEC). For example, a misconfigured HEC SSL certificate, disabled HEC token, or message processing error by Dataflow.
      • Primary: <topic-name>
      • Secondary: <topic-name>
3. Create your subscription to query the both topics created in the last step.
   • Enter any name for your subscription
   • Select the Pub/Sub primary topic created in the previous step
   • Leave the rest of the values default or customize to your organization’s preference
   • Repeat the same steps for your dead-letter topic

4. Create an organization-level aggregated log sink. This lets administrators configure one aggregated sink, and capture all logs across an organization, and projects that should be sent to the Pub/Sub topic created above.

   You cannot create aggregated sinks through the Google Cloud Console. They must be configured and managed through either the API or gcloud CLI tool. Only project-level (non-aggregated) sinks show up in Google Cloud Console at this time.

5. Open a Cloud Shell in the active project

6. Enter the following in the Cloud Shell to create the aggregated sink:

   gcloud logging sinks create <sample-sink> \
   pubsub.googleapis.com/projects/<sample-project-sink>/topics/topic-name --include-children \
   --organization=[organization_id] \
   --log-filter='logName:"organizations/<unique organization identifier>/logs/cloudaudit.googleapis.com"'
   

   (Optional) If you want to export more than GSuite events, modify the --log-filter to capture any additional logs you want to export.

   See the Google Cloud documentation for more information on creating aggregated log sinks.

7. Update permissions for the service account that you created in the previous step. Updating permissions on your service account allows the sink service account to publish messages to your previously created Pub/Sub input topics. To update the permissions, copy the entire name and run the following in the Google Cloud Console:

   1. Open a cloud shell in the active project, or use the existing shell.
   2. Enter the following into the shell.

      gcloud pubsub topics add-iam-policy-binding my-logs \
      --member serviceAccount:<service account name from previous step>\
      --role roles/pubsub.publisher
      

   3. (Optional)Validate the service account and permission association with the following command:

      gcloud logging sinks describe kitchen-sink --organization=organization_id
      

8. Referencing the logging configurations that you set up in the previous steps, configure the Dataflow template to output the logs to the Splunk HEC.

   1. Navigate to Dataflow and select Create New Job From Template

   2. Populate the following fields:

      • Job name (Any)
      • Preferred Region
      • Cloud Dataflow Template: Cloud Pub/Sub to Splunk
      • Pub/Sub Subscription name created in previous steps
      • HEC token, created in previous steps
      • HEC URL, created in previous steps
      • DLT Topic, created in previous steps
      • Source, The token default source value. The Splunk Software assigns this value to data that doesn’t already have a source value set.
      • Sourcetype, The token default sourcetype value. The Splunk software assigns this value to data that doesn’t already have a sourcetype value set.
      • Any bucket name. If you have not created a bucket, navigate to Storage and create a new bucket. The syntax for the bucket name is gs://bucketName.

      • Expand Optional Parameters

        • Set Batch size for sending multiple events to Splunk HEC to 2 (can be adjusted later depending on your volume)
        • Set Maximum Number of Parallel Requests to 8 (can be adjusted later depending on your volume)
        • Set Max workers to 2 (can be adjusted later depending on your volume).

        The default is 20 which will incur unnecessary total Persistent Disk cost if not fully utilized.

   3. Enter any additional settings pertinent to your organization.

   4. Run job.

Configure Compute Engine inputs for Splunk Add-on for Google Cloud Platform

Configure Compute Engine inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

The Resource Metadata input has been renamed to Compute Engine starting in version 4.0.

Configure Compute Engine inputs using Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata, and then choose Compute Engine
2. Enter the Name, Credentials, Project, Zones, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.
3. Save your changes.

Do not navigate to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Compute Engine inputs using configuration files

1. Create a file named google_cloud_resource_metadata_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. See the google_cloud_resource_metadata_inputs.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference).

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   google_zones = <value>
   index = <value>
   sourcetype = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Compute Engine input.
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google Project ID
google_zones	Zones	A deployment area for Google Cloud resources within a region.
google_apis	APIs	Resources of Compute Engine.
index	Index	The index where your Google Cloud Platform data is stored.
sourcetype	Sourcetype	Name of the sourcetype.

Configure Cloud Storage inputs for Splunk Add-on for Google Cloud Platform

Configure Cloud Storage inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Configure Cloud Storage inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata, and then choose Cloud Storage.
2. Enter the Name, Credentials, Projects, Buckets, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.
3. Save your changes.

Do not navigate to the Splunk Add-on for Google Cloud Platform configuration page at Settings, and then Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Storage inputs using configuration file

1. Create a file named google_cloud_resource_metadata_inputs_cloud_storage.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. See the google_cloud_resource_metadata_inputs_cloud_storage.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference.

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   bucket_name = <value>
   index = <value>
   sourcetype = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Cloud Storage input.
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google project ID
bucket_name	Bucket	Google Bucket Name. To collect the data for BucketAccessControls,DefaultObjectAccessControls and ObjectAccessControls the bucket should be non-uniform. For more information, see the “Behavior when enabled” section of the Uniform bucket-level access topic in the Google Cloud documentation.
google_apis	APIs	Resources of Cloud Storage.
index	Index	The index in which Google Cloud Platform Data should be stored.
sourcetype	Sourcetype	Name of the sourcetype.

Configure Kubernetes inputs for Splunk Add-on for Google Cloud Platform

Configure Kubernetes inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Configure Kubernetes inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata, and then choose Kubernetes.
2. Enter the Name, Credentials, Project, Location, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.
3. Save your changes.

Do not navigate to the Splunk Add-on for Google Cloud Platform configuration page at Settings, then Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Kubernetes inputs using configuration file

1. Create a file named google_cloud_resource_metadata_inputs_kubernetes.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. See the google_cloud_resource_metadata_inputs_kubernetes.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference.

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   location_name = <value>
   index = <value>
   sourcetype = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Kubernetes input.
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google project ID
location_name	Location	Supported Google Cloud Platform location. This parameter is optional. If no location is selected then data will be collected for all supported locations.
google_apis	APIs	Resources of Kubernetes.
index	Index	The index in which Google Cloud Platform data should be stored.
sourcetype	Sourcetype	Name of the sourcetype.

Configure VPC Access inputs for Splunk Add-on for Google Cloud Platform

Configure VPC Access inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Configure VPC Access inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata, and then choose VPC Access.
2. Enter the Name, Credentials, Project, Location, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.
3. Save your changes.

Do not navigate to the Splunk Add-on for Google Cloud Platform configuration page at Settings, then Data Inputs, to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure VPC Access inputs using configuration file

1. Create a file named google_cloud_resource_metadata_inputs_vpc_access.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. See the google_cloud_resource_metadata_inputs_vpc_access.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference.

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   location_name = <value>
   index = <value>
   sourcetype = <value>
   

   3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the VPC Access input.
google_credentials_name	Credentials	The stanza name defined in google_cloud_credentials.conf
google_project	Project	Google Project ID
location_name	Location	Supported Location by Google Cloud Platform.This parameter is optional.If no location is selected then data will be collected for all supported locations.
google_apis	APIs	Resources of VPC Access.
index	Index	The index in which Google Cloud Platform data should be stored.
sourcetype	Sourcetype	Name of the sourcetype.

Configure Cloud Pub/Sub Based Bucket inputs for the Splunk Add-on for Google Cloud Platform

Complete the following steps to configure Cloud Pub/Sub Based Bucket inputs for the Splunk Add-on for Google Cloud Platform using Splunk Web or a configuration file.

1. You must manage Google Credentials for the add-on as a prerequisite.
2. Configure Google services for the Pub/Sub Based Bucket input.
3. Configure Google services permissions for the Pub/Sub Based Bucket input.
4. Configure Cloud Pub/Sub Based Bucket inputs either through Splunk Web or configuration files.

Configuration prerequisites

Before you configure Pub/Sub Based Bucket inputs, perform the following tasks:

• Create a Pub/Sub Topic and Subscription to receive notifications, and a second Pub/Sub Topic and Subscription to serve as a dead letter topic.
• Configure Pub/Sub notifications for the Cloud Storage Bucket to send notifications to the topic on the object creation and updation. This lets the bucket notify the add-on that new events were written to the bucket.
• Add Google Credentials to the Splunk Add-on for Google Cloud Platform.

Attach a Notification Configuration to Bucket to receive notifications

Attach notification configuration to the bucket using glcoud CLI in order to allow your Bucket to send notifications to a Pub/Sub Topic. Please follow gcloud storage buckets notifications create Google documentation for the notification configuration.

Best practices

Please take into consideration the following points while configuring your inputs:

• Pub/Sub Based Bucket input only collects data for objects that meet the following criteria:
  • The input must be stored in the Storage Bucket
  • The input must have a notification sent to Pub/Sub Topic
• To achieve high throughput data ingestion from a Storage Bucket, configure multiple Pub/Sub Based Bucket inputs to scale out data collection.
• Set up a Pub/Sub Dead Letter Topic for the Pub/Sub Topic to be used for the input for storing invalid messages. For information about Pub/Sub Dead Letter Topic and how to configure it, see https://cloud.google.com/pubsub/docs/handling-failures in the Google Cloud Pub/Sub documentation.
• Configure the Subscription Acknowledgement deadline to prevent multiple inputs from receiving and processing messages in a subscription more than once. Though the add-on will modify the Acknowledgement deadline when it is close to expiring for a message in processing, It is recommended to set the Acknowledgement deadline to 5 minutes or longer. If the Acknowledgement deadline for a message is reached before the message is fully processed by the input, the message reappears in the subscription and is retrieved and processed again, resulting in duplicate data.

Configure Cloud Pub/Sub Based Bucket inputs using the Splunk Web

Follow these steps to configure Cloud Pub/Sub Based Bucket inputs.

1. Follow the menu path Create New Input > Cloud Storage Bucket > Cloud Pub/Sub Based Bucket.
2. Enter the Name, Credentials, Projects, Pub/Sub Subscription, Index, Sourcetype, Message Batch Size, and Number of Threads using the input parameter table information.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Storage Bucket inputs using configuration file

Follow these steps to configure Cloud Pub/Sub Based Bucket inputs.

1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.
2. Create a file named inputs.conf, if it does not already exist.

3. Add the following stanza for Cloud Pub/Sub Based Bucket input:

   [google_cloud_pubsub_based_bucket://<input_stanza_name>]
   google_credentials_name = <value>
   google_project = <value>
   google_subscriptions = <value>
   index = <value>
   message_batch_size = <value>
   number_of_threads = <value>
   sourcetype = google:gcp:buckets:data
   

   4. Save and restart the Splunk platform.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
input_stanza_name	Name	Enter a unique name of the Cloud Pub/Sub Based Bucket input.
google_credentials_name	Credentials	Google Credentials configured using Configuration > Google Credentials or stanza defined in the google_cloud_credentials.conf
google_project	Project	The project from which you want to collect data.
google_subscriptions	Pub/Sub Subscription	The subscription from which you want to pull messages.
index	Index	The index in which to store the Google Cloud Pub/Sub Based Bucket data.
sourcetype	Sourcetype	The sourcetype to use for this input.
message_batch_size	Message Batch Size	Max number of messages to pull from Pub/Sub in one batch. The default value is 10.
number_of_threads	Number of Threads	The number of threads used to collect data in parallel. The dafault value is 10.

Configure Cloud Pub/Sub Lite inputs for Splunk Add-on for Google Cloud Platform

Configure Cloud Pub/Sub Lite inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

Scaling

• To scale the data ingestion, It is recommended to configure the number of Pub/Sub Lite inputs which should be equal to or less than the number of partitions that you have in the Pub/Sub Lite Topic.
• You might observe data duplication less than 1%

Configure Cloud Pub/Sub Lite inputs using the Splunk Web

Follow these steps to configure Cloud Pub/Sub Lite inputs.

1. Click Create New Input in the Inputs tab, and then choose Cloud Pub/Sub Lite.
2. Enter the Name, Credentials, Projects, Location Type, Region/Zone, Pub/Sub Lite Subscription, Index, Sourcetype, Number of Threads, Messages Outstanding, and Bytes Outstanding using the information in the inputs parameter table.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Cloud Pub/Sub Lite inputs using the configuration file

See the following steps to configure Cloud Pub/Sub Lite inputs:

1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.
2. Create a file named inputs.conf, if it does not already exist.
3. Add the following stanza for Cloud Pub/Sub Lite input:

   1. Input configuration for Regional Location Type

      [google_cloud_pubsub_lite://<input_stanza_name>]
      bytes_outstanding = <value>
      google_credentials_name = <value>
      google_project = <value>
      location = regional
      messages_outstanding = <value>
      number_of_threads = <value>
      pubsublite_regions = <value>
      pubsublite_subscriptions = <value>
      sourcetype = google:gcp:pubsublite:message
      

      2. Input configuration for Zonal Location Type

      [google_cloud_pubsub_lite://<input_stanza_name>]
      bytes_outstanding = <value>
      google_credentials_name = <value>
      google_project = <value>
      location = zonal
      messages_outstanding = <value>
      number_of_threads = <value>
      pubsublite_subscriptions = <value>
      pubsublite_zones = <value>
      sourcetype = google:gcp:pubsublite:message
      

      4. Save and return to your Splunk instance.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web:

Attribute	Corresponding field in Splunk Web	Description
input_stanza_name	Name	Enter a unique name of the Google Cloud Pub/Sub Lite input.
google_credentials_name	Credentials	Stanza name defined in google_cloud_credentials.conf.
google_project	Projects	google project ID.
location	Location Type	Select the Regional Topic or Zonal Topic. If Regional topic is selected then Region dropdown will be available and if Zonal topic is selected then Zone dropdown will be available. The default is Regional.
pubsublite_regions	Region	Select the supported region for Pub/Sub Lite service.
pubsublite_zones	Zone	Select the supported zone for Pub/Sub Lite service.
pubsublite_subscriptions	Pub/Sub Lite Subscription	Google Cloud Pub/Sub Lite subscriptions list based on the selected region/zone.
index	Index	The index in which to store Google Cloud Pub/Sub Lite data.
sourcetype	Sourcetype	The sourcetype to use for this input.
number_of_threads	Number of Threads	The number of threads used to collect Cloud Pub/Sub Lite data in parallel. The default value is 10.
messages_outstanding	Messages Outstanding	Count of after how many max messages, TA will pause receiving messages if we have not acknowledged 1st message. The default value is 1000.
bytes_outstanding	Bytes Outstanding	Count of after how much Megabyte, TA will pause receiving messages if we have not acknowledged 1st message. The default value is 10.

Configure Resource Metadata inputs for Splunk Add-on for Google Cloud Platform

Configure Resource Metadata inputs for Splunk Add-on for Google Cloud Platform using Splunk Web or via configuration file, using the information in the inputs parameters table below.

This input is only supported by versions 3.2.0 and earlier of the Splunk Add-on for Google Cloud Platform. The Resource Metadata input has been renamed to Compute Engine starting in version 4.0.0.

Configure Resource Metadata inputs using the Splunk Web

1. Click Create New Input in the Inputs tab, and then choose Resource Metadata.

2. Enter the Name, Credentials, Project, Zones, APIs with suitable intervals, Index, and Sourcetype using the information in the inputs parameter table.

3. Save your changes.

Do not go to the Splunk Add-on for Google Cloud Platform configuration page under Settings > Data Inputs to configure Google Cloud Platform inputs. This page is not supported for this type of input.

Configure Resource Metadata inputs using configuration file

1. Create a file named google_cloud_resource_metadata_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/local.

2. Create stanzas using the following template. Use the google_cloud_resource_metadata_inputs.conf.spec file in $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/README for reference).

   [<name>]
   google_apis= <value>
   google_credentials_name = <value>
   google_project = <value>
   google_zones = <value>
   index = <value>
   sourcetype = <value>
   

3. Save and return to your Splunk instance.

Restart your Splunk platform after making changes to configuration (.conf) files.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute	Corresponding field in Splunk Web	Description
name	Name	Enter a unique name of the Resource Metadata input.
google_credentials_name	Credentials	The stanza name defined in google_credentials.conf
google_project	Project	Google resource metadata project ID
google_zones	Zones	A deployment area for Google Cloud resources within a region.
google_apis	APIs	Resources of Compute Engine.
index	Index	The index in which to store Google Cloud Resource Metadata.
sourcetype	Sourcetype	Name of the sourcetype.

Ended: Configuration

Troubleshoot

Troubleshoot the Splunk Add-on for Google Cloud Platform

General troubleshooting

For troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.

Data not showing up

If you are upgrading from a version of the Splunk Add-on for Google Cloud Platform earlier than 1.2.0, you must run the following upgrade script in order for your data to be properly sent. python3 $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/bin/tools/migrate_pusbub_input.py

Accessing logs

You can access internal log data for help with troubleshooting. Data collected with these source types does not appear in any dashboards.

Data source	Source type
Logs from *google_cloud_pubsub*.log	google:gcp:pubsub:log
Logs from *google_cloud_monitoring*.log	google:gcp:monitoring:log
Logs from *google_cloud_billing*.log	google:gcp:billing:log
Logs from *google_cloud_pubsub_lite*.log	google:gcp:pubsublite:log
Logs from *google_cloud_pubsub_based_bucket*.log	google:gcp:pubsubbasedbucket:log
Logs from *google_cloud_bucket_metadata*.log	google:gcp:buckets:log
Logs from *google_cloud_resource_metadata*.log	google:gcp:resource_metadata:log
Logs from *google_cloud_resthandler*.log	google:gcp:resthandler:log

Configure log levels

1. Click Splunk Add-on for Google Cloud Platform in your left navigation bar on Splunk Web’s home page.
2. Click Configuration in the app navigation bar.
3. Click the Logging tab.
4. Adjust the log levels for each of the Google Cloud Platform services as needed by changing the default of INFO to one of the other available options, DEBUG or ERROR.
5. (Optional) If you are using pub/sub, restart your Splunk instance to apply changes.

These log level configurations apply only to runtime logs. Some REST endpoint logs from configuration activity log at DEBUG, and some validation logs log at ERROR. These levels cannot be configured.

Large pub/sub subscriptions

For large pub/sub subscriptions, we recommend cloning existing inputs that are ingesting the same subscriptions to increase data throughput and performance. These identical inputs can be in the same instance or in different instances.

To manage a large number of subscriptions to one Splunk instance, aggregate subscriptions belonging to the same Google Cloud Service account into one input to save resources.

Exceed Request Limit

If you see any insufficient tokens for quota group errors such as the following, then you have exceeded the Google Cloud Monitoring request limit, which is 50000 per day. You should limit your request or apply for higher quota in Google:

<"Insufficient tokens for quota group and limit DefaultGroupCLIENT_PROJECT-1d using the limit by ID 342432.">

Python version issues

If you are upgrading from versions 1.2.0 or lower, upgrade your Splunk Enterprise deployment to work with Python 3

1. Plan your Splunk Enterprise upgrade to work with the Python 3 migration.
2. Clean up all pycache files from your add-on’s directory location. For example, $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform

Billing data not ingesting

If you encounter the following message in your internal logs:

Billing ingestion error. You must use the Cloud BigQuery Billing input in order to ingest billing data. You must delete your existing billing inputs, upgrade to version 3.2.0 or later of this add-on, and recreate your billing inputs using the Cloud BigQuery Billing input in order to ingest billing data.

Google has deprecated regular file export of your Cloud Billing data to CSV and JSON. To export your Cloud Billing data for analysis, use Cloud Billing export to BigQuery.

See the Configure Cloud BigQuery Billing inputs for the Splunk Add-on for Google Cloud Service topic in this manual.

Missing fields in Events/Larger events not structured in Pub/Sub input

If you get a larger size of events and face an issue with some missing fields in the Events or Events not structured correctly then perform the steps below:

1. Click on Settings
2. Click on Source types under the DATA section
3. Unclick Show only popular checkbox if it is already checked
4. Search google:gcp:pubsub:message sourcetype
5. Click on Edit, it will open the Edit Source Type: google:gcp:pubsub:message dialogue box
6. Click on Advanced
7. Increase the TRUNCATE value based on Event size and click on the Save button
8. Perform 4 to 7 steps for google:gcp:pubsub:audit:change sourcetype

Account Type information not getting updated in Google Credentials Configuration Tab after editing the existing credentials

Reload the configuration page to reflect the updated Account Type value will.

Manually enable/disable the scripted input for Cloud Storage Bucket

In case of a migration failure for specific cloud storage bucket inputs, the best practice is to initiate a rerun of the migration script. To manually enable or disable the input migration script $SPLUNK_HOME/etc/apps/Splunk_TA_google-cloudplatform/bin/google_cloud_storage_bucket_conf_migrator.py for Cloud Storage Bucket input, navigate to Settings>Data inputs>Scripts.

Ended: Troubleshoot

Reference

Performance reference for the Splunk Add-on for Google Cloud Platform

This page provides reference information on performance testing of the pub/sub, pub/sub lite, and pub/sub based bucket inputs for the Splunk Add-on for Google Cloud Platform. Use this information to enhance the performance of your own Google Cloud Platform collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results should be used as reference information and do not represent performance in all environments.

Version 4.5.0 Cloud Storage Bucket input performance statistics for JSON files

Common architecture setup	Number of inputs	Number of Threads	Number of files	File Size	Events Count	Data Collection Time (in minutes)	Avg CPU (by input)	Avg Memory (by input)
CO2 Stack - Victoria Search Head Cluster 3 search heads (m5.4xlarge) 16 CPU 64 GiB memory 3 indexers (im4gn.large) 1 cluster master (c5.xlarge)	1 input	1	10k	~3MB	6180001	53	3%	0.1%
	1 input	10	10k	~3MB	6180001	12	15%	0.1%
CO2 Stack - Classic Cluster (1 IDM) 1 cluster master (c5.xlarge) 3 search heads (c5.2xlarge) 3 indexers (i3.large) 1 Input Data Manager (IDM) (m5.4xlarge) 16 CPU 64 GiB memory	1 input	1	10k	~3MB	6180001	48	4%	0.1%
	1 input	10	10k	~3MB	6180001	15	13%	0.1%
On Prem Heavy Forwarder (m5.4xlarge) 16 CPU 64 GiB memory	1 input	1	10k	~3MB	6180001	108	2%	0.1%
	1 input	10	10k	~3MB	6180001	14	12%	0.1%

Version 4.5.0 Cloud Storage Bucket input performance statistics for CSV files

Common architecture setup	Number of inputs	Number of Threads	Number of files	File Size	Events Count	Data Collection Time (in minutes)	Avg CPU (by input)	Avg Memory (by input)
CO2 Stack - Victoria Search Head Cluster 3 search heads (m5.4xlarge) 16 CPU 64 GiB memory 3 indexers (im4gn.large) 1 cluster master (c5.xlarge)	1 input	1	100k	~50KB	86159944	225	6%	0.1%
	1 input	10	100k	~50KB	86159944	169	15%	0.1%
CO2 Stack - Classic Cluster (1 IDM) 1 cluster master (c5.xlarge) 3 search heads (c5.2xlarge) 3 indexers (i3.large) 1 Input Data Manager (IDM) (m5.4xlarge) 16 CPU 64 GiB memory	1 input	1	100k	~50KB	86159944	223	5%	0.1%
	1 input	10	100k	~50KB	86159944	167	15%	0.1%
On Prem Heavy Forwarder (m5.4xlarge) 16 CPU 64 GiB memory	1 input	1	100k	~50KB	86159944	445	3%	0.1%
	1 input	10	100k	~50KB	86159944	155	15%	0.1%

Version 4.2.0 Pub/Sub Lite input performance statistics

There following tests a configuration of 20 partitions for Version 4.2.0 Pub/Sub Lite input performance.

Common architecture setup	Scenario	Event type	Event Size	Number of threads	Ingest Rate (KB/min)	CPU	RAM
CO2 Stack - Victoria Search Head Cluster 1 cluster master (c5.xlarge) 3 search heads (c6i.4xlarge) 16 CPU 16 GiB Memory 3 indexer clusters (is4gen.4xlarge) 16 CPU core 96 GiB Memory Default TA configs	1 input	JSON	1 KB	10	212274	13.58%	11.8%
	1 input	JSON	1 KB	64	2202370	14.25%	11.7%
	20 input	JSON	1 KB	10	2020845	22.88%	9.74%
	20 input	JSON	1 KB	64	2067952	23.25%	8.79%
	1 input	JSON	10KB	10	3166630	14.32%	6.77%
	20 input	JSON	10KB	10	6197910	22.05%	11.07%
	1 input	Non-JSON	1KB	10	414976	13.69%	8.05%
	1 input	Non-JSON	1KB	64	379032	13.78%	3.27%
	20 input	Non-JSON	1KB	10	2069770	23.9%	8.63%
	20 input	Non-JSON	1KB	64	2064460	23.2%	7.06%
	1 input	Non-JSON	10KB	10	3795480	14.48%	8.92%
	20 input	Non-JSON	10KB	10	6129498	23.7%	5.57%
CO2 Stack - Classic Cluster (1 IDM) 1 cluster master (c5.xlarge) 3 search heads (c6i.xlarge) 4 CPU core 8 GB RAM 3 indexers (is4gen.4xlarge) 16 CPU 96 GiB Memory 1 Input Data Manager (IDM) (c6i.4xlarge) 16 CPU core 32 GB RAM Default TA configs
	1 input	JSON	1KB	10	231987	13.61%	6.6%
	1 input	JSON	1KB	64	237228	13.65%	6.65%
	20 input	JSON	1KB	10	1022357	22.4%	13.1%
	20 input	JSON	1KB	64	1019737	22.7%	12.98%
	1 input	JSON	10KB	10	1887750	14.21%	12.9%
	20 input	JSON	10KB	10	3228910	23.38%	13.1%
	1 input	Non-JSON	1KB	10	247171	13.6%	12.96%
	1 input	Non-JSON	1KB	64	241652	9.17%	13.55%
	20 input	Non-JSON	1KB	10	1021291	22.5%	3.9%
	20 input	Non-JSON	1KB	64	987298	23%	4.02%
	1 input	Non-JSON	10KB	10	2171011	14.35%	4%
	20 input	Non-JSON	10KB	10	291004	22%	4.05%

Version 4.2.0 Pub/Sub Based Bucket input performance statistics

The following tests file sizes of 1, 5, and 10 KB for Version 4.2.0 Pub/Sub Based Bucket input. The number of threads tested is 10.

Common architecture setup	Scenario	File type	Message batch	Ingest rate (KB/min)	CPU	RAM
CO2 Stack - Victoria Search Head Cluster 1 cluster master (c5.xlarge) 3 search head (c6i.4xlarge) 16 CPU 32 GiB Memory 3 indexer cluster (is4gen.4xlarge) 16 CPU 96 GiB Memory Default TA configs
	1 input	CSV	10	1649	3.75%	11.55%
	5 inputs	CSV	10	7869	3.7%	11.54%
	10 inputs	CSV	10	16670	4.65%	11.53%
	20 inputs	CSV	10	33327	3.75%	11.49%
	40 inputs	CSV	10	64360	4.3%	11.48%
	1 input	CSV	100	9783	3.9%	11.56%
	5 inputs	CSV	100	49756	3.65%	11.54%
	10 inputs	CSV	100	100888	3.85%	11.54%
	20 inputs	CSV	100	192448	4.49%	11.52%
	40 inputs	CSV	100	385078	7.46%	11.54%
	1 input	JSON	10	1655	4.25%	11.59%
	5 inputs	JSON	10	7853	4.5%	11.58%
	10 inputs	JSON	10	16882	3.65%	11.59%
	20 inputs	JSON	10	34611	4.65%	11.57%
	40 inputs	JSON	10	65909	3.9%	11.57%
	1 input	JSON	100	9813	3.85%	11.57%
	5 inputs	JSON	100	48865	3.55%	11.55%
	10 inputs	JSON	100	99252	4.4%	11.52%
	20 inputs	JSON	100	201040	4.24%	11.46%
	40 inputs	JSON	100	397136	7.89%	11.39%
CO2 Stack - Classic Cluster (1 IDM) 1 cluster master (c5.xlarge) 3 search heads (c6i.xlarge) 4 CPU core 8 GB RAM 3 indexers (is4gen.4xlarge) 16 CPU 96 GiB Memory 1 Input Data Manager (IDM) (c6i.4xlarge) 16 CPU core 32 GB RAM Default TA configs	1 input	CSV	10	1596	4.5%	9.87%
	5 inputs	CSV	10	7670	4.25%	9.84%
	10 inputs	CSV	10	15129	4.75%	9.48%
	20 inputs	CSV	10	30278	4.45%	9.06%
	40 inputs	CSV	10	60220	6.5%	8.68%
	1 input	CSV	100	8708	5.3%	9.89%
	5 inputs	CSV	100	45657	4.45%	9.46%
	10 inputs	CSV	100	92601	4.91%	9.15%
	20 inputs	CSV	100	185263	6.57%	8.86%
	40 inputs	CSV	100	365956	9.09%	8.45%
	1 input	JSON	10	1591	5.7%	6.98%
	5 inputs	JSON	10	7550	6.95%	4.4%
	10 inputs	JSON	10	15376	4.75%	6.88%
	20 inputs	JSON	10	30534	5.5%	6.7%
	40 inputs	JSON	10	60647	8.05%	6.23%
	1 input	JSON	100	11024	4.3%	8.43%
	5 inputs	JSON	100	48298	4%	7.97%
	10 inputs	JSON	100	93452	4.81%	7.53%
	20 inputs	JSON	100	182911	6.92%	7.44%
	40 inputs	JSON	100	372861	6.78%	7.54%

Version 1.3.0 Pub/Sub input performance statistics

Use the following information on version 1.3.0 Pub/Sub input performance statistics to enhance the performance of your own Google Cloud Platform collection tasks.

Testing architecture

The throughput data and conclusions provided here are based on performance testing using single-instance Splunk Enterprise 7.0.1 running on the following environment.

Instance type	n1-standard-8
Memory	30 GB
vCPU	8
Cores	4 CPU cores
Storage Type	standard persistent disk (2000GB)

Measured performance data

The throughput data provided here is the average performance for different subscription numbers achieved in performance testing under specific operating conditions and is subject to change when any of the hardware and software variables changes. Use this data for a rough reference only.

Subscriptions	Input numbers	Throughput (KB/s)	Throughput (GB/day)
1	1	1800	150
4	4	9000	740
8	8	17000	1400
16	16	24000	1970
200	16	16320	1340

For heavy data ingestion scenarios, change the default Acknowledgment Deadline of your subscription from 10 seconds to 60 seconds for optimal performance and to avoid data duplication.

REST API reference

Resource Metadata (Compute Engine) REST API reference

Collect Google Cloud Platform Cloud Resource Metadata Compute Engine from the following APIs.

Resources	Description	API endpoint
Instances	Lists currently alive Instances in given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instances
Accelerator Types	Lists GPU(s) available for the project in given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/acceleratorTypes
Autoscalers	Lists instance groups with autoscaling capabilities.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/autoscalers
Disk Types	Lists Disk types available for the project in the given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/diskTypes
Disks	Lists disks in the given region.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/disks
Managed Instance Groups	Lists managed instance groups.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroupManagers
Instance Groups	Lists unmanaged instance groups.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroups
Machine Types	Lists types of machines available for the project in the given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/machineTypes
Network Endpoint Groups	Lists Network Endpoint Group for the given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/networkEndpointGroups
Node Groups	Lists node groups for the given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeGroups
Node Types	Lists types of nodes available for the project in given zone.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeTypes
Reservations	Lists reservations made.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/reservations
Target Instance	Lists target instances.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/targetInstances
Operation Resources	Lists operation for the given account for the day for selected zones.	GET https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/operations
Firewall Resources	Retrieves the list of firewalls rules available to the specified project.	GET https://compute.googleapis.com/compute/v1/projects/{project}/global/firewalls

projects/{project}/zones/{zone}/instances

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instances

GET

Lists currently alive Instances in given zone

Request parameters None

Returned values Lists currently alive Instances in given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instances

projects/{project}/zones/{zone}/acceleratorTypes

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/acceleratorTypes

GET

Lists GPU(s) available for the project in given zone.

Request parameters None

Returned values Lists GPU(s) available for the project in given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/acceleratorTypes

projects/{project}/zones/{zone}/autoscalers

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/autoscalers

GET

Lists instance groups with autoscaling capabilities.

Request parameters None

Returned values Lists instance groups with autoscaling capabilities.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/autoscalers

projects/{project}/zones/{zone}/diskTypes

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/diskTypes

GET

Lists Disk types available for the project in the given zone.

Request parameters None

Returned values Lists Disk types available for the project in the given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/diskTypes

projects/{project}/zones/{zone}/disks

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/disks

GET

Lists disks in the given region.

Request parameters None

Returned values Lists disks in the given region.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/disks

projects/{project}/zones/{zone}/instanceGroupManagers

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroupManagers

GET

Lists managed instance groups.

Request parameters None

Returned values Lists managed instance groups.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroupManagers

projects/{project}/zones/{zone}/instanceGroups

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroups

GET

Lists unmanaged instance groups.

Request parameters None

Returned values Lists unmanaged instance groups.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/instanceGroups

projects/{project}/zones/{zone}/machineTypes

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/machineTypes

GET

Lists types of machines available for the project in the given zone.

Request parameters None

Returned values Lists types of machines available for the project in the given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/machineTypes

projects/{project}/zones/{zone}/networkEndpointGroups

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/networkEndpointGroups

GET

Lists Network Endpoint Group for the given zone.

Request parameters None

Returned values Lists Network Endpoint Group for the given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/networkEndpointGroups

projects/{project}/zones/{zone}/nodeGroups

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeGroups

GET

Lists node groups for the given zone.

Request parameters None

Returned values Lists node groups for the given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeGroups

projects/{project}/zones/{zone}/nodeTypes

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeTypes

GET

Lists types of nodes available for the project in given zone.

Request parameters None

Returned values Lists types of nodes available for the project in given zone.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/nodeTypes

projects/{project}/zones/{zone}/reservations

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/reservations

GET

Lists reservations made.

Request parameters None

Returned values Lists reservations made.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/reservations

projects/{project}/zones/{zone}/targetInstances

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/targetInstances

GET

Lists target instances.

Request parameters None

Returned values Lists target instances.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/targetInstances

projects/{project}/zones/{zone}/operations

https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/operations

GET

Lists operation for the given account for the day for selected zones.

Request parameters None

Returned values Lists operation for the given account for the day for selected zones.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/zones/{zone}/operations

projects/{project}/global/firewalls

https://compute.googleapis.com/compute/v1/projects/{project}/global/firewalls

GET

Lists target instances.

Request parameters None

Returned values Lists target instances.

Example request

curl —k u admin:password https://compute.googleapis.com/compute/v1/projects/{project}/global/firewalls

Resource Metadata (Cloud Storage) REST API reference

Collect Google Cloud Platform Cloud Resource Metadata Cloud Storage from the following APIs.

Resources	Description	API endpoint
Buckets	List buckets available for the selected projects.	GET https://storage.googleapis.com/storage/v1/b
Bucket Access Controls	List ACLs for the selected bucket.	GET https://storage.googleapis.com/storage/v1/b/{bucket}/acl
Default Object Access Controls	List Default Object ACL for selected bucket	GET https://storage.googleapis.com/storage/v1/b/{bucket}/defaultObjectAcl
Notifications	List Pub/Sub notifications for the selected bucket.	GET https://storage.googleapis.com/storage/v1/b/{bucket}/notificationConfigs
Object Access Controls	List ACL of all the selected object.	GET https://storage.googleapis.com/storage/v1/b/{bucket}/o/{object}/acl

b

https://storage.googleapis.com/storage/v1/b

GET

List buckets available for the selected projects.

Request parameters project: A valid API project identifier.

Returned values List of buckets of selected project.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/b

b/{bucket}/acl

https://storage.googleapis.com/storage/v1/b/{bucket}/acl

GET

List ACLs for the selected bucket.

Request parameters None

Returned values List of ACL for the selected bucket. Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/b/{bucket}/acl

b/{bucket}/defaultObjectAcl

https://storage.googleapis.com/storage/v1/b/{bucket}/defaultObjectAcl

GET

List Default Object ACL for selected bucket

Request parameters None

Returned values List of Default object ACL for the selected bucket.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/b/{bucket}/defaultObjectAcl

b/{bucket}/notificationConfigs

https://storage.googleapis.com/storage/v1/b/{bucket}/notificationConfigs

GET

List Pub/Sub notifications for the selected bucket.

Request parameters None

Returned values List of Pub/Sub notifications for the selected bucket.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/b/{bucket}/notificationConfigs

b/{bucket}/o/{object}/acl

https://storage.googleapis.com/storage/v1/b/{bucket}/o/{object}/acl

GET

List ACL of all the selected object.

Request parameters None

Returned values List of ACL for the selected object.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/b/{bucket}/o/{object}/acl

Resource Metadata (Kubernetes) REST API reference

Collect Google Cloud Platform Cloud Resource Metadata Kubernetes from the following APIs.

Resources	Description	API endpoint
Subnetworks	Lists subnetworks that are usable for creating clusters in a project.	GET https://container.googleapis.com/v1/projects/{project}/aggregated/usableSubnetworks
Clusters	Lists all clusters owned by a project.	GET https://container.googleapis.com/v1/projects/{project}/locations/{location}/clusters
Node pools	Lists the node pools for a cluster.	GET https://container.googleapis.com/v1/projects/{project}/locations/{location}/clusters/{cluster}/nodePools
Operations	Lists all operations in a project in the specified zone or all zones.	GET https://container.googleapis.com/v1/projects/{project}/locations/{location}/operations

projects/{project}/aggregated/usableSubnetworks

https://storage.googleapis.com/storage/v1/projects/{project}/aggregated/usableSubnetworks

GET

` Lists subnetworks that are usable for creating clusters in a project.

Request parameters None

Returned values List of Default subnetworks for the selected project.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/aggregated/usableSubnetworks

projects/{project}/locations/{location}/clusters

https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/clusters

GET

Lists all clusters owned by a project.

Request parameters None

Returned values List of clusters owned by a project.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/clusters

projects/{project}/locations/{location}/clusters/{cluster}/nodePools

https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/clusters/{cluster}/nodePools

GET

Lists the node pools for a cluster.

Request parameters None

Returned values List of node pools for the selected cluster.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/clusters/{cluster}/nodePools

projects/{project}/locations/{location}/operations

https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/operations

GET

Lists all operations in a project in the specified zone or all zones.

Request parameters None

Returned values List of all operations for the selected project and location.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/operations

Resource Metadata (VPC Access) REST API reference

Collect Google Cloud Platform Cloud Resource Metadata VPC Access from the following APIs.

Resources	Description	API endpoint
Locations	Lists information about the supported locations for selected project.	GET https://vpcaccess.googleapis.com/v1/projects/{project}/locations
Connectors	Lists Serverless VPC Access connectors.	GET https://vpcaccess.googleapis.com/v1/projects/{project}/locations/{location}/connectors
Operations	Lists operations that match the specified filter in the request.	GET https://vpcaccess.googleapis.com/v1/projects/{project}/locations/{location}/operations

projects/{project}/locations

https://storage.googleapis.com/storage/v1/projects/{project}/locations

GET

Lists information about the supported locations for selected project.

Request parameters None

Returned values List of supported locations for the selected project.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations

projects/{project}/locations/{location}/connectors

https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/connectors

GET

Lists Serverless VPC Access connectors.

Request parameters None

Returned values List of Serverless VPC Access connectors.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/connectors

projects/{project}/locations/{location}/operations

https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/operations

GET

Lists operations that match the specified filter in the request.

Request parameters None

Returned values List of operations for selected project and location.

Example request

curl —k u admin:password https://storage.googleapis.com/storage/v1/projects/{project}/locations/{location}/operations

Ended: Reference