Skip to content

Release notes ↵

Release notes for the Splunk Add-on for Microsoft Cloud Services

Version 5.4.3 of the Splunk Add-on for Microsoft Cloud Services was released on February 18, 2025.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.4.3 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 9.1.x, 9.2.x, 9.3.x, 9.4.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Microsoft Entra ID, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

Fixed issues

Version 5.4.3 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issues appear, then there are no bug fixes reported:

Known issues

Version 5.4.3 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.4.3 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Release history for the Splunk Add-on for Microsoft Cloud Services

The latest version of the Splunk Add-on for Microsoft Cloud Services is version 5.4.3. See Release notes for the Splunk Add-on for Microsoft Cloud Services for the release notes of this latest version.

Version 5.4.2

Version 5.4.2 of the Splunk Add-on for Microsoft Cloud Services was released on January 30, 2025.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.4.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 9.1.x, 9.2.x, 9.3.x, 9.4.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

Fixed issues

Version 5.4.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issues appear, then there are no bug fixes reported:

Known issues

Version 5.4.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.4.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.4.1

Version 5.4.1 of the Splunk Add-on for Microsoft Cloud Services was released on October 21, 2024.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.4.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 9.1.x, 9.2.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Microsoft Entra ID, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

Fixed issues

Version 5.4.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issues appear, then there are no bug fixes reported:

Known issues

Version 5.4.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.4.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.4.0

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services was released on September 11, 2024.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x, 9.2.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Added support for exporting EventHub modular inputs snapshot in JSON format for cloud environments. These snapshots can be imported into Data Manager. See <Generate JSON snapshot for Event Hubs>.
  • FIPS support for Metrics modular inputs.
  • GUI to manage global settings for modular inputs.
  • Support in Metrics modular inputs for metric gathering on the same resource types with different metrics available.
  • Additional checkbox for EventHub modular inputs to enable AMQP mode for EventHub even if the global proxy is configured.

Fixed issues

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.4.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.3.2

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services was released on August 1, 2024.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • No new features

Fixed issues

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.3.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.3.1

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services was released on June 7, 2024.

Compatibility

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • No new features

Fixed issues

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.3.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.3.0

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services was released on April 12, 2024.

Compatibility

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Added support for compressed log sets for Storage Blobs
  • Added support for Unicode/ASCII in EventHub collector event raw view
  • Removed limitation of 64 partition for EventHub inputs

Fixed issues

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.3.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.2.2

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services was released on February 5, 2024.

Compatibility

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent (MacOS is not supported)
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Bug fixes.

Fixed issues

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.2.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.2.1

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services was released on October 6, 2023.

Versions 5.1.0 and 5.2.0 are dependent on version 5.0 for upgrade. Upgrade to version 5.0 first before upgrading these versions. Please note that this dependency has been eliminated in versions 5.1.2 and 5.2.1. See the release notes topic for more details.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Removed the dependency of version 5.0.0 during upgrade for Storage Blob input.

Fixed issues

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.2.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.2.0

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services was released on September 17, 2023.

After upgrading to version 5.0.0 or later of this add-on, you might observe a rise in the usage of memory and CPU resources within your deployment.

Check the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.2.x, 9.0.x, 9.1.x
CIM version 5.1.0
Supported OS for data collection Platform independent
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Updated Azure Resource, Azure Consumption APIs and the Azure Storage Blob SDK to their latest versions.
  • Fixed security related issues.
  • Updated the read_timeout parameter’s default value for the Azure Storage Blob input to 60 seconds.
  • Automatic deletion of obsolete Storage Blob file checkpoints after successful migration to KV store.

Fixed issues

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.2.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.1.2

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services was released on October 3, 2023.

See the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM version 5.1.0
Supported OS for data collection Platform independent
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Removed Dependency for Storage Blob Input in v5.0.0 Step Upgrade

Fixed issues

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.1.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.1.1

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services was released on May 2, 2023.

See the release notes for version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM version 5.1.0
Supported OS for data collection Platform independent
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Improved CPU utilization for eventhub inputs.
  • Improved logging mechanism for eventhub inputs.
  • Added a warning message to the Azure App account update, proxy, and logging pages, informing users that they will be required to re-enable EventHub inputs upon account, proxy, and log level changes.

Fixed issues

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.1.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.1.0

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services was released on May 2, 2023.

Please also check the release notes for Splunk Add-on for Microsoft Cloud Services v5.0.0 before upgrading to the latest version as breaking changes were introduced in the Storage Blob input.

Compatibility

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM version 5.1.0
Supported OS for data collection Platform independent
Vendor products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • The following inputs were migrated from Splunk Add on for Microsoft Azure to Splunk Add-on for Microsoft Cloud Services. If these inputs are configured in Splunk Add-on for Microsoft Cloud Services, then they will be treated as new inputs. It is recommended to disable those inputs in the Splunk Add-on for Microsoft Azure:
    • Introduced the Azure Metrics input
    • Introduced the Azure KQL Log Analytics input
    • Introduced the Azure Consumption(Billing) input
    • Introduced new Resource Types (Disk Data, Image Data, Snapshot Data, Resource Groups, Security Groups and Subscriptions) in the Azure Resource input
  • Security related issue have been fixed
  • Introduced the Read Timeout parameter to the Storage Blob input, which can be used to resolve the data ingestion stuck issue. See the Storage Blob input configuration manual for more information.
  • Added UI support to the Blob Mode parameter.

Provided CIM 5.1.0 support for the following:

Sourcetype Category
mscs:resource:securityGroup Azure Resource
mscs:resource:disk Azure Resource
mscs:resource:image Azure Resource
mscs:resource:snapshot Azure Resource
mscs:resource:subscriptions Azure Resource
mscs:resource:resourceGroup Azure Resource

Fixed issues

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear, then there are no bug fixes reported:

Known issues

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear, no issues have yet been reported:

Third-party software attributions

Version 5.1.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 5.0.0

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services was released on March 21, 2023.

Compatibility

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM version 5.0.2
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  1. The following enhancements were made on the Eventhub Input. See Input Parameters for more details:
    1. Resolved the memory leak issue for the input.
    2. Introduced load balancing support across multiple instances. See Horizontal Scaling Across Multiple Splunk Environment section in the Eventhub input manual. See Horizontal Scaling for more information.
    3. Introduced debug loggers to the input execution. See Input Parameters for more details.

  2. Enhancements were made on the Storage Blob Input. The Storage Blob checkpoint will be migrated from the File checkpoint mechanism to the KV Store mechanism.

    If inputs are interrupted during the checkpoint migration in the first interval after upgrading the add-on to Version 5.0.0, it may lead to data duplication.

    1. The checkpoint mechanism was migrated to the Splunk KV Store.
    2. Introduced Horizontal Scaling that would allow parallel data ingestion via multiple inputs on a common KV Store architecture. See Horizontal Scaling for more information.
    3. Introduced a new field called Prefix to optimize the execution time of the input.
    4. Introduced an Advanced Tab in the Configuration Tab to control the File Based Checkpoint deletion for Storage Blob.

Provided CIM 5.0.2 support for the following:

Sourcetype Category
azure:monitor:aad AzureActiveDirectory
azure:monitor:activity Administrative

See the following table for the CIM fields removed for 5.0.0:

Source-type operationName Fields removed Reason for removed fields
azure:monitor:aad Add a deletion-marked app role assignment grant to user as part of link removal object The event is not mapped to any Datamodel
azure:monitor:aad Add blocked user object_id There is no ID for the target user present in the raw event.
azure:monitor:aad Clear block on user object_id There is no ID for the target user present in the raw event.
azure:monitor:aad POST Tenant.RemoveBlockedUser, POST Tenant.CreateBlockedUser, Update StsRefreshTokenValidFrom Timestamp, Process role update request, User started security info registration object The event is not mapped to any datamodel.
azure:monitor:aad Sign-in activity, Validate user authentication, Risky user, User Risk Detection object The object field is not part of the datamodels mapped to the events.
['azure:monitor:aad'] Start applying group based license to users object The event is not mapped to any datamodel.

See the following table for a list of CIM fields modified for 5.0.0:

Source-type CIM Field operationName Comment
['azure:monitor:aad'] object Access review ended, Add app role assignment grant to user, Add blocked user, Add conditional access policy, Add label, Add owner to group, Add owner to service principal, Add role definition, Add role from template, Add user, Clear block on user, Consent to application, Create access package catalog, Create business flow, Create connected organization, Delete access package catalog, Delete application, Delete business flow, Delete conditional access policy, Delete group, Delete policy, Delete role definition, Delete user, Disable account, Enable account, Finish applying group based license to users, Get resource properties of a tenant, Get tenant details, Hard Delete application, Hard Delete group, Hard Delete user, Hard delete service principal, Initialize tenant, POST Tenant.CreateTenant, Remove app role assignment from user, Remove eligible member from role in PIM completed (permanent), Remove eligible member from role in PIM completed (timebound), Remove member from role, Remove member from role in PIM completed (permanent), Remove member from role in PIM completed (timebound), Remove member from role in PIM requested (permanent), Remove member from role in PIM requested (timebound), Remove owner from application, Remove owner from group, Remove service principal, Restore application, Set Company Information, Set directory feature on tenant, Set group license, Set user manager, Update access package catalog, Update application, Update authorization policy, Update business flow, Update conditional access policy, User registered all required security info, User registered security info The object field is changed, the extraction is now more accurate, i.e. having more specific values, e.g. the object was the generic Microsoft Entra ID, and now it has more specific and meaningful value.
['azure:monitor:aad'] object_attrs Add app role assignment grant to user, Add label, Add owner to group, Add owner to service principal, Add role from template, Add user, Create connected organization, Delete user, Disable account, Enable account, Hard Delete user, Hard delete service principal, POST Tenant.CreateTenant, Remove app role assignment from user, Remove eligible member from role in PIM completed (permanent), Remove eligible member from role in PIM completed (timebound), Remove member from role, Remove member from role in PIM completed (permanent), Remove member from role in PIM completed (timebound), Remove member from role in PIM requested (permanent), Remove member from role in PIM requested (timebound), Remove owner from application, Remove owner from group, Remove service principal, Update access package catalog, Update business flow, Verify domain The object_attrs field got now more meaningful (and sometime more concise) value than before.
['azure:monitor:aad'] user Add blocked user, Clear block on user, Disable account, Enable account, Hard Delete user, Remove eligible member from role in PIM completed (permanent), Remove eligible member from role in PIM completed (timebound), Remove member from role in PIM completed (permanent), Remove member from role in PIM completed (timebound), Remove member from role in PIM requested (permanent), Remove member from role in PIM requested (timebound), Set user manager, User registered all required security info, User registered security info The user field value is now corrected and extracted properly reflecting the CIM definitions of this field in the Change Datamodel (All_changes and Account_management Datasets).

Fixed issues

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 4.5.2

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services was released on February 15, 2023.

Compatibility

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.x
CIM version 5.0.1
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New features

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Security related issue have been fixed, No new features added.

Fixed issues

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.5.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 4.5.1

Version 4.5.1 of the Splunk Add-on for Microsoft Cloud Services was released in November 15, 2022.

Compatibility

Version 4.5.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.1.x, 8.2.x, 9.0.0
CIM version 5.0.1
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

Fixed issues

Eventhub input does not support “Transport Type” as “AMQP” in Splunk Cloud.

Version 4.5.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.5.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.5.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.5.0

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services was released on July 31, 2022.

Compatibility

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms:
Splunk platform versions 8.0.x, 8.1.x, 8.2.x, 9.0.0
CIM version 5.0.1
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, and other cloud services.

New Features

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Provided CIM support for Azure Data Share events
  • Updated Azure Audit API, Azure Storage Blob, and Storage Table client SDK to the latest version

Note: A high-level overview of differences between Audit API version 2015-04-01 and the old 2014-04-01 version:

  • The key name was changed for the following fields of the audit events, but the value remains the same:
    • eventSource → category
    • resourceUri → resourceId
  • The following fields were added in response to the latest Audit API version::
    • "resourceType":{"value": "<value>", "localizedValue": "<localizedValue>"}
    • "tenantId": "<tenant_id>"

Fixed issues

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.5.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Third-party software attributions for the Splunk Add-on for Microsoft Cloud Services

Version 4.3.3

Version 4.3.3 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • The Microsoft Azure Event Hubs input in the previous version of the Splunk Add-on for Microsoft Cloud Services had an additional level of nesting for ingested events that had a records key. The additional nesting has been removed to provide a simpler and faster query experience. Previous versions of the Splunk Add-on for Microsoft Cloud Services:

    {
"body":
   {
      "records": {
         "field1": value1
       }
   }
}

    Current version of the Splunk Add-on for Microsoft Cloud Services:

    {
"body":
   "field1": value1
}
    • Bug fixes.
    • Fixed a memory leak issue that was affecting the performance of the Event Hub input.

In this release, the existing lookups are updated for the Self Service App Install (SSAI) upgrade. Lookups do not update with the latest values automatically. To fix this issue, upgrade the Splunk Add-on for Microsoft Cloud Services, then manually update the lookup files using the latest version of this add-on.

Fixed issues

Version 4.3.3 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.3.3 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Version 4.2.0

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services was released on September 13, 2021.

Compatibility

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x, 8.1.x, 8.2.x
CIM version 4.20
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • UI component upgrades for compatibility with future versions of the Splunk software (jQuery upgrade).
  • Bug fixes.
  • Common Information Model (CIM) Release Notes:
    • Compatibility with CIM version 4.20.
    • The following CIM mapping enhancements:
      • Added support for Alert and Change data models in the mscs:azure:audit sourcetype.
      • Added support for Inventory_Network data model in the mscs:azure:networkInterfaceCard sourcetype.
      • Fixed existing field mapping issue for image_name and severity fields in mscs:resource:virtualMachine and mscs:azure:security:recommendation sourcetypes respectively.
      • The following mscs:azure:audit sourcetype enhancements:
        • Added an extra field event_description to retain the existing description values from the event and updated the description field values as per the Alert CIM data model recommendations.
        • Added new lookup mscs_audit_change_cim_fields_with_status_code.csv for populating CIM fields.
      • Updated the values in the lookup mscs_security_alert_object_category.csv for the mscs:azure:security:alert sourcetype.

In this release, the existing lookups are updated for the Self Service App Install (SSAI) upgrade. Lookups do not update with the latest values automatically. To fix this issue, upgrade the Splunk Add-on for Microsoft Cloud Services, then manually update the lookup files using the latest version of this add-on.

Fixed issues

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.2.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.5

Fixed issues

Version 4.1.5 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.1.5 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.1.5 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.4

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services was released on July 28, 2021.

Compatibility

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.18
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Bug fixes

Fixed issues

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.1.4 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.3

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services was released on May 14, 2021.

Compatibility

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.15
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • When Event Hub data is ingested by the Splunk software, different events are generated in the Splunk platform for each record.
  • Each record from Event Hub data is now split into separate Splunk events.
  • Fixed an Event Hub input bug where Event Hub data isn’t ingested due to the following client secret error:

AADSTS7000215: Invalid client secret is provided.

  • The upper limit for max_batch_size is increased to be 10000.

Fixed issues

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.1.3 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.2

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services was released on April 20, 2021.

Compatibility

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.15
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Changes to the Blob Storage input to address a data duplication issue with Append Blobs.

Fixed issues

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues. If no issue appear below, then there are no bug fixes reported:

Known issues

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services introduced a blob storage duplication solution that conflicts with the Event Hub input, leading to the following error:

AADSTS7000215: Invalid client secret is provided.

If you do not need the blob storage duplication fix, the best practice is to continue using version 4.1.1 of this add-on instead of upgrading to version 4.1.2.

Third-party software attributions

Version 4.1.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.1

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services was released on February 12, 2021.

Compatibility

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.15
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • The 4.1.0 release of MSCS included a new SDK and libraries to support EventHubs. Due to some underlying Splunk Python behavior some customers who had other Microsoft TAs installed noted that the GUI configuration was failing for MSCS, This release solves this library clash issue.
  • Improvements to proxy configuration enforcing an integer value.
  • Fix for an exception UnicodeDecodeError that some customers where seeing for the Event Hubs Modular Input

Fixed issues

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.1.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.1.0

It is a best practice to use either version 4.1.1 and later or versions 4.0.2 and earlier of this add-on.

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services was released on January 9, 2020.

Compatibility

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.15
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services cannot be installed on the same Splunk platform instance as one that has the Microsoft Azure Add-on for Splunk installed.

New Features

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Support for the Microsoft Azure Event Hubs input type.

Fixed issues

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.1.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.0.2

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services was released on August 31, 2020.

Compatibility

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 8.0.x
CIM version 4.15
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

New Features

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Improved support for the Splunk Enterprise Security Assets & Identities Framework interface.
  • Additional storage blob input capability and security compatibility.
  • Federal Information Processing Standard (FIPS) compliance.
  • Additional Python3 library support.

For more information on migrating your deployment to a Python 3 deployment, see Upgrade using the Python 3 runtime and dual-compatible Python syntax in custom scripts in the Splunk Enterprise Installation manual.

Fixed issues

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

The Splunk Add-on for Microsoft Cloud Services version 4.0.2 is incompatible with Splunk Enterprise versions 7.x.x and earlier.

Third-party software attributions

Version 4.0.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.0.1

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services was released on August 31, 2020.

Compatibility

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 7.0.x, 7.1.x, 7.2.x, 7.3.x, 8.0.x
CIM version 4.12
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

Upgrade

The following migration guide is supported for upgrading from version 3.0.0 to version 4.0.0 or later. Upgrading from any version older than 3.0.0 requires a fresh installation of version 3.0.0.

A best practice for upgrading the Splunk Add-on for Microsoft Cloud Services is to remove your older version before re-installing version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services.

  1. Disable all your inputs before you upgrade the add-on. Otherwise you may see errors in the log files which may results data loss against your already configured inputs.
  2. Install the Splunk Add-on for Microsoft Cloud Services version 3.1.0 and up from the Splunk Web UI (make sure Upgrade App checkbox is selected).
  3. Restart the Splunk platform.
  4. Navigate to the input page of the Splunk Add-on for Microsoft Cloud Service. Alerts will appear, indicating incomplete account authorization.
  5. Edit each required input by clicking the click here link to navigate to the account configuration page or by directly navigating to the account configuration page.
  6. Complete the authorization of your account by adding your account secret key/account token.
  7. Repeat above steps for all inputs which have alert sign against them.
  8. Enable each desired input to start data collection.

In previous versions, settings including proxy, logging, and performance were stored in splunk_ta_o365_client_setting.conf and splunk_ta_o365_server_setting.conf. In version 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services, all setting and performance tuning configurations are in splunk_ta_mscs_setting.conf. The default log level is INFO.

Versions 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services removes the Microsoft Office 365 module. See the Splunk Add-on for Microsoft 0ffice 365.

New Features

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Default support for Python 3

For more information on migrating your deployment to a Python 3 deployment, see Choose your Splunk Enterprise upgrade path for the Python 3 migration in the Splunk Enterprise Installation manual.

Fixed issues

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.0.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 4.0.0

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services was released on October 21, 2019.

Compatibility

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 7.0.x, 7.1.x, 7.2.x, 7.3.x, 8.0.x
CIM version 4.12
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

Upgrade

The following migration guide is supported for upgrading from version 3.0.0 to version 4.0.0. Upgrading from any version older than 3.0.0 requires a fresh installation of version 3.0.0.

A best practice for upgrading the Splunk Add-on for Microsoft Cloud Services is to remove your older version before re-installing version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services.

  1. Disable all your inputs before you upgrade the add-on. Otherwise you may see errors in the log files which may results data loss against your already configured inputs.
  2. Install the Splunk Add-on for Microsoft Cloud Services version 3.1.0 from the Splunk Web UI (make sure Upgrade App checkbox is selected).
  3. Restart the Splunk platform.
  4. Navigate to the input page of the Splunk Add-on for Microsoft Cloud Service. Alerts will appear, indicating incomplete account authorization.
  5. Edit each required input by clicking the click here link to navigate to the account configuration page or by directly navigating to the account configuration page.
  6. Complete the authorization of your account by adding your account secret key/account token.
  7. Repeat above steps for all inputs which have alert sign against them.
  8. Enable each desired input to start data collection.

In previous versions, settings including proxy, logging, and performance were stored in splunk_ta_o365_client_setting.conf and splunk_ta_o365_server_setting.conf. In version 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services, all setting and performance tuning configurations are in splunk_ta_mscs_setting.conf. The default log level is INFO.

Versions 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services removes the Microsoft Office 365 module. See the Splunk Add-on for Microsoft 0ffice 365.

New Features

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Support for Python 3

For more information on migrating your deployment to a Python 3 deployment, see Choose your Splunk Enterprise upgrade path for the Python 3 migration in the Splunk Enterprise Installation manual.

Fixed issues

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 4.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 3.1.0

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services was released on April 8, 2019.

Compatibility

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.6,x, 7.0.x, 7.1.x, 7.2.x, 7.3.x
CIM version 4.12
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

Upgrade

The following migration guide is supported for upgrading from version 3.0.0 to version 3.1.0. Upgrading from any version older than 3.0.0 requires a fresh installation of version 3.0.0.

A best practice for upgrading the Splunk Add-on for Microsoft Cloud Services is to remove your older version before re-installing version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services.

  1. Disable all your inputs before you upgrade the add-on. Otherwise you may see errors in the log files which may results data loss against your already configured inputs.
  2. Install the Splunk Add-on for Microsoft Cloud Services version 3.1.0 from the Splunk Web UI (make sure Upgrade App checkbox is selected).
  3. Restart the Splunk platform.
  4. Navigate to the input page of the Splunk Add-on for Microsoft Cloud Service. Alerts will appear, indicating incomplete account authorization.
  5. Edit each required input by clicking the click here link to navigate to the account configuration page or by directly navigating to the account configuration page.
  6. Complete the authorization of your account by adding your account secret key/account token.
  7. Repeat above steps for all inputs which have alert sign against them.
  8. Enable each desired input to start data collection.

In previous versions, settings including proxy, logging, and performance were stored in splunk_ta_o365_client_setting.conf and splunk_ta_o365_server_setting.conf. In version 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services, all setting and performance tuning configurations are in splunk_ta_mscs_setting.conf. The default log level is INFO.

Versions 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services removes the Microsoft 0ffice 365 module. See the Splunk Add-on for Microsoft 0ffice 365.

New Features

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features:

  • Credential validation of Account Name and Account secret key on Account configuration page.

Fixed issues

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 3.1.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 3.0.0

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.6,x, 7.0.x, 7.1.x, 7.2.x
CIM version 4.12
Supported OS for data collection Platform independent
Vendor Products Azure Active Directory, Azure Storage Table, Azure Storage Blob, Azure Audit, and other cloud services

Upgrade

A best practice for upgrading the Splunk Add-on for Microsoft Cloud Services is to remove your older version before re-installing version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services.

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services removes the Microsoft Office 365 module. See the Splunk Add-on for Microsoft Office 365.

In previous versions, settings including proxy, logging, and performance were stored in splunk_ta_o365_client_setting.conf and splunk_ta_o365_server_setting.conf. In version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services, all setting and performance tuning configurations are in splunk_ta_mscs_setting.conf. The default log level is INFO.

After you install version 3.0.0, you must clear the cache on the host of your Splunk platform instance or force refresh the input and configuration page the first time you use Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services.

New Features

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services has the following new feature:

  • Support for XML and JSON field extractions via the mscs:storage:blob:xml and mscs:storage:blob:json sourcetypes.

Fixed issues

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues:

Known issues

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues. If no issues appear below, no issues have yet been reported:

Third-party software attributions

Version 3.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 2.1.0

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.5.x, 6.6,x, 7.0.x, 7.1.x, 7.2.x
CIM 4.11
Platforms Platform independent
Vendor Products Microsoft Office 365, Azure Active Directory, Sharepoint Online, Exchange Online, Azure Storage Table, Azure Storage Blob, Azure Audit, and Azure Resource Group.

New Features

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Service has the following new features and enhancements.

  • Support for Office365 Government Cloud
  • Support for Azure Government Cloud
  • Support for the Audit General class of Office365 events

Fixed issues

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services fixes the following issues.

Known issues

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues.

Third-party software attributions

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 2.0.3

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.4 and later
CIM 4.4 and later
Platforms Platform independent
Vendor Products Microsoft Office 365, Azure Active Directory, Sharepoint Online, Exchange Online, Azure Storage Table, Azure Storage Blob, Azure Audit, and Azure Resource Group.

New Features

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Service has the following new features and enhancements.

  • Enhanced stability and performance in data collection through the O365 Management APIs
  • Updates to pagination handling for the O365 Management Activity APIs
  • Added proxy support for Audit and Resource data inputs
  • Optimized performance for the Diagnostics and websitesapplogs tables

Fixed issues

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Service fixes the following issues.

Known issues

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues.

Third-party software attributions

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 2.0.2

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.4 and 6.5
CIM 4.4 or later
Platforms Platform independent
Vendor Products Microsoft Office 365, Azure Active Directory, Sharepoint Online, Exchange Online, Azure Storage Table, Azure Storage Blob, Azure Audit, and Azure Resource Group.

Fixed issues

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Service fixes the following issues.

Publication Date Issue number Description
2017/02/20 ADDON-12556 Cannot use proxy without Authentication in Storage channel.
2017/02/20 ADDON-12665 The length of the checkpoint file name exceeds the limitation of the operating system.
2017/02/20 ADDON-12666 Cannot parse SAS token which is not start with ‘?’.

Known issues

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services contains the following new known issues.

Date

Issue number

Description

2017/06/02

ADDON-14969

Truncated Key/value pairs in Splunk Add-on for Microsoft Cloud Services.

2017/02/07

ADDON-13487

The proxy value you configured in this add-on cannot be used for the Azure resource and Azure audit input channel.
Workaround: Configure the proxy on the local system for Azure resource and Azure audit input channel.

2017/02/06

ADDON-13476

Error occurs during upgrading Splunk add-on for Microsoft cloud service on Windows platform.
Workaround: If you want to upgrade this add-on on Windows platform, disable the add-on first, then enable it after upgrading.

For the known issues in the previous release, see release history of the Splunk add-on for Microsoft cloud service.

Third-party software attributions

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 2.0.1

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services is compatible with the same software, CIM versions and platforms as Version 2.0.2.

Fixed issues

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Service fixes the following issues.

Resolved Date

Issue number

Description

2016/10/14

ADDON-10454

Only the first 30 inputs (in the alphabet order) of Azure Storage Table (including Virtual Machine Metrics) can work.
Only the first 30 Azure Storage Blob inputs (in the alphabet order) can work.

Known issues

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services contains the following known issues.

Date

Issue number

Description

2016-10-13

ADDON-11638

This add-on does not check the input name stanza at the frontend.

2016-10-12

ADDON-11609

This add-on fails to configure the certificate in the latest Firefox browser.

2016-09-24

ADDON-11423

This add-on can only get data when blob name in Microsoft Cloud Service only contains ASCII code. It cannot get data if the blob name contains multibyte character set, such as Latin characters, Japanese characters.

2016-09-20

ADDON-11419

If the names of the Azure storage blob inputs under the same account are the same except the case, such as INPUTS and inputs, the checkpoint conflicts to each other on Windows platform. This issue also exists in other modular inputs.

2016-09-20

ADDON-11409

The changes in the inputs.conf won't take effect until restarting Splunk platform.

2016-09-20

ADDON-11400

If you set the log level to ERROR for Azure Audit and Azure Blob input, there are still some INFO level logs recorded in the log file.

2016-09-19

ADDON-11349

The error message error_message=The range specified is invalid for the current size of the resource exists in the log file if the blob input has been collected and revised later to a smaller size. The error message can be ignored.

2016-09-19

ADDON-11316

There will be some errors, such as Failed to load endpoint, Refresh token failed, Failed to init ServerInfo or Failed to send rest request in the log file when you restart Splunk platform. But it does not effect data collection.

2016-09-15

ADDON-11298

There will be some data loss if the Splunk platform restart or shutdown accidently.
Workaround: If you need to restart Splunk platform, you have to disable the inputs beforehand to prevent the data loss.

2016-09-09

ADDON-11178

You can only add the Office365 account via Splunk web, you can not add it using the configuration file.

2016-09-05

ADDON-11164

The Proxy Type and DNS Resolution settings do not work for Azure Storage Table and Azure Storage Blob input.

2016-08-23

ADDON-10984

This add-on cannot get Virtual Machine (classic) metadata.

2016/03/30

ADDON-8505

Splunk searches sometimes display duplicate events. This is a known issue with the Microsoft Office 365 Management API.

2016/03/30

ADDON-8504

Splunk searches sometimes display events out of order. This is a known issue with the Microsoft Office 365 Management API.

2016/03/29

ADDON-8432

Stanza "o365_certificate_setting" in splunk_ta_ms_o365_server_ucc_system_setting.conf.spec has incorrect default values.

2016/03/29

ADDON-8424

Certificate status messages "* but invalid" should not appear until a longer time has passed.

2016/03/08

ADDON-8221

If you configure an X.509 certificate and private key and upload the keyCredentials JSON for any integration account configured in the add-on, you also need to be uploaded it for all other accounts configured in the add-on, or any accounts not using the certificate cannot collect data.

2016/01/31

ADDON-7653

Management log reports rest request error during Splunk platform stop/restart immediately after a configuration change. This error can be ignored.

2016/01/26

ADDON-7597

Input will stop when the proxy_url exists but is invalid as a proxy. Workaround: Change your proxy URL to a valid proxy value.

Third-party software attributions

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 2.0.0

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the same software, CIM versions and platforms as Version 2.0.1.

New features

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features.

Date Issue number Description
2016/09/20 ADDON-10883 Mapping to Cloud of ITSI data model.
2016/09/20 ADDON-10728 Add modular input for Azure Storage Blob data.
2016/09/20 ADDON-10727 Add modular input for Azure Storage Table data.
2016/09/20 ADDON-10129 Add modular input for Azure Audit data.
2016/09/20 ADDON-10696 Add modular input for Azure Resource data.
2016/09/20 ADDON-10222 Add modular input for Azure Virtual Machine Metrics data.

Fixed issues

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Service fixes the following issues.

Resolved Date Issue number Description
2016-09-05 ADDON-11033 If there is space in the name of inputs or account, this add-on will fail to ingest data.
2016-07-19 ADDON-9329 This add-on does not work if you install the add-on under /etc/apps/SPLUNK_HOME/ect/apps folder
2016-08-30 ADDON-8735 If the global proxy is enabled in splunk-launch.conf, the add-on cannot display the Account or Proxy tab under Configuration.

Known issues

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the following known issues.

Date

Issue number

Description

2016-09-27

ADDON-10454

Only the first 30 inputs (in the alphabet order) of Azure Storage Table (including Virtual Machine Metrics) can work.
Only the first 30 Azure Storage Blob inputs (in the alphabet order) can work.
Workaround: You can reduce the number of inputs by using wildcard or regex expression in the Blob list.

2016-09-24

ADDON-11423

This add-on can only get data when blob name in Microsoft Cloud Service only contains ASCII code. It cannot get data if the blob name contains multibyte character set, such as Latin characters, Japanese characters.

2016-09-20

ADDON-11419

If the names of the Azure storage blob inputs under the same account are the same except the case, such as INPUTS and inputs, the checkpoint conflicts to each other on Windows platform. This issue also exists in other modular inputs.

2016-09-20

ADDON-11409

The changes in the inputs.conf won't take effect until restarting Splunk platform.

2016-09-20

ADDON-11400

If you set the log level to ERROR for Azure Audit and Azure Blob input, there are still some INFO level logs recorded in the log file.

2016-09-19

ADDON-11349

The error message error_message=The range specified is invalid for the current size of the resource exists in the log file if the blob input has been collected and revised later to a smaller size. The error message can be ignored.

2016-09-19

ADDON-11316

There will be some errors, such as Failed to load endpoint, Refresh token failed, Failed to init ServerInfo or Failed to send rest request in the log file when you restart Splunk platform. But it does not effect data collection.

2016-09-15

ADDON-11298

There will be some data loss if the Splunk platform restart or shutdown accidently.
Workaround: If you need to restart Splunk platform, you have to disable the inputs beforehand to prevent the data loss.

2016-09-09

ADDON-11178

You can only add the Office365 account via Splunk web, you can not add it using the configuration file.

2016-09-05

ADDON-11164

The Proxy Type and DNS Resolution settings do not work for Azure Storage Table and Azure Storage Blob input.

2016-08-23

ADDON-10984

This add-on cannot get Virtual Machine (classic) metadata.

2016/03/30

ADDON-8505

Splunk searches sometimes display duplicate events. This is a known issue with the Microsoft Office 365 Management API.

2016/03/30

ADDON-8504

Splunk searches sometimes display events out of order. This is a known issue with the Microsoft Office 365 Management API.

2016/03/29

ADDON-8432

Stanza "o365_certificate_setting" in splunk_ta_ms_o365_server_ucc_system_setting.conf.spec has incorrect default values.

2016/03/29

ADDON-8424

Certificate status messages "* but invalid" should not appear until a longer time has passed.

2016/03/08

ADDON-8221

If you configure an X.509 certificate and private key and upload the keyCredentials JSON for any integration account configured in the add-on, you also need to be uploaded it for all other accounts configured in the add-on, or any accounts not using the certificate cannot collect data.

2016/01/31

ADDON-7653

Management log reports rest request error during Splunk platform stop/restart immediately after a configuration change. This error can be ignored.

2016/01/26

ADDON-7597

Input will stop when the proxy_url exists but is invalid as a proxy. Workaround: Change your proxy URL to a valid proxy value.

Third-party software attributions

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Version 1.0.0

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services was released on April 1, 2016. Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible with the following software, CIM versions, and platforms.
Splunk platform versions 6.3.X or later
CIM 4.4 or later
Platforms Platform independent
Vendor Products Microsoft Office 365, Azure Active Directory, Sharepoint Online, Exchange Online, and other cloud services.

New features

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services has the following new features.

Date Issue number Description
2016/03/10 ADDON-3941 Create a new add-on for Microsoft cloud services.

Known issues

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the following known issues.

Date Issue number Description
2016/03/30 ADDON-8505 Splunk searches sometimes display duplicate events. This is a known issue with the Microsoft Office 365 Management API.
2016/03/30 ADDON-8504 Splunk searches sometimes display events out of order. This is a known issue with the Microsoft Office 365 Management API.
2016/03/29 ADDON-8432 Stanza “o365_certificate_setting” in splunk_ta_ms_o365_server_ucc_system_setting.conf.spec has incorrect default values.
2016/03/29 ADDON-8424 Certificate status messages “* but invalid” should not appear until a longer time has passed.
2016/03/15 ADDON-8280 Add-on throws “Failed to send rest request” errors during restart after initial installation unless the user waits for about one minute after installing the add-on and before restarting the Splunk platform. Workaround: Restart the Splunk platform a second time.
2016/03/08 ADDON-8221 If you configure an X.509 certificate and private key and upload the keyCredentials JSON for any integration account configured in the add-on, you also need to be upload it for all other accounts configured in the add-on, or any accounts not using the certificate cannot collect data.
2016/01/31 ADDON-7653 Management log reports rest request error during Splunk platform stop/restart immediately after a configuration change. This error can be ignored.
2016/01/26 ADDON-7597 Input will stop when the proxy_url exists but is invalid as a proxy. Workaround: Change your proxy URL to a valid proxy value.

Third-party software attributions

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the following third-party software or libraries.

Ended: Release notes

Overview ↵

Splunk Add-on for Microsoft Cloud Services
Version 5.4.3
Vendor products Microsoft Entra ID, Azure Event Hubs, Azure Storage Table, Azure Storage Blob, Azure Audit, Azure Resource Group, Azure Metrics, Azure Log Analytics, Azure Consumption etc.
Add-on has a web UI Yes. This add-on contains views for configuration.

The Splunk Add-on for Microsoft Cloud Services allows a Splunk platform administrator to pull Azure audit, Azure resource data, and Azure Storage Table and Blob data from a variety of Microsoft Cloud services using the Azure Service Management APIs and Azure Storage APIs.

This add-on provides the inputs and CIM-compatible knowledge to use with other Splunk apps, such as Splunk Enterprise Security, the Splunk App for PCI Compliance, and Splunk IT Service Intelligence.

Download the Splunk Add-on for Microsoft Cloud Services from Splunkbase.

For a summary of new features, fixed issues, and known issues, see Release notes for the Splunk Add-on for Microsoft Cloud Services.

For information about installing and configuring the Splunk Add-on for Microsoft Cloud Services, see Installation overview for the Splunk Add-on for Microsoft Cloud Services.

See the Splunk Answers page for more information about this add-on.

Source types for the Splunk Add-on for Microsoft Cloud Services

The Splunk Add-on for Microsoft Cloud Services provides the index-time and search-time knowledge for Microsoft Cloud Services data in the following formats:

The ms:o365:management source type is for backward compatibility. A similar source type, o365:management:activity, is in the Splunk Add-on for Microsoft Office 365.

Data source

Source type

Event type

API

CIM data models

ITSI data models

Notes

Azure Event Hubs

mscs:azure:eventhub

n/a

Microsoft Azure Event Hubs Client Library for Python

n/a

n/a

Azure Event Hubs

mscs:azure:security:alert

n/a

Microsoft Azure Event Hubs Client Library for Python

Alerts

n/a

Azure Event Hubs

mscs:azure:security:recommendation

n/a

Microsoft Azure Event Hubs Client Library for Python

Alerts

n/a

Azure Event Hubs

azure:monitor:aad

mscs_audit_auth_account_management, mscs_audit_auth_all_changes, mscs_audit_auth_authentication, mscs_audit_auth_alerts, mscs_azure_aad_auditlogs, mscs_azure_aad_signinlogs, mscs_azure_aad_provisionlogs, mscs_azure_aad_userlogs

Microsoft Azure Event Hubs Client Library for Python

Alerts, Authentication, Change

n/a

Azure Event Hubs

azure:monitor:resource

n/a

Microsoft Azure Event Hubs Client Library for Python

Change, Databases DataAccess

n/a

Azure Event Hubs

azure:monitor:activity

mscs_azure_activity_all_changes, mscs_azure_activity_instance_changes, mscs_azure_activity_administrative_logs

Microsoft Azure Event Hubs Client Library for Python

Change

n/a

Azure Resource virtualMachine

mscs:resource:virtualMachine

mscs_inventory_vm

Azure Virtual Machines REST — List
Azure Virtual Machines REST — Get VM information

n/a

Inventory

Azure Resource network
InterfaceCard

mscs:resource:networkInterfaceCard

mscs_inventory_vm

Azure Network REST — List network interface cards

n/a

Inventory

Azure Resource public
IPAddress

mscs:resource:publicIPAddress

n/a

Azure Network REST — List public IP addresses

n/a

n/a

Resource virtualNetwork

mscs:resource:virtualNetwork

n/a

Azure Network REST — List virtual networks

n/a

n/a

Azure Resource Disk

mscs:resource:disk

mscs_azure_resource_disk

n/a

Inventory, Storage

n/a

Azure Resource Image

mscs:resource:image

mscs_azure_resource_image

n/a

Inventory, Virtual

n/a

Azure Resource Snapshot

mscs:resource:snapshot

mscs_azure_resource_snapshot

n/a

Inventory, Virtual, Snapshot

n/a

Azure Resource Group

mscs:resource:resourceGroup

mscs_azure_resource_resourceGroup

n/a

Inventory

n/a

Azure Resource Subscription

mscs:resource:subscriptions

mscs_azure_resource_subscriptions

n/a

Inventory

n/a

Azure Resource SecurityGroup

mscs:resource:securityGroup

mscs_azure_resource_securityGroup

n/a

Inventory

n/a

Azure Audit log

mscs:azure:audit

n/a

Azure Insights — List events for an Azure subscription

Alerts, Change

n/a

Azure Storage Table

mscs:storage:table

n/a

Azure SDK for Python

n/a

n/a

Azure Storage Blob

mscs:storage:blob

n/a

Azure SDK for Python

n/a

n/a

Azure Storage Blob

mscs:storage:blob:json

n/a

Azure SDK for Python — Storage Table query_
entities

n/a

n/a

When selected in the input, XML and JSON fields for the mscs:storage:blob:xml and mscs:storage:blob:json source types are automatically extracted. You can configure the settings for these source types in their respective stanzas in your local props.conf file.

Azure Storage Blob

mscs:storage:blob:xml

n/a

Azure SDK for Python — Storage Table query_
entities

n/a

n/a

When selected in the input, XML and JSON fields for the mscs:storage:blob:xml and mscs:storage:blob:json source types are automatically extracted. You can configure the settings for these source types in their respective stanzas in your local props.conf file.

Virtual Machine Metrics

mscs:vm:metrics

mscs_perf_vm_cpu

Azure SDK for Python — Storage Table query_
entities

n/a

Performance

Azure Metrics

mscs:metrics

n/a

n/a

n/a

n/a

Azure Metrics

mscs:metrics:events

n/a

n/a

n/a

n/a

Azure KQL Log Analytics

mscs:kql

n/a

n/a

n/a

n/a

Azure KQL Log Analytics

mscs:kql:stats

n/a

n/a

n/a

n/a

Azure Consumption (Billing)

mscs:consumption:billing

n/a

n/a

n/a

n/a

Azure Consumption (Billing)

mscs:consumption:reservation:recommendation

n/a

n/a

n/a

n/a

Hardware and software requirements for the Splunk Add-on for Microsoft Cloud Services

To install and configure the Splunk Add-on for Microsoft Cloud Services, you must be a member of the admin or sc_admin role.

You do not need a special role to use this add-on’s troubleshooting dashboard.

The Splunk Add-on for Microsoft Cloud Services uses two types of Microsoft accounts to collect data: the Azure App account and the Azure Storage account.

  • If you want to collect data from the Azure Audit, Azure Resource, Azure Metrics, Azure Consumption (Billing) and Azure KQL inputs, you must first apply for an Azure app account. Then, connect your Azure app account to the Splunk Add-on for Microsoft Cloud Services.
  • If you want to collect data from the Azure Storage Table input, which includes virtual machine metrics, or the Azure Storage Blob input, you must first apply for an Azure Storage account. Then, connect your Azure Storage account to the Splunk Add-on for Microsoft Cloud Services.

Microsoft account permission requirements

To collect data from Azure audit and Azure resources, you must configure an Microsoft Entra ID Application with read permissions. See Add permissions to your Active Directory Application.

To collect data from Azure storage table and Azure storage blob, see configure the storage account to get data.

Azure Government Cloud limitations

The Splunk Add-on for Microsoft Cloud Services has not been tested with Azure Government Cloud. The functionality of the Splunk Add-on for Microsoft Cloud Services responsible for Azure Government Cloud data is not supported and is provided “as is”, and should be used at your own risk.

Performance reference for the Splunk Add-on for Microsoft Cloud Services

For reference information about each performance tested inputs for the Splunk Add-on for Microsoft Cloud services, see the following topics in this manual:

Many factors impact performance results, including file size, file compression, event size, deployment architecture, batch size for Event Hub file size, and hardware. Results represent reference information and do not represent performance in all environments.

Splunk platform requirements

Because this add-on runs on the Splunk platform, all the system requirements apply for the Splunk software that you use to run this add-on.

For information about installation locations and environments, see Install the Splunk Add-on for Microsoft Cloud Services.

Support for macOS

The Splunk Add-on for Microsoft Cloud Services has not been tested with any versions of the macOS operating system. Running the Splunk Add-on for Microsoft Cloud Services is not supported and is provided “as is”, and should be used at your own risk.

Installation overview for the Splunk Add-on for Microsoft Cloud Services

Complete the following steps to install and configure this add-on on your supported platform.

  1. Install the Splunk Add-on for Microsoft Cloud Services.
  2. If you want to collect Azure Resource, Azure Audit, Azure Event Hub, Azure Metrics, Azure KQL Log Analytics, and Azure Consumption(Billing) data, see the following steps:
    1. Configure an application in Microsoft Entra ID for the Splunk Add-on for Microsoft Cloud Services.
    2. Connect to your Azure App Account with Splunk Add-on for Microsoft Cloud Services.
    3. Depending on the data you want to collect, see the following links:

  3. If you want to collect Azure Storage Table, Azure Storage Blob or Azure Virtual Machine Metrics data, perform the following steps:

    1. Configure a Storage Account in Microsoft Cloud Service.

    2. Connect to your Azure Storage account with the Splunk Add-on for Microsoft Cloud Services.

    3. Depending on the data you want to collect, see the following links:

Ended: Overview

Installation ↵

Install the Splunk Add-on for Microsoft Cloud Services

Follow these high-level steps to install the Splunk Add-on for Microsoft Cloud Services:

  1. Get the Splunk Add-on for Microsoft Cloud Services by downloading it from Splunkbase or browsing to it using the app browser within Splunk Web.
  2. Determine where and how to install this add-on in your deployment, using the tables on this page.
  3. Perform any prerequisite steps before installing, if required and specified in the following tables.
  4. Complete your installation.

If you need step-by-step instructions on how to install an add-on in your specific deployment environment, see the Installation walkthroughs section for links to installation instructions specific to a single-instance deployment, distributed deployment, or Splunk Cloud Platform.

Distributed deployment

Use the following tables to determine where and how to install this add-on in a distributed deployment of Splunk Enterprise or any deployment for which you are using forwarders to get your data in. Depending on your environment, your preferences, and the requirements of the add-on, you might need to install the add-on in multiple places.

Where to install this add-on

Unless otherwise noted, you can safely install all supported add-ons to all tiers of a distributed Splunk platform deployment. See Where to install Splunk add-ons in Splunk Add-ons for more information.

This table provides a reference for installing this specific add-on to a distributed deployment of the Splunk platform.

Splunk platform instance type

Supported

Required

Actions required / Comments

Splunk Cloud

Yes

No

To install the Splunk Add-on for Microsoft Cloud Services on your Splunk Cloud instance, file an installation request with Splunk Support.

Search Heads

Yes

Yes

Install this add-on to all search heads where Microsoft Cloud Services knowledge management is required.

As a best practice, turn off add-on visibility on your search heads to prevent data duplication errors that can result from running inputs on your search heads instead of or in addition to your data collection node.

Indexers

Yes

Conditional

Not required. Parsing operations occur on the heavy forwarders. If using a HEC token, indexer build installation is required on indexers.

Heavy forwarders

Yes

No

This add-on supports heavy forwarders and Inputs Data Manager (IDM) for data collection.

Universal forwarders

No

No

This add-on supports only heavy forwarders and Inputs Data Manager (IDM) for data collection because the modular inputs require Python and the Splunk REST handler.

Inputs Data Manager (IDM)

Yes

No

This add-on supports heavy forwarders and Inputs Data Manager (IDM) for data collection. IDM is required for Splunk Cloud.

Distributed deployment feature compatibility

This table describes the compatibility of this add-on with Splunk distributed deployment features.

Distributed deployment feature

Supported

Actions required / Comments

Search Head Clusters

Yes

Turn off add-on visibility on search heads.

You can install this add-on on a search head cluster for all search-time functionality, but configure inputs on forwarders to avoid duplicate data collection.

Before you install this add-on to a cluster, remove the inputs.conf file.

Indexer Clusters

Yes

Before you install this add-on to a cluster, remove the inputs.conf file.

Deployment Server

No

Supported for deploying unconfigured add-ons only. Using a deployment server to deploy configured add-ons to multiple forwarders acting as data collectors causes duplication of data.

Installation walkthroughs

The Splunk Add-Ons manual includes an About installing Splunk add-ons guide that helps you successfully install any Splunk-supported add-on to your Splunk platform.

For a walkthrough of the installation procedure, follow the link that matches your deployment scenario:

Upgrade the Splunk Add-on for Microsoft Cloud Services

The following migration guide is required for upgrading from version 4.0.1 or later. Upgrading from any version older than 3.0.0 requires a fresh installation of version 4.0.1 or later.

A best practice for upgrading the Splunk Add-on for Microsoft Cloud Services is to remove your older version before re-installing the latest version of the Splunk Add-on for Microsoft Cloud Services.

Upgrade from a version older than 3.0.0

  1. Install the Splunk Add-on for Microsoft Cloud Services version 4.0.1 and later from the Splunk Web UI (make sure Upgrade App checkbox is selected).
  2. Restart the Splunk platform.
  3. Navigate to the input page of the Splunk Add-on for Microsoft Cloud Services. Alerts will appear, indicating incomplete account authorization.
  4. Edit each required input by clicking the click here link to navigate to the account configuration page or by directly navigating to the account configuration page.
  5. Complete the authorization of your account by adding your account secret key/account token.
  6. Repeat the above steps for all inputs with alert signs against them.

In previous versions, settings including proxy, logging, and performance were stored in splunk_ta_o365_client_setting.conf and splunk_ta_o365_server_setting.conf. In version 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services, all setting and performance tuning configurations are in splunk_ta_mscs_setting.conf. The default log level is INFO.

Versions 3.0.0 and above of the Splunk Add-on for Microsoft Cloud Services removes the Microsoft Office 365 module. See the Splunk Add-on for Microsoft Office 365.

Upgrade from a version older than 4.4.0

  1. If Eventhub inputs were configured using a version earlier than 4.4.0 and any third-party apps that use Event Hub data formatting should follow the below-mentioned steps:
    1. Before upgrading, disable the Event Hub inputs.
    2. Upgrade the TA to the latest version.
    3. For the Event Hub inputs add event_format_flags = 1
    4. Enable the Event Hub inputs.
  2. While creating a new Event Hub input, add event_format_flags = 1 for the Apps which are dependent on the EventHub data formatting.

Upgrade from a version older than 5.0.0

  1. Follow the Standard Upgrade Guide.
  2. After enabling the Storage Blob inputs, wait for the completion of file-based checkpoint to KV Store migration by following the successful migration notification in the Splunk Messages.
    1. The following SPL query is used to verify the successful KV Migration for Storage Blob inputs:

    Search

    index=\_internal source=\*storage_blob\* "Checkpoint has been migrated to KVstore"

Standard Upgrade Guide

  1. Verify that you are running version 8.0.0 or later of the Splunk software.
  2. (Optional) Plan your Splunk Enterprise upgrade to work with the Python 3 migration.
  3. Disable all your inputs before you upgrade the add-on. Otherwise, you might see errors in the log files, resulting in data loss against your already configured inputs.
  4. Upgrade the Splunk Add-on for Microsoft Cloud Services to the required version, and follow the version-specific upgrade guide.
  5. Enable each desired input to start data collection. Enable Storage Blob inputs in small batches.

Migrate from the Splunk Add-on for Microsoft Azure

To collect Microsoft Entra ID data using an Azure Event Hub, migrate from the Splunk Add-on for Microsoft Azure to the Splunk Add-on for Microsoft Cloud Services. See the following steps:

  1. Install the latest version of Splunk Add-on for Microsoft Cloud Services.
  2. Configure an application in Microsoft Entra ID for the Splunk Add-on for Microsoft Cloud Services.
  3. Configure a Storage Account in Microsoft Cloud Services.
  4. Connect to your Azure App Account with Splunk Add-on for Microsoft Cloud Services.
  5. Configure Azure Event Hub inputs for the Splunk Add-on for Microsoft Cloud Services.
  6. Run the following search to verify data collection: index=* sourcetype="azure:monitor:*".

Source type changes

See the following source type changes from the Splunk Add-on for Microsoft Azure to the Splunk Add-on for Microsoft Cloud Services:

Azure source type MSCS event type MSCS source type
azure:aad:user mscs_azure_aad_userlogs azure:monitor:aad
azure:aad:signin mscs_azure_aad_signinlogs azure:monitor:aad
azure:aad:audit mscs_azure_aad_auditlogs azure:monitor:aad

CIM field changes

See the following CIM Field Changes from the Splunk Add-on for Microsoft Azure to the Splunk Add-on for Microsoft Cloud Services:

CIM field The Splunk Add-on for Microsoft Azure Extraction The Splunk Add-on for MSCS Extraction
Vendor Product Microsoft Entra ID Microsoft Entra ID
src Event field: ipAddress
Instead of ipAddress, properties.ipAddress was found. So assume the current add-on field is not getting extracted.		 Event field: callerIpAddress
src_ip Event field: ipAddress
Instead of ipAddress, properties.ipAddress was found. So assume the current add-on field is not getting extracted.		 Event field: callerIpAddress
user_agent Event field: UserAgent
Instead of UserAgent, properties.userAgent was found. So assuming in the current add-on field is not getting extracted.		 Event field: properties.userAgent
app Event field: appDisplayName
Instead of appDisplayName we found properties.appDisplayName. So assuming in the current TA field is not getting extracted.		 Event field: properties.appDisplayName
dest Event field: resourceDisplayName Event field: tenantId
enabled Event field: accountEnabled Instead of accountEnabled we found provisioningSteps.details.dynamicProperties.accountEnabled. So assume the current TA field is not getting extracted. Event field: provisioningSteps.details.dynamicProperties.accountEnabled
authentication_method Event field: authenticationDetails{}.authenticationMethod
Sample values: Previously satisfied, Password		 Event field: properties.isInteractive
If properties.isInteractive is true, then it is Interactive. Otherwise, it is nonInteractive.
user Event Field: userPrincipalName (Authentication Event), displayName(User event) case(operationName IN ("Add service principal","Update service principal"),mvindex('properties.targetResources{}.displayName',mvfind('properties.targetResources{}.type',"^ServicePrincipal$")), \ operationName IN ("Provisioning activity"),'properties.provisioningSteps{}.details.dynamicProperties.userPrincipalName', \ operationName IN ("Redeem external user invite","Delete external user","Viral user creation"),UPN, \ like(operationName,"Add member to role in PIM%") OR like(operationName,"Add eligible member to role in PIM%") OR operationName IN ("Add member to role","Add member to group","Add owner to application","Update user","Invite external user","Reset user password","Restore user","Add member to role outside of PIM (permanent)","Change password (self-service)","Reset password (by admin)","Add eligible member to role","Remove eligible member from role","Remove member from group","Change user password"),'properties.targetResources{}.userPrincipalName',operationName IN ("Add device"),'properties.initiatedBy.app.displayName', \ true(),coalesce('properties.initiatedBy.user.userPrincipalName','properties.userPrincipalName','properties.servicePrincipalName'))
user_id Event Field: userPrincipalName (Authentication Event), displayName(User event) case(isnotnull('properties.servicePrincipalId') AND 'properties.servicePrincipalId' != "", 'properties.servicePrincipalId', \ true(), 'properties.userId')

Ended: Installation

Configuration ↵

Configure an application in Microsoft Entra ID for the Splunk Add-on for Microsoft Cloud Services

To gather data from the Windows Azure Service Management APIs, you must first create an application in Microsoft Entra ID.

Follow the instructions in the Microsoft documentation to create an application in Microsoft Entra ID: Use portal to create an Microsoft Entra ID application and service principal that can access resources for either your Azure portal or Azure Government portal.

When prompted, select or enter the following parameters:

  • Client ID: Required for Azure App account.
    • Copy this value. You need this value and a valid secret key to connect to your account from the add-on.
  • Key: Required for Azure App account.
    • Copy this value to a secure location as soon as the Microsoft Entra ID admin console displays it.
  • Tenant ID: Required for Azure App account.
    • Copy this value for the future use.

Application permissions to access Windows Azure Service Management APIs

Select Access Azure Service Management as organization under Delegated Permissions.

Grant the application read access in Microsoft Entra ID

After creating the application in Microsoft Entra ID, login to either https://portal.azure.com or https://portal.azure.us on the Azure website, and perform the following steps:

  1. You must have a Premium P1 level edition or higher to perform this operation. See https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/ on the Microsoft Azure website for more information on who can access resources.
  2. Navigate to Home > Subscriptions.
  3. Select the active subscription that you want to use from the Subscription Name column.
  4. Select Access control (IAM)
  5. Select Role assignments
  6. Select Add role assignment.
  7. In the Add role assignment drop-down list, perform the following steps:
    1. Select Reader from the Role dropdown list.
    2. Select User, group, or service principal from the Assign access to dropdown list, if it has not already been selected.
    3. Select your application by searching for it by name in the dropdown.
  8. Save your changes.

Configure a storage account in Microsoft Cloud Services

To gather data from Azure Storage Table, Azure Storage Blob, and Azure Virtual Machine Metrics, you need to create or configure a storage account in Microsoft Azure.

Create and manage a storage account

See https://docs.microsoft.com/azure/storage/storage-create-storage-account in the Microsoft documentation for the instructions to create and manage the storage account.

Configure the Storage Account to get data

The Splunk Add-on for Microsoft Cloud Services provides two methods for you to get Azure storage table and Azure virtual machine metrics data. You can use either an Access Key or Account Token (SAS: Shared access signature) with the following steps.

If you want to get Azure storage blob data, you can also use None Secret to get the data without inputting a key or token.

Get storage account access key

  1. Log in to your Azure portal or Azure Government portal.
  2. Select the storage account you want to use.
  3. Copy either Key1 or Key2 of Access Key under Settings.

Get your storage account token

  1. Log in to your Azure portal or Azure Government portal.
  2. Select the storage account you want to use.
  3. In the Shared access signature drop-down list, configure your shared access signature based on the data you want to collect, such as allowed services, allowed resource types and start and expiry date/time under Settings. In the Allowed resource type section, verify that the Service box is checked.
  4. Generate your SAS and copy it to your clipboard.

Get storage blob data without key or token

  1. Login to your Azure portal or Azure Government portal.
  2. Select All resources in the menu, then select the storage account that that you want to use.
  3. Select Overview and then select Blobs under Services.
  4. Select the container you want to configure and then select Access policy.

# Select Container for the Access type.

Connect to your Azure app account with the Splunk Add-on for Microsoft Cloud Services

Connect between the Splunk Add-on for Microsoft Cloud Services and your Azure App account so that you can ingest your Microsoft Cloud Services data into the Splunk platform. You can configure this connection using Splunk Web on your data collection node, or using the configuration files. To create an Azure App account, you need a Global Administrator to grant you the permissions.

Prerequisites

Before you complete these steps, follow the directions in Configure an application in Microsoft Entra ID for the Splunk Add-on for Microsoft Cloud Services to prepare your Microsoft account for this integration.

Connect to your account using Splunk Web

Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on.

  1. Launch the add-on, then select Configuration.
  2. Select Azure App Account > Add Azure App Account.
  3. Enter a friendly Name for the account.
  4. Enter the Client ID , Key (Client Secret) and Tenant ID using the following Account attributes table.
  5. Select Add.

Connect to your account using configuration files

If you do not have access to Splunk Web on your data collection node, you can configure the connection to your account using the configuration files.

  1. Create or open $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_azure_accounts.conf.
  2. Add the following stanza: 
    [<account_stanza_name>]
client_id = <value>
client_secret = <value>
tenant_id = <value>

Account attributes

Attribute Corresponding name in Splunk Web Description
account_stanza_name Name Enter a friendly name for your Azure app account. Account name cannot contain any whitespace.
client_id Client ID Use the Client ID that Microsoft Entra ID automatically assigned to your integration application
client_secret Key (Client Secret) Enter the password for Client ID
tenant_id Tenant ID Enter the Tenant ID when you Create an application in the Splunk add-on for Microsoft Azure

Configure Azure resource modular inputs for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice, or by using the configuration files.

Prerequisites

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure Resource.
  3. Fill out the Name, Azure App Account, Subscription ID, Resource Type, Resource Group List, Interval and Index fields using the Input parameters.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. Create a file called mscs_azure_resource_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Add the following stanza: 
    [<input_stanza_name>]
account = <value>
subscription_id = <value>
resource_type = <value>
resource_group_list = <value>
index = <value>
interval = <value>
  3. Save and restart Splunk platform.

Input parameters

Attributes Corresponding Fields in Splunk Web Description
input_stanza_name Name A friendly name for your input. Name cannot contain any whitespace.
account Azure App Account The Azure App Account from which you want to gather data. Name cannot contain any whitespace.
subscription_id Subscription ID The instance queries that manage events belonging to this subscription. The subscription ID is the one you configured in Microsoft account
resource_type Resource Type You can choose from Virtual Machine, Public IP Address, Network Interface Card, Virtual Network, Disk Data, Image Data, Snapshot Data, Resource Groups, Security Groups and Subscriptions using Splunk Web, or set resource_type to virtual_machine, public_ip_address, network_interface_card, virtual_network, disk_data, image_data, snapshot_data, resource_groups, security_groups, subscriptionsin the configuration file.
resource_group_list Resource Group List The resource group list is defined by subscription ID and resource type. If you leave this field blank, this add-on will query all resource lists under the subscription ID and the resource type you choose. You can add multiple resource group list separated by commas.
interval Interval The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.
index Index The index where the Microsoft Cloud Services data is stored.

Configure Azure audit modular inputs for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice, or by using the configuration files.

Prerequisites

Before you enable inputs, complete the previous steps in the configuration process:

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure Audit.
  3. Enter the Name, Azure Account, Subscription ID, Start Time, Interval and Index using the information in the Input parameters table.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. Create a file named mscs_azure_audit_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Add the following stanza: 
    [<input_stanza_name>]
account = <value>
subscription_id = <value>
start_time = <value>
index = <value>
interval = <value>
  3. Save and restart the Splunk platform.

Verify that the value listed for account matches the account entry in mscs_azure_accounts.conf.

Input parameters

Create Azure Eventhub input for each log category e.g.: Microsoft Entra ID, Resource, and Activity.

Attribute

Corresponding field in Splunk Web

Description

input_stanza_name

Name

A friendly name for your input. Name cannot contain any whitespace.

account

Azure Account

The Azure App account from which you want to gather data. Name cannot contain any whitespace.

subscription_id

Subscription ID

The instance queries the management events belong to this subscription. The subscription ID is the one you configured in Microsoft account requirements.

start_time

Start Time

The add-on starts collecting data with a date later than this time. The format is YYYY-MM-DDThh:mm:ssTZD and the default is 30 days before the configuration, e.g. 2016-07-15T09:00:00+08:00 stands for fetching data from 2016-07-15 09:00:00 in UTC+8 time zone.

The maximum start time of Azure Audit inputs is 90 days before the configuration.

interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.

index

Index

The index in which to store Azure audit data.

Connect to your Azure Storage account with the Splunk Add-on for Microsoft Cloud Services

Connect the Splunk Add-on for Microsoft Cloud Services and your Azure Storage account so that you can ingest your Azure storage table, Azure storage blob and Azure virtual machine metrics data into the Splunk platform. You can configure this connection using Splunk Web on your data collection node as a best practice, or by using the configuration files.

Prerequisites

Before you complete these steps, follow the directions in Configure a Storage Account in Microsoft Cloud Services to prepare your Microsoft account for this integration.

When the Splunk Add-on for Microsoft Cloud Services is used in conjunction with Azure storage, the number of inodes available can fill up quickly, creating pointer files on your operating system for every single blob. This results in a file directory containing extremely large numbers of files, and potentially resulting in a “no space left on device” error.

Connect to your account using Splunk Web

Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on.

  1. Open the add-on, then select Configuration.
  2. Select Azure Azure Storage Account and enter the corresponding fields using the Input parameter table.

There are three Account Secret Types that you can select to configure an Azure storage account: Access Key, Account Token, and None Secret.

  • If you want to collect Azure storage table Azure virtual machine metrics data, you have to configure the account with the Access Key or Account Token.
  • If you want to collect Azure storage blob data, you can use any of three types.

Connect to your account using configuration files

If you do not have access to Splunk Web on your data collection node, you can configure the connection to your account using the configuration files.

  1. Create or open $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_storage_accounts.conf.
  2. Add the following stanza: 
    [<account_stanza_name>]
account_name = <value>
account_secret = <value>
account_secret_type = <value>
account_class_type = <value>

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute

Corresponding field in Splunk Web

Description

account_name

Account Name

The name for the storage account. Name cannot contain any whitespace.

account_secret

Account Secret

You can enter the key or token generated when you Configure a Storage Account in Microsoft Cloud Service.

account_secret_type

Access Key, Account Token or None Secret

If you set account_secret_type=0, it means the storage account use the None Secret type. You do not have to set Account Name and Account Secret. If you configure the inputs using a configuration file, you can leave account_name and account_secret blank.

If you set account_secret_type=1, it means the storage account uses Access Key type. You have to enter the key generated when you Configure a Storage Account in Microsoft Cloud Service.

If you set account_secret_type=2, it means the storage account use Account Token type. You have to enter the token generated when you Configure a Storage Account in Microsoft Cloud Service.

account_class_type

Account class type

Type of account class. The integer is either 1 or 2, 1 for Azure public cloud, and 2 for Azure government cloud.

Configure the Azure Storage Table modular Input for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice, or by using the configuration files.

Prerequisites

Before you enable inputs, complete the previous steps in the configuration process:

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input.
  3. Select Azure Storage Table.
  4. Select Input type as Storage table, and fill out the Name, Azure Storage Account, Table List, Start Time, Interval, Index and Sourcetype fields using the Input parameters table.

Configure inputs using configuration files

  1. Create a file called inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Configure Azure storage table inputs with the following stanza: 
    [mscs:storage:table://<input_name>]
account = <value>
collection_interval = <value>
storage_table_type = storage_table
table_list = <value>
start_time = <value>
index = <value>
sourcetype = <value>
  3. Save and restart the Splunk platform.

Input parameters

Attributes

Corresponding field in Splunk Web

Description

mscs:storage:table://<input_name>

Name

A friendly name for your input. Name cannot contain any whitespace.

account

Azure Storage Account

Choose a Storage Account you have configured. Name cannot contain any whitespace.

table_list

Table List

The table list under the storage account. You can enter multiple table names separated by commas. You can also use wildcards (*) or regular expression in the table name. If the table name uses regular expressions, add a colon in front of the table name. For example: table*, :table\d+.

start_time

Start Time

The add-on starts collecting data with a date later than this time. The format is YYYY-MM-DDThh:mm:ssTZD and the default is 30 days before the configuration. For example, 2016-07-15T09:00:00+08:00 collects data from 2016-07-15 09:00:00 in the UTC+8 time zone.

collection_interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.

index

Index

The index in which to store Azure Storage Table data.

sourcetype

Sourcetype

The default source type is mscs:storage:table.

If you want to change the default source type, Splunk software detects the time field of the event, which can cause errors in the timestamp field. To prevent this issue, configure the timestamp under SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/props.conf.

storage_table_type

Input Type, with Storage Table as selection value.

Choose Storage Table' for the input type.

Configure Azure Storage Blob modular inputs for the Splunk Add-on for Microsoft Cloud Services

Before you enable inputs, complete the previous steps in the configuration process:

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web as a best practice, or you can use the configuration files.

Versions 5.0.0 and higher of the Splunk Add-on for Microsoft Cloud Services contain changes to the checkpoint mechanism for the Storage Blob input. See the upgrade steps in this manual for more information.

The Azure Storage Blob modular input for Splunk Add-on for Microsoft Cloud Services does not support the ingestion of gzip files. Only plaintext files are supported.

Since the format of the data in Azure Storage Blob channel varies, use source types to make the event data more effective. See Overview of Event Processing for more information.

Horizontal Scaling

Introduced Horizontal Scaling in the Splunk Add-on for Microsoft Cloud Services Version 5.0.0. Horizontal scaling provides functionality to collect data from the same Storage Container using multiple inputs in parallel to reduce data ingestion delays.

Analyze the user-case before opting for Horizontal Scaling. Horizontal Scaling is designed for containers containing a huge number of files. However, if there is a small number of large files in the container, then scaling up might be limited by the indexing rate of the environment.

Horizontal Scaling is not directly proportional to ingestion rate. For instance, if 1 input is capable of collecting the entire container’s data in 1 hour, then creating 2 inputs will not necessarily make the net collection time to 30 mins, and 3 inputs will not necessarily bring it down to 20 minutes.

Scale the inputs incrementally and monitor the ingestion rate before scaling up the environment again. If the number of inputs starts filling up the indexing queue of Splunk, then the health of the environment might be adversely affected.

The horizontal scaling should only be used after the file based checkpoint for the input has been successfully migrated to KV Store. Otherwise, it may lead to data duplication.

Prerequisites

  • All input should use the same index.
  • All Splunk instances should use the centralized KVStore. In Victoria stack, there is a centralized KVStore so this feature can be used there. If Splunk instances use a different KVStore, there will be data duplication. If one Heavy Forwarder uses its own KVStore and another Heavy Forwarder uses a different KVStore, and both Heavy Forwarders have their inputs collecting data from the same Storage Container, then there will be data duplication.

Risks

  • There is a small chance of data duplication, up to 5%.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and select Azure Storage Blob.
  3. Enter the Name, Storage Account, Container Name, Blob list, Interval, Index and Sourcetype using the Inputs parameters table.

Configure inputs using Configuration File

  1. Create a file called inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Configure the Azure Storage Blob input with the following stanza: 
        [mscs_storage_blob://<input_name>]]
    account = <value>
    application_insights = <value>
    blob_mode = <value>
    collection_interval = <value>
    container_name = <value>
    prefix = <value>
    blob_list = <value>
    exclude_blob_list = <value>
    decoding = <value>
    guids = <value>
    index = <value>
    log_type = <value>
    sourcetype = <value>
    disabled = <value>
    read_timeout = <value>
    blob_compression = <value>

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute

Corresponding field in Splunk Web

Description

mscs:storage:blob://<input_name>

Name

Enter a friendly name of your inputs. Do not use whitespaces in your input names. Name cannot contain any whitespace.

account

Azure Storage Account

Select the storage account name you configured. Name cannot contain any whitespace.

application_insights

Application Insights Check

Indicates whether the Azure storage blob ingests data from application_insights. Set the value to 1 to ingest data, otherwise set it to 0. You must configure Log type and GUIDs if application_insights is set to 1.

container_name

Container Name

Enter the container name under the storage account. You can only add one container name for each input.

prefix

Prefix

Specify the prefix string for the blobs. Input will only collect the data from the blobs whose names begin with the specified prefix.

For instance, if user wants to collect data from h=08 blob, and the blob name is /y=2022/m=10/d=05/h=08/m=00/blob1.txt, then the prefix can be specified as /y=2022/m=10/d=05/h=08.

Constraints:

  • Blob name should not contain the trailing and leading spaces.
  • Prefix should not contain spaces in path segment. For instance, the below prefix would not provide the data from blob. Prefix:/y=2022 / m=10 / d=05 /.

blob_list

Blob List

Enter the Blob name, wildcard, or regular expression for the data you want to collect. You can add multiple blob names separated by commas. If you leave this field empty, this add-on collects all the blob lists under the Container Name you configured. If you want to collect data from a specific blob list, enter the name of the blob list, such as blob_name. You can use wildcards, such as blob*, to collect data from the blob lists starting from blob. Use commas to separate multiple blob names, such as blob, name*. To use regular expressions, use thisJSON format syntax: {"regex syntax":3}, where 3 stands for the regular expression file. If you want to enter the blob list which has both a wildcard and a regular expression, enter both separated by commas, for example, {"regex syntax" :3, blob* :2}, where 2 stands for a wildcard list. To enter the blob list using all of the expressions, use this syntax: {"regex syntax" :3, blob* :2, blob :1}, where 1 stands for using a specific blob list name.

  • The blob name must be at least one character long but cannot be more than 1,024 characters.
  • Blob names are case-sensitive.
  • Reserved URL characters must be properly escaped.
  • The number of path segments comprising the blob name cannot exceed 254.

blob_mode

Blob Mode

Select the following blob mode:
append: Makes the blob storage input retrieve only the incremental changes.
random: Makes the blob storage input retrieve the entire blob again after the add-on is updated.
Default is random.

blob_compression

Blob Compression Type

Select the following blob compression type values:
Not compressed: Tells modular input that blob content is not compressed, i.e. the input will not try to inflate blob content after download.
Gzip: Tells modinput to inflate downloaded blob content using gzip algorithm. So far this is the only supported compression type. If inflation fails, modinput will log this with an error message and blob content will be ingested as is. To avoid this situation, ensure that the scope of container blobs is correctly defined using prefix, blob_list and/or exclude_blob_list configuration parameters. Note that Gzip is only supported with random blobs. For append blob mode only Not compressed compression type value is allowed.
Default is Not compressed.

collection_interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.

decoding

Decoding

Specify the character set of the file, such as UTF-8 or UTF-32. If you leave this field blank, this add-on uses the default character set of the file.

exclude_blob_list

Excluded Blob List

Optional. Enter the Blob name or regular expressions for the data you do not want to collect. You can add multiple blob names separated by commas. The syntax of the Excluded Blob List is the same as Blob List.
For example, if you do not want to include blobs from 2020, and your blob name is /y=2020/m=10/d=05/h=08/m=00/blob1.txt, then you enter this regular expression: .*y=2020\/.*.

guids

GUIDs

Indicates the GUID identifier used for application insights data with this format: <application insights resource name>_<instrumentation key>. Required if application_insights is turned on. Enter individual GUIDs as comma-separated values.

index

Index

The index in which to store Azure Storage Blob data.

log_type

Log type

Filters the results to return only blobs whose names begin with the specified log type. Use the following application Insights blob format: <container_name>/<guid>/<arbitrary_log_type_value>/<yyyy-mm-dd>/<hh>/<blob_file>. Use only one log type value per input. Required if application_insights is turned on.

sourcetype

Sourcetype

The default is mscs:storage:blob. To simplify field extraction, enter one of the following predefined sourcetypes: mscs:storage:blob:json, or mscs:storage:blob:xml.

read_timeout

Read Timeout

Specify the maximum amount of time (in seconds) to wait for a response from the Azure Storage service when reading data. The default value is 60 seconds.

If a file matches the syntax both in Blob List and Exclude Blob List, Exclude Blob List takes priority. For example, if there is a blob list name blob1, and it matches the syntax you set in Blob List and Exclude Blob List, this add-on will exclude this list because Exclude Blob List is in higher priority.

Configure ingestion mode

Configure ingestion mode by selecting a blob mode that aligns with the blob type that you selected while creating the blob in your Azure storage account.

  1. On your Splunk platform deployment, navigate to the $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local directory.
  2. Open inputs.conf file with a text editor.
  3. Navigate to the stanza of the blob storage input that you created.
  4. Change the blob_mode attribute to append or random, based on the following table:

    blob_type\ingestion_mode

    Incremental

    Full

    append

    blob_mode is irrelevant.
    You always receive incremental changes to your blob.

    N/A

    block or page


    If you use a block_blob to append data to the blob and only want the incremental changes, set blob_mode = append.

    blob_mode = random.
    After a blob is complete or closed, the contents are ingested to the Splunk platform.
  5. Save your changes.

Advanced Configuration

Introduced the “Allow Storage Blob Deletion” option in the Configuration -> Advanced tab in the Version 5.0.0 of the Splunk Add-on for Microsoft Cloud Services. This option allows the deletion of checkpoint files from the Splunk environment after migration to KVStore. Enable this option after all the Storage Blob inputs are migrated to KVStore successfully and the System is stable.

Configure Azure Virtual Machine metrics modular input for Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web as a best practice, or by using configuration files.

Prerequisites

Before you enable inputs, complete the previous steps in the configuration process:

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You cannot configure Table List, Interval, or Sourcetype using Splunk Web.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and select Azure Storage Table.
  3. Select Input type as Virtual Machine Metrics and type the Name, Storage Account, Start Time and Index using the Input parameters.
  4. Select Add.

Configure inputs using configuration file

  1. Create a file called inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Configure Azure virtual machine metrics inputs with the following stanza: 
    [mscs_storage_table://<input_name>]
account = <value>
storage_table_type = vm_metrics
table_list = WADMetricsPT1M*
start_time = <value>
index = <value>
collection_interval = 60
sourcetype = mscs:vm:metrics
  3. Save and restart Splunk platform.

Input parameters

Attribute

Corresponding field in Splunk Web

Description

mscs_storage_table://<input_name>

Name

A friendly name for your input. Name cannot contain any whitespace.

account

Azure Storage Account

Choose a Storage Account you have configured. Account name cannot contain any whitespace.

table_list

Table List

Enter a table list name under the storage account. You cannot change the Table List name in Splunk Web, which is WADMetricsPT1M*.

The best practice is to keep the default value WASMetricsPT1M* in the table list.

start_time

Start Time

The add-on starts collecting data with a date later than this time. The format is YYYY-MM-DDThh:mm:ssTZD and the default is 30 days before the configuration, e.g. 2016-07-15T09:00:00+08:00 stands for fetching data from 2016-07-15 09:00:00 in UTC+8 time zone.

collection_interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 60 seconds, and you cannot change this interval in Splunk Web. If you want to change the interval time, you have to configure it using the configuration file.

If you want to use ITSI data models, the best practice is to set the interval to 60 seconds.

index

Index

The index in which to store Azure Storage Table data.

sourcetype

Sourcetype

The default is mscs:vm:metrics. You cannot change the sourcetype in Splunk Web. If you want to change the sourcetype, you have to configure it using the configuration file.

storage_table_type

Input Type, with Virtual Machine Metrics as the selection value.

Choose data input as Virtual Machine Metrics.

Configure Office 365 Management APIs inputs for the Splunk Add-on for Microsoft Cloud Services

This functionality has moved to the Splunk Add-on for Microsoft Office 365. See the Configure Office 365 Management APIs inputs for the Splunk Add-on for Microsoft Office 365 section of the Splunk Add-on for Microsoft Office 365 manual.

Connect to your Microsoft Office 365 account with the Splunk Add-on for Microsoft Cloud Services

Set up integration between the Splunk Add-on for Microsoft Cloud Services and your Microsoft Office 365 account so that you can ingest your Microsoft Cloud Services data into the Splunk platform.

You can connect to your account only using Splunk Web. Configuring your Microsoft Office 365 account with configuration files is not supported.

Prerequisite

Before you complete these steps, follow the directions in Configure an application in Microsoft Entra ID for the Splunk Add-on for Microsoft Cloud Services to prepare your Microsoft account for this integration.

Connect to your account using Splunk Web

  1. Clear your cache, start a new browser session, or use a different browser than the one you use to sign in to the Microsoft Entra ID admin console. This best practice helps to avoid issues with incorrectly cached credentials that interfere with OAuthentication.

  2. Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on.

  3. Open the add-on, then select Configuration > O365 account.

  4. Select Account > Add Account.

  5. Enter a friendly Name for the account.

  6. Choose the public account type, or choose GCC High if you are using the high-security government version.

  7. Enter the Client ID that Microsoft Entra ID automatically assigned to your integration application.

  8. In the Key (Client Secret) field, enter the secret key that you created for your application in the Microsoft Entra ID console.

  9. Select Add.

  10. The Splunk Add-on for Microsoft Cloud Services authenticates using the client ID and secret you provided. Microsoft prompts you to log in with your account credentials to complete the authentication.

Enable a saved search

You can populate the vm_id, private_ip, and public_ip fields in a lookup file that works with mscs:resource:virtualMachine events.

Enable a saved search in the Splunk Add-on for Microsoft Cloud Services Object view in Splunk Web or in default/savedsearches.conf. You can set a schedule for the search to run on, and users can also run it manually.

Configure Azure Event Hub inputs for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice, or by using the configuration files.

Add-on prerequisites

Azure Event Hub prerequisites

Perform the following prerequisites before configuring an Azure Event Hub input:

  • Configure an Azure Event Hub for each log category in Azure, such as Microsoft Entra ID, Resource, and Activity. For more information, see the Quickstart: Create an Event Hub using Azure portal topic in the Microsoft Azure documentation for more information.
  • Authorize access to Azure Event Hubs by giving Azure Event Hubs Data Receiver permissions to each applicable Azure application. See the Authorize access to Azure Event Hubs topic in the Microsoft Azure documentation for more information..
  • Splunk Cloud customers who are installing this add-on on the Inputs Data Manager (IDM) and want to collect Event Hub data, must use the Admin Configuration Service (ACS) to configure outbound ports 5671/tcp and 5672/tcp (Advanced Message Queuing Protocol (AMQP) specification) to connect to their target Azure address. By default IDM’s can only go out on port 443.

Scaling

On your Azure deployment, a scaling best practice is to configure a ratio of at least one Event Hub throughput unit for each partition. For example, if you have 20 throughput units, the best practice is to configure 20 partitions. For more information on Event Hub throughput scalability, see the https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-scalability#throughput-units in the Microsoft Azure documentation.

On the Splunk software side, the number of Event Hub inputs that you create as consumers on an Event Hub must be less than or equal to the number of partitions that you have on the Event Hub. For more information, see the https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-features Event Hubs in the Microsoft Azure documentation.

Limitations

The Splunk Add-on for Microsoft Cloud Services does not support multiple Inputs Data Managers (IDMs) or heavy forwarders reading from a single Event Hub.

The Azure EventHubs input for the Splunk Add-on for Microsoft Cloud Services is not compatible with the Event Hubs input in the Splunk Add-on for Microsoft Azure, when listening to the same Event Hub namespace. The Event Hubs input in the Splunk Add-on for Microsoft Azure needs to be disabled for this input to run.

Horizontal Scaling Across Multiple Splunk Environment

Version 5.0.0 and higher of the Splunk Add-on for Microsoft Cloud Services supports multiple Eventhub inputs configuration across multiple Splunk environments to collect data from the same Azure Eventhub using the Storage Blob checkpoint store mechanism. To use the horizontal scaling, while creating the Eventhub input, enter “Enable Blob Checkpoint Store”, “Azure Storage Account” and “Container Name”.

Prerequisites

Risks

  • There is a small chance of data duplication, up to 5%.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure Event Hub.
  3. Enter the Name, Azure App Account, Event Hub namespace, Event Hub name, Consumer group, Max Wait Time, Max Batch Size, Transport Type, Interval and Index , “Enable Blob Checkpoint Store” then enter “Container Name”, and “Storage Account” using the information in the following Input parameters table.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Create a file named inputs.conf , if it does not already exist.
  3. Add the following stanza for the Event Hub input: 
    [<input_stanza_name>]
account = <value>
blob_checkpoint_enabled = <value>
storage_account = <value>
container_name = <value>
consumer_group = <value>
event_hub_name = <value>
event_hub_namespace = <value>
container_name = <value>
index = <value>
interval = <value>
max_batch_size = <value>
max_wait_time = <value>
use_amqp_over_websocket = 1
sourcetype = mscs:azure:eventhub
  4. Save and restart the Splunk platform.

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute Corresponding field in Splunk Web Description
input_stanza_name Name A friendly name for your input. Name cannot contain any whitespace.
account Azure Account The Azure App account from which you want to collect data. Name cannot contain any whitespace.
consumer_group Consumer Group The Azure Event Hub Consumer Group.
event_hub_name Event Hub Name The Azure Event Hub Name.
event_hub_namespace Event Hub Namespace (Fully Qualified Domain Name (FQDN)) The Azure Event Hub Namespace (FQDN). On portal.azure.com, on your Event Hubs Namespace page, the event_hub_namespace is displayed as Host Name in the Essentials section. It has the following formatting: .servicebus.windows.net.
index Index The index in which to store Azure Event Hub data.
interval Interval The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.
max_batch_size Max Batch Size The maximum number of events to retrieve in one batch. The default is 300.
max_wait_time Max Wait Time The maximum interval in seconds that the event processor will wait before processing. The default is 300 seconds.
use_amqp_over_websocket Transport Type The switch that allows use of Advanced Message Queuing Protocol (AMQP) over WebSocket. The default is AMQP over WebSocket.
The Event Hub input does not support AMQP as the transport type in Splunk Cloud Platform.
sourcetype Sourcetype Select the source type based on the configured Event Hub. Supported source types are mscs:azure:eventhub, azure:monitor:aad, azure:monitor:resource and azure:monitor:activity. The default sourcetype is mscs:azure:eventhub
blob_checkpoint_enabled Blob Checkpoint Store Enable storage blob as checkpoint for eventhub input.
It is important to note that if you use this option, there will be no backward compatibility for the File Checkpoint. If this option is checked once, and then disabled in future; there will be data duplication.
storage_account Azure Storage Account The Azure Storage account in which Container is created to store eventhub checkpoint.
container_name Container Name Enter the container name under the storage account. You can only add one container name for each input.
ensure_ascii Enforce ASCII encoding (JSON) If set to Strict ASCII the json events (and only those) are going to be encoded in ASCII. Native encoding doesn’t change the events encoding.

Configure Azure Metrics inputs for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice.

Prerequisites

Complete the following steps in the configuration process:

  1. Configure an Active Directory Application in Azure Active Directory for the Splunk Add-on for Microsoft Cloud Services, if you have not already done so.
  2. Connect to your Azure App Account with Splunk Add-on for Microsoft Cloud Services, if you have not already done so.
  3. Create an Azure App Account in the Splunk Add-on for Microsoft Cloud Services.
  4. Azure Metrics input provides support for the metric index. See Create metric indexes to create a metrics index.

The Azure Metrics input for the Splunk Add-on for Microsoft Cloud Services is not compatible with the Metrics input in the Microsoft Azure Add-on for Splunk.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure Metrics.
  3. Enter the Name, Azure App Account, Subscription IDs, Namespaces, Metric Statistics, Preferred Time Aggregation, Interval, Use Metric Index?, Index, Sourcetype, and Number of Threads using the information in the following Input parameters table.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Create a file named inputs.conf , if it does not already exist.
  3. Add the following stanza for Azure Metrics input:
    1. Input configuration for event index 
      [mscs_azure_metrics://<input_stanza_name>]
account = <value>
index = <value>
interval = <value>
metric_index_flag = no
metric_statistics = <value>
namespaces = <value>
number_of_threads = <value>
preferred_time_aggregation = <value>
sourcetype = mscs:metrics:events
subscription_id = <value>
    2. Input configuration for metrics index 
      [mscs_azure_metrics://<input_stanza_name>]
account = <value>
index = <value>
interval = <value>
metric_index_flag = yes
metric_statistics = <value>
namespaces = <value>
number_of_threads = <value>
preferred_time_aggregation = <value>
sourcetype = mscs:metrics
subscription_id = <value>
  4. Save and restart the Splunk platform.

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute Corresponding field in Splunk Web Description
input_stanza_name Name A friendly name for your input. . Input name cannot contain any whitespace.
account Azure Account The Azure App account from which you want to collect data. Account name cannot contain any whitespace.
subscription_id Subscription IDs The Azure Subscription containing the resources to query metrics.
Comma-separated list of subscriptions.
namespaces Namespaces Comma-separated list of metric namespaces to query. Refer to section ‘Supported metrics with Azure Monitor’ in microsoft document for list of available metrics namespaces.
Example: Microsoft.Compute/virtualMachines
metric_statistics Metric Statistics The type of statistic to gather. Valid options are average, minimum, maximum, total, and count
preferred_time_aggregation Preferred Time Aggregation The preferred aggregation type. If the preferred time period is not available for a specific metric in the namespace, the next available time grain will be used.
Valid options are PT1M, PT5M, PT15M, PT30M, PT1H, PT6H, PT12H, and P1D.
interval Interval The number of seconds to wait before the Splunk platform runs the command again. The default is 300 seconds.
metric_index_flag Use Metric Index? Use Metrix Index is for using metric index or event index. The default is yes (using metric index).
index Index The index that stores Azure Metrics data. It can be metrics, indexes, or events indexes based on the metric_index_flag value.
sourcetype Sourcetype The sourcetype to use for this input.
If metric index the sourcetype value is mscs:metrics.
If event index the sourcetype value is mscs:metrics:events.
number_of_threads Number of Threads The number of threads used to collect metric data in parallel. The default value is 5.

Configure Azure consumption (billing) inputs for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice.

Prerequisites

Before you enable inputs, complete the following steps in the configuration process:

The Azure Consumption(Billing) input for the Splunk Add-on for Microsoft Cloud Services is not compatible with Azure Reservation Recommendation and Azure Billing and Consumption inputs in the Microsoft Azure Add-on for Splunk.

The Azure Consumption (Billing) input for the Usage Details data type collects data until one day prior to the current UTC time at every interval invocation.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure Consumption(Billing).
  3. Enter the Name, Azure App Account, Subscription ID, Data Type, Interval, Index, Sourcetype, Max days to query and Start Date using the information in the following Input parameters table.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Create a file named inputs.conf, if it does not already exist.
  3. Add the following stanza for consumption input:
    1. Input configuration for the Usage Details data type 
      [mscs_azure_consumption://<input_stanza_name>]
account = <value>
data_type = Usage Details
index = <value>
interval = 86400
query_days = <value>
sourcetype = mscs:consumption:billing
start_date = <value>
subscription_id = <value>
    2. Input configuration for Reservation Recommendation data type 
      [mscs_azure_consumption://<input_stanza_name>]
account = <value>
data_type = Reservation Recommendation
index = <value>
interval = 86400
sourcetype = mscs:consumption:reservation:recommendation
subscription_id = <value>
  4. Save and restart the Splunk platform.

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web:

Attribute

Corresponding field in Splunk Web

Description

input_stanza_name

Name

A friendly name for your input. Name cannot contain any whitespace.

account

Azure Account

The Azure App account from which you want to collect data. Name cannot contain any whitespace.

subscription_id

Subscription ID

The Azure Subscription ID.

data_type

Data Type

Data Types:

  • Usage Details: To collect usage details data
  • Reservation Recommendation: To collect reservation recommendation data

The default is Usage Details

interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 86400 seconds.

index

Index

The index in which to store Azure Consumption data.

sourcetype

Sourcetype

Select the respective sourcetype based on the configured Data Types

  • Usage Details: mscs:consumption:billing
  • Reservation Recommendation: mscs:consumption:reservation:recommendation

The default is mscs:consumption:billing

query_days

Max days to query

Specify the maximum number of days to query The default is 10 days

Only visible and applicable when data type is Usage Details

When Usage Details data type is selected, each time this input runs a start date, it is calculated for the Usage Details API query. The end date for the Usage Details API query will be calculated as the start date plus the number of days specified by this parameter.

For example, if the calculated start date is 2022-01-01T00:00:00 (midnight on January 1, 2022), the end date for the query will be 2022-01-11T00:00:00 if the Max days to query is 10 days.

start_date

Start Date

Select a Start Date to specify how far back to go when initially collecting data The default is 90 days in the past

Only visible and applicable when data type is Usage Details

Configure Azure KQL Log Analytics input for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web, which is a best practice.

Prerequisites

Complete the following steps in the configuration process:

The Azure Log Analytics KQL input for the Splunk Add-on for Microsoft Cloud Services is not compatible with the Azure Log Analytics KQL input in the Microsoft Azure Add-on for Splunk.

During the data collection of the input, memory usage is directly proportional to the total response size of the provided KQL query. If the response size is very large, then it is expected to use high memory.

In each invocation of the input, it will ingest all the events returned by the KQL Query. Configure the input interval field based on how frequently the input should keep getting all its events.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In the Splunk Add-on for Microsoft Cloud Services, select Inputs.
  2. Select Create New Input and then select Azure KQL Log Analytics.
  3. Enter the Name, Azure App Account, Workspace ID, KQL Query, Interval, Index, Sourcetype, Index KQL Statistics and Index Empty Field Values using the information in the following Input parameters.

Configure inputs using configuration files

Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder.

  1. In your Splunk platform deployment, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  2. Create a file named inputs.conf, if it does not already exist.
  3. Add the following stanza to the Azure Log Analytics KQL input: 
    [mscs_azure_kql://<input_stanza_name>]
interval = <value>
index = <value>
account = <value>
workspace_id = <value>
kql_query = <value>
sourcetype = mscs:kql
index_stats = 0/1
index_empty_values = 0/1
  4. Save and restart the Splunk platform.

Input parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Attribute

Corresponding field in Splunk Web

Description

input_stanza_name

Name

A friendly name for your input. Name cannot contain any whitespace.

account

Azure Account

The Azure App account from which you want to collect data. Name cannot contain any whitespace.

workspace_id

Workspace ID

The ID of Azure Log Analytics Workspace on which the provided KQL Query will run.
Sample workspace ID: 12345678-da78-4b5c-a034-22463f5b8639

kql_query

KQL Query

The KQL Query to run on given workspace.
Sample KQL Query: SigninLogs | project UserDisplayName, Identity

interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds. In each invocation of the input, it will ingest all the events returned by the KQL Query. So configure the input interval field based on how frequently the input should keep getting all the events.

index

Index

The index in which to store Azure KQL Log Analytics data.

sourcetype

Sourcetype

The Sourcetype to use for this input.

index_stats

Index KQL Statistics

If enabled, then input will index a statistics event about the provided KQL query. The term :stats will be appended to the provided Sourcetype for the statistical event.

index_empty_values

Index Empty Field Values

If enabled, then input will also index KQL Log Analytic event's fields having empty values.

If Index Empty Field Values is not enabled then following example shows how raw event in Log Analytics Workspace will be ingested in Splunk. It will help in reducing event size by excluding empty fields.

Sample Raw Event in Log Analytics Workspace:

{
  "user": "test",
  "email": "email@test.com"
  "location": ""
  "mobile": ""
}

Sample Ingested Event in Splunk:

{
  "user": "test",
  "email": "email@test.com"
}

Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services

This step is only used when you need to configure Microsoft Office 365 Management APIs inputs. If you don’t have to configure Microsoft Office 365 Management APIs inputs, you can skip this step.

This add-on uses OAuth to authenticate from the Splunk platform to your Microsoft Office 365 account using an authorization token refreshed automatically with a refresh token. This authorization token has a mandatory expiration set by Microsoft, so the refresh token only keeps your integration current for a limited period. To avoid having to periodically re-enter a secret key manually, you can upload a Base64-encoded X.509 Certificate and private key to enable service-to-service calls and use the key credentials to update the manifest of your integration application in Microsoft Entra ID.

If you are using the configuration files to configure your connection to your Microsoft cloud services, this procedure is mandatory. If you are using Splunk Web, this procedure is highly recommended.

If you skip this step, then when your authorization token expires, you will need to edit your account configuration that handles your connection to Microsoft Office 365 by entering a new secret key from the Microsoft Entra ID admin console.

Configure a certificate and private key

You can configure the certificate and private key in Splunk Web on your data collection node (recommended), or in the configuration files.

Configure a certificate in Splunk Web

  1. In Splunk Web on the instance responsible for data collection with this add-on, go to the Splunk Add-on for Microsoft Cloud Services > Configuration.
  2. Click Certificate.
  3. Choose one of the two options.

    Option 1: Upload your own certificate and private key

    Using your preferred tool, generate a X.509 certificate file and a private key with a length of at least 2048 characters and upload them on this screen. For more information about using self-signed certificates, see How to self-sign certificates. Click Choose a Certificate and browse to the certificate file (.cer) in your file system.

    You need to decrypt the private key before you upload it on Splunk add-on for Microsoft Cloud Service.

    Option 2: Use an auto-generated certificate

    Choose this option if you want to use a certificate that the Splunk Add-on for Microsoft Cloud Services auto-generates for you.
  4. The add-on displays the keyCredentials JSON object for your certificate.
  5. Copy the entire JSON object to your clipboard.

Next, see Upload the certificate credentials to your integration application in Microsoft Entra ID.

Configure a certificate using the configuration files

  1. Generate a Base64-encoded X.509 certificate and put it in $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/certificate.cer.

Make sure the certificate is a X.509 certificate and the key length is at least 2048 . Shorter key lengths are not accepted by Microsoft Office 365 as valid keys.

  1. Create $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_certificate.conf and add the following stanza. 
    [certificate]
private_key = <Your private key, using '\' as link breaker>
  2. Next, you need to obtain the keyCredentials JSON object. Run 
    bin/splunk cmd python $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/bin/splunktamscs/key_credentials_generator.py $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/certificate.cer
  3. Copy the results to the manifest_json field in $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_certificate.conf.

Once your certificate is created using Splunk Web or using the configuration files, it will look like the following example. 

"keyCredentials": [{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977",
"customKeyIdentifier": "<custom-key-identifier>", "value": "<value>", "type": "AsymmetricX509Cert", "usage": "Verify"}]

For more information about using self-signed certificates, see How to self-sign certificates.

Next, continue with the procedure in the next section.

Upload the certificate credentials to your integration application in Microsoft Entra ID

  1. Sign in to the Azure management portal and navigate to the integration application that you created in Connect to your Microsoft Office 365 account with the Splunk Add-on for Microsoft Cloud Services.
  2. Click Manage Manifest > Download Manifest. It will look similar to the below example. 
    {
  "appId": "0399fdb3-c651-4360-ae33-97ed0598b5af",
  "appRoles": [],
  "availableToOtherTenants": false,
  "displayName": "zliang-test-app",
  "errorUrl": null,
  "groupMembershipClaims": null,
  "optionalClaims": null,
  "acceptMappedClaims": null,
  "homepage": "http://localhost:8000",
  "identifierUris": [
    "https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641b1b33f"
  ],
  "keyCredentials": [],
  "knownClientApplications": [],
  "logoutUrl": null,
  "oauth2AllowImplicitFlow": false,
  "oauth2AllowUrlPathMatching": false,
  "oauth2Permissions": [
    {
      "adminConsentDescription": "Allow the application to access zliang-test-app on behalf of the signed-in user.",
      "adminConsentDisplayName": "Access zliang-test-app",
      "id": "8448c8ef-a250-481e-ba5c-d877badd3e07",
      "isEnabled": true,
      "type": "User",
      "userConsentDescription": "Allow the application to access zliang-test-app on your behalf.",
      "userConsentDisplayName": "Access zliang-test-app",
      "value": "user_impersonation"
    }
  ],
  "oauth2RequirePostResponse": false,
  "objectId": "aa082da8-0f43-4a09-a364-630f4df75a62",
  "passwordCredentials": [],
  "publicClient": false,
  "replyUrls": [
    "http://localhost:8000"
  ],
  "requiredResourceAccess": [
    {
      "resourceAppId": "00000002-0000-0000-c000-000000000000",
      "resourceAccess": [
        {
          "id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
          "type": "Scope"
        }
      ]
    }
  ],
  "samlMetadataUrl": null
}
  3. Open the manifest in a text editor.
  4. Place your cursor inside the empty brackets after "keyCredentials": and replace the keyCredentials key-value pair with the one generated in your configured certificate. See the below example. 
    {
  "appId": "0399fdb3-c651-4360-ae33-97ed0598b5af",
  "appRoles": [],
  "availableToOtherTenants": false,
  "displayName": "zliang-test-app",
  "errorUrl": null,
  "groupMembershipClaims": null,
  "optionalClaims": null,
  "acceptMappedClaims": null,
  "homepage": "http://localhost:8000",
  "identifierUris": [
    "https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641b1b33f"
  ],
  "keyCredentials": [{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977", "customKeyIdentifier": "<custom-key-identifier>", "value": "<value>", "type": "AsymmetricX509Cert", "usage": "Verify"}],
  "knownClientApplications": [],
  "logoutUrl": null,
  "oauth2AllowImplicitFlow": false,
  "oauth2AllowUrlPathMatching": false,
  "oauth2Permissions": [
    {
      "adminConsentDescription": "Allow the application to access zliang-test-app on behalf of the signed-in user.",
      "adminConsentDisplayName": "Access zliang-test-app",
      "id": "8448c8ef-a250-481e-ba5c-d877badd3e07",
      "isEnabled": true,
      "type": "User",
      "userConsentDescription": "Allow the application to access zliang-test-app on your behalf.",
      "userConsentDisplayName": "Access zliang-test-app",
      "value": "user_impersonation"
    }
  ],
  "oauth2RequirePostResponse": false,
  "objectId": "aa082da8-0f43-4a09-a364-630f4df75a62",
  "passwordCredentials": [],
  "publicClient": false,
  "replyUrls": [
    "http://localhost:8000"
  ],
  "requiredResourceAccess": [
    {
      "resourceAppId": "00000002-0000-0000-c000-000000000000",
      "resourceAccess": [
        {
          "id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
          "type": "Scope"
        }
      ]
    }
  ],
  "samlMetadataUrl": null
}
  5. Check to make sure the edited JSON is valid.
  6. (Optional) If the keyCredentials array in your application’s manifest is not empty, copy the value, from your generated keyCredentials array, and paste it inside your existing keyCredentials array in your manifest, along with a “,” in between the copied value and the existing values in order to construct a valid JSON array. See the below example. 
    {"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977",
"customKeyIdentifier": "<custom-key-identifier>",
"value": "<value>", "type": "AsymmetricX509Cert", "usage": "Verify"}
  7. Save the file. Do not change the file name.
  8. In the Azure management portal, click Manifest > Upload Manifest.
  9. Upload the edited JSON file that you just saved.
  10. On the Splunk platform instance responsible for data collection for this add-on, click on Troubleshooting.

If the Certificate Status panel says anything other than “Uploaded and verified as valid”, wait a moment and refresh the page. If the certificate is still not reported as valid, try again with a new certificate and key file.

Generate JSON snapshot for Event Hubs

You can migrate event hubs from the Splunk Add-on for Microsoft Cloud Services to Data Manager using the Migrate from Splunk Add-on for Microsoft Cloud Services version 5.4 or higher option in Data Manager. Migration allows you to automate and accelerate the data ingestion process.

The migration process consists of two parts:

  • In the Splunk Add-on for Microsoft Cloud Services, export Event Hubs to a JSON file template. It ensures appropriate formatting of data so that it can be ingested to Data Manager.
  • In Data Manager, migrate data inputs. It includes ingesting complex Event Hub data and reviewing created data inputs. Data inputs are created from modular inputs. Migration does not affect modular inputs in the Splunk Add-on for Microsoft Cloud Services.

Prerequisites to exporting Event Hub data from the Splunk Add-on for Microsoft Cloud Services

To be able to use the export feature, you have upgraded the Splunk Add-on for Microsoft Cloud Services to version 5.4.0 or higher. Download the latest version of the add-on from the Splunk Add-on for Microsoft Cloud Services from Splunkbase.

Export Event Hub data from the Splunk Add-on for Microsoft Cloud Services to a JSON file

In the Splunk Add-on for Microsoft Cloud Services, export Event Hubs by taking the following steps:

  1. Navigate to Configuration > Export tab.
  2. In the Input status column, set the status of the Event Hubs that you want to export to Inactive. Deactivating these inputs prevent data loss.
  3. In the Health status column, check that the status of Event Hubs that you want to export is Ready.

Configure all modular inputs associated with an Event Hub under the same Inputs Data Manager (IDM). Otherwise, the Splunk Add-on for Microsoft Cloud Services does not include them.

To learn how to work with IDM, see Work with Inputs Data Manager in the Splunk Cloud Platform Admin Manual.

  1. In the Ready for export column, select one or more Event Hubs with the Ready health status to export. After selecting the Event Hubs, on the Inputs tab, in the Name column, the Migration in progress status appears next to the Event Hubs selected for export. You can’t activate inputs with this status until they are selected for export.
  2. Select the Export button to download a JSON file that contains the point-in-time state of the Event Hubs.

Ended: Configuration

Troubleshooting ↵

Troubleshoot the Splunk Add-on for Microsoft Cloud Services

For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.

Accessing logs of Azure inputs

The following table describes the logs for different inputs:

Log File Sourcetype Description Troubleshooting SPL search
splunk_ta_microsoft-cloudservices_storage_table.log mscs:storage:table:log Azure Storage Table and VIrtual Machine Metrics channel log index=_internal sourcetype=”mscs:storage:table:log” ERROR
splunk_ta_microsoft-cloudservices_storage_blob.log mscs:storage:blob:log Azure Storage Blob channel log index=_internal sourcetype=”mscs:storage:blob:log” ERROR
splunk_ta_microsoft-cloudservices_azure_resource.log mscs:azure:resource:lo Azure Resource channel log index=_internal sourcetype=”mscs:azure:resource:log” ERROR
splunk_ta_microsoft-cloudservices_azure_audit.log mscs:azure:audit:log Azure Audit Log Channel related log index=_internal sourcetype=”mscs:azure:audit:log” ERROR

Checkpoint directories

The following data sources are stored in the following directories:

Data source Directory
Azure Storage Table $SPLUNK_HOME/var/lib/splunk/modinputs/mscs_storage_table
Azure Storage Blob $SPLUNK_HOME/var/lib/splunk/modinputs/mscs_storage_blob
Azure Resource n/a
Azure Audit Log $SPLUNK_HOME/var/lib/splunk/modinputs/mscs_azure_audit

Cannot get data in

If you can’t get data in using Azure Resource and Azure Audit, follow these steps:

  1. If you can’t get data, check that you are using the correct Client ID, Client Secret, and Tenant ID. See Grant the Microsoft Entra ID Application Read Access.
  2. Use the search in the Accessing logs of Azure inputs table to check for errors.
  3. If you have no errors and cannot collect data, remove the checkpoint file and try again.

If you can’t get data in using Azure Storage Table, Azure Storage Blob and Azure Virtual Machine Metrics:

  1. If you can’t get data, check that you are using the correct Account Name and Account Secret.
  2. Use the search in the Accessing logs of Azure inputs table to check for errors.
  3. If you have no errors and cannot collect data, remove the checkpoint file and try again.

Truncated events

The default number of maximum lines for any event in the Splunk software is 256. If the number of lines in an event exceeds this limit, then the Splunk software truncates the event. If the maximum number of lines in a file exceeds the default, change the max_events setting in the props.conf file under the file’s source type stanza.

To increase the character limit beyond 10K bytes in a single line, use the truncate setting in the props.conf file to define the size of the line. See Props.conf in the Splunk Enterprise Admin manual.

Scripted inputs causing a spike in CPU percentage

If your Microsoft Cloud Services deployment experiences a CPU spike after you install and configure the Splunk Add-on for Microsoft Cloud Services, your deployment might have too many inputs enabled and too short an interval in the code.

To fix this issue, follow these steps:

  1. Navigate to your Task Manager, and verify a high amount of python.exe tasks.
  2. Increase intervals in proportion to the number of inputs you have configured in your deployment.
  3. Save your changes.

Event Hub input using the old proxy or account configuration even if the configurations are changed from UI

If the Event hub input is using the old proxy/account configuration, turn off and then turn on the Event Hub input so the new configurations are reloaded.

Azure KQL Log Analytics Input - PartialError

If you are seeing “PartialError” in logs, then a possible cause is that the Azure Log Analytics workspace API used in the input has limits on the maximum number of events and maximum size of responses returned from the API. If configured KQL Query has results that exceed the default API limits, then partial events will be returned and ingested into Splunk. Check for error messages in the input log files for more information on possible ways to optimize KQL Query.

Azure Metrics Input - ‘code’: ‘RateLimiting’ Error

If you repeatedly face the Rate Limiting error, try to resolve it by merging the multiple metrics inputs to one input. Add the comma separated metric namespaces in the namespaces field while configuring the input.

Azure Storage Blob Input - Data Ingestion stuck issue

If the Storage Blob input data ingestion gets stuck, it may occur when the API service accepts the request but fails to return a response, causing the thread to get stuck until a response is received. To address this, the SDK includes 80000 seconds (around 22 hours) read timeout so that it fails and retries the same request, resuming the data collection. As a workaround to this API issue, Input configuration includes the Read Timeout parameter which can be used to set lower read timeout value (instead of 80000 seconds) to resume data collection sooner. If Read Timeout parameter is set to a very small value, the input might start to report the read timeout error, which will cause a data ingestion issue. This is because it increases the value to the point which works best without causing the read timeout error. Please refer to Storage Blob input configuration manual for more details about the Read Timeout parameter.

Error message: azure.core.exceptions.ResourceModifiedError: The condition specified using HTTP conditional header(s) is not met

If you receive the following error in your Storage Blob input:

azure.core.exceptions.ResourceModifiedError: The condition specified using HTTP conditional header(s) is not met.

This is expected behavior from your Storage Blob because the Splunk Add-on for Microsoft Cloud Services is trying to download the blob file at the same time that blob file is being updated or modified in the Azure Portal.

After the blob file’s modification is completed in your Azure portal, the blob data is collected at the next scheduled interval.

Increased CPU usage for enabled Event Hub inputs due to constant occurrence of errors

Whenever Event Hub inputs are enabled and errors constantly occur, CPU usage will increase. The workaround is to disable the corresponding Event Hub inputs that are experiencing these errors, resolve the errors, and then re-enable the Event Hub inputs.

Authentication Put-Token failed. Retries exhausted.

This error can occur for multiple different reasons, but one common reason is that the configured Event Hub has been deleted from your Azure portal while data collection is still happening.

An error occurred while load-balancing and claiming ownership. The exception is AssertionError(). Retrying after 10.527246428161302 seconds.

This error occurs when the configured Event Hub has more than 64 partitions. The Event Hub input for the Splunk Add-on for Microsoft Cloud Services is only be able to collect data if the configured Event Hub has a maximum of 64 partitions. The fix for this error is to delete the affected Event Hub and create a new one with up to 64 partitions.

Password corruption for Azure App Account upon add-on version downgrade

There is a potential risk of password corruption for your Azure App Account when downgrading your version of the Splunk Add-on for Microsoft Cloud Services.

To resolve this issue, reconfigure the affected app accounts and then re-enable the associated inputs with that app account.

Configure the Splunk Add-on for Microsoft Cloud Services for Azure endpoints for international regions

Configure the Splunk Add-on for Microsoft Cloud Services for Azure endpoints from different international regions.

  1. On your Azure deployment, configure your desired region.
  2. On the machine that contains the Splunk Add-on for Microsoft Cloud Services, navigate to $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.
  3. Using a text editor, edit the following files, depending on each data collection API endpoint, to update the regional API endpoints to match the region that you configured in your Azure deployment:
Endpoint Affected configuration files Comments
Office 365 login endpoint URL splunk_ta_ms_o365_server_ucc_system_setting.conf
Office 365 management endpoint API URL splunk_ta_ms_o365_server_ucc_system_setting.conf and splunk_ta_ms_o365_api_settings.conf
Azure account setting schema (for ingesting Azure audit events) mscs_azure_accounts.conf Set the variable account_class_type stanza to 3
Azure storage account setting schema mscs_storage_accounts.conf Set the variable account_class_type stanza to 3
  1. Save your changes.
  2. Restart your Splunk Platform instance to apply the changes.

Ended: Troubleshooting

Reference ↵

Lookups for the Splunk Add-on for Microsoft Cloud Services

The Splunk Add-on for Microsoft Cloud Services has the following lookups that map fields from Microsoft Cloud Services systems to Common Information Model (CIM)-compliant values in the Splunk platform. The lookup files are located in $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/lookups.

Filename Description
o365_certficate_status_lookup.csv Maps a status field to a friendly description.
o365_management_api_data_lookup.csv Maps the management_api_data field to a friendly name.
o365_model_lookup.csv Maps Operation and ResultStatus to model_type,action, change_type, and object_category fields.
o365_model_operation_only_lookup.csv Maps Operation to model_type, action, change_type, and object_category fields.
o365_status_lookup.csv Maps ResultStatus to a CIM-compliant status value.
o365_troubleshooting_error_code_lookup.csv Maps o365_error to Problem, Problem Detail, Possible Reason, and Proposal values for the Troubleshooting dashboard.
o365_troubleshooting_microsoft_error_code_lookup.csv Maps microsoft_error_code to o365_error, Problem, Problem Detail, Possible Reason, and Proposal values for the Troubleshooting dashboard.
mscs_vm_cpu_mem_storage.csv Maps vm_size to cpu_cores, mem_capacity and storage_capacity.
mscs_vm_ip.csv Maps vm_id to private_ip and public_ip.
mscs_vm_power_state.csv Maps a power_state field to a common description.

Performance reference for the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services

The following tables contain reference information about performance testing of the Azure Event Hub input in the Splunk Add-on for Microsoft Cloud Services. Use this information to enhance the performance of your own Azure Event Hub data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, batch size for Event Hub file size, and hardware. Results represent reference information and do not represent performance in all environments.

Version 5.0.0 Event Hub performance characteristics

Common architecture setup

Scenario

Event type

Event Size

Ingest Rate

IDM CPU

Splunk platform environment - Victoria Search Head Cluster

  • 1 cluster master (c5.xlarge)
  • 3 search heads (m5.2xlarge)
    • 8 CPU core
    • 16 GB RAM
  • 3 indexers (i3.4xlarge)
    • 16 CPU core
    • 122 GB RAM
  • Default TA configs

1 input 16 partition eventhub

Non-JSON

1 KB

2124 kb/s

N/A

16 inputs 16 partition eventhub

Non-JSON

1 KB

7018 kb/s

N/A

1 input 32 partition eventhub

Non-JSON

1 KB

2124 kb/s

N/A

32 inputs 32 partition eventhub

Non-JSON

1 KB

4093 kb/s

N/A

Splunk platform environment - Classic Cluster (1 IDM)

  • 1 cluster master (c5.xlarge)
  • 3 search heads (c6i.xlarge)
    • 4 CPU core
    • 8 GB RAM
  • 3 indexers (i3.large)
    • 2 CPU core
    • 15.25 GB RAM
  • 1 Input Data Manager (IDM) (r6i.xlarge)
    • 4 CPU core
    • 32 GB RAM
  • Default TA configs

1 input 16 partition eventhub

Non-JSON

1 KB

1947 kb/s

30.60%

16 inputs 16 partition eventhub

Non-JSON

1 KB

1949 kb/s

13%

1 input 32 partition eventhub

Non-JSON

1 KB

1845 kb/s

30%

32 inputs 32 partition eventhub

Non-JSON

1 KB

1380 kb/s

20%

Version 4.1.5 Event Eub performance characteristics

See the following performance testing data for the previous version 4.1.5.

Event Hub input performance characteristics

Common Setup

Event type

Event size

Scenario

Ingest rate

Data lag

IDM CPU

Index CPU

Bottleneck
  • 2 add-on inputs
  • 2 Event Hub partitions
  • Default Splunk Cloud platform environment
    • 1 cluster master (c5.xlarge)
    • 1 search head (c5.2xlarge)
    • 3 indexer cluster (i3.large)
    • 1 Inputs Data Manager (IDM) (r5.xlarge)
  • default TA configs
  • 1 IDM (4 cores, 32 GB memory)

JSON

1 KB

Sweet Spot

187 GB/Day (2200 eps)

4 seconds

50-89%

86-92%

IDX CPU utilized

Non-JSON (mcas-cef)

994 bytes

Sweet Spot

133 GB/day (2000 eps)

8 seconds

65-94%

99%

IDX CPU utilized

Event Hub Scale Up performance characteristics

Environment setup

Cluster setup

Event Hub namespace

Event Hub

Add-on inputs

Splunk software configurations

Scaled up Splunk Cloud platform environment

  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.9xlarge)
  • 3 indexer cluster (i3.8xlarge)
    • 32 CPU cores
    • 244 GB memory
  • 2 Input Data Manager (c5.9xlarge)
    • 36 CPU cores
    • 72 GB memory
  • 2 Event Hub namespaces
  • 40 throughput units (TU) on each namespace
  • 80 throughput units (TU) in total (2*40)
  • 2 Event Hubs per namespace
  • 4 Event Hubs in total (2*2)
  • 20 partitions per Event Hub
  • 80 partitions in total (4*20)
  • 80 inputs in total (2*40)
  • 40 inputs per add-on
  • 2 add-on installed on Inputs Data Manager
  • default add-on configurations
  • max wait 10s
  • batch size 300
  • 1 add-on pulls from 1 namespace only
  • 1 add-on pulls from 2 Event Hubs on same namespace
  • 40 inputs on each add-on
  • 20 inputs on each Event Hub (20 partitions)

Inputs Data Manager and indexer configuration
*parallelIngestionPipelines = 8

Scale up result summary

Event type Number of inputs Event size Scenario Ingest rate Ingest Events per second Max index rate Data lag in seconds Inputs Data Manager (IDM) CPU % IDM CPU Cores % (Percentage of total IDM cores) Indexer CPU %
JSON 80 1 Kb Sweet Spot 6.106 TB/day 78K 7.55 TB/day 4s 51% 36% 30.5%
JSON 10 1 Kb Sweet Spot 1.764 TB/day 20.8K 2.19 TB/day 9s 24% 8.6% 10%
Non JSON 80 0.998 Kb Sweet Spot 5.555 TB/day 83.2K 7.22 TB/day 4s 67% 48% 47%
Non JSON 10 0.998 Kb Sweet Spot 1.595 TB/day 24K 2.07 TB/day 3s 29% 10.4% 11%

Performance reference for the Azure Storage Table input in the Splunk Add-on for Microsoft Cloud Services

The following tables show reference information about performance testing of the Azure storage input in the Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 2.0.0, when the Azure storage input was first introduced. Use this information to enhance the performance of your own Azure storage data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Testing architecture

Splunk tested the performance of the Storage input using a single-instance Splunk Enterprise 6.4.3 on an C4 High-CPU Double Extra Large instance to ensure CPU, memory, storage, and network do not introduce any bottlenecks. See the following instance specifications:

The EC2 in the testing environment is in the same area of Azure storage input, the network latency is low.

Instance type C4 High-CPU Double Extra Large
Memory 15 GB
Compute Units (ECU) 31 Units
Cores 8
Storage Type GP2(SSD)
Architecture 64-bit
Network performance High
EBS Optimized: Max Bandwidth 1000 Mbps

Test environment

Deployment Type Role EC2 Type Count
Standalone Deployment Standalone C4 High-CPU Double Extra Large 1

Testing result

The detailed test result is as follows.

  • The input number stands for the number of the inputs, where one input collects one table.
  • Each table contains 2,131,200 events.
  • Each event is 500 Bytes.
Input number Throughput (MB/s) Throughput (GB/day) Throughput (Event/s)
1 3.44 290 7045
2 5.7 480 11670
4 6.84 577 14000
8 6.12 516 12533

The maximum throughput is 6.8 MB/s with 4 data inputs, which is 100% higher than one data input. The data input can scale by 100% with max throughput 577 GB/day for single instances.

Performance reference for the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services

The following tables show reference information about performance testing of the Azure Storage Blob input in the Splunk Add-on for Microsoft Cloud Services. Use this information to enhance the performance of your own Azure Storage Blob data collection tasks.

Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Version 5.2.1 and above Blob input performance characteristics

Common architecture setup

Scenario

Blob type

File size

Ingest rate

IDM CPU

CO2 Stack - Victoria Search Head Cluster

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (im4gn.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

16324 kb/s

10.95%

5 inputs

AppendBlob

1MB

39814 kb/s

27.69%

1 input

BlockBlob

1MB

24364 kb/s

9.42%

5 inputs

BlockBlob

1MB

40810 kb/s

16.34%

CO2 Stack - Classic Cluster (1 IDM)

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (im4gn.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 1 Inputs Data Manager (IDM) (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

8717 kb/s

8.44%

5 inputs

AppendBlob

1MB

39815 kb/s

28.41%

1 input

BlockBlob

1MB

11577 kb/s

6.4%

5 inputs

BlockBlob

1MB

41857 kb/s

28%

Version 5.1.2 and above Blob input performance characteristics

Common architecture setup

Scenario

Blob type

File size

Ingest rate

IDM CPU

CO2 Stack - Victoria Search Head Cluster

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (im4gn.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

8277 kb/s

25.34%

5 inputs

AppendBlob

1MB

37963 kb/s

28.09%

1 input

BlockBlob

1MB

11532 kb/s

18.45%

5 inputs

BlockBlob

1MB

40810 kb/s

29.82%

CO2 Stack - Classic Cluster (1 IDM)

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (im4gn.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 1 Inputs Data Manager (IDM) (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

7405 kb/s

9.45%

5 inputs

AppendBlob

1MB

39815 kb/s

28.64%

1 input

BlockBlob

1MB

20030 kb/s

10.12%

5 inputs

BlockBlob

1MB

40810 kb/s

29.09%

Version 5.0.0 and above Blob input performance characteristics

Common architecture setup

Scenario

Blob type

File size

Ingest rate

IDM CPU

CO2 Stack - Victoria Search Head Cluster

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (is4gen.4xlarge)
    • 16 CPU
    • 96 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

8155.26 kb/s

N/A

5 inputs

AppendBlob

1MB

21390.76 kb/s

N/A

1 input

BlockBlob

1MB

13100.21 kb/s

N/A

5 inputs

BlockBlob

1MB

24328.95 kb/s

N/A

CO2 Stack - Classic Cluster (1 IDM)

  • 1 cluster master (c5.xlarge)
  • 3 search head (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • 3 indexer cluster (is4gen.4xlarge)
    • 16 CPU
    • 96 GiB Memory
  • 1 Inputs Data Manager (IDM) (m6i.4xlarge)
    • 16 CPU
    • 64 GiB Memory
  • Default TA configs

1 input

AppendBlob

1MB

6850.66 kb/s

55.85%

5 inputs

AppendBlob

1MB

8049.20 kb/s

33.35%

1 input

BlockBlob

1MB

3808.87 kb/s

44.55%

5 inputs

BlockBlob

1MB

3838.31 kb/s

32.00%
Continuous append scenario
Append 6 files sequentially to 1GB.
  • A single user creates an append blob.
  • Append 1MB each time in each file.
  • Pause up to 1 second after each append.
  • Append 1024 times to get to 1GB for the file.
  • In total, append 6 files to 1GB in parallel.
  • Total event size matches the original file size in Azure.

Version 4.1.4 Blob input performance characteristics

Block blob

Common architecture setup

Blob type

File size

Ingest rate

Events per minute

Data lag

IDM CPU Cores % (Percentage of total IDM cores)

Scaled up stack

  • 20 containers
  • 20 add-on inputs
  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.2xlarge)
  • 3 indexer cluster (i3.large)
    • 32 CPU cores
    • 244 GB memory
  • 1 input data manager (c5.9xlarge)
    • 32 CPU cores
    • 72 GB memory
  • 20 Splunk Add-on for Microsoft Cloud Services inputs
  • 120 seconds collection interval

block blob

10kb

48GB/day

3522

~120 seconds

1.8%

1MB

50GB/day

36

~121 seconds

1.84%

Append blob

Common architecture setup

Blob type

File size

Ingest rate

Events per minute

Data lag

IDM CPU Cores % (Percentage of total IDM cores)

Scaled up stack

  • 20 containers
  • 20 add-on inputs
  • 1 cluster master (c5.xlarge)
  • 1 search head (c5.2xlarge)
  • 3 indexer cluster (i3.large)
    • 32 CPU cores
    • 244 GB memory
  • 1 input data manager (c5.9xlarge)
    • 32 CPU cores
    • 72 GB memory
  • 20 Splunk Add-on for Microsoft Cloud Services inputs
  • 120 seconds collection interval

append blob

1 kb

33GB/day

21522

~120 seconds

3.7%

10 kb

281GB/day

20496

~121 seconds

2.67%

100 kb

2.69TB/day

18402

~129 seconds

6.5%

Additional Blob input scenarios

Scenario

Workflow

Result

Append file up to 4GB
  • A single user creates an append blob.
  • Append 100 kb each time.
  • Append 40000 times.
  • Pause up to 100 ms in each append.
  • All 40000 append blocks pulled back.
  • Total event size matches original file size in Azure.
  • Minimal to no data lag.
  • Collection interval is 12 seconds.

Append 3 files sequentially to 100 MB
  • A single user creates an append blob.
  • 100000 append each time.
  • Pause up to 100 ms after each append.
  • Append 1000 times to get to 100 MB for the file.
  • Move on to the next file and repeat.
  • In total, append 3 files to 100 MB sequentially.
  • All 3000 append blocks pulled back.
  • Total event size matches original file size in Azure.

APIs used in the Splunk Add-on for Microsoft Cloud Services

The following table lists the APIs that the Splunk Add-on for Microsoft Cloud Services uses:

Inputs name Method Link to Microsoft site
Azure Storage Table Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Azure Storage Blob List Blobs https://msdn.microsoft.com/en-us/library/azure/dd135734.aspx
Get Blob https://msdn.microsoft.com/en-us/library/azure/dd179440.aspx
Azure Audit Azure Insights - List of management events https://msdn.microsoft.com/en-us/library/azure/dn931934.aspx
Azure Resource - Virtual Machine List of virtual machines in a resource group https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/list
Get the instance view of a virtual machine https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/instanceview
Azure Resource - Public IP Address List of public IP addresses within a resource group https://msdn.microsoft.com/en-us/library/azure/mt163657.aspx
Azure Resource - Network Interface Card List of network interface cards within a resource group. https://msdn.microsoft.com/en-us/library/azure/mt163627.aspx
Azure Resource - Virtual Network List of virtual networks within a resource group https://msdn.microsoft.com/en-us/library/azure/mt163587.aspx
Azure Resource - Disk List of disks within a resource group n/a
Azure Resource - Image List of images within a resource group n/a
Azure Resource - Snapshot List of snapshots within a resource group n/a
Azure Resource - Resource Group List of resource groups within a subscription n/a
Azure Resource - Subscriptions List subscriptions within a tenant n/a
Azure Resource - Security Groups List of security groups within a resource group n/a
Azure KQL Log Analytics Query KQL Log Analytics Events n/a
Azure Virtual Machine Metrics Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx

Ended: Reference