Troubleshoot the Splunk Add-on for Okta Identity Cloud¶

For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support and resource links for add-ons in Splunk Add-ons.

The following troubleshooting topics are specific to this add-on.

Authentication Error After Upgrade to Okta TA v4.0.0¶

Issue¶

After upgrading to Okta TA version 4.0.0 or later, some accounts may fail authentication and prevent data collection. This happens because the auth_type field is now strictly validated. If the field is missing in your account configuration, the input will not run and you may encounter authentication errors.

This situation is most commonly seen in accounts that were originally created in much older versions of the TA (prior to v1.2.0), when the auth_type parameter was not automatically required. Strict enforcement of this parameter only began in version 4.0.0. As a result, older accounts without the auth_type field may be affected after the upgrade.

Solution¶

To resolve this issue:

Open each affected account configuration.
Re-enter the token and save the account.
Once saved, the auth_type field will be automatically added to the account stanza.

After completing this update, inputs should resume normal operation.

Monitor the troubleshooting dashboard¶

Starting in version 2.1.0, the add-on provides a monitoring dashboard that lets you quickly spot possible issues and metrics on ingested events:

The panels are visible on the landing page of the TA under the Monitoring Dashboard tab:

Currently, three panels are supported:

Add-on version - This can be used to easily identify the Add-on version.
Events ingested by sourcetype - This helps to get the count of events ingested for a specific sourcetype under the filtered time range.
Errors in the add-on - This helps to get to see the errors associated with the Add-on under the filtered time range.

If you change the dashboard page (Edit button) after the add-on is installed, the changes go to local folder, and you will see your version of the dashboard even if you update an add-on.

Monitor the Okta System Logs Streaming Dashboard¶

This dashboard provides a time-series graph of the Okta System Logs events ingested in Splunk based on hostname and source name. It enables users to determine when Okta System logs coming in via Log Streaming are missed and not ingested in their Splunk environment. After figuring out the time range, the user can utilize the modular input and specify the Start and End dates between which they want to collect their missing data. The time-series graph will be populated based on the “published” time of the System log event which is also the _time of the event. The user can select multiple hosts and sources, based on which the search will populate the results in the graph.

Missing Data¶

Inputs troubleshooting steps: If the input is created successfully and you do not see the data in Splunk, Make sure the index uses an input available in Splunk and the API Token is valid and up to date. You can run this query to check your input data collection logs and troubleshoot the issue: index=_internal source=splunk_ta_okta_identity_cloud_input-.log

If the proxy is enabled, make sure it is working properly, if the proxy is wrong you may not see data in Splunk. You can run the following query to find this information in Splunk:

index=_internal ProxyError

Data loss after Upgrading from the Splunk Add-on for Okta Identity Cloud v1.0.1 to later version¶

When upgrading the add-on you must disable inputs. If data ingestion is in progress, disabling inputs can lead to data loss in Splunk. Once the add-on is successfully upgraded and inputs are enabled, the data collection will continue without any issues.

Account not configured in case of OAuth2 mechanism¶

If you cannot save the account after providing all the details in the Account configuration tab, please verify that the Okta Web App created has all the necessary scopes required to collect the data. To collect system logs, okta.logs.read scope should be granted to the web app. To collect groups data, okta.groups.read scope should be granted to the web app. To collect users data, okta.users.read scope should be granted to the web app. To collect app data, okta.apps.read scope should be granted to the web app.

If you see a 400 Bad Request in the popup window, then make sure that you have added the given Redirect URI value (while configuring the Account in Splunk add-on) in the Okta Web App’s Sign-in Redirect URL section.

Data Collection stopped in the add-on¶

If the data collection uses the OAuth2 mechanism, then the reason for data collection getting stopped can be:

“Expired Refresh Token” - To solve this, search index=_internal "Error occurred while regenerating the access token" or directly search the respective input log file. If this search shows results for that particular input, then reconfigure the account in the add-on that is mentioned in the respective input log file.
“Incorrect API Scopes” - To solve this, search index=_internal "Failure caused due to incorrect Okta Web App Scopes" or directly search the respective input log file. If this search shows results for that particular input, then reconfigure the account in the add-on that is mentioned in the respective input log file.

For further troubleshooting, check the input log files.