Configure CollectD to send data to the Splunk Add-on for Linux¶

The Splunk Add-on for Linux depends on data sent from CollectD to the Splunk HTTP Event Collector (HEC) or a TCP input. CollectD is a daemon which includes a rich set of plugins for gathering system and application performance metrics. The following picture illustrates how CollectD gathers data from the Linux host (as CollectD client) and sends data to Splunk (as CollectD server).

Data is gathered from a Linux host and sent through a CollectD server and then to Splunk.

You can customize your CollectD deployment based on your needs and environment. You can configure the CollectD client and CollectD server on the same Linux host, or you can configure several CollectD clients to send data to a single CollectD server.

Download and install CollectD¶

Prerequisite

Review the hardware and software requirements for the Splunk Add-on for Linux. See Hardware and software requirements.

Go to https://collectd.org/download.html and download CollectD.
Follow the instructions from https://collectd.org/wiki/index.php/First_steps to install CollectD.

Configure CollectD for Linux¶

You must configure CollectD to collect data and send the data to Splunk. The default location for collectd.conf is /etc/collectd.conf or /etc/collectd/collectd.conf.

See the CollectD manpage to learn more about collectd.conf.

Configure CollectD client to collect data from Linux¶

Data collected	Plugin in CollectD	Configuration suggestion
CPU metrics	Plugin cpu Enable the plugin by removing the `#` in front of the plugin line. Example: change `#LoadPlugin cpu` to `LoadPlugin cpu`.	`<Plugin cpu>` `# ReportByCpu true` `# ReportByState true` `ValuesPercentage true` `</Plugin>`
Memory metrics	Plugin memory	`<Plugin memory>` `ValuesAbsolute true` `ValuesPercentage true` `</Plugin>`
Swap metrics	Plugin swap	`<Plugin swap>` `ReportByDevice true` `# ReportBytes true` `# ValuesAbsolute true` `ValuesPercentage true` `</Plugin>`
VMEM metrics	Plugin vmem	`<Plugin vmem>` `Verbose false` `</Plugin>`
Mountpoint usage / FS usage	Plugin df	`<Plugin df>` `# Device "/dev/hda1"` `# Device "192.168.0.2:/mnt/nfs"` `# MountPoint "/home"` `# FSType "ext3"` `ReportByDevice true` `# ReportInodes false` `# ValuesAbsolute true` `ValuesPercentage true` `</Plugin>`
Network interface traffic	Plugin interface	None. Use the default configuration.
Disk utilization	Plugin disk	None. Use the default configuration.
System load	Plugin load	`<Plugin load>` `ReportRelative true` `</Plugin>`
Process information	Plugin processes	`<Plugin processes>` `ProcessMatch "all" "(.*)"` `</Plugin>`
Network protocols information	Plugin protocols	None. Use the default configuration.
IRQ metrics	Plugin irq	None. Use the default configuration.
TCP connections information	Plugin tcpconns	None. Use the default configuration.
Thermal information	Plugin thermal	None. Use the default configuration.
System uptime statistics	Plugin uptime	None. Use the default configuration.

Configure the CollectD client to send data to the CollectD server¶

Note

If you configure the CollectD client and the CollectD server on the same machine, you can skip this step.

See Plugin network in the CollectD manpage at https://collectd.org/documentation/manpages/collectd.conf.html for information on how to configure the Plugin network. See Networking introduction on the CollectD Wiki at https://collectd.org/documentation/ for a detailed walkthrough.

Configure the CollectD server to send data to Splunk¶

Plugin write_http and Plugin write_graphite submit values to Splunk. Plugin write_http sends data via HTTP and encoding metrics with JSON, and Plugin write_graphite writes data to Graphite via TCP.

Configure plugin write_http¶

If you want to send Linux performance metrics data to Splunk in JSON format via HTTP, configure URL, Header and Format as follows:

Field name	Description	Syntax	Example
URL	URL to which the values are submitted to. The values for `IP`, `Port`, and `Token Value` must be the same as the values you define for the HEC inputs. See Configure HEC inputs for the Splunk Add-on for Linux.	URL “https://Splunk Server IP:Port Number/services/collector/raw?channel=Token Value“	URL “https://10.66.104.127:8088/services/collector/raw?channel=693E90D4-91A5-49A3-99B1-CFE8828A0711”
Header	A HTTP header to add to the request.	Header “Authorization: Splunk Token Value“	Header “Authorization: Splunk 693E90D4-91A5-49A3-99B1-CFE8828A0711”
Format	The data format.	Format “JSON”	Format “JSON”

Example

LoadPlugin write_http
<Plugin write_http>
   <Node "node-http-1">
      URL "https://10.66.104.127:8088/services/collector/raw?channel=693E90D4-91A5-49A3-99B1-CFE8828A0711"
      Header "Authorization: Splunk 693E90D4-91A5-49A3-99B1-CFE8828A0711"
      Format "JSON"
      Metrics true
      StoreRates true
    </Node>
</Plugin>

Configure plugin write_graphite¶

If you want to send Linux performance metrics data to Splunk in Graphite format, configure plugin write_graphite as follows:

Set AlwaysAppendDS to true.
Set SeparateInstances to false.
Make sure the values for Host and Port are the same as the values you define for the TCP inputs. See Configure TCP inputs for the Splunk Add-on for Linux.

Note

If dots (.) are used in the metric name (including prefix, EscapeCharacter, hostname, and postfix), Splunk cannot recognize the key-value pair in the data.

Example

LoadPlugin write_graphite
<Plugin write_graphite>
   <Node "node-graphite-1">
      Host "10.66.108.127"
      Port "2104"
      Protocol "tcp"
      EscapeCharacter "_"
      AlwaysAppendDS true
      SeparateInstances false
    </Node>
</Plugin>