Configure enhanced log formatting on the Apache Web Server using httpd.conf¶

Configure log formatting on the Apache Web Server using httpd.conf. This lets the Splunk Add-on for Apache Web Server monitor your deployment’s log files.

To make sure you have all the required fields present in apache access_log, Splunk best practice is to use an enhanced custom log format in the apache httpd.conf file. For more information, see Configure monitor inputs for the Splunk Add-on for Apache Web Server.

The Splunk Add-on for Apache Web Server now supports the default out-of-the-box log format of Apache Web Server; the add-on extracts all possible CIM fields where present and makes them searchable. However, this log format is missing many critical CIM fields (for example, user, hostname, and so on), so note that the add-on does not apply CIM Data Model tags to those events. Splunk best practice is still to use an enhanced custom log format which requires modifications to httpd.conf, and use the apache:access:kv or apache:access:json sourcetypes. To use the default out-of-the-box events, ignore the following procedure and proceed with Configure monitor inputs for the Splunk Add-on for Apache Web Server.

Open the httpd.conf in the Apache Web Server installation folder $APACHE_HOME/etc/apache/conf/httpd.conf

Note

The default location of httpd.conf might be different on different platforms.

Look for the statement <IfModule log_config_module> and replace the whole block with the following stanza:

<IfModule log_config_module>
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

 <IfModule logio_module>
    # You need to enable mod_logio.c to use %I and %O
    LogFormat "time=%{%s}t.%{usec_frac}t, bytes_in=%I, bytes_out=%O, cookie=\"%{Cookie}i\", server=%v, dest_port=%p, http_content_type=\"%{Content-type}i\", http_method=\"%m\", http_referrer=\"%{Referer}i\", http_user_agent=\"%{User-agent}i\", ident=\"%l\", response_time_microseconds=%D, client=%h, status=%>s, uri_path=\"%U\", uri_query=\"%q\", user=\"%u\"" splunk_kv

    #LogFormat "{\"time\":\"%{%s}t.%{usec_frac}t\", \"bytes_in\":\"%I\", \"bytes_out\":\"%O\", \"cookie\":\"%{Cookie}i\", \"server\":\"%v\", \"dest_port\":\"%p\", \"http_content_type\":\"%{Content-type}i\", \"http_method\":\"%m\", \"http_referrer\":\"%{Referer}i\", \"http_user_agent\":\"%{User-agent}i\", \"ident\":\"%l\", \"response_time_microseconds\":\"%D\", \"client\":\"%h\", \"status\":\"%>s\", \"uri_path\":\"%U\", \"uri_query\":\"%q\", \"user\":\"%u\"}" splunk_json

  #LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

</IfModule>
    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    # CustomLog "logs/access_log" common
    #
    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #
    CustomLog "logs/access_log" splunk_kv
    #CustomLog "logs/access_log" splunk_json
    #CustomLog "logs/access_log" combined
</IfModule>

Choose either the splunk_kv or splunk_json format for access logs. You can enable only one format at a time. The information provided by either splunk_kv and splunk_json is the same. The difference is only in formatting. By default, splunk_kv is enabled and the splunk_json is disabled. To enable the splunk_json format:

a. Comment out the splunk_kv definition and log file directives.

b. Uncomment the splunk_json definition and log file directives.

Note

The KV pair formatting is simpler when compared to JSON formatting. The best practice is to include new custom fields, if required, in KV format. If applied in JSON format, then it must comply with strict JSON formatting.

Validate the syntax of the conf file after you make your changes:

sudo apache2ctl configtest

or

sudo httpd -t

If the output says syntax ok, proceed.
Restart Apache Web Server. If the log format in Apache Web Server is configured correctly, you receive log files that look like this:

For splunk_kv

time=###TIME###.000000, bytes_in=###BYTES_IN###, bytes_out=###BYTES_OUT###, cookie="###COOKIE###", server=C6852495051.domain, dest_port=###DEST_PORT###, http_content_type="###HTTP_CONTENT_TYPE###", http_method="###HTTP_METHOD###", http_referrer="###HTTP_REFERRER###", http_user_agent="###HTTP_USER_AGENT###", ident="1", response_time_microseconds=###RESPONSE_TIME_MICROSECONDS###, client=###CLIENT###, status=###STATUS###, uri_path="###URI_PATH###", uri_query="###URI_QUERY###", user="xyz123"

For splunk_json

{"time":"###TIME###.000000", "bytes_in":"###BYTES_IN###",
    "bytes_out":"###BYTES_OUT###", "cookie":"###COOKIE###", "server":"C6852495051.domain", "dest_port":"###DEST_PORT###", "http_content_type":"###HTTP_CONTENT_TYPE###", "http_method":"###HTTP_METHOD###", "http_referrer":"###HTTP_REFERRER###", "http_user_agent":"###HTTP_USER_AGENT###", "ident":"1", "response_time_microseconds":"###RESPONSE_TIME_MICROSECONDS###", "client":"###CLIENT###", "status":"###STATUS###", "uri_path":"###URI_PATH###", "uri_query":"###URI_QUERY###", "user":"xyz123"}

Note

The new fields are auto extracted without making any change in the add-on’s field extraction.