Implement a Container Runtime and SC4S¶
- Linux host with Docker (CE 19.x or greater) or Podman enabled, depending on runtime choice (below).
- A network load balancer (NLB) configured for round-robin. Note: Special consideration may be required when more advanced products are used. The optimal configuration of the load balancer will round-robin each http POST request (not each connection).
- The host linux OS receive buffer size should be tuned to match the sc4s default to avoid dropping events (packets) at the network level.
The default receive buffer for sc4s is set to 16 MB for UDP traffic, which should be OK for most environments. To set the host OS kernel to
match this, edit
/etc/sysctl.confusing the following whole-byte values corresponding to 16 MB:
net.core.rmem_default = 17039360 net.core.rmem_max = 17039360
and apply to the kernel:
- Ensure the kernel is not dropping packets by periodically monitoring the buffer with the command
netstat -su | grep "receive errors".
- NOTE: Failure to account for high-volume traffic (especially UDP) by tuning the kernel will result in message loss, which can be very unpredictable and difficult to detect. See this helpful discussion in the syslog-ng Professional Edition documentation regarding tuning syslog-ng in particular (via the SC4S_SOURCE_*_SO_RCVBUFF environment variable in sc4s) as well as overall host kernel tuning. The default values for receive kernel buffers in most distros is 2 MB, which has proven inadequate for many.
In many distributions (e.g. CentOS provisioned in AWS), IPV4 forwarding is not enabled by default. This needs to be enabled for container networking to function properly. The following is an example to check and set this up; as usual this needs to be vetted with your enterprise security policy:
sudo sysctl net.ipv4.ip_forward
sudo sysctl net.ipv4.ip_forward=1
To ensure the change survives a reboot:
- sysctl settings are defined through files in
- To override only specific settings, you can either add a file with a lexically later name in
/etc/sysctl.d/and put following setting there:
- or find this specific setting in one of existing configuration files (mentioned above) and set value to
Select a Container Runtime and SC4S Configuration¶
|Container Runtime and Orchestration||Operating Systems|
|MicroK8s||Ubuntu with Microk8s|
|Podman 1.7 & 1.9 + systemd||RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS|
|Docker CE 19 (and greater) + systemd||RHEL or CentOS >7.7 (best option), Debian or Ubuntu 18.04LTS|
|Docker Desktop + Compose||MacOS|
|Docker Desktop + Compose||RHEL or CentOS 8.1 & 8.2 (best option)|
|Bring your own Environment||RHEL or CentOS 8.1 & 8.2 (best option)|
|Offline Container Installation||RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS|
|Ansible+Docker Swarm||RHEL 8.x or CentOS 8.x (best option), Debian or Ubuntu 18.04LTS|
|Ansible+Podman||RHEL 7.x/8.x or CentOS 7.x/8.x (best option), Debian or Ubuntu 20.10LTS(and higher)|
|Ansible+Docker||RHEL 7.x/8.x or CentOS 7.x/8.x (best option), Debian or Ubuntu 18.04LTS(and higher)|
Docker and Podman basic configurations¶
- To run properly sc4s you need to create directories:
/opt/sc4s/localwill be used as a mount point for local overrides and configurations. The empty
localdirectory created above will populate with defaults and examples at the first invocation of SC4S for local configurations and context overrides. Do not change the directory structure of the files that are laid down; change (or add) only individual files if desired. SC4S depends on the directory layout to read the local configurations properly. See the notes below for which files will be preserved on restarts. In the
local/config/directory there are four subdirectories that allow you to provide support for device types that are not provided out of the box in SC4S. To get you started, there is an example log path template (
lp-example.conf.tmpl) and a filter (
example.conf) in the
filterssubdirectories, respectively. These should not be used directly, but copied as templates for your own log path development. They will get overwritten at each SC4S start.
local/contextdirectory, if you change the “non-example” version of a file (e.g.
splunk_metadata.csv) the changes will be preserved on a restart.
/opt/sc4s/archivewill be used as a mount point for local storage of syslog events (if the optional mount is uncommented above). The events will be written in the syslog-ng EWMM format. See the “configuration” document for details on the directory structure the archive uses.
/opt/sc4s/tlswill be used as a mount point for custom TLS certificates (if the optional mount is uncommented above).
IMPORTANT: When creating the directories above, ensure the directories created match the volume mounts specified in the sc4s.service unit file. Failure to do this will cause SC4S to abort at startup.
Dedicated (Unique) Listening Ports¶
For certain source technologies, categorization by message content is impossible due to the lack of a unique “fingerprint” in the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. For collection of such sources, we provide a means of dedicating a unique listening port to a specific source.
Follow this step to configure unique ports for one or more sources:
- Modify the
/opt/sc4s/env_filefile to include the port-specific environment variable(s). Refer to the “Sources” documentation to identify the specific environment variables that are mapped to each data source vendor/technology.
Modify index destinations for Splunk¶
Log paths are preconfigured to utilize a convention of index destinations that are suitable for most customers.
- If changes need to be made to index destinations, navigate to the
/opt/sc4s/local/contextdirectory to start.
splunk_metadata.csvto review or change the index configuration as required for the data sources utilized in your environment. The key (1st column) in this file uses the syntax
vendor_product. Simply replace the index value (the 3rd column) in the desired row with the index appropriate for your Splunk installation. The “Sources” document details the specific
vendor_productkeys (rows) in this table that pertain to the individual data source filters that are included with SC4S.
- Other Splunk metadata (e.g. source and sourcetype) can be overridden via this file as well. This is an advanced topic, and further information is covered in the “Log Path overrides” section of the Configuration document.
Configure source filtering by source IP or host name¶
Legacy sources and non-standard-compliant sources require configuration by source IP or hostname as included in the event. The following steps apply to support such sources. To identify sources that require this step, refer to the “sources” section of this documentation. See documentation for your vendor/product to determine if specific configuration is required
Configure compliance index/metadata overrides¶
In some cases, devices that have been properly sourcetyped need to be further categorized by compliance, geography, or other criterion.
The two files
compliance_meta_by_source.csv can be used for this purpose. These operate similarly to
the files above, where the
conf file specifies a filter to uniquely identify the messages that should be overridden, and the
lists one or more metadata items that can be overridden based on the filter name. This is an advanced topic, and further information is
covered in the “Override index or metadata based on host, ip, or subnet” section of the Configuration document.