SC4S “Bring Your Own Environment”¶
- FOREWORD: The BYOE SC4S deliverable should be considered as a self/community supported option for SC4S deployment, and should be considered only by those with specific needs based on advanced understanding of syslog-ng architectures and linux/syslog-ng system administration and the ability to develop and automate testing in non-production environments. The container deliverable is the most often correct deliverable of SC4S for almost all enterprises. If you are simply trying to “get syslog working”, the turnkey, container approach described in the other runtime documents will be the fastest route to success.
The “Bring Your Own Environment” instructions that follow allow expert administrators to utilize the SC4S syslog-ng config files directly on the host OS running on a hardware server or virtual machine. Administrators must provide an appropriate host OS (RHEL 8 used in this document) as well as an up-to-date syslog-ng installation either built from source (not documented here) or installed from community-built RPMs. Modification of the base configuration will be required for most customer environments due to enterprise infrastructure variations. Once installed preparing an upgrade requires evaluation of the current environment compared to this reference then developing and testing a installation specific install plan. This activity is the responsibility of the administrator.
NOTE: Installing or modifying system configurations can have unexpected consequences, and advanced linux system administration and syslog-ng configuration experience is assumed when using the BYOE version of SC4S.
NOTE: Do not depend on the distribution-supplied version of syslog-ng, as it will likely be far too old. Read this explanation for the reason why syslog-ng builds are so dated in almost all RHEL/Debian distributions.
BYOE Installation Instructions¶
These installation instructions assume a recent RHEL or CentOS-based release. Minor adjustments may have to be made for
Debian/Ubuntu. In addition, almost all pre-compiled binaries for syslog-ng assume installation in
/etc/syslog-ng; these instructions
will reflect that.
The following installation instructions are summarized from a blog maintained by a developer at One Identity (formerly Balabit), who is the owner of the syslog-ng Open Source project. It is always adivisable to review the blog for the latest changes to the repo(s), as changes here are quite dynamic.
Install CentOS or RHEL 8.0
Enable EPEL (Centos 8)
dnf install 'dnf-command(copr)' -y dnf install epel-release -y dnf copr enable czanik/syslog-ng334 -y dnf install syslog-ng syslog-ng-python syslog-ng-http syslog-ng-afsnmp net-snmp python3-pip gcc python3-devel -y
- Disable the distro-supplied syslog-ng unit file, as the syslog-ng process configured here will run as the
sc4sservice. rsyslog will continue to be the system logger, but should be left enabled only if it is configured to not listen on the same ports as sc4s. sc4s BYOE can be configured to provide local logging as well if desired.
sudo systemctl stop syslog-ng sudo systemctl disable syslog-ng
Download the latest bare_metal.tar from releases on github and untar the package in
/etc/syslog-ngusing the command example below.
wgetprocess below will unpack a tarball with the sc4s version of the syslog-ng config files in the standard
/etc/syslog-nglocation, and will overwrite existing content. Ensure that any previous configurations of syslog-ng are saved if needed prior to executing the download step.
NOTE: At the time of writing, the latest major release is
v1.33. The latest release is typically listed first on the page above, unless there is an
-rcrelease that is newer (which will be clearly indicated). For production use, select the latest that does not have an
sudo wget -c https://github.com/splunk/splunk-connect-for-syslog/releases/download/<latest release>/baremetal.tar -O - | sudo tar -x -C /etc/syslog-ng
- Install python requirements
sudo pip3 install -r /etc/syslog-ng/requirements.txt
- (Optional, for monitoring): Install
gossand confirm that the version is v0.3.16 or newer.
/usr/local/binby default, so ensure that 1)
entrypoint.shis modified to include
/usr/local/binin the full path, or 2) move the
curl -L https://github.com/aelsabbahy/goss/releases/latest/download/goss-linux-amd64 -o /usr/local/bin/goss chmod +rx /usr/local/bin/goss curl -L https://github.com/aelsabbahy/goss/releases/latest/download/dgoss -o /usr/local/bin/dgoss # Alternatively, using the latest # curl -L https://raw.githubusercontent.com/aelsabbahy/goss/latest/extras/dgoss/dgoss -o /usr/local/bin/dgoss chmod +rx /usr/local/bin/dgoss
There are two main options for running SC4S via systemd, the choice of which largely depends on administrator preference and orchestration methodology: 1) the
entrypoint.shscript (identical to that used in the container) can be run directly via systemd, or 2) the script can be altered to preconfigure SC4S (after which only the syslog-ng and snmp executables are run via systemd). These are by no means the only ways to run BYOE – as the name implies, the method you choose will be based on your custom needs.
To run the
entrypoint.shscript directly in systemd, create the sc4s unit file
/lib/systemd/system/sc4s.serviceand add the following content:
[Unit] Description=SC4S Syslog Daemon Documentation=https://splunk-connect-for-syslog.readthedocs.io/en/latest/ Wants=network.target network-online.target After=network.target network-online.target [Service] Type=simple ExecStart=/etc/syslog-ng/entrypoint.sh ExecReload=/bin/kill -HUP $MAINPID EnvironmentFile=/etc/syslog-ng/env_file StandardOutput=journal StandardError=journal Restart=on-abnormal [Install] WantedBy=multi-user.target
- To run
entrypoint.shas a “preconfigure” script, modify the script by commenting out or removing the stanzas following the
OPTIONAL for BYOEcomments in the script. This will prevent syslog-ng (and optionally snmptrapd) from being launched by the script. Then create the sc4s unit file
/lib/systemd/system/syslog-ng.serviceand add the following content:
[Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) After=network.target [Service] Type=notify ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid ExecReload=/bin/kill -HUP $MAINPID EnvironmentFile=-/etc/default/syslog-ng EnvironmentFile=-/etc/sysconfig/syslog-ng StandardOutput=journal StandardError=journal Restart=on-failure [Install] WantedBy=multi-user.target
- Create the file
/etc/syslog-ng/env_fileand add the following environment variables (adjusting the URL/TOKEN appropriately):
# The following "path" variables can differ from the container defaults specified in the entrypoint.sh script. # These are *optional* for most BYOE installations, which do noot differ from the install location used. # in the container version of SC4S. Failure to properly set these will cause startup failure. #SC4S_ETC=/etc/syslog-ng #SC4S_VAR=/etc/syslog-ng/var #SC4S_BIN=/bin #SC4S_SBIN=/usr/sbin #SC4S_TLS=/etc/syslog-ng/tls # General Options SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://splunk.smg.aws:8088 SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=a778f63a-5dff-4e3c-a72c-a03183659e94 # Uncomment the following line if using untrusted (self-signed) SSL certificates # SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY=no
- Reload systemctl and restart syslog-ng (example here is shown for systemd option (1) above)
sudo systemctl daemon-reload sudo systemctl enable sc4s sudo systemctl start sc4s
Configure SC4S Listening Ports¶
Most enterprises use UDP/TCP port 514 as the default as their main listening port for syslog “soup” traffic, and TCP port 6514 for TLS.
The standard SC4S configuration reflect these defaults. These defaults can be changed by adding the following
additional environment variables with appropriate values to the
SC4S_LISTEN_DEFAULT_TCP_PORT=514 SC4S_LISTEN_DEFAULT_UDP_PORT=514 SC4S_LISTEN_DEFAULT_RFC6587_PORT=601 SC4S_LISTEN_DEFAULT_RFC5426_PORT=601 SC4S_LISTEN_DEFAULT_RFC5425_PORT=5425 SC4S_LISTEN_DEFAULT_TLS_PORT=6514
Dedicated (Unique) Listening Ports¶
For certain source technologies, categorization by message content is impossible due to the lack of a unique “fingerprint” in the data. In other cases, a unique listening port is required for certain devices due to network requirements in the enterprise. For collection of such sources we provide a means of dedicating a unique listening port to a specific source.
Refer to the “Sources” documentation to identify the specific environment variables used to enable unique listening ports for the technology in use.