Advanced OpenTelemetry Collector
Pre-requisites
Prerequisites #
- Proficiency in editing YAML files using
vi,vim,nano, or your preferred text editor. - Supported Environments:
- A provided Splunk Workshop Instance (preferred). Outbound access to port
2222is required forsshaccess. - Apple Mac (Apple Silicon). Installation of
jqis required - https://jqlang.org/download/
- A provided Splunk Workshop Instance (preferred). Outbound access to port
1
Initial Setup
In your environment create a new directory and change into it:
bash
mkdir advanced-otel-workshop && \
cd advanced-otel-workshopWe will refer to this directory as [WORKSHOP] for the remainder of the workshop.
Remove any existing OpenTelemetry Collectors
If you have completed the Splunk IM workshop, please ensure you have deleted the collector running in Kubernetes before continuing. This can be done by running the following command:
bash
helm delete splunk-otel-collectorThe EC2 instance in that case may also run some services that can interfere with this workshop , so run the following command to make sure they are stopped if present:
bash
kubectl delete ~/workshop/apm/deployment.yaml2
Download workshop binaries
Change into your [WORKSHOP] directory and download the OpenTelemetry Collector, Load Generator binaries and setup script:
bash
curl -L https://github.com/signalfx/splunk-otel-collector/releases/download/v0.136.0/otelcol_linux_amd64 -o otelcol && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/loadgen/build/loadgen-linux-amd64 -o loadgen && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/setup-workshop.sh -o setup-workshop.sh && \
chmod +x setup-workshop.shbash
curl -L https://github.com/signalfx/splunk-otel-collector/releases/download/v0.136.0/otelcol_darwin_arm64 -o otelcol && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/loadgen/build/loadgen-darwin-arm64 -o loadgen && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/setup-workshop.sh -o setup-workshop.sh && \
chmod +x setup-workshop.sh3
Run the setup
Run the setup-workshop.sh script which will configure the correct permissions and also create the initial configurations for the Agent and the Gateway:
bash
curl -L https://github.com/signalfx/splunk-otel-collector/releases/download/v0.136.0/otelcol_linux_amd64 -o otelcol && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/loadgen/build/loadgen-linux-amd64 -o loadgen && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/setup-workshop.sh -o setup-workshop.sh && \
chmod +x setup-workshop.shbash
curl -L https://github.com/signalfx/splunk-otel-collector/releases/download/v0.136.0/otelcol_darwin_arm64 -o otelcol && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/loadgen/build/loadgen-darwin-arm64 -o loadgen && \
curl -L https://github.com/splunk/observability-workshop/raw/refs/heads/main/workshop/ninja/advanced-otel/setup-workshop.sh -o setup-workshop.sh && \
chmod +x setup-workshop.shbash
./setup-workshop.shtext
███████╗██████╗ ██╗ ██╗ ██╗███╗ ██╗██╗ ██╗ ██╗
██╔════╝██╔══██╗██║ ██║ ██║████╗ ██║██║ ██╔╝ ╚██╗
███████╗██████╔╝██║ ██║ ██║██╔██╗ ██║█████╔╝ ╚██╗
╚════██║██╔═══╝ ██║ ██║ ██║██║╚██╗██║██╔═██╗ ██╔╝
███████║██║ ███████╗╚██████╔╝██║ ╚████║██║ ██╗ ██╔╝
╚══════╝╚═╝ ╚══════╝ ╚═════╝ ╚═╝ ╚═══╝╚═╝ ╚═╝ ╚═╝
Welcome to the Splunk Advanced OpenTelemetry Workshop!
======================================================
macOS detected. Removing quarantine attributes...
otelcol version v0.126.0
Usage: loadgen [OPTIONS]
Options:
-base Send base traces (enabled by default)
-health Send health traces
-security Send security traces
-logs Enable logging of random quotes to quotes.log
-json Output logs in JSON format (only applicable with -logs)
-count Number of traces or logs to send (default: infinite)
-h, --help Display this help message
Example:
loadgen -health -security -count 10 Send 10 health and security traces
loadgen -logs -json -count 5 Write 5 random quotes in JSON format to quotes.log
Creating workshop directories...
✓ Created subdirectories:
├── 1-agent-gateway
├── 2-building-resilience
├── 3-dropping-spans
├── 4-sensitive-data
├── 5-transform-data
├── 6-routing-data
└── 7-sum-count
Creating configuration files for 1-agent-gateway...
Creating OpenTelemetry Collector agent configuration file: 1-agent-gateway/agent.yaml
✓ Configuration file created successfully: 1-agent-gateway/agent.yaml
✓ File size: 4355 bytes
Creating OpenTelemetry Collector gateway configuration file: 1-agent-gateway/gateway.yaml
✓ Configuration file created successfully: 1-agent-gateway/gateway.yaml
✓ File size: 3376 bytes
✓ Completed configuration files for 1-agent-gateway
Creating configuration files for 2-building-resilience...
Creating OpenTelemetry Collector agent configuration file: 2-building-resilience/agent.yaml
✓ Configuration file created successfully: 2-building-resilience/agent.yaml
✓ File size: 4355 bytes
Creating OpenTelemetry Collector gateway configuration file: 2-building-resilience/gateway.yaml
✓ Configuration file created successfully: 2-building-resilience/gateway.yaml
✓ File size: 3376 bytes
✓ Completed configuration files for 2-building-resilience
Workshop environment setup complete!
Configuration files created in the following directories:
1-agent-gateway/
├── agent.yaml
└── gateway.yaml
2-building-resilience/
├── agent.yaml
└── gateway.yamltext
[WORKSHOP]
├── 1-agent-gateway
├── 2-building-resilience
├── 3-dropping-spans
├── 4-sensitive-data
├── 5-transform-data
├── 6-routing-data
├── 7-sum-count
├── loadgen
├── otelcol
└── setup-workshop.sh