Hands-On OpenTelemetry, Docker, and K8s

Deploy Application to K8s

15 minutes

Update the Dockerfile

With Kubernetes, environment variables are typically managed in the .yaml manifest files rather than baking them into the Docker image. So let’s remove the following two environment variables from the Dockerfile:

bash 
vi /home/splunk/workshop/docker-k8s-otel/helloworld/Dockerfile

Then remove the following two environment variables:

dockerfile 
ENV OTEL_SERVICE_NAME=helloworld
ENV OTEL_RESOURCE_ATTRIBUTES='deployment.environment=otel-$INSTANCE'

To save your changes in vi, press the esc key to enter command mode, then type :wq! followed by pressing the enter/return key.

Build a new Docker Image

Let’s build a new Docker image that excludes the environment variables:

bash 
cd /home/splunk/workshop/docker-k8s-otel/helloworld 

docker build -t helloworld:1.2 .

Note: we’ve used a different version (1.2) to distinguish the image from our earlier version. To clean up the older versions, run the following command to get the container id:

bash 
docker ps -a | grep helloworld

Then run the following command to delete the container:

bash 
docker rm <old container id> --force

Now we can get the container image id:

bash 
docker images | grep 1.1

Finally, we can run the following command to delete the old image:

bash 
docker image rm <old image id>

Import the Docker Image to Local Container Repository

Normally we’d push our Docker image to a repository such as Docker Hub. But for this workshop, we’ll push the Docker image to the local container repository running on our EC2 instance at localhost:9999

bash 
# Update the image tag
docker tag helloworld:1.2 localhost:9999/helloworld:1.2

# Import the image into the local repository
docker push localhost:9999/helloworld:1.2

Deploy the .NET Application

Hint: To enter edit mode in vi, press the ‘i’ key. To save changes, press the esc key to enter command mode, then type :wq! followed by pressing the enter/return key.

To deploy our .NET application to K8s, let’s create a file named deployment.yaml in /home/splunk:

bash 
vi /home/splunk/deployment.yaml

And paste in the following:

yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld
spec:
  selector:
    matchLabels:
      app: helloworld
  replicas: 1
  template:
    metadata:
      labels:
        app: helloworld
    spec:
      containers:
        - name: helloworld
          image: localhost:9999/helloworld:1.2
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
          env:
            - name: PORT
              value: "8080"

What is a Deployment in Kubernetes?

The deployment.yaml file is a kubernetes config file that is used to define a deployment resource. This file is the cornerstone of managing applications in Kubernetes! The deployment config defines the deployment’s desired state and Kubernetes then ensures the actual state matches it. This allows application pods to self-heal and also allows for easy updates or roll backs to applications.

Then, create a second file in the same directory named service.yaml:

bash 
vi /home/splunk/service.yaml

And paste in the following:

yaml 
apiVersion: v1
kind: Service
metadata:
  name: helloworld
  labels:
    app: helloworld
spec:
  type: ClusterIP
  selector:
    app: helloworld
  ports:
    - port: 8080
      protocol: TCP

What is a Service in Kubernetes?

A Service in Kubernetes is an abstraction layer, working like a middleman, giving you a fixed IP address or DNS name to access your Pods, which stays the same, even if Pods are added, removed, or replaced over time.

Then, create a third file in the same directory named ingress.yaml:

bash 
vi /home/splunk/ingress.yaml

And paste in the following:

yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-ingress
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  ingressClassName: traefik
  rules:
    - host: helloworld.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: helloworld
                port:
                  number: 8080

What is an Ingress in Kubernetes?

An Ingress in Kubernetes is a Kubernetes API object that manages external access to services within a cluster, typically HTTP and HTTPS traffic. It acts as a set of rules for routing incoming connections to the correct internal services and pods, handling functions like load balancing, SSL/TLS termination, and name-based virtual hosting

We can then use these manifest files to deploy our application:

bash 
cd /home/splunk

# create the deployment
kubectl apply -f deployment.yaml

# create the service
kubectl apply -f service.yaml

# create the ingress
kubectl apply -f ingress.yaml

Test the Application

Use the following command to access the application:

bash 
curl http://helloworld.localhost/hello/Kubernetes

Configure OpenTelemetry

The .NET OpenTelemetry instrumentation was already baked into the Docker image. But we need to set a few environment variables to tell it where to send the data.

Add the following to deployment.yaml file you created earlier:

IMPORTANT replace $INSTANCE in the YAML below with your instance name, which can be determined by running echo $INSTANCE.

yaml 
          env:
            - name: PORT
              value: "8080"
            - name: NODE_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://$(NODE_IP):4318"
            - name: OTEL_SERVICE_NAME
              value: "helloworld"
            - name: OTEL_RESOURCE_ATTRIBUTES 
              value: "deployment.environment=otel-$INSTANCE"

The complete deployment.yaml file should be as follows (with your instance name rather than $INSTANCE):

yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld
spec:
  selector:
    matchLabels:
      app: helloworld
  replicas: 1
  template:
    metadata:
      labels:
        app: helloworld
    spec:
      containers:
        - name: helloworld
          image: localhost:9999/helloworld:1.2
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
          env:
            - name: PORT
              value: "8080"
            - name: NODE_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://$(NODE_IP):4318"
            - name: OTEL_SERVICE_NAME
              value: "helloworld"
            - name: OTEL_RESOURCE_ATTRIBUTES 
              value: "deployment.environment=otel-$INSTANCE"

Apply the changes with:

bash 
kubectl apply -f deployment.yaml

Then use the following command to generate some traffic:

bash 
curl http://helloworld.localhost/hello/Kubernetes

After a minute or so, you should see traces flowing in the o11y cloud. But, if you want to see your trace sooner, we have …

A Challenge For You

If you are a developer and just want to quickly grab the trace id or see console feedback, what environment variable could you add to the deployment.yaml file?

Click here to see the answer

If you recall in our challenge from Section 4, Instrument a .NET Application with OpenTelemetry, we showed you a trick to write traces to the console using the OTEL_TRACES_EXPORTER environment variable. We can add this variable to our deployment.yaml, redeploy our application, and tail the logs from our helloworld app so that we can grab the trace id to then find the trace in Splunk Observability Cloud. (In the next section of our workshop, we will also walk through using the debug exporter, which is how you would typically debug your application in a K8s environment.)

First, open the deployment.yaml file in vi:

bash 
vi deployment.yaml

Then, add the OTEL_TRACES_EXPORTER environment variable:

yaml 
          env:
            - name: PORT
              value: "8080"
            - name: NODE_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://$(NODE_IP):4318"
            - name: OTEL_SERVICE_NAME
              value: "helloworld"
            - name: OTEL_RESOURCE_ATTRIBUTES 
              value: "deployment.environment=YOURINSTANCE"
            # NEW VALUE HERE:
            - name: OTEL_TRACES_EXPORTER
              value: "otlp,console"

Save your changes then redeploy the application:

bash 
kubectl apply -f deployment.yaml

Tail the helloworld logs:

bash 
kubectl logs -l app=helloworld -f

Then, in your other terminal window, generate a trace with your curl command. You will see the trace id in the console in which you are tailing the logs. Copy the Activity.TraceId: value and paste it into the Trace search field in APM.

Last Modified ·