Getting Data In (GDI) with OTel and UF

Code to Kubernetes - Python

7 min

Code to Kubernetes - Python

Objective: Understand activities to instrument a python application and run it on Kubernetes.

Note: these steps do not involve Splunk

Duration: 15 Minutes

1. Verify the code - Review service

Navigate to the review directory

bash 
cd /home/splunk/realtime_enrichment/flask_apps/review/

Inspect review.py (realtime_enrichment/flask_apps/review)

bash 
cat review.py
python 
from flask import Flask, jsonify
import random
import subprocess

review = Flask(__name__)
num_reviews = 8635403
num_reviews = 100000
reviews_file = '/var/appdata/yelp_academic_dataset_review.json'

@review.route('/')
def hello_world():
    return jsonify(message='Hello, you want to hit /get_review. We have ' + str(num_reviews) + ' reviews!')

@review.route('/get_review')
def get_review():
    random_review_int = str(random.randint(1,num_reviews))
    line_num = random_review_int + 'q;d'
    command = ["sed", line_num, reviews_file] # sed "7997242q;d" <file>
    random_review = subprocess.run(command, stdout=subprocess.PIPE, text=True)
    return random_review.stdout

if __name__ == "__main__":
    review.run(host ='0.0.0.0', port = 5000, debug = True)

Inspect requirements.txt

text 
Flask==2.0.2

Create a virtual environment and Install the necessary python packages

bash 
cd /home/splunk/realtime_enrichment/workshop/flask_apps_start/review/

pip freeze #note output
pip install -r requirements.txt
pip freeze #note output

Start the REVIEW service. Note: You can stop the app with control+C

bash 
python3 review.py

 * Serving Flask app 'review' (lazy loading)
 * Environment: production
         ...snip...
 * Running on http://10.160.145.246:5000/ (Press CTRL+C to quit)
 * Restarting with stat
127.0.0.1 - - [17/May/2022 22:46:38] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [17/May/2022 22:47:02] "GET /get_review HTTP/1.1" 200 -
127.0.0.1 - - [17/May/2022 22:47:58] "GET /get_review HTTP/1.1" 200 -

Verify that the service is working

bash 
curl http://localhost:5000
bash 
curl localhost:5000
{
  "message": "Hello, you want to hit /get_review. We have 100000 reviews!"
}

curl localhost:5000/get_review
{"review_id":"NjbiESXotcEdsyTc4EM3fg","user_id":"PR9LAM19rCM_HQiEm5OP5w","business_id":"UAtX7xmIfdd1W2Pebf6NWg","stars":3.0,"useful":0,"funny":0,"cool":0,"text":"-If you're into cheap beer (pitcher of bud-light for $7) decent wings and a good time, this is the place for you. Its generally very packed after work hours and weekends. Don't expect cocktails. \n\n-You run into a lot of sketchy characters here sometimes but for the most part if you're chilling with friends its not that bad. \n\n-Friendly bouncer and bartenders.","date":"2016-04-12 20:23:24"}

Workshop Question

  • What does this application do?
  • Do you see the yelp dataset being used?
  • Why did the output of pip freeze differ each time you ran it?
  • Which port is the REVIEW app listening on? Can other python apps use this same port?

2. Create a REVIEW container

To create a container image, you need to create a Dockerfile, run docker build to build the image referencing the Docker file and push it up to a remote repository so it can be pulled by other sources.

Dockerfile for review

dockerfile 
FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt /app
RUN pip install -r requirements.txt
COPY ./review.py /app
EXPOSE 5000
CMD [ "python", "review.py" ]

Create a container image (locally) Run ‘docker build’ to build a local container image referencing the Dockerfile

bash 
docker build -f Dockerfile -t localhost:8000/review:0.01 .

Push the container image into a container repository Run ‘docker push’ to place a copy of the REVIEW container to a remote location

bash 
docker push localhost:8000/review:0.01

Verify that the image is in Docker Hub. The same info can be found in Docker Desktop

bash 
curl -s http://localhost:8000/v2/_catalog

3. Run REVIEW in Kubernetes

Create K8 deployment yaml file for the REVIEW app

Reference: Creating a Deployment

review.deployment.yaml

yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: review
  labels:
    app: review
spec:
  replicas: 1
  selector:
    matchLabels:
      app: review
  template:
    metadata:
      labels:
        app: review
    spec:
      imagePullSecrets:
      - name: regcred
      containers:
      - image: localhost:8000/review:0.01
        name: review
        volumeMounts:
        - mountPath: /var/appdata
          name: appdata
      volumes:
      - name: appdata
        hostPath:
          path: /var/appdata

Notes regarding review.deployment.yaml:

Create a K8 service yaml file for the review app.

Reference: Creating a service:

review.service.yaml

yaml 
apiVersion: v1
kind: Service
metadata:
  name: review
spec:
  type: NodePort
  selector:
    app: review
  ports:
    - port: 5000
      targetPort: 5000
      nodePort: 30000

Notes about review.service.yaml:

Apply the review deployment and service

bash 
kubectl apply -f review.service.yaml -f review.deployment.yaml

Verify that the deployment and services are running:

bash 
kubectl get deployments
bash 
kubectl get services
bash 
curl localhost:30000
bash 
curl localhost:30000/get_review

Workshop Question

What changes are required if you need to make an update to your Dockerfile now?
Last Modified ·