Fix RabbitMQ Propagation

ReDeploy RabbitMQ Fix

5 minutes

Rebuild and Redeploy after Saving

bash 
cd ~/workshop/context-propagation
bash scripts/build-images.sh payment-api fulfillment-worker
bash scripts/import-images-k3d.sh payment-api fulfillment-worker
kubectl -n cosmic-shop rollout restart deployment/payment-api
kubectl -n cosmic-shop rollout restart deployment/fulfillment-worker
kubectl -n cosmic-shop rollout status deployment/payment-api --timeout=180s
kubectl -n cosmic-shop rollout status deployment/fulfillment-worker --timeout=180s

Validation Checklist

Run these commands after redeploy completes.

1. Confirm health endpoints report propagation enabled

bash 
kubectl -n cosmic-shop get pods -l app=fulfillment-worker
kubectl -n cosmic-shop exec deploy/fulfillment-worker -- wget -qO- http://localhost:3006/health
kubectl -n cosmic-shop exec deploy/payment-api -- wget -qO- http://localhost:3005/health

2. Confirm rollouts completed

bash 
kubectl -n cosmic-shop rollout status deployment/payment-api
kubectl -n cosmic-shop rollout status deployment/fulfillment-worker

3. Confirm startup logs mention propagation

bash 
kubectl -n cosmic-shop logs deployment/payment-api --tail=3
kubectl -n cosmic-shop logs deployment/fulfillment-worker --tail=3

Troubleshooting

Click here for Troubleshooting Guidance

Potential Issue 1. Fulfillment worker still shows orphan traces

  • Confirm both files were edited (producer inject + consumer extract)
  • Confirm you ran make fix-rabbitmq (rebuilds both images — restart alone is not enough)
  • Verify both health endpoints show "propagation": true
  • Generate new orders after the fix

Potential Issue 2. Only payment-api health is true

The consumer stub must be replaced with extractTraceContext() - fixing only the producer is not sufficient.

Potential Issue 3. Alternate pattern: auto-instrumentation

Some teams use @opentelemetry/instrumentation-amqplib (OpenTelemetry JS contrib) instead of manual inject/extract.

The failure mode is the same - instrumentation disabled or misconfigured - but the issue is resolved by enabling the library in instrumentation.js rather than editing publish/consume code.

This workshop uses raw amqplib because that is still common in Node.js services and makes the inject/extract contract explicit.

Last Modified ·