Published on

Debugging Kubernetes Ingress Issues

Authors

Today I finished writing the Kubernetes configuration for my application. I deployed this and tried to hit it using my ingress but instead ended up on a page with a blank screen.

I debugged this using an inward out approach as generally the culprit is the app that sits behind the ingress. A Kubernetes app with an ingress will have the following layers:

http://www.example.com/api
     -> ingress(host: www.example.com , path: /api)
          -> your-apps-api-svc:appPort
               -> deployment/statefulset/daemonset

When someone hits your URL the process is:

  1. Your DNS provider resolves the URL to an IP
  2. The request is passed to that IP
  3. The request hits the ingress (which is generally an Nginx backed operator)
  • Look at what path the request is coming in on
  • Redirect the request to the service associated with the path. In this case the path is /api and the service is your-apps-api-svc:appPort
  1. Pass the request to your app deployed as a deployment/statefulset/daemonset
  2. Your app handles the request

I use an inward out approach to debugging as generally the layers above your app are fairly boiler plate and do not change very often - your app is most likely to have the issue in as a result.

To debug inward out the process is:

  1. Check the pod is running: kubectl get pods -n yourNamespace -o wide
  • Confirm that STATUS is RUNNING and READY is 1/1
  1. Port forward to the pod: kubectl port-forward -n yourNamespace your-pod 3000:3000
  • Change the ports at the end to the ports relevant to your pod
  1. In another shell tail the logs of your-pod: kubectl logs -n yourNamespace your-pod -f
  2. Try hit the URL locally, in my case it would be http://localhost:3000
  3. If you get the app back without issues the problem likely is higher up
  • In my case I could hit the app locally, but it turns out the issue was actually in my React app still as the issue did not manifest locally
  1. Check the service configuration associated with the pod
  • Are the ports on the service the same as the one exposed by the pod?
  • Do the labels match? In your service you should have a labels section and in your deployment you should have a matchLabels section which should have the exact same labels in order for the service to be able to link up with the pod
  1. Check the ingress configuration is correctly associated with the service
  • The service name in the ingress should match the service name in your service configuration
  • The service port in the ingress should be the same as the service port in your service configuration
  1. Check that your DNS provider has the host and IP configured correctly
  • As a workaround you can add the host and IP to your /etc/hosts file if you cannot access the DNS entry for this site (this will only work on your machine)

In the end the issue on my side was caused by a misconfigured entry in my React apps .env file. The PUBLIC_URL had a different value to what the ingress had configured. For example the ingress had the path /api but the React PUBLIC_URL was set to /ui. This resulted in the React app prepending /ui instead of /api to all routes. I simply changed the ingress to use /ui as the path and the issue was resolved.