Running a web with HTTPS in kubernetes

Carles Barrobés

2020-03-10 09:39

For a couple of recent projects (a personal one and a client's) I've been setting up a couple of typical web applications in kubernetes. Since I see this as a very common pattern, which is becoming ever easier to replicate, I will share my experience and the scaffolding code that can get you up and running as well.

I have been using both Linode and Digital Ocean, and the experience with both is almost identical. In particular once you create your kubernetes cluster using either of their UIs (which is a very easy process), the rest is exactly the same.

My assumptions (and target audience)

The ideal target audience for this article is someone who has already been running their own web applications already (either in bare metal or a virtual machine) and wants to know how to translate that setup to kubernetes. Or has just done the basic tutorials on how to deploy a container to k8s and wants to round out their production setup.

A typical "bare metal" implementation of a web app would have something like an nginx/apache proxy doing SSL termination, that forwards traffic to your backend and serves static pages. You might also have letsencypt's certbot running off a cron job to generate your SSL certificates, or have set up your certificates manually.

In this article I will assume you are familiar with containers and know how to build docker images to some extent. I also presume you've at least read about kubernetes and have a basic idea of what it is for. A simplistic metaphor for kubernetes that may work for you is "a cloud operating system that runs containers".

Tools

You will need kubectl and helm. And a kubernetes cluster you can provision with a few clicks (or some terraform if you are into it) on either Linode or Digital Ocean, or your provider of choice.

The pieces you need to put together

Regardless of the overall structure of your application, frontend and APIs, monolith or microservices, you need these in place:

An HTTP server that does SSL termination and serves traffic for your application domain.
To serve your front-end code as static files (HTML, CSS, JS, images...)
To run your backend API service(s)
Some form of database or persistence

I will illustrate these with a relatively simple app with a UI and a single backend API. But the principles apply the same if e.g. your backend is split across multiple microservices or you have multiple UI applications.

TL;DR

If you just want to get something working without reading all my explanations, you can find all of the definitions and scripts for basic bootstrapping of the generic cluster elements in kubestrap

Just clone that repo, and follow instructions, which are as simple as:

EMAIL=your@email.here ./configure-cluster

(the email is required for letsencrypt to send you expiry notifications).

The Kubernetes approach

This is how these elements translate in kubernetes:

ingress-nginx: its role is to do the job of the proxy http server - it will indeed be an nginx proxy that will do SSL termination and forward requests to our frontend or backend
cert-manager: it will handle the generation of SSL certificates, we will be setting it up for letsencrypt
For each of your frontend and API, you will need:
- A docker image you know how to dockerise your backend service(s)
- A deployment that specifies the container image and version that runs your application. You may run as many instances (replicas) as you need.
- A service definition (which you can think of as a load balancer for your application replicas).
- An ingress definition: this will be the equivalent of the nginx virtual host, where you tell the proxy where to route requests for each domain
For your database, you will need a deployment and a persistent volume for your data

Another typical thing you may need to solve is being able to download images from a secured private registry (one that requires docker login).

Ingress setup

You can find a full installation guide in https://kubernetes.github.io/ingress-nginx/deploy/

However after reading through a lot of stuff it turns out these are the only 2 commands I had to run, and they worked well for both Linode and Digital Ocean:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/cloud-generic.yaml

Only I did not really run these commands. I downloaded both YAML, concatenated them and put them in version control in ingress.yml and applied that.

SSL setup with letsencrypt

You will need to install cert-manager, and define letsencrypt as a certificate issuer. This will install the cert manager:

kubectl create namespace test
helm repo add jetstack https://charts.jetstack.io
helm install --atomic cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v0.13.1

In order to add the letsencypt issuer to cert manager, download letsencrypt.yml, replace both __EMAIL__ placeholders with your own email, and run:

kubectl apply -f letsencrypt.yml

I'm setting up two certificate issuers, one for letsencrypt staging (for testing purposes) and one for production. You can do away with the staging one if you think you won't need it.

Setting up your service to use SSL ingress

Now you will want your services to take advantage of this ingress with SSL termination. You will tie things together when you create an ingress resource for your service. You may already have a deployment and a service in place to run your application - if not, set them up.

For instance, my service name is kemendius. It needs at least:

A Deployment where I specify the container image and version to run, and which ports to expose, etc. Upon creation this will end up instantiating the pods for your application.
A Service, where I expose the pods generated by the deployment
An Ingress resource where I declare the virtual host configuration for this service.

The ingress is where we will tell it to use the SSL certificate issuer and which domain this is for (ingress.yml):

    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: kemendius
      annotations:
        # This annotation makes it use letsencrypt as issuer:
        cert-manager.io/cluster-issuer: letsencrypt
        nginx.ingress.kubernetes.io/rewrite-target: /
    spec:
      rules:
        - host: kemendius.cat
          http:
            paths:
              - path: /
                backend:
                  serviceName: kemendius
                  servicePort: 80
      tls:
        # Placing a host in the TLS config will indicate that a 
        # certificate should be created:
        - hosts:
            - kemendius.cat
          # cert-manager will store the certificate in this secret:
          secretName: kemendius-cert

The annotation cert-manager.io/cluster-issuer: letsencrypt is important because otherwise it will use a fake kubernetes self-signed CA to issue the certificate.

Replace kemendius everywhere with your service name, save as app-ingres.yml and run:

kubectl apply -f app-ingress.yml

Carles Barrobés, Software Delivery