Configure Ingress Controller / Resource to route traffic

[batpad]

This is similar to #46, but keeping that separate for SSL configuration, and this issue to handle general configuration of an Ingress controller.

The starting point for documentation would be https://kubernetes.io/docs/concepts/services-networking/ingress/

The idea is that first, one needs to define an Ingress Controller - on GKE, there is an Ingress controller present on the master, but on other clusters, one needs to choose and create a deployment for the Ingress Controller, which essentially runs as a DaemonSet, with one container running on each of your nodes, to handle Ingress traffic and forward it to your Ingress rules to handle routing. One option is the nginx Ingress controller: https://github.com/kubernetes/ingress-nginx/blob/master/README.md and another that seems like a popular option is traefik: https://github.com/containous/traefik

Once one has an Ingress Controller running on the cluster, one can setup Ingress rules, mapping different host names and URL paths, to different Services.

Then one can write Ingress configuration definitions, where we would likely have the domain name be defined in the values.yaml file so the user can customize it.

An Ingress yaml file would look roughly like (from the docs):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /foo
        backend:
          serviceName: s1
          servicePort: 80
      - path: /bar
        backend:
          serviceName: s2
          servicePort: 80

Essentially, one is able to configure different hosts and paths to direct to different services. Then all we need to do is configure A records for the domains to point to the IP address of the cluster, and the Ingress rules should handle routing to the correct service based on the URL.

The documentation includes instructions for configuring TLS over port 443, providing certificate files as secrets: https://kubernetes.io/docs/concepts/services-networking/ingress/#tls

cc @Rub21

[batpad]

With the setup on EKS creating separate ELBs for our services, I think this is less urgent. To get custom domains + SSL working, all we need to do now is point the domain CNAME record to the ELB address and generate an SSL certificate and pass it to the ELB as SSLCertificateID or so.

cc @Rub21

[batpad]

Closing in favour of #216