Skip to content

Instantly share code, notes, and snippets.

@fabianoalmeida

fabianoalmeida/cert-manager-nginx-ingress-gke-recipe.md

Last active February 18, 2020 18:54
Show Gist options
  • Save fabianoalmeida/23fbbec74a8a7ff44fa6b39ed7dcffaf to your computer and use it in GitHub Desktop.
Save fabianoalmeida/23fbbec74a8a7ff44fa6b39ed7dcffaf to your computer and use it in GitHub Desktop.
Download ZIP
A quickstart guide to deploying cert-manager and nginx-ingress on GKE
Raw
cert-manager-nginx-ingress-gke-recipe.md

This is a quick recipe for deploying cert-manager and nginx-ingress on GKE to obtain SSL certificates from Lets Encrypt. Whilst this recipe is designed for Google Cloud Platform, it can easily be adapted for other cloud platforms.

We'll begin with a Kubernetes cluster, and we'll obtain authentication credentials.

Step 0 - Get Credentials and Install Helm Client

$ gcloud container clusters get-credentials my-test-app

Increasing this gist I based on, I used this link to make all works fine: https://cert-manager.io/docs/tutorials/acme/ingress/

So, first of all, we need to install helm (if your OS is not Unix based, search how to install it for your case):

$ brew install kubernetes-helm

And after that, just update your repo:

$ helm repo update

Step 1 - Create Nginx Ingress Controller

When you have your repo updated, we can start on deploying our Nginx Ingress Controller:

$ helm install stable/nginx-ingress --generate-name

This command will create two services:

$ kubectl get services
NAME                                       TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
kubernetes                                 ClusterIP      10.63.240.1     <none>           443/TCP                      23m
nginx-ingress-1581685044-controller        LoadBalancer   10.63.248.177   35.233.154.161   80:31345/TCP,443:31376/TCP   16m
nginx-ingress-1581685044-default-backend   ClusterIP      10.63.250.234   <none>           80/TCP                       16m

Step 2 - Deploy Deployment and Service

When you see an external IP after this command, you can configure your DNS with this IP and test accessing your domain.

Using this Deployment file:

# deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: your-app
spec:
  selector:
    matchLabels:
      app: your-app
  replicas: 1
  selector:
    matchLabels:
      app: your-app
  template:
    metadata:
      labels:
        app: your-app
    spec:
      containers:
      - name: your-app
        image: gcr.io/your-project/your-app:v1
        imagePullPolicy: Always
        ports:
        - containerPort: 3000

You can apply to create your first deployment.

$ kubectl apply -f deployment.yaml
deployment.extensions "your-app" created

Using this Service file:

# service.yaml
apiVersion: v1
kind: Service
metadata:
  name: your-app
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 3000
      protocol: TCP
  selector:
    app: your-app

You can apply to create your first service.

$ kubectl apply -f service.yaml
service "your-app" created

Step 3 - Deploy Ingress

Now you can create your Ingress file and create a connection between your internal pods with an external world using a name.

# ingress.yaml
apiVersion: v1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: your-app
  annotations:
    kubernetes.io/ingress.class: "nginx"    
    #cert-manager.io/cluster-issuer: "letsencrypt-staging"

spec:
  tls:
  - hosts:
    - example.example.com
    secretName: your-app-example-tls
  rules:
  - host: example.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: your-app
          servicePort: 80

Take a look... We're creating a new ingress but we're not defining a cert-manager cluster-issuer.

$ kubectl apply -f ingress.yaml
ingress.extensions "your-app" created

You can execute

$ kubectl get ingress

To see your external IP like something like that:

NAME      HOSTS     ADDRESS         PORTS     AGE
your-app  *         35.199.170.62   80        9m

And test a real communication to your domain:

$ curl -kivL -H 'Host: example.your-domain.com' 'http://35.199.164.14'

Step 4 - Deploy Cert Manager

Now, we can deploy cert manager for our domain and pods. First of all, I used this link https://cert-manager.io/docs/installation/kubernetes/ to guide me in the intallations of Kubernetes.

Listing the steps... Creating a new namespace

$ kubectl create namespace cert-manager

Installing the CustomResourceDefinitions and cert-manager itself (look for the version you want to install):

$ kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.13.0/cert-manager.yaml

And then, create a ClusterRoleBinding:

$ kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user=$(gcloud config get-value core/account)

Step 5 - Configure Let’s Encrypt Issuer

It's recommended you create an staging cluster-issuer first and then create a prod. So...

# letsencrypt-staging.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
 name: letsencrypt-staging
 namespace: cert-manager
spec:
 acme:
   # The ACME server URL
   server: https://acme-staging-v02.api.letsencrypt.org/directory
   # Email address used for ACME registration
   email: YOU_EMAIL_HERE
   # Name of a secret used to store the ACME account private key
   privateKeySecretRef:
     name: letsencrypt-staging
   # Enable the HTTP-01 challenge provider
   solvers:
   - http01:
       ingress:
         class: nginx

And then, apply it:

$ kubectl create -f letsencrypt-staging.yaml
issuer.cert-manager.io "letsencrypt-staging" created

At this moment you can create and apply your prod cluster-issuer

# letsencrypt-prod.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
 name: letsencrypt-prod
 namespace: cert-manager
spec:
 acme:
   # The ACME server URL
   server: https://acme-v02.api.letsencrypt.org/directory
   # Email address used for ACME registration
   email: YOU_EMAIL_HERE
   # Name of a secret used to store the ACME account private key
   privateKeySecretRef:
     name: letsencrypt-prod
   # Enable the HTTP-01 challenge provider
   solvers:
   - http01:
       ingress:
         class: nginx

And then, apply it:

$ kubectl create -f letsencrypt-prod.yaml
issuer.cert-manager.io "letsencrypt-prod" created

You can execute this command to check if everything is really ok

$ kubectl describe issuer letsencrypt-staging

And you can see something like that:

Name:         letsencrypt-staging
Namespace:    default
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"cert-manager.io/v1alpha2","kind":"Issuer","metadata":{"annotations":{},"name":"letsencrypt-staging","namespace":"default"},"spec":{"a...
API Version:  cert-manager.io/v1alpha2
Kind:         Issuer
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-11-17T18:03:54Z
  Generation:          0
  Resource Version:    9092
  Self Link:           /apis/cert-manager.io/v1alpha2/namespaces/default/issuers/letsencrypt-staging
  UID:                 25b7ae77-ea93-11e8-82f8-42010a8a00b5
Spec:
  Acme:
    Email:  [email protected]
    Private Key Secret Ref:
      Key:
      Name:  letsencrypt-staging
    Server:  https://acme-staging-v02.api.letsencrypt.org/directory
    Solvers:
      Http 01:
        Ingress:
          Class:  nginx
Status:
  Acme:
    Uri:  https://acme-staging-v02.api.letsencrypt.org/acme/acct/7374163
  Conditions:
    Last Transition Time:  2018-11-17T18:04:00Z
    Message:               The ACME account was registered with the ACME server
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>

Step 6 - Deploy a TLS Ingress Resource

Now we're able to execute the last step to put our domain on a secure stage.

Fist, update your ingress file:

# ingress.yaml
apiVersion: v1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: your-app
  annotations:
    kubernetes.io/ingress.class: "nginx"    
    cert-manager.io/cluster-issuer: "letsencrypt-staging"

spec:
  tls:
  - hosts:
    - example.example.com
    secretName: your-app-example-tls
  rules:
  - host: example.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: your-app
          servicePort: 80

And apply it:

$ kubectl apply -f ingress.yaml

You can check if your certificate is generated

$ kubectl get certificate
NAME                     READY   SECRET                   AGE
your-app-example-tls     True    your-app-example-tls   16m

Or use kubectl describe to see more details

$ kubectl describe certificate your-app-example-tls
Name:         your-app-example-tls
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1alpha2
Kind:         Certificate
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-11-17T17:58:37Z
  Generation:          0
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  your-app
    UID:                   a3e9f935-ea87-11e8-82f8-42010a8a00b5
  Resource Version:        9295
  Self Link:               /apis/cert-manager.io/v1alpha2/namespaces/default/certificates/your-app-example-tls
  UID:                     68d43400-ea92-11e8-82f8-42010a8a00b5
Spec:
  Dns Names:
    example.your-domain.com
  Issuer Ref:
    Kind:       ClusterIssuer
    Name:       letsencrypt-staging
  Secret Name:  your-app-example-tls
Status:
  Acme:
    Order:
      URL:  https://acme-staging-v02.api.letsencrypt.org/acme/order/7374163/13665676
  Conditions:
    Last Transition Time:  2018-11-17T18:05:57Z
    Message:               Certificate issued successfully
    Reason:                CertIssued
    Status:                True
    Type:                  Ready
Events:
  Type     Reason          Age                From          Message
  ----     ------          ----               ----          -------
  Normal   CreateOrder     9m                 cert-manager  Created new ACME order, attempting validation...
  Normal   DomainVerified  8m                 cert-manager  Domain "example.example.com" verified with "http-01" validation
  Normal   IssueCert       8m                 cert-manager  Issuing certificate...
  Normal   CertObtained    7m                 cert-manager  Obtained certificate from ACME server
  Normal   CertIssued      7m                 cert-manager  Certificate issued Successfully

And you can see the Secret too

$ kubectl describe secret your-app-example-tls
Name:         your-app-example-tls
Namespace:    default
Labels:       cert-manager.io/certificate-name=your-app-example-tls
Annotations:  cert-manager.io/alt-names=example.example.com
              cert-manager.io/common-name=example.example.com
              cert-manager.io/issuer-kind=Issuer
              cert-manager.io/issuer-name=letsencrypt-staging

Type:  kubernetes.io/tls

Data
====
tls.crt:  3566 bytes
tls.key:  1675 bytes

Now we can deploy our production certificate. Update your Ingress file and apply it.

# ingress.yaml
apiVersion: v1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: your-app
  annotations:
    kubernetes.io/ingress.class: "nginx"    
    cert-manager.io/cluster-issuer: "letsencrypt-prod"

spec:
  tls:
  - hosts:
    - example.example.com
    secretName: your-app-example-tls
  rules:
  - host: example.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: your-app
          servicePort: 80

Then

$ kubectl apply -f ingress.yaml

Before you test your domain with a TLS certificate, do this

$ kubectl delete secret your-app-example-tls
secret "your-app-example-tls" deleted

This action will remove the secret created to letsencrypt-staging and force to generate another one.

Listing certificates

$ kubectl describe certificate
Name:         your-app-example-tls
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1alpha2
Kind:         Certificate
Metadata:
  Cluster Name:
  Creation Timestamp:  2018-11-17T18:36:48Z
  Generation:          0
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  your-app
    UID:                   a3e9f935-ea87-11e8-82f8-42010a8a00b5
  Resource Version:        283686
  Self Link:               /apis/cert-manager.io/v1alpha2/namespaces/default/certificates/your-app-example-tls
  UID:                     bdd93b32-ea97-11e8-82f8-42010a8a00b5
Spec:
  Dns Names:
    example.your-domain.com
  Issuer Ref:
    Kind:       ClusterIssuer
    Name:       letsencrypt-prod
  Secret Name:  your-app-example-tls
Status:
  Conditions:
    Last Transition Time:  2019-01-09T13:52:05Z
    Message:               Certificate does not exist
    Reason:                NotFound
    Status:                False
    Type:                  Ready
Events:
  Type    Reason        Age   From          Message
kubectl describe certificate your-app-example-tls   ----    ------        ----  ----          -------
  Normal  Generated     18s   cert-manager  Generated new private key
  Normal  OrderCreated  18s   cert-manager  Created Order resource "your-app-example-tls-889745041"

Describing your certificate

$ kubectl describe order your-app-example-tls-889745041
...
Events:
  Type    Reason      Age   From          Message
  ----    ------      ----  ----          -------
  Normal  Created     90s   cert-manager  Created Challenge resource "your-app-example-tls-889745041-0" for domain "example.example.com"

So, that's it. Your app is now served via TLS support.

@fabianoalmeida
Copy link
Author

fabianoalmeida commented Feb 18, 2020

Cause you want to config correctly you gitlab project with a pipeline to delivery your project to GKE, follow these steps.

Basically, create an account and cluster on GCP and create a Service Account and configure it with these roles:

image

#1 - Kubernetes Engine Developer
#2 - Cloud Build Service Account
#3 - Storage Admin
#4 - Viewer

After that, create a key as json file and apply it:

base64 /path/to/credential.json | tr -d \\n

Copy the result of it and paste in your gitlab as an environment variable and use it inside your .gitlab-ci.yaml file.

Happy Depploying! ⏩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment