Skip to main content
Uncategorized

Changing the default SSL certificate on your OpenShift routers

By July 15, 2021No Comments

Changing the default SSL certificate on your OpenShift routers

The OpenShift platform make extensive use of SSL certificates. The most common deployments leverage the automatically generated SSL certificates for back-end (inter-cluster) communication, but will almost always use a public CA signed certificate for the console access, as well as the wildcard certificate applied to application routes that require TLS.

While it may not be a frequent task, updating SSL certificates is necessary from time to time. A few of these reasons include:

  • Changing providers (cost, value, features, etc.)
  • Changing server/service/organizational details
  • Lifecycle management (expired certificates, rotation policies, etc)
  • Lost, stolen, or compromised keys

In this post, we will illustrate one method of changing these certificates with minimal impact to the production cluster.

During Install

In a standard OpenShift deployment, a wildcard certificate is created and deployed into the router pods. This default certificate is used for any TLS enabled route that does not specify a specific certificate. The process to add a default certificate during deployment is well documented here.

In a Production Environment

Once the OpenShift PaaS is running in production, updating the certificate requires a different process than the one followed during the install. This process will help to ensure that services remain online during the change, and will use the native OpenShift blue-green deployment strategy to ensure that the router pods operate properly before cutting over users/services.

The Process

In order to change the default SSL certificate, we will simply modify the certificate contents that are loaded into the router and then re-deploy the router. The certificate contents are stored in OpenShift as a secret, which means that the secret can be updated with the new contents, and will take effect upon router redeploy. This script will:

  • Back up the existing secret (sometimes you just need a safety net)
  • Create a new .pem file out of the new certificate files
  • Delete the existing secret
  • Create the new secret
  • Start a new router deployment
  • Clean up the .pem file
  • Display pods in the default project (new containers should be spawning)

This script assumes you have cluster-admin privileges, and are running this from one of the master hosts (likely where you maintain your SSL certificates).

The Script

The script is maintained in the ArctiqTeam public GitHub repository, and can be found here: https://github.com/ArctiqTeam/openshift/tree/master/ssl_management

{% highlight shell

!/bin/bash

This script is intended to replace the default SSL certificate assigned to the OpenShift router pods

This process can be peformed without service disruption, rather than re-deploying the routers (which has a service impact)

The SSL certificate contents are mounted as a secret inside of the router pods, which this script will replace and re-deploy

You must have cluster-admin privileges, and this assumes the router deployment config has the default name

Pass the new certificate file names in this order: NEW_CERT NEW_PRIVATE_KEY NEW_CA_CERT

ie. ./change_router_default_cert.sh apps_cert.crt apps_key.key ca_cert.crt

NEW_CERT=’$1′ NEW_PRIVATE_KEY=’$2′ NEW_CA_CERT=’$3′ NEW_PEM=new_cert.pem

date=date +%y%m%d%H%M

Select the project

oc project default

Backup existing secret data

oc export secret router-certs -o yaml > router-certs.backup.$date.yaml

Create a pem file with the new files

cat $NEW_CERT $NEW_CA_CERT > $NEW_PEM

Remove existing secret

oc delete secret router-certs

Create new secret

oc secrets new router-certs tls.crt=$NEW_PEM tls.key=$NEW_PRIVATE_KEY –type=’kubernetes.io/tls’ –confirm

Redeploy router pods

oc deploy dc/router –latest

Clean up temp file

rm $NEW_PEM

Display running pods for verificaiton

oc get pods

{% endhighlight

Additional considerations

This has been tested in environments with the default HAProxy router deployment that has previously used the –default-cert flag, with and without the ipfailover deployment options (outlined here ). No changes are required to the ipfailover deployment configuration for this to take effect. If the current router config does not use the –default-cert flag, it will need to be redeployed with the oadm command as outlined in the installation instructions.