This tool will automatically rotate/update the older Diego root certificate authority and Intermediate Identity Cert used by Diego for application instance identity. This tool is not needed if you have already upgraded to Tanzu Application Service 2.7 or later, as that version automatically rotates those certs on upgrade.
Riic operates on all BOSH deployments that have Diego cells, including TAS, TASW, and Isolation Segments for TAS.
Note: we only support the full TAS deployment - small footprint runtime is not currently supported.
The rotation process is a 2 step process:
At the end of each stage the tool will select a Diego cell and a router from each deployment and validate the certs match what is in Credhub.
Download the latest binary from the
releases
page. This is a linux binary that will only run on the Operations Manager VM.
If you can’t directly download the file from Operations Manager, download the file to your
workstation and then scp the binary to your Operations Manager VM:
$ scp ~/Downloads/riic -i ops-manager-private-key.pem ubuntu@ops-manager-fqdn:~/riic
You will need to make the binary executable and move it into your PATH.
$ chmod +x riic
$ sudo mv riic /usr/local/bin/riic
We recommend running the rotate command using nohup. This is important because
if you run a rotate command directly from your SSH session and your connection
terminates, so does the rotation process potentially leaving your foundation in
an indeterminite state. To start the tool in this way run the following:
$ nohup riic <subcommand> <args> &
After running the command with nohup you must hit Enter a second time to drop
you back the shell. The command will keep running even if your SSH session
terminates. The standard error and standard out are viewable in the nohup.out
log file:
$ tail -f nohup.out
There are no flags to pass in secret values. This is so that they will not
appear in your shell history. You will be prompted to enter the Operations
Manager admin user’s/client’s password and the Operations Manager decryption
passphrase. If you prefer this to be non-interactive like when running with
nohup, set the RIIC_PASSWORD and RIIC_DECRYPTION_PASSPHRASE environment
variables.
If your Operations Manager does not use username/password authentication, simply
set the --use-client-secret (-c) flag, and put your client ID in the
--username argument and your client secret as the password (see above for
an explanation of password handling).
Before attempting to rotate your instance identity certificates it’s a good
idea to check their expiration dates. This can easily be done using the
riic check-expiry command.
From your Operations Manager VM, run the following command, passing in your Operations Manager username/clientid and then enter the password/secret when prompted.
$ riic check-expiry --username admin
This will display the Diego CA and Intermediate Identity Cert expiration dates along with a warning if they’re close to expiring.
Once you’ve checked the expiration dates and are ready to rotate certificates, we recommend that you first ensure that all VMs in your TAS, isolation segment, and TASW deployments are healthy. You can do this by looking at the status tab for each tile in Operations Manager, or from the bosh CLI:
$ bosh -d DEPLOYMENT_NAME vms
If there are VMs that are unhealthy, resolve those issues before proceeding with the rotation. This may require you to recreate failing VMs. When you’ve confirmed that the deployment is healthy, proceed to rotate the certificates.
NOTE - When rotating certificates, the tool must run as the tempest-web user
to ensure there are no permission errors during the deployment. Make sure to
switch to the correct user before proceeding with the rotation:
$ sudo su - tempest-web
If you forget this step, the operation will fail and ask you to switch users
before trying again. Once you’re running as tempest-web, run the following
command to begin the rotation:
The following examples run the rotate command using nohup as noted above
and assume we’ve previously set the RIIC_PASSWORD and
RIIC_DECRYPTION_PASSPHRASE environment variables.
$ nohup riic rotate --username admin &
$ tail -f nohup.out
This will rotate the Diego CA and Intermediate Identity certs and update all BOSH jobs that reference these certs.
As an additional check you can run the validate command when the rotation completes. This will check all the certs on the diego cells and routers. Unlike the validation done at the end of rotate, this will be done on every single VM, not just the first. We don’t recommend running this on very large deployments.
$ riic validate --username admin