How to renew Puppet CA certificate

We’ve written about Puppet a couple of times before, but this time we’ll cover the topic of renewing Puppet CA certificate. While the official documentation recommends regenerating the Puppet CA and re-issueing certificates for all Puppet agents, in some cases that’s not neccessary nor convenient. In this post, we’ll explain how you can renew Puppet CA certificate, without the need to re-sign all Puppet agent certificates.

In our scenario, Puppet Server is working in tandem with PuppetDB and Foreman. Therefore, beside renewing Puppet CA, we’ll also renew client certificate that’s used by Puppet agent, PuppetDB, and Foreman (all running on the same server as Puppet Server). Don’t worry, the procedure is the same even if you’re not using PuppetDB and Foreman.

First things first. Let’s backup existing SSL configuration.

mkdir -p /root/puppet-ssl-backup/{puppet,puppetserver,puppetdb}
cp -prv /etc/puppetlabs/puppet/ssl/ /root/puppet-ssl-backup/puppet/
cp -prv /etc/puppetlabs/puppetserver/ca/ /root/puppet-ssl-backup/puppetserver/
cp -prv /etc/puppetlabs/puppetdb/ssl/ /root/puppet-ssl-backup/puppetdb/

Next, let’s note the serial number of the current CA certificate.

openssl x509 -in /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem -noout -serial

We can now generate new CA CSR (certificate signing request) using existing CA details and private key.

openssl x509 -x509toreq -in /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem -signkey /etc/puppetlabs/puppet/ssl/ca/ca_key.pem -out /etc/puppetlabs/puppet/ssl/ca/ca_csr.pem

Before generating the new CA, we’ll need to configure custom OpenSSL extensions.

cat > /root/puppet-ca-extension.cnf << EOF
[CA_extensions]
basicConstraints = critical,CA:TRUE
nsComment = "Puppet Ruby/OpenSSL Internal Certificate"
keyUsage = critical,keyCertSign,cRLSign
authorityKeyIdentifier=keyid,issuer
subjectKeyIdentifier = hash
EOF

Now we’re ready to generate new CA using the same serial from one the previous commands (don’t forget to insert it at the end of the command). If necessary, adjust the validity period for the CA. In our example, the CA will be valid for 10 years.

openssl x509 -req -days 3650 -in /etc/puppetlabs/puppet/ssl/ca/ca_csr.pem -signkey /etc/puppetlabs/puppet/ssl/ca/ca_key.pem -out /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem -extfile /root/puppet-ca-extension.cnf -extensions CA_extensions -set_serial <enter_serial_here>

We also need to regenerate Puppet agent certificate using offline method. But first, remove existing SSL certificate and keys as otherwise Puppet Server will return an error. 

rm -fv /etc/puppetlabs/puppet/ssl/{certs,private_keys,public_keys}/$(hostname).pem
puppetserver ca generate --certname $(hostname) --subject-alt-names $(hostname) --ttl 3650 --ca-client --force

Optional: if you are using PuppetDB, copy new CA and client certificate to PuppetDB SSL directory.

cp -pv /etc/puppetlabs/puppetserver/ca/ca_crt.pem /etc/puppetlabs/puppetdb/ssl/ca.pem
cp -pv /etc/puppetlabs/puppet/ssl/private_keys/$(hostname).pem /etc/puppetlabs/puppetdb/ssl/private.pem
cp -pv /etc/puppetlabs/puppet/ssl/certs/$(hostname).pem /etc/puppetlabs/puppetdb/ssl/public.pem
chown -R puppetdb. /etc/puppetlabs/puppetdb/ssl/
chmod 600 /etc/puppetlabs/puppetdb/ssl/*

Restart all services that are using renewed SSL certificates. 

systemctl restart apache2 foreman foreman-proxy puppetserver puppetdb

At this point, Puppet agents will continue to use old CA during Puppet runs. To prevent failed Puppet runs once the old CA expires, it’s enough to simply remove /etc/puppetlabs/puppet/ssl/certs/ca.pem file on every Puppet agent. On the next Puppet run, the agent will automatically download renewed CA from Puppetserver and store it at the same location.

If you are using Puppet Bolt, you can automate this step with a single command:

bolt command run 'rm -f /etc/puppetlabs/puppet/ssl/certs/ca.pem' --run-as root -t all

That was easy, right? With the CA certificate all set for the next decade, you can get back to Puppet module development, or sharpen your unit and acceptance testing skills.

If you are in the mood for a scary Puppet campfire story, we recommend the blog post about a hugry regex that brought the Puppet server to its knees. It can’t get any scarier than regexes, right? 👻️