Puppet acceptance testing - intro

Puppet acceptance testing – intro

By Saša Teković Sysbee Blog acceptance testing, configuration management, docker, puppet 0 Comments

Hi there. In this post, we’ll cover the introduction to Puppet acceptance testing. If you are new to Puppet, we recommend that you check out Easy Puppet module development with PDK. And if you are curious about testing Puppet modules, the Puppet unit testing for beginners post is a good place to kick things off.

Praise the PDK

By definition, Puppet acceptance testing is a method by which we verify if Puppet actually ensured the desired state on the target system. In practice, this means that we apply changes on an actual test node, and then verify that Puppet configured the system exactly how we declared it our module.

Deploying a test node, triggering Puppet runs on it, and inspecting the applied state is a tedious process, so once again we’ll rely on the good old PDK (Puppet Development Kit). Smart folks at Puppet Labs developed Litmus – a testing tool bundled with the PDK that automates the whole acceptance testing process. Well, everything except writing the acceptance tests, but you get the point ?

We won’t go deep into how Litmus works, but if you are interested in that sort of thing, technical details are explained in the official documentation.

Tools of the trade

Although we’re going to invoke all commands through the PDK, it’s useful to know what tools we are using behind the scenes. We’ll start with Litmus which pretty much glues everything together. Litmus will handle the provisioning and destruction of test nodes. Supported provisioners are Vagrant, Docker, ABS (AlwaysBeScheduling), and vmpooler (internal to Puppet). In this post, we’ll focus only on Docker.

So, once Litmus provisions the test node, it’ll utilize the Bolt orchestration tool to install Puppet agent on the test node, package, and push Puppet module (with its dependencies) to it. Pretty straightforward. The actual acceptance testing is performed using Serverspec utility. Serverspec supports many resource types and it’s not Puppet-specific. You are free to use it with other configuration management systems as well.

Syntactically, acceptance tests are very similar to unit tests. But unlike unit tests, which we use to describe what we want to have in a compiled Puppet catalog, with acceptance tests we’re describing the desired system state once our Puppet code has been executed. For example, if we had a module that manages Apache web server on a Debian system, with acceptance tests we can verify if Puppet successfully installed the apache2 package and started apache2 service without issues.

Prerequisites for Puppet acceptance testing

The module that you wish to test should be PDK compatible. If it’s not, you’ll want to run the pdk convert command.
Next, we’ll need to define development dependencies in the .fixtures.yml file located in the root directory of your module.

---
fixtures:
  forge_modules:
    stdlib: "puppetlabs/stdlib"
  repositories:
    facts: 'https://github.com/puppetlabs/puppetlabs-facts.git'
    puppet_agent: 'https://github.com/puppetlabs/puppetlabs-puppet_agent.git'
    provision: 'https://github.com/puppetlabs/provision.git'

Bare minimum is facts, puppet_agent, and provision. If your module depends on other modules, you will have to define them too.In this tutorial, we’ll use the same “rsyslog” module we used for unit testing.

Next, we’ll create spec/spec_helper_acceptance.rb helper script with the following contents:

# frozen_string_literal: true

require 'puppet_litmus'
require 'spec_helper_acceptance_local' if File.file?(File.join(File.dirname(__FILE__), 'spec_helper_acceptance_local.rb'))

PuppetLitmus.configure!

Writing tests for Puppet acceptance testing

We should place acceptance tests within the spec/acceptance/ folder. The most simple form of acceptance test is the idempotency test. Let’s go ahead and create spec/acceptance/rsyslog_spec.rb file with the following content:

# frozen_string_literal: true

require 'spec_helper_acceptance'

describe 'rsyslog class' do
  # Apply the module with default parameters.
  context 'with default parameters' do
    let(:pp) { "class { 'rsyslog': }" }

    # Manifest should be idempotently applied (successfully the first time and
    # without changes the second time).
    it 'behaves idempotently' do
      idempotent_apply(pp)
    end
  end
end

idempotent_apply helper function tests the defined Puppet manifest twice. The first time to apply changes on the target system and ensure there were no errors. The second run is used to verify that there were no changes. Obviously, we should consider test successful if both Puppet runs complete without errors and the second run completes without having to apply any changes.

If you recall, our example module was managing /etc/rsyslog.d/10_modules.conf configuration file. The configuration file should have specific permissions, ownership, and specific content. We can easily test all those things if we expand our acceptance test to something like this:

# frozen_string_literal: true

require 'spec_helper_acceptance'

describe 'rsyslog class' do
  # Apply the module with default parameters.
  context 'with default parameters' do
    let(:pp) { "class { 'rsyslog': }" }

    # Manifest should be idempotently applied (successfully the first time and
    # without changes the second time).
    it 'behaves idempotently' do
      idempotent_apply(pp)
    end

    # Ensure that /etc/rsyslog.d/10_modules.conf exist, that has the correct
    # permissions, ownership, and content.
    describe file('/etc/rsyslog.d/10_modules.conf') do
      it { is_expected.to be_file }
      it { is_expected.to be_mode 640 }
      it { is_expected.to be_owned_by 'root' }
      it { is_expected.to be_grouped_into 'root' }
      its(:content) { is_expected.to match %r{\$ModLoad( )+imudp\.so} }
    end
  end
end

By the way, the permissions check doesn’t contain a typo. We intentionally omitted the leading 0 (zero) when testing 0640 permissions, because the numbers with the leading zero are automatically converted from octal to decimal notation by Ruby. Under the hood, Serverspec will indeed check that file permissions are set to 0644.

Fire away!

Okay, we now have our acceptance test and we’re ready to go. Let’s take it step-by-step.

1. Install gems required by Puppet module
If you Puppet module depends on specific gems, now it’s the time to install them using the following command:

$ pdk bundle install

2. provision the Docker container

$ pdk bundle exec rake 'litmus:provision[docker, litmusimage/debian:11]'

On this occasion, we’ll use the Debian 11 image. We recommend using the litmusimage Docker images built by Puppet Labs, because they are available in various Linux distribution flavors, and they come with preinstalled Systemd and OpenSSH server. The latter is required because Litmus uses SSH as a communication protocol. On the other hand, Systemd is required if you intend to emulate regular service installs within a Docker container.

3. Install Puppet agent on the target system

$ pdk bundle exec rake litmus:install_agent

But, if you’d like to perform tests with a different Puppet agent version, you can run the following command instead:

$ pdk bundle exec rake 'litmus:install_agent[puppet6]

If you’d like to verify that we have install Puppet agent correctly on target system, you can do that by executing

$ pdk bundle exec bolt command run 'puppet --version' --targets localhost:2222 --inventoryfile spec/fixtures/litmus_inventory.yaml

4. Install Puppet module on the target system

$ pdk bundle exec rake litmus:install_module

This command will archive your module and its dependecies and install them on the target system. You can verify that we have install module correctly by executing the command below, and reviewing its output:

$ pdk bundle exec bolt command run 'puppet module list' --targets localhost:2222 -i spec/fixtures/litmus_inventory.yaml

5. Run Puppet acceptance tests

$ pdk bundle exec rake litmus:acceptance:parallel

If all tests were successful, your output should look similar to this one:

pdk (INFO): Using Ruby 2.7.3
pdk (INFO): Using Puppet 7.12.0
┌ [✔] Running against 1 targets.
└── [✔] localhost:2222, litmusimage/debian:11
================
localhost:2222, litmusimage/debian:11
......

Finished in 5.18 seconds (files took 0.56565 seconds to load)
6 examples, 0 failures

pid 20130 exit 0
Successful on 1 nodes: ["localhost:2222, litmusimage/debian:11"]
Checking connectivity for ["localhost:2222"]
Connectivity check PASSED.

If we were to test that the file has permissions set to 1644, the test would fail and we’d get some pointers on what exactly failed.

pdk (INFO): Using Ruby 2.7.3
pdk (INFO): Using Puppet 7.12.0
┌ [✖] Running against 1 targets.
└── [✖] localhost:2222, litmusimage/debian:11
================
localhost:2222, litmusimage/debian:11
..F…

Failures:

1) rsyslog class with default parameters File "/etc/rsyslog.d/10_modules.conf" is expected to be mode 1644
  On host `localhost:2222'
  Failure/Error: it { is_expected.to be_mode 1644 }
    expected `File "/etc/rsyslog.d/10_modules.conf".mode?(1644)` to be truthy, got false
    /bin/sh -c stat\ -c\ \%a\ /etc/rsyslog.d/10_modules.conf\ \|\ grep\ --\ \\\^1644\\\

  # ./spec/acceptance/rsyslog_spec.rb:20:in `block (4 levels) in <top (required)>'

Finished in 4.96 seconds (files took 0.57632 seconds to load)
6 examples, 1 failure

Failed examples:

rspec ./spec/acceptance/rsyslog_spec.rb:20 # rsyslog class with default parameters File "/etc/rsyslog.d/10_modules.conf" is expected to be mode 1644

pid 20502 exit 1
Failed on 1 nodes: ["localhost:2222, litmusimage/debian:11"]
Checking connectivity for ["localhost:2222"]
Connectivity check PASSED.

6. Clean up

$ pdk bundle exec rake litmus:tear_down

Luckily, Litmus will clean everything up once we finish testing. With one simple command, we can remove the provisioned Docker container and the generated fixtures.

What’s next for Puppet acceptance testing?

We’ve just scratched the surface. To improve your acceptance tests, we recommend familiarizing yourself with Serverspec by reading the official documentation and peeking inside other people’s Puppet modules to see how they implemented their acceptance tests. We also recommend automating the process of acceptance testing and integrating it with your CI/CD pipeline.

That’s all folks! Feel free to share your tips and experiences in the comments, we’d love to hear them! ?