By Piotr Gaczkowski, IOD expert
Almost every modern programming language comes with some testing support built-in. Yet, there aren’t so many frameworks for testing servers and deployments. “Wait a second,” you might be thinking right now. “Servers and deployments aren’t code. They should be provisioned by IT operations, not tested along the application lifecycle, right?”

Unfortunately, the answer is, “No.” What used to be a separate job left to dedicated specialists is evolving in an agile manner. When development happens fast, there is no room for separate stages of development, testing, and deployment.

Deployment as Code

When you take a DevOps approach, deployment becomes a part of an application’s code. But is it tested in a similar way? Most of the time it’s not. One reason is the lack of appropriate tooling. Most developers could name several ways of implementing unit tests or end-to-end tests, but when it comes to testing deployments, most fall back to some standard UNIX tool like ssh, netcat, and sed.
No matter which of the many tools you use for deployment — Packer and Terraform for infrastructure provisioning; Chef, Puppet, Ansible, SaltStack, and StackStorm for configuration management; and Docker Swarm Mode, Kubernetes, or Nomad for workload orchestration — their results should be tested.
Of course, end-to-end tests may succeed in discovering some problems resulting from improper deployment, but they are unreliable. Plus, analyzing end-to-end tests is really cumbersome as the root cause of an error can lie literally anywhere: from one end to the other. Checking the assertions earlier on can lead to discovery of bugs even before e2e tests are executed.
Serverspec, however, shows you the exact low level parts of your deployment that fail to behave as intended.

Using Serverspec

Part of Serverspec’s origin is woven into its name. It evolved from Rspec, a Ruby testing framework, so anyone familiar with the language should quickly be able to write appropriate tests. Even though Ruby has many followers in the web development space, non-practitioners often find the syntax to be rather quirky. Don’t take my word for it, have a look yourself:

describe package('git'), :if => os[:family] == 'debian' do
  it { should be_installed.with_version('1:2.1.4') }
end

describe file('/etc/letsencrypt/config/example.com.conf') do
  it { should be_file }
  it { should be_owned_by 'letsencrypt' }
  it { should be_mode 600 }
  it { should contain('example.com')
end

Does it look weird? Maybe. But you can get the meaning behind this Domain Specific Language, right? Most probably you do and that’s the point!
Test cases can cover behavior of the target machine, as well as facts about it. The official documentation shows the possible data you can retrieve and assert against. Supported operating systems include the popular Linux distros, Mac OS X, FreeBSD, OpenBSD, AIX, Solaris, SmartOS, and Microsoft Windows. The host OS can either be detected automatically or specified in tests manually. Note that not all facts and features are available for every supported OS.
Probably the weakest part are the available resources to test against. They certainly pale in comparison to  Ansible modules. For example, the port resource allows referencing only a single port, which might be inconvenient if we wanted to verify that all ports apart from 22 and 80 were not listening.
These shortcomings mean Serverspec is not yet a one-stop shop for all your testing needs.  Love it or hate it, though, the DSL used for describing cases is still much more readable than most shell scripts.

Maintaining Equilibrium

Even though having 100% code coverage may look nice on paper, in the real world it is often impractical. Same goes for trying to cover similar code paths through different layers of testing. It’s not a bad idea to verify the assumption first on a unit level and then on an integration level, but checking the same thing again and again in different test-suites is counterproductive. When in doubt, I recommend sanity over zealousness.
In my opinion, Serverspec is a nice addition to the tools for Continuous Testing, but only when used properly. Check the services running on a target machine, find out if only the necessary ports are open to the world, and verify that the landing page actually opens. Do not try to rewrite e2e tests, unit tests or integration tests here as this is not where they belong. Tests should reside along their source code, and Serverspec’s related code is the deployment code that you maintain.

Summary

As Serverspec is mostly a standalone tool, it can also be used to verify manual deployments or perform scheduled audits. (To check if all company servers adhere to the standards, for example.) While this particular use case has not been covered in this article, you are free to experiment according to your needs. Keep in mind that the tests are designed to be run against a single machine or a container. If you want to check an entire cluster or multiple microservices, you need to coordinate the test runs. You can find some suggestions in Advanced Tips.
Leave a comment below after you try it out for yourself. I’m curious to know if you agree with me that you should be using Serverspec for deployments.
***
Piotr is an automation enthusiast who aims to replace all repeatable tasks with code. He exercises his urges working as a DevOps Enforcement Agent. Never without headphones around.
He is also part of the IOD family of experts. If you’re interested in blogging about the IT topics YOU are an expert in, leave a comment or join us by clicking on the yellow + sign featured throughout this web site.