Testing Docker Images

In Taskotron, we support two approaches on how to test Docker images: shell script or pytest.

Shell script

Shell scripts are commonly used in testing, Docker official images have checks written in bash and we assume that shell script approach will be the default choice.

Here’s an example of how a docker check in shell often looks like. The script makes sure that docker service is running, that image is pulled from repository and then runs two checks. Finally, the container is stopped and removed.

#!/bin/bash

# exit on any error
set -eo pipefail

# setup
systemctl start docker
docker pull httpd
container=`docker run -d -p 80:80 httpd`
# wait for container
sleep 1s

# test 1
output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'`
[ "$output" != "" ]

# test 2
curl localhost

# teardown
docker rm -f $container

For running this check in Taskotron, we need slight adjustments. We will no longer crash the whole script on any individual command error, instead we will check their output/exit code and generate results accordingly (an alternative is to put parts of code into separate shell scripts and run them in a subshell). We will check a particular docker image version, instead of the latest one, and identify it in task results. We will save the results in a file (which is later passed to a reporting directive). In order to save task results easily, we’ll use taskotron_result, a tool that generates Task Result Format. Consecutive usage of taskotron_result will append another result to the specified output file.

#!/bin/bash

# test item is passed from task formula and is in "name:tag" format, e.g. "httpd:2.4"
ITEM=$1
# where to save results, again passed from task formula
OUTFILE=$2
# how to name our task results
CHECKNAME="httpd_sanity"

# setup
systemctl start docker
docker pull $ITEM
container=`docker run -d -p 80:80 $ITEM`
# wait for container
sleep 1s

# test 1
output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'`
outcome1=$([ "$output" != "" ] && echo "PASSED" || echo "FAILED")
taskotron_result --file $OUTFILE --item $ITEM --report_type docker_image --outcome $outcome1 \
  --checkname "$CHECKNAME.netstat"

# test 2
curl localhost
outcome2=$([ "$?" == 0 ] && echo "PASSED" || echo "FAILED")
taskotron_result --file $OUTFILE --item $ITEM --report_type docker_image --outcome $outcome2 \
  --checkname "$CHECKNAME.curl"

# teardown
docker rm -f $container

Output file now contains this ResultYAML output:

results:
  - item: <item from command line>
    type: docker_image
    checkname: httpd_sanity.netstat
    outcome: PASSED
  - item: <item from command line>
    type: docker_image
    checkname: httpd_sanity.curl
    outcome: PASSED

We’ll use this task formula to execute this task in Taskotron:

name: httpd_sanity

environment:
  rpm:
    - docker

actions:
  - shell:
      - [bash, test.sh, '${item}', '${workdir}/test_output']

  - resultsdb:
      file: ${workdir}/test_output

Pytest

Pytest provides richer coding experience (e.g. fixtures, setup methods, teardown methods etc.) and there are projects that already use pytest for docker image testing. For ease of reusability, Taskotron can run test suites written in pytest. For seamless integration we support xUnit format to propagate check results further.

Example of the aforementioned check, now written in Python.

#!/usr/bin/env python

from time import sleep
import subprocess

class TestDockerHttpdSimple(object):
    def setup_method(self, test_method):
        self.container = subprocess.check_output(['docker', 'run', '-d', '-p', '80:80', 'httpd'])
        sleep(1) # wait for container

    def teardown_method(self, test_method):
        subprocess.check_call(['docker', 'rm', '-f', self.container])

    def test_listening_port80(self):
        out = subprocess.check_output("netstat -lnt | awk '$6 == \"LISTEN\" && $4 ~ \"80$\"'",
                                      shell=True)
        assert out != ""

    def test_working_http(self):
        subprocess.check_call(['curl', 'localhost'])

We’ll use this task formula to execute this task in Taskotron:

name: httpd_sanity

environment:
  rpm:
    - docker
    - python2-pytest

actions:
  - shell:
      - systemctl start docker
      - [docker, pull, '${item}']
      - [py.test, dockertest.py, '--junitxml=${artifactsdir}/output.xml']

  - xunit:
      file: ${artifactsdir}/output.xml
      aggregation: none
    export: test_results

  - resultsdb:
      results: ${test_results}

Other possibilities

Taskotron is capable of running arbitrary code (e.g. via bash directive) in any language. If you cannot choose between shell script and pytest, you can always write your docker tests in any other language you prefer. Just make sure that the test results are in format we understand – Task Result Format or xUnit.