Task Formula Format

One of the core bits of Taskotron is the formula yaml file used to describe tasks to be run. As Taskotron is still a very young project, some components of the task formulae are not yet used by the task runner but are useful to human readers of the task. Those sections will be noted and those notes updated as things change.

The documentation here is a description of how things are implemented but are a little light on the practical creation of tasks. Writing Taskotron Tasks and some existing tasks are also good references:

Task description

Metadata

Some metadata is required to be in the task formula:

  • task name
  • description
  • maintainer
---
name: rpmlint
desc: download specified rpms and run rpmlint on them
maintainer: tflink

Note

Please use a FAS account name for the maintainer

Input

Most tasks require some form of input. The required input must be declared in order to verify that the task can be run. These variables are populated from the runtask command line parameters.

input:
    args:
        - koji_build
        - arch

Valid args include:

  • Item type name. Variable name matches --type from the command line, variable value is taken from --item parameter. See the full list of available item types and their example values in libtaskotron.main.

    Examples: koji_build with value cups-2.2.0-9.fc25, or bodhi_update with value FEDORA-2017-04459ef8cf, or koji_tag with value f25-updates-testing-pending.

  • arch. Populated from the --arch parameter. Value examples: x86_64, i386, noarch.

Dependencies

A task may also require the presence of other code to support execution. Those dependencies are specified as part of the environment description. It is recommended to only use package names or file paths.

environment:
    rpm:
        - python-solv
        - python-librepo
        - /usr/bin/xz

Note

You might also use advanced syntax like package version comparisons or group names (as long as it’s supported by dnf install), but such tasks might not work properly when executed under a non-root user.

Note

Future versions will also support specifying dependencies in other formats than RPM.

Environment

Note

This is only applicable when running in the disposable-minion mode, which is the default on our deployment.

By default, runtime environment (distro, release, arch and flavor) is devised from the item under test for Koji builds and Bodhi updates. This affects the base-image used to spawn the new virtual machine in the disposable-minion mode.

When the environment can not be devised, either because of unsupported item_type or unrecognized value of the item, default environment (defined in configuration by the default_disposable_{distro, release, arch, flavor} options) is used.

For example, runtask.py -i xchatex-2.8.8-21.fc20 -t koji_build -a x86_64 rpmlint.yml would use x86_64 (arch) Fedora (distro) 20 (release).

On top of that, environment requirements can be specified in the Task Formula, although we do not think that these should be broadly used, there might be a valid reason to, for example, aways run a task on x86_64 disregarding the item under test.

All of the distro, release, arch and flavor can be defined in the environment section of the task formula, and take precedence over the auto-devised values.

To specify environment in the formula use the environment section like this:

environment:
    distro: fedora
    release: 24
    arch: x86_64

Note

The task execution fails, when there is no base image fitting the distro/release/arch/flavor requirements.

Task execution

Every task is defined in the actions block of the formula yaml file. The task consists of one or more actions which consist of a name, directive and optional export of data.

actions:
    - name: using directivename for something
      directivename:
          arg1: value1
          arg2: ${some_variable}
      export: somestep_output

Variable storage and syntax

The formula yaml file uses modified string.Template syntax for variables. As in the standard, you can use $variable or ${variable} format, and you need to make the dollar sign doubled if you want to include it literally ($$not_a_variable).

On top of that, you can use a dot (.) to access attributes of a variable. For the sake of convenience, ${foo.bar} does the following things on the Python layer:

  • check for an item bar in foo (foo.__getitem__('bar'))
  • if there is not, check for an attribute bar in foo (getattr(foo, 'bar'))
  • if there is not, raises TaskotronYamlError
actions:
    - name: first step
      directive_one:
          arg1: value1
      export: firststep_output

    - name: second step
      directive_two:
          arg1: ${firststep_output.bar}

Variables can be created during task execution or provided by default by the task runner.

Provided variables

The task runner provides the following variables without a need for them to be explicitly specified in Input section:

  • type denotes the type of the item being tested (as described in Input), so e.g. a string koji_build, koji_tag, etc.
  • taskdir is the path where the task itself is stored, after it is checked out from git. If you need to run any helper scripts or load some libraries you store in your git alongside your task, this is where you find the files.
  • workdir is often used by Directives to store output files by default (downloaded RPMs, etc). The task can also use it for temporary files during its execution.
  • artifactsdir contains a path to the artifacts directory. It is used by tasks to store their various outputs like logs, archives, images or any other output. All stored artifacts are then exposed and accessible via HTTP after the task is finished.
  • checkname contains a task name of the task currently being run by the runner.
  • namespace is the namespace this task wants to report to, as defined in the task Metadata.
  • jobid is used primarially for reporting to render proper log and job urls. It can be specified on the command line but isn’t required for local execution. If it is not explicitly provided at execution time, a sane default value will be provided by the runner.
  • uuid is an identification of a run which is unique across all parts of Taskotron. It can be specified on the command line. If it is not explicitly provided at execution time, a sane default value will be provided by the runner.

Created variables

There are many situations in which the output from one step is used as input for another step which is executed later. In order for a step’s output to be used later, it must first be exported.

actions:
    - name: first step
      directive_one:
          arg1: value1
      export: firststep_output

    - name: second step
      directive_two:
          arg1: ${firststep_output}

In this example, the output from first step is stored as firststep_output and later used by second step. Once the first step is executed, its output is stored as a variable using the name following export:.

Directives

You can see the full list of available directives that you can use for your actions in Directives Modules.