Writing Taskotron Tasks

While the eventual target for most tasks will be to run in a production system, one of libtaskotron’s features is that it can be run without the overhead of a full production-like system.

To learn more about what can be done with taskotron, see What can I do with Taskotron?.

In order to write tasks, you must install libtaskotron on your local machine if you aren’t running from git. Also, see Taskotron Quick Start if you haven’t already.

Examples

examplebodhi

examplebodhi task git repository

The examplebodhi task is a trivial example which takes an update and determines a PASS/FAIL results in a pseudo-random fashion. This task wasn’t designed to be used in production but is useful as a starting point and for development purposes.

rpmlint

rpmlint task git repository

rpmlint is the simplest of the production tasks. It takes a koji build and runs rpmlint on it, reporting results as result-YAML.

Creating a new task

While running rpmlint is an easy and trivial example, that doesn’t help with creating something new. To write a new task, you’ll need some information:

  • What type of object will the task be run against?
  • What will be the name of the task?
  • What are the dependencies?

The taskotron runner works on specially formatted YAML files (see Task Result Format for details) called formulae which describe the task that is to be executed. In addition to the task formula, any other code or resources needed for task execution should be accessible by the runner.

A directory layout could look like:

mytask/
    mytask.yml
    mytask.py
    someresource.txt

Note

Taskotron was designed to work with tasks stored in git repositories and while this is not strictly required for the local execution case, it is highly recommended that you limit tasks to one per directory.

Non-executed task information

For this example, let’s write a task that runs on bodhi IDs. Looking at the non-executed task data needed in mytask.yml:

---
name: mytask
desc: do something stupendous
maintainer: somefasuser

input:
    args:
        - arch
        - bodhi_id

environment:
    rpm:
        - rpm_that_task_uses

Executed task information

Let’s have this new task do the following:

  • download the rpms contained in the indicated bodhi update
  • run the function run_mytask in mytask.py using a list of the downloaded rpms as input
  • report the results to resultsdb
# this is a trivial example that doesn't do everything we want it to do, more
# details will be added later in the example
def run_mytask(rpmfiles, bodhi_id):
    print "Running mytask on %s" % bodhi_id
actions:
    # we can use bodhi_id and arch as variables here because they're defined
    # for us by the runner which also verifies that all required input listed
    # above is present before executing the task
    - name: download bodhi update
      bodhi:
          action: download
          update_id: ${bodhi_id}
          arch: ${arch}
      export: bodhi_downloads


    # notice how the third item under python is 'rpmfiles' and this matches
    # the arg name in mytask.py. Anything outside of 'file' and 'callable'
    # are passed in as kwargs to the specified python callable.
    - name: run mytask
      python:
          file: mytask.py
          callable: run_mytask
          rpmfiles: ${bodhi_downloads}
          bodhi_id: ${bodhi_id}
      export: mytask_output


    # this assumes that the output from mytask is valid result-YAML
    - name: report mytask results to resultsdb
      resultsdb:
          results: ${mytask_output}

We can execute this new task using:

$ runtask -i FEDORA-2016-8760e32d9b -t bodhi_id -a x86_64 ../task-mytask/mytask.yml

The output from this task should end with:

Running mytask on FEDORA-2016-8760e32d9b
[libtaskotron] 15:00:40 CRITICAL Traceback (most recent call last):
  File "runtask.py", line 10, in <module>
    main.main()
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/main.py", line 151, in main
    overlord.start()
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/overlord.py", line 95, in start
    runner.execute()
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 56, in execute
    self._run()
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 93, in _run
    self._do_actions()
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 131, in _do_actions
    self._do_single_action(action)
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 152, in _do_single_action
    self.arg_data)
  File "/home/mkrizek/devel/libtaskotron/libtaskotron/directives/resultsdb_directive.py", line 196, in process
    raise TaskotronDirectiveError("Failed to load 'results': %s"  % e.message)
TaskotronDirectiveError: Failed to load 'results': Failed to parse YAML contents: empty input

We’re not actually passing result-YAML output from our task, so let’s get that fixed.

Actually doing something

The Python code for mytask.py above doesn’t really do anything at all, so let’s make it do something:

  • Change the task to create a result from the input bodhi update id and list all of the rpms contained in that update as details.

Generating result-YAML from python tasks in Taskotron isn’t very difficult.

Our original task looked like:

# this is a trivial example that doesn't do everything we want it to do, more
# details will be added later in the example
def run_mytask(rpmfiles):
    for rpmfile in rpmfiles['downloaded_rpms']:
        print "Running mytask on %s" % rpmfile

To create valid result-YAML, we want to populate a libtaskotron.check.CheckDetail object and use libtaskotron.check.export_YAML() to generate valid result-YAML with the data required for reporting.

Note

If you can not use the libtaskotron.check.CheckDetail directly, then create the YAML output according to Task Result Format.

from libtaskotron import check

def run_mytask(rpmfiles, bodhi_id):
    """run through all passed in rpmfiles and emit result-YAML saying they all passed"""

    print "Running mytask on %s" % bodhi_id

    result = 'PASSED'
    note = 'from testing-pending to testing'
    detail = check.CheckDetail(bodhi_id, check.ReportType.BODHI_UPDATE, result, note)
    for rpmfile in rpmfiles['downloaded_rpms']:
        detail.store(rpmfile, printout=False)

    return check.export_YAML(detail)

Now if we run the task we get output that ends with:

[libtaskotron] 10:01:03 INFO    Reporting to ResultsDB is disabled. Once enabled, the following would get reported:
results:
- item: FEDORA-2016-8760e32d9b
  note: from testing-pending to testing
  outcome: PASSED
  type: bodhi_update

Working with CheckDetail objects

Check can have one of these outcomes (with increasing priority): PASSED, INFO, FAILED, NEEDS_INSPECTION, ABORTED and CRASHED. These are available as a list (where higher index means bigger priority) in CheckDetail.outcome_priority. You can set outcome during libtaskotron.check.CheckDetail creation, by setting CheckDetail.outcome directly or by using libtaskotron.check.CheckDetail.update_outcome() - it changes task outcome only if it has higher priority than current outcome.

libtaskotron.check.CheckDetail.store() is a convenience method if you receive some output gradually and you want to store it and also print it at the same time. You can also set check’s output by setting CheckDetail.output or in CheckDetail constructor.

import os
import random

from libtaskotron import check

def random_choice(_):
    return random.choice(['PASSED', 'FAILED', 'ABORTED'])

def run_mytask(rpmfiles, bodhi_id):
    """run through all passed in rpmfiles and emit result-YAML. randomly report failure or pass"""

    print "Running mytask on %s" % bodhi_id

    seed = random.random()
    random.seed(seed)

    results = [(rpmfile, random_choice(rpmfile)) for rpmfile in rpmfiles['downloaded_rpms']]

    detail = check.CheckDetail(bodhi_id, check.ReportType.BODHI_UPDATE)

    for rpmfile, outcome in results:
        # e.g. "xchat-1.2-3.fc20.x86_64.rpm: FAILED"
        detail.store('%s: %s' % (os.path.basename(rpmfile), outcome))
        detail.update_outcome(outcome)

    detail.note = "seed was %s" % seed
    detail.checkname = "mytask"

    return check.export_YAML(detail)

Running this code as python runtask.py -i tzdata-2014f-1.fc20 -t bodhi_id -a x86_64 ../task-mytask/mytask.yml ends with:

results:
  - item: tzdata-2014f-1.fc20
    outcome: ABORTED
    note: seed was 0.45189
    type: bodhi_update
    checkname: mytask

For more information about APIs and CheckDetail usage, see CheckDetail class.

Note

Reporting is disabled in the default configuration for Taskotron. While it is possible to set up a local resultsdb instance to check reporting, we want to make it easier than that and will update these docs when that easier method is available.

For now, if the resultsdb directive doesn’t throw exceptions, it’s reasonable to assume that the result-YAML output was constructed correctly and is reasonably valid.