Task Result Format¶
The required output format for tasks in Taskotron is YAML with particular values, which we call ResultYAML.
We recommend using libtaskotron
Python library or the provided
taskotron_result
script to create the YAML output. Read
Saving task results in ResultYAML format to see how. If you can not use it, then please adhere
to the specification described below.
Minimal version¶
If you don’t need anything fancy, then this is the minimalistic version of the output which, at the same time, is valid for results reporting in Taskotron:
results:
- item: xchat-0.5-1.fc20
type: koji_build
checkname: mytest
outcome: PASSED
You just need to provide the item
, type
, checkname
and outcome
values in the YAML block.
The outcome
field defines whether the check result was OK or not OK.
Values allowed at the moment, are PASSED
, INFO
, FAILED
, ERROR
,
WAIVED
, and NEEDS_INSPECTION
. Using other values will cause the
ResultsDB reporting to fail.
type
describes “what” was tested on a high level - whether it was a build
from Koji, DVD image, update in Bodhi, or something completely different. You
can see the supported list of items and their types in
libtaskotron.main
.
You can, of course, use any other value for the type
parameter, but if you
have a check for repositories, Koji builds or Bodhi updates, please stick to
the already defined ones, so the tools next in the chain will recognize your
results properly.
item
is a string representation of the specific piece, that was tested -
NVR for the koji_build
type, update-name or update-id for the
bodhi_update
, etc. Once again - you can set whatever value you think is
right, but sticking to the reasonable/obvious choices will ensure seamless
processing.
checkname
represents the test case identification. For development
purposes, it doesn’t really matter what you use. Once you want to have your
task run in production, the admins will suggest an appropriate checkname
for you. You can provide test case subresults by using dot as a nesting
separator.
Note
item
, type
, outcome
and checkname
are just some of the
“reserved” key-names in the YAML block. See Note on reserved field names.
Full version¶
Although the minimal version of the output is sufficient, you can use additional fields to provide more information:
results:
- item: tzdata-2014f-1.fc20
type: bodhi_update
outcome: ABORTED
artifact: upgradepath_details.html
checkname: dist.upgradepath
arch: x86_64
package: tzdata
- item: bluez-5.34-1.fc23
type: bodhi_update
outcome: PASSED
artifact: upgradepath_details.html
checkname: dist.upgradepath
arch: x86_64
package: bluez
- item: tzdata-2014f-1.fc20
type: bodhi_update
outcome: PASSED
artifact: depcheck_tzdata-2014f-1.fc20.txt
checkname: dist.depcheck
arch: x86_64
Going top-down, the new fields are artifact
, checkname
, note
,
arch
and package
.
artifact
contains the name of a file in the $artifactsdir
, that
contains detailed information about the check run. In this example the
upgradepath check produced an user-friendly report in HTML format, and the
author decided to link it as an artifact.
note
adds a short valuable information that should be easily visible in the
result overview.
arch
and package
are custom fields that do not have any special meaning
for the Taskotron tooling, but provide additional information about the check
results as such.
Note on reserved field names¶
The full list of fields with a pre-defined meaning is available at
libtaskotron.check.RESERVED_KEYS
.
Apart from those, you can add any number of custom key-value pairs (like the
package
in the above example), which will get stored in ResultsDB, and can
be then used for searching. Those fields will be specific for your task and you
can give them any meaning and use them for any purpose that you need.