Taskotron Development Guide¶
So, you’re interested in helping out with libtaskotron? Great, this will help get you started on that.
General information¶
Libtaskotron is written mostly in Python.
Source code, bugs, tasks¶
The source code for libtaskotron is available at: https://pagure.io/taskotron/libtaskotron
This project is also used to track bugs and tasks. If you submit patches, please use the process of Submitting code.
Installing a Development Environment¶
Consider whether you really need a libtaskotron development environment. Maybe you simply want to develop tests that can be executed by libtaskotron runner, or you want to use the libtaskotron Python library in them? In that case, please read Running Taskotron Tasks and Writing Taskotron Tasks. If you really want to develop libtaskotron itself, please continue.
For the moment, libtaskotron can’t be fully installed by either pip
or
rpm
and needs a bit of both for now.
On your Fedora system, install the necessary packages:
sudo dnf install \
createrepo \
gcc \
git \
libtaskotron-config \
python3-doit \
python2-hawkey \
python3-hawkey \
python2-koji \
python3-koji \
python2-libvirt \
python3-libvirt \
python2-pip \
python3-pip \
python3-progressbar \
python2-rpm \
python3-rpm \
python2-rpmfluff \
python3-rpmfluff \
python3-tox \
python2-virtualenv \
python3-virtualenv \
rpm-build \
rsync \
testcloud
If you have not yet cloned the repository, do it now:
git clone https://pagure.io/taskotron/libtaskotron.git
cd libtaskotron
Then, set up the virtualenv:
virtualenv --system-site-packages env_taskotron
source env_taskotron/bin/activate
pip install -r requirements.txt
If you encounter any installation issues, it’s possible that you don’t have
gcc
and necessary C development headers installed to compile C extensions
from PyPI. Either install those based on the error messages, or install
the necessary packages directly to your system. See requirements.txt
to
learn how.
Finally, you should install libtaskotron in editable mode. This way you don’t need to reinstall the project every time you make some changes to it, the code changes are reflected immediately:
pip install -e .
Before running any task, you also need to manually create a few required
directories. First, create a taskotron
group if you don’t have it already,
and add your user to it (you’ll need to re-login afterwards):
getent group taskotron || sudo groupadd taskotron
sudo usermod -aG taskotron <user>
Now create the directories with proper permissions:
sudo install -d -m 775 -g taskotron /var/tmp/taskotron /var/log/taskotron \
/var/cache/taskotron /var/lib/taskotron /var/lib/taskotron/artifacts \
/var/lib/taskotron/images
Configuration¶
The libtaskotron-config
package installs config files with default values
into /etc/taskotron
. If you need to change those default values, you can
either change the files in /etc/taskotron
or you can create config files
inside your checkout with:
cp conf/taskotron.yaml.example conf/taskotron.yaml
cp conf/yumrepoinfo.conf.example conf/yumrepoinfo.conf
The configuration files in conf/
take precedence over anything in /etc
,
so make sure that you’re editing the correct file if you create local copies.
In the development environment, it’s also useful to have taskotron-generated
files automatically cleaned up, so that they don’t occupy disk space in vain.
There is a tmpfiles.d template prepared for you, look into conf/tmpfiles.d
.
Running the Test Suite¶
We place a high value on having decent test coverage for the libtaskotron code. In general, tests are written using pytest and are broken up into two types:
- Unit Tests test the core logic of code. They do not touch the filesystem or interact with any network resources. The unit tests should run very quickly
- Functional Tests are a set of more comprehensive tests which are allowed to touch the filesystem and/or interact with networked resources. Functional tests are often much slower than unit tests but they offer coverage which is not present in the unit tests.
To run the unit tests, execute this in the checkout directory:
tox
The testsuite is seamlessly executed in both Python 2.7 and 3.6 environments,
as defined in tox.ini
.
A nice HTML-based representation is available if you add --cov-report=html
command line parameter.
If you write new code, be sure to run this to see whether the code is sufficiently covered by the tests.
Submitting code¶
Libtaskotron follows the gitflow branching model and with the exception of
hotfixes, all changes made should be against the develop
branch.
If you want to use the gitflow plugin for git to make this process more
user-friendly, simply install the gitflow
package.
Start a new feature¶
To start work on a new feature, use:
git checkout -b feature/XXX-short-description develop
or if you want to use gitflow, use:
git flow feature start XXX-short-description
where XXX
is the issue number in Pagure and short-description
is a
short, human understandable description of the change contained in this branch.
In general, short reviews are better than long reviews. If you can, please break up features into chunks that are smaller and easier to manage.
Submitting code for review¶
Make sure to run all unit and functional tests before submitting code for review. Any code that causes test failure will receive requests to fix the offending code or will be rejected. See Running the Test Suite for information on running unit and functional tests.
Updating code reviews¶
There will often be requests for changes to the code submitted for review. Once
the requested changes have been made in your feature branch, commit them and
make sure that your branch is still up to date with respect to origin/develop
.
Pushing code¶
Note
Changes to git are very difficult to undo or cleanup once they have been pushed to a central repository. If you are not comfortable going through the process listed here, please ask for help.
If you run into any problems before pushing code to origin, please ask for help before pushing.
Once the review has been accepted, the code needs to be merged into develop
and pushed to origin
.
Make sure that your local develop
branch is up-to-date with origin/develop
before starting the merge process, else messy commits and merges may ensue.
Once develop
is up-to-date, the basic workflow to use is:
git checkout feature/XXX-some-feature
git rebase develop
To merge the code into develop, use one of two commands. If the feature can be reasonably expressed in one commit (most features), use:
git flow feature finish --squash XXX-some-feature
Else, if the feature is larger and should cover multiple commits (less common), use:
git flow feature finish XXX-some-feature
After merging the code, please inspect git commit description and make it prettier if needed. Groups of commits should at least have a short description of their content and a link to the issue in Pagure. Once the feature is ready, push to origin:
git push origin develop
Documentation¶
The documentation for libtaskotron is made up of several sources which are combined together into a single document using Sphinx.
Most of the libtaskotron docs are written in reStructuredText (reST) and unless otherwise mentioned, follow the Python documentation style guide.
Building documentation¶
The documentation is easy to build once deps are installed. Have the virtualenv active and run:
doit builddocs
Docstrings¶
Docstrings should be formatted using a reST-like format which Sphinx’s autodoc extension can easily render into HTML.
Sphinx has several built-in info fields which should be used to document function/method arguments and return data. Read the Sphinx documentation on info fields.
reST documentation¶
The majority of libtaskotron’s documentation is in the form of reST files which
live in the docs/source
directory in the git repository.
For more information see:
ReST headings¶
We use the suggested reST headers as outlined in the Python documentation style guide on section headers.
Build Automation¶
There are several development related tasks which are at least somewhat automated using doit.
You can see a list of available tasks and a short description of those tasks
by running doit list
. Some of the available tasks are:
buildsrpm
takes a snapshot of current git repo and uses the in-repo spec file to build a matching srpm in thebuilds/<version>
directory. Note that if a snapshot already exists for a given version, a new snapshot will not be generated until the existing one is deleted.chainbuild
uses mockchain and the existing COPR repo to build a noarch binary rpm from the latest srpm.builddocs
builds documentation from current git sources
By default, the tool is pretty quiet but if you would like to see more verbose
output, add --verbosity 2
to the doit command and all stdout/stderr output
will be shown.