From a79ba59b9d4edc5af67b73125940c731194213cb Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Nov 10 2017 10:20:39 +0000 Subject: [PATCH 1/2] Fill in the module testing section --- diff --git a/source/development/building-modules/installation-dependency-issues.rst b/source/development/building-modules/installation-dependency-issues.rst index 7e47591..9b5f180 100644 --- a/source/development/building-modules/installation-dependency-issues.rst +++ b/source/development/building-modules/installation-dependency-issues.rst @@ -1,4 +1,6 @@ +.. _resolving-dependency-issues: + Examining dependency issues during the installation of modules ============================================================== diff --git a/source/development/building-modules/testing.rst b/source/development/building-modules/testing.rst index 1a775eb..bb28205 100644 --- a/source/development/building-modules/testing.rst +++ b/source/development/building-modules/testing.rst @@ -1,10 +1,112 @@ Testing modular content in Fedora ================================= -We use the `Meta Test Family `__ -project to test modules and their resulting artifacts for Fedora. +Automated Testing +----------------- -It allows writing tests for the RPM packages that are built as part of a -module, or a container created from these packages. When a module is built in -Fedora infrastructure and MTF tests are available in its repository, they will -be run afterwards by `Taskotron `__. +As described in the +`Continuous Integration `__ page on the +Fedora wiki, Fedora is currently working to improve the level of automated +test coverage for the entire distribution. + + +### Modular compose tests + +A base set of automated tests check that the core set of modules defined in +Fedora's system profile can all be installed and updated. + +These tests are written as Ansible playbooks, and can be found in the +`compose-tests repository `__. + +These tests are executed in the CentOS CI infrastructure: + +* `Last successful build `__ +* `Latest build `__ + + +### Per-module tests in dist-git + +Modules may also define their own individual automated tests, in the ``tests/`` +subdirectory of their respective +`dist-git module repositories `__ + +The `Meta Test Family `__ +project is used to define test cases for whole modules and their resulting RPMs +and container images. + +When a module is built in Fedora's infrastructure and MTF tests are available +in its repository, they will be run afterwards by +`Taskotron `__. + + +### Per-package tests + +Independently of the modular build process, individual packages may also +define their own automated tests, both as part of the ``%check`` section in +their RPM spec file, as well as in the ``tests/`` subdirectory of their +respective +`dist-git RPM repositories `__ + + +Interactive Testing +------------------- + +If the automated tests are failing, or if a new module is being built for the +first time, it is useful to be able to check modules interactively before they +make their way into an official compose. + +The interactive testing guide below assumes that ``fedpkg module-build`` is +succeeding - if that isn't the case, then the module and package build logs +should provide details as to what is going wrong. + +### Boltron testing images + +The original Fedora 26 Boltron prototype lives on as a set of +`test container images `__, +which build atop the Fedora modular composes, but include some additional +utililties to help out with testing modules prior to their release in Fedora. + +Two variants of the image are provided, a ``boltron-27`` image with module +definitions from the Fedora 27 Modular Server release, and a +``boltron-bikeshed`` with the latest module definitions from the module build +service that have passed their self-tests in Taskotron:: + + $ docker run --rm -it jamesantill/boltron-27 bash + $ docker run --rm -it jamesantill/boltron-bikeshed bash + +See the ``README`` in the image source repository for more information on +the available utilities. + + +### Enabling access to unreleased modules + +Module releases may take a day or more to become available in the default image, +so the Boltron image provides a helper script to download built modules directly +from the Module Build Service and incorporate them into the currently running +image. For example: + + # /LOCAL.sh postgresql:9.6:20171018083530 + +This will download the binary artifacts for that particular build of the +PostreSQL 9.6 stream, set up a local repository for them (including the module +metadata), enable that repository, and then install the module with its default +profile. + +A non-default install profile can be requested by appending the profile name to +the build identifer, separated by a ``/``: + + # /LOCAL.sh postgresql:9.6:20171018083530/client + + +### Finding module build identifiers + +The list of currently build modules and their identifiers is available +at http://modularity.fedorainfracloud.org/modularity/mbs/. + + +### Resolving dependency issues + +The most common problems that are encountered despite a successful build and +passing integration tests are dependency resolution issues when attempting to +install a module. The :ref:`resolving-dependency-issues` section provides more +guidance on dealing with such situations. From 8e97de78be1e7ecc838915dd4fbe0539579194e9 Mon Sep 17 00:00:00 2001 From: Nick Coghlan Date: Nov 10 2017 10:31:17 +0000 Subject: [PATCH 2/2] Use ReST headings, not Markdown ones --- diff --git a/source/development/building-modules/testing.rst b/source/development/building-modules/testing.rst index bb28205..212bc53 100644 --- a/source/development/building-modules/testing.rst +++ b/source/development/building-modules/testing.rst @@ -10,7 +10,8 @@ Fedora wiki, Fedora is currently working to improve the level of automated test coverage for the entire distribution. -### Modular compose tests +Modular compose tests +~~~~~~~~~~~~~~~~~~~~~ A base set of automated tests check that the core set of modules defined in Fedora's system profile can all be installed and updated. @@ -24,7 +25,8 @@ These tests are executed in the CentOS CI infrastructure: * `Latest build `__ -### Per-module tests in dist-git +Per-module tests in dist-git +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Modules may also define their own individual automated tests, in the ``tests/`` subdirectory of their respective @@ -39,7 +41,8 @@ in its repository, they will be run afterwards by `Taskotron `__. -### Per-package tests +Per-package tests +~~~~~~~~~~~~~~~~~ Independently of the modular build process, individual packages may also define their own automated tests, both as part of the ``%check`` section in @@ -59,7 +62,8 @@ The interactive testing guide below assumes that ``fedpkg module-build`` is succeeding - if that isn't the case, then the module and package build logs should provide details as to what is going wrong. -### Boltron testing images +Boltron testing images +~~~~~~~~~~~~~~~~~~~~~~ The original Fedora 26 Boltron prototype lives on as a set of `test container images `__, @@ -78,7 +82,8 @@ See the ``README`` in the image source repository for more information on the available utilities. -### Enabling access to unreleased modules +Enabling access to unreleased modules +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Module releases may take a day or more to become available in the default image, so the Boltron image provides a helper script to download built modules directly @@ -98,13 +103,15 @@ the build identifer, separated by a ``/``: # /LOCAL.sh postgresql:9.6:20171018083530/client -### Finding module build identifiers +Finding module build identifiers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The list of currently build modules and their identifiers is available at http://modularity.fedorainfracloud.org/modularity/mbs/. -### Resolving dependency issues +Resolving dependency issues +~~~~~~~~~~~~~~~~~~~~~~~~~~~ The most common problems that are encountered despite a successful build and passing integration tests are dependency resolution issues when attempting to