From 092a928881023134d22294a933a417a4ac4dd16b Mon Sep 17 00:00:00 2001
From: ph0zzy <arthur-fayzullin@yandex.ru>
Date: Mar 11 2019 21:51:10 +0000
Subject: [PATCH 1/2] add initial data for tests page


---

diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc
index 420e55e..649d128 100644
--- a/modules/ROOT/nav.adoc
+++ b/modules/ROOT/nav.adoc
@@ -5,4 +5,5 @@
 * xref:standard-test-interface.adoc[Standard Test Interface]
 * xref:standard-test-roles.adoc[Standard Test Roles]
 * xref:examples.adoc[Examples]
+* xref:tests.adoc[Tests]
 * xref:faq.adoc[FAQ]
diff --git a/modules/ROOT/pages/tests.adoc b/modules/ROOT/pages/tests.adoc
new file mode 100644
index 0000000..e02c084
--- /dev/null
+++ b/modules/ROOT/pages/tests.adoc
@@ -0,0 +1,238 @@
+:toc:
+
+= Tests =
+
+[cols="1", options="header"]
+|===
+|Quick links to automated test reports
+
+|https://fedoraproject.org/wiki/CI/Tests/stat[stat]
+
+|https://fedoraproject.org/wiki/CI/Tests/new-stat[new-stat]
+
+|https://fedoraproject.org/wiki/CI/Tests/stat_atomic[stat atomic]
+
+|https://fedoraproject.org/wiki/CI/Tests/recent_builds[recent builds]
+
+|https://fedoraproject.org/wiki/CI/Tests/stat_everything_subset[stat everything subset]
+
+|https://fedoraproject.org/wiki/CI/Tests/stat_fedoraserver[stat fedoraserver]
+|===
+
+== Enabling ==
+
+Tests may be written in different ways, but are exposed and invoked in a standard way as defined by the xref:standard_test_interface.adoc[Standard Test Interface] directly in the package https://src.fedoraproject.org/projects/rpms/%2A[git repository].
+It is also possible to enable pipeline for the tests namespace, see [[CI/Share_Test_Code#Testing_Tests|Testing Tests]] for details.
+To start working on tests you can clone a package repo directly:
+
+ git clone https://src.fedoraproject.org/rpms/qrencode.git
+
+You can also use the `fedpkg` to clone the repo.
+See the [[Package_maintenance_guide|Package Maintenance Guide]] for more info about the tool:
+
+ fedpkg clone -a qrencode
+
+Tests are enabled by including the <code>tests.yml</code> file under the <code>tests</code> directory:
+
+ cd qrencode/tests
+ cat tests.yml
+
+Tests are wrapped or written as [http://docs.ansible.com/ansible/playbooks.html Ansible playbooks].
+Here is an example of a simple playbok which enables a single <code>smoke</code> test of the <code>qrencode</code> package:
+
+ - hosts: localhost
+   roles:
+   - role: standard-test-beakerlib
+     tags:
+     - classic
+     - container
+     - atomic
+     tests:
+     - smoke
+     required_packages:
+     - qrencode
+     - file
+
+Let's now briefly look at the playbook to see which variables are defined in order to enable the smoke test:
+
+* *role* — this test uses role <code>standard-test-beakerlib</code> from [[CI/Standard_Test_Roles|Standard Test Roles]] to run a BeakerLib test
+* *tags* — all three test subjects ([[CI/Standard_Test_Roles#Atomic|classic]] rpm, docker [[CI/Standard_Test_Roles#Container|container]] and [[CI/Standard_Test_Roles#Atomic|atomic]] host) are relevant for this test
+* *tests* — list of tests to be executed (here we have just a single smoke test)
+* *required_packages* — list of rpm packages required for test execution
+
+There may by multiple files ending in <code>.yml</code> in the <code>tests/</code> subdirectory and each of them can represent a test or a part of a test.
+All of them need to be included in the main <code>tests.yml</code> file.
+Let's have a look at the [https://src.fedoraproject.org/rpms/gzip/blob/master/f/tests gzip] example:
+
+ > fedpkg clone -a gzip
+ Cloning into 'gzip'...
+
+ > cd gzip/tests/
+ > ls
+ test-simple  test_simple.yml  tests.yml
+
+ > cat tests.yml
+ - include: test_simple.yml
+
+== Executing ==
+
+Before running tests make sure you have the following dependencies installed on your system:
+
+ dnf install ansible python2-dnf libselinux-python standard-test-roles
+
+Although some playbooks may function without sudo, tests are always invoked as root.
+The test itself may set up users and/or drop permissions if a part of that test.
+But in general be sure to be root when invoking tests.
+
+{{admon/important|Tests may modify or destroy your environment|It's recommended to use a virtual machine for testing to prevent any unwated changes performed by the test to your system.}}
+
+Running a test directly on the current system is easy:
+
+ ansible-playbook tests.yml
+
+To only run tests that are suited for classic systems installed by <code>yum</code> or <code>dnf</code> use the <code>--tags</code> argument:
+
+ ansible-playbook --tags=classic tests.yml
+
+See [[CI/Standard_Test_Roles|Standard Test Roles]] documentation for detailed instructions how to run tests for a specific [[CI/Standard_Test_Roles#Package|Rpm Package]], [[CI/Standard_Test_Roles#Container|Docker Container]] or [[CI/Standard_Test_Roles#Atomic|Atomic Host]].
+
+== Writing ==
+
+Test code itself can be stored directly in the dist-git (recommended as default) or fetched from another repository hosted in the Fedora infrastructure such as the [[CI/Share_Test_Code|Test Namespace]].
+The simplest way to add a new test is by using one of the existing [[CI/Standard_Test_Roles|Standard Test Roles]] which take care of many implementatin details.
+If you want to create a custom test follow instructions below.
+
+Once you've identified a dist-git repository you will be adding new tests to (above), you can start to write a new Ansible test.
+Create an [http://docs.ansible.com/ansible/latest/playbooks.html Ansible playbook] with a new name.
+Make sure the extension is <code>.yml</code>.
+Lets place the following example in <code>test_pid_1.yml</code> file.
+
+<pre>
+---
+- hosts: localhost
+  vars:
+  - artifacts: ./artifacts
+  tags:
+  - atomic
+  - classic
+  - container
+  tasks:
+  - name: Test block
+    block:
+      - name: Test that /proc/1 exists
+        shell: ls /proc > /tmp/test.log && grep -qw 1 /tmp/test.log
+
+    always:
+      - name: Pull out the artifacts
+        fetch:
+          dest: "{{ artifacts }}/"
+          src: "/tmp/test.log"
+          flat: yes
+</pre>
+
+All tests have an artifacts directory where they place their output.
+The testing or CI system that invokes the test will fill in this variable with a directory that it will archive.
+We ensure this directory exists in the test.
+
+By use of <code>tags</code> we note what kind of systems this test is suitable to run on.
+
+The <code>block</code> is the section that runs the actual test.
+In this example, we use a rather convoluted way of checking that PID 1 exists.
+However, by doing so, we place an extra test artifact in the artifacts directory.
+
+Lastly, we download the artifacts.
+Remember that the test is not always running on the same system that it was invoked on.
+Try running this example test against an [[CI/Standard_Test_Roles#Atomic|Atomic Host]] or [[CI/Standard_Test_Roles#Container|Docker Container]].
+It should pass.
+Try changing the <code>/proc/1</code> argument to another value, and the test should fail.
+
+You can use most of the Ansible techniques in your playbooks.
+Take a look at the [[CI/Standard_Test_Roles|Standard Test Roles]] for Ansible roles to make writing your tests easier.
+
+*Marking the test to be run*
+
+Just having a <code>.yml</code> file in the right directory doesn't yet mean it will be invoked.
+Make sure to reference or add it from a <code>tests.yml</code> playbook.
+This is the entry point that the testing or CI system will use to invoke all the tests for a given package.
+
+If the <code>tests.yml</code> file doesn't yet exist, create it.
+Lets continue with our above example and create a <code>tests.yml</code> with the following content:
+
+ - import_playbook: test_pid_1.yml
+
+You can now run this test with the standard commands above.
+
+See the [[CI/Quick_Start_Guide#Contributing|Quick Start Guide]] to get recommendations for contributing new tests.
+
+== Wrapping ==
+
+Let's say you have a script that runs a test.
+Its stdout and stderr is the test output, and an exit status of zero indicates success.
+Here's how we would wrap that test to be invoked.
+Lets say we have a simple script like in a file called <code>test-simple</code>
+
+ #!/bin/sh
+ set -ex
+ # exercise installed gzip/gunzip programs
+ echo "Bla" > bla.file
+ cp bla.file bla.file.orig
+ gzip bla.file
+ gunzip bla.file.gz
+ cmp bla.file bla.file.orig
+ rm bla.file bla.file.orig
+
+We can write an Ansible wrapper for this script like this in <code>test_simple.yml</code>:
+
+<pre>
+---
+- hosts: localhost
+  vars:
+  - artifacts: ./artifacts
+  tags:
+  - atomic
+  - classic
+  - container
+  remote_user: root
+  tasks:
+  - name: Install the test files
+    copy: src={{ item.file }} dest=/usr/local/bin/{{ item.dest }} mode=0755
+    with_items:
+    - {file: test-simple, dest: test-simple }
+
+  - name: Test block
+    block:
+      - name: Execute the tests
+        shell: exec > /tmp/test.log 2>&1 && /usr/local/bin/test-simple
+
+    always:
+      - name: Pull out the logs
+        fetch:
+          dest: "{{ artifacts }}/"
+          src: "/tmp/test.log"
+          flat: yes
+</pre>
+
+All tests have an artifacts directory where they place their output.
+The testing or CI system that invokes the test will fill in this variable with a directory that it will archive.
+We create ensure this directory exists in the test.
+
+The <code>block</code> is the section that runs the actual test.
+
+Lastly, we download the artifacts.
+Remember that the test is not always running on the same system that it was invoked on.
+
+
+If the <code>tests.yml</code> file doesn't yet exist, create it.
+Lets continue with our above example and create a <code>tests.yml</code> with the following content:
+
+ - import_playbook: test_simple.yml
+
+Try running this example test against an [[CI/Standard_Test_Roles#Atomic|Atomic Host]] or [[CI/Standard_Test_Roles#Container|Docker Container]].
+It should pass.
+
+See [[CI/Standard_Test_Roles|Standard Test Roles]] documentation for instructions how to wrap a [[CI/Standard_Test_Roles#BeakerLib|BeakerLib]] and [[CI/Standard_Test_Roles#RHTS|RHTS]] tests.
+
+See the [[CI/Quick_Start_Guide#Contributing|Quick Start Guide]] to get recommendations for contributing new tests.
+
+[[Category:FedoraAtomicCi]]
+

From e3e6b84e66237e4f75e1d6f812f5c0ac26dcee0b Mon Sep 17 00:00:00 2001
From: ph0zzy <arthur-fayzullin@yandex.ru>
Date: Mar 13 2019 21:05:07 +0000
Subject: [PATCH 2/2] change formating from md to adoc


---

diff --git a/modules/ROOT/pages/tests.adoc b/modules/ROOT/pages/tests.adoc
index e02c084..dd82c8c 100644
--- a/modules/ROOT/pages/tests.adoc
+++ b/modules/ROOT/pages/tests.adoc
@@ -21,48 +21,51 @@
 
 == Enabling ==
 
-Tests may be written in different ways, but are exposed and invoked in a standard way as defined by the xref:standard_test_interface.adoc[Standard Test Interface] directly in the package https://src.fedoraproject.org/projects/rpms/%2A[git repository].
-It is also possible to enable pipeline for the tests namespace, see [[CI/Share_Test_Code#Testing_Tests|Testing Tests]] for details.
+Tests may be written in different ways, but are exposed and invoked in a standard way as defined by the xref:standard-test-interface.adoc[Standard Test Interface] directly in the package https://src.fedoraproject.org/projects/rpms/%2A[git repository].
+It is also possible to enable pipeline for the tests namespace, see xref:share-test-code.adoc#_testing_tests[Testing Tests] for details.
 To start working on tests you can clone a package repo directly:
 
  git clone https://src.fedoraproject.org/rpms/qrencode.git
 
 You can also use the `fedpkg` to clone the repo.
-See the [[Package_maintenance_guide|Package Maintenance Guide]] for more info about the tool:
+See the xref:package-maintenance-guide.adoc[Package Maintenance Guide] for more info about the tool:
 
  fedpkg clone -a qrencode
 
-Tests are enabled by including the <code>tests.yml</code> file under the <code>tests</code> directory:
+Tests are enabled by including the `tests.yml` file under the `tests` directory:
 
  cd qrencode/tests
  cat tests.yml
 
-Tests are wrapped or written as [http://docs.ansible.com/ansible/playbooks.html Ansible playbooks].
-Here is an example of a simple playbok which enables a single <code>smoke</code> test of the <code>qrencode</code> package:
-
- - hosts: localhost
-   roles:
-   - role: standard-test-beakerlib
-     tags:
-     - classic
-     - container
-     - atomic
-     tests:
-     - smoke
-     required_packages:
-     - qrencode
-     - file
+Tests are wrapped or written as http://docs.ansible.com/ansible/playbooks.html[Ansible playbooks].
+Here is an example of a simple playbok which enables a single `smoke` test of the `qrencode` package:
+
+[source,ansible]
+----
+- hosts: localhost
+  roles:
+  - role: standard-test-beakerlib
+    tags:
+    - classic
+    - container
+    - atomic
+    tests:
+    - smoke
+    required_packages:
+    - qrencode
+    - file
+----
 
 Let's now briefly look at the playbook to see which variables are defined in order to enable the smoke test:
 
-* *role* — this test uses role <code>standard-test-beakerlib</code> from [[CI/Standard_Test_Roles|Standard Test Roles]] to run a BeakerLib test
-* *tags* — all three test subjects ([[CI/Standard_Test_Roles#Atomic|classic]] rpm, docker [[CI/Standard_Test_Roles#Container|container]] and [[CI/Standard_Test_Roles#Atomic|atomic]] host) are relevant for this test
+* *role* — this test uses role `standard-test-beakerlib` from xref:standard-test-roles.adoc[Standard Test Roles] to run a BeakerLib test
+* *tags* — all three test subjects (xref:standard-test-roles.adoc#_classic[classic] rpm, docker xref:standard-test-roles.adoc#_container[container] and xref:standard-test-roles.adoc#_atomic[atomic] host) are relevant for this test
 * *tests* — list of tests to be executed (here we have just a single smoke test)
 * *required_packages* — list of rpm packages required for test execution
 
-There may by multiple files ending in <code>.yml</code> in the <code>tests/</code> subdirectory and each of them can represent a test or a part of a test.
-All of them need to be included in the main <code>tests.yml</code> file.
-Let's have a look at the [https://src.fedoraproject.org/rpms/gzip/blob/master/f/tests gzip] example:
+There may by multiple files ending in `.yml` in the `tests/` subdirectory and each of them can represent a test or a part of a test.
+All of them need to be included in the main `tests.yml` file.
+Let's have a look at the https://src.fedoraproject.org/rpms/gzip/blob/master/f/tests[gzip] example:
 
  > fedpkg clone -a gzip
  Cloning into 'gzip'...
@@ -84,30 +87,35 @@ Although some playbooks may function without sudo, tests are always invoked as r
 The test itself may set up users and/or drop permissions if a part of that test.
 But in general be sure to be root when invoking tests.
 
-{{admon/important|Tests may modify or destroy your environment|It's recommended to use a virtual machine for testing to prevent any unwated changes performed by the test to your system.}}
+[NOTE]
+====
+*Tests may modify or destroy your environment* +
+It's recommended to use a virtual machine for testing to prevent any unwated changes performed by the test to your system.
+====
 
 Running a test directly on the current system is easy:
 
  ansible-playbook tests.yml
 
-To only run tests that are suited for classic systems installed by <code>yum</code> or <code>dnf</code> use the <code>--tags</code> argument:
+To only run tests that are suited for classic systems installed by `yum` or `dnf` use the `--tags` argument:
 
  ansible-playbook --tags=classic tests.yml
 
-See [[CI/Standard_Test_Roles|Standard Test Roles]] documentation for detailed instructions how to run tests for a specific [[CI/Standard_Test_Roles#Package|Rpm Package]], [[CI/Standard_Test_Roles#Container|Docker Container]] or [[CI/Standard_Test_Roles#Atomic|Atomic Host]].
+See xref:standard-test-roles.adoc[Standard Test Roles] documentation for detailed instructions how to run tests for a specific xref:standard-test-roles.adoc#_package[Rpm Package], xref:standard-test-roles.adoc#_container[Docker Container] or xref:standard-test-roles.adoc#_atomic[Atomic Host].
 
 == Writing ==
 
-Test code itself can be stored directly in the dist-git (recommended as default) or fetched from another repository hosted in the Fedora infrastructure such as the [[CI/Share_Test_Code|Test Namespace]].
-The simplest way to add a new test is by using one of the existing [[CI/Standard_Test_Roles|Standard Test Roles]] which take care of many implementatin details.
+Test code itself can be stored directly in the dist-git (recommended as default) or fetched from another repository hosted in the Fedora infrastructure such as the xref:share-test-code.adoc[Test Namespace].
+The simplest way to add a new test is by using one of the existing xref:standard-test-roles.adoc[Standard Test Roles] which take care of many implementatin details.
 If you want to create a custom test follow instructions below.
 
 Once you've identified a dist-git repository you will be adding new tests to (above), you can start to write a new Ansible test.
-Create an [http://docs.ansible.com/ansible/latest/playbooks.html Ansible playbook] with a new name.
-Make sure the extension is <code>.yml</code>.
-Lets place the following example in <code>test_pid_1.yml</code> file.
+Create an http://docs.ansible.com/ansible/latest/playbooks.html[Ansible playbook] with a new name.
+Make sure the extension is `.yml`.
+Lets place the following example in `test_pid_1.yml` file.
 
-<pre>
+[source,ansible]
+----
 ---
 - hosts: localhost
   vars:
@@ -128,48 +136,51 @@ Lets place the following example in <code>test_pid_1.yml</code> file.
           dest: "{{ artifacts }}/"
           src: "/tmp/test.log"
           flat: yes
-</pre>
+----
 
 All tests have an artifacts directory where they place their output.
 The testing or CI system that invokes the test will fill in this variable with a directory that it will archive.
 We ensure this directory exists in the test.
 
-By use of <code>tags</code> we note what kind of systems this test is suitable to run on.
+By use of `tags` we note what kind of systems this test is suitable to run on.
 
-The <code>block</code> is the section that runs the actual test.
+The `block` is the section that runs the actual test.
 In this example, we use a rather convoluted way of checking that PID 1 exists.
 However, by doing so, we place an extra test artifact in the artifacts directory.
 
 Lastly, we download the artifacts.
 Remember that the test is not always running on the same system that it was invoked on.
-Try running this example test against an [[CI/Standard_Test_Roles#Atomic|Atomic Host]] or [[CI/Standard_Test_Roles#Container|Docker Container]].
+Try running this example test against an xref:standard-test-roles.adoc#_atomic[Atomic Host] or xref:standard-test-roles.adoc#_container[Docker Container].
 It should pass.
-Try changing the <code>/proc/1</code> argument to another value, and the test should fail.
+Try changing the `/proc/1` argument to another value, and the test should fail.
 
 You can use most of the Ansible techniques in your playbooks.
-Take a look at the [[CI/Standard_Test_Roles|Standard Test Roles]] for Ansible roles to make writing your tests easier.
+Take a look at the xref:standard-test-roles.adoc[Standard Test Roles] for Ansible roles to make writing your tests easier.
 
 *Marking the test to be run*
 
-Just having a <code>.yml</code> file in the right directory doesn't yet mean it will be invoked.
-Make sure to reference or add it from a <code>tests.yml</code> playbook.
+Just having a `.yml` file in the right directory doesn't yet mean it will be invoked.
+Make sure to reference or add it from a `tests.yml` playbook.
 This is the entry point that the testing or CI system will use to invoke all the tests for a given package.
 
-If the <code>tests.yml</code> file doesn't yet exist, create it.
-Lets continue with our above example and create a <code>tests.yml</code> with the following content:
+If the `tests.yml` file doesn't yet exist, create it.
+Lets continue with our above example and create a `tests.yml` with the following content:
 
- - import_playbook: test_pid_1.yml
+[source,ansible]
+----
+- import_playbook: test_pid_1.yml
+----
 
 You can now run this test with the standard commands above.
 
-See the [[CI/Quick_Start_Guide#Contributing|Quick Start Guide]] to get recommendations for contributing new tests.
+See the xref:quick-start-guide.adoc#_contributing[Quick Start Guide] to get recommendations for contributing new tests.
 
 == Wrapping ==
 
 Let's say you have a script that runs a test.
 Its stdout and stderr is the test output, and an exit status of zero indicates success.
 Here's how we would wrap that test to be invoked.
-Lets say we have a simple script like in a file called <code>test-simple</code>
+Lets say we have a simple script like in a file called `test-simple`
 
  #!/bin/sh
  set -ex
@@ -181,9 +192,10 @@ Lets say we have a simple script like in a file called <code>test-simple</code>
  cmp bla.file bla.file.orig
  rm bla.file bla.file.orig
 
-We can write an Ansible wrapper for this script like this in <code>test_simple.yml</code>:
+We can write an Ansible wrapper for this script like this in `test_simple.yml`:
 
-<pre>
+[source,ansible]
+----
 ---
 - hosts: localhost
   vars:
@@ -210,29 +222,32 @@ We can write an Ansible wrapper for this script like this in <code>test_simple.y
           dest: "{{ artifacts }}/"
           src: "/tmp/test.log"
           flat: yes
-</pre>
+----
 
 All tests have an artifacts directory where they place their output.
 The testing or CI system that invokes the test will fill in this variable with a directory that it will archive.
 We create ensure this directory exists in the test.
 
-The <code>block</code> is the section that runs the actual test.
+The `block` is the section that runs the actual test.
 
 Lastly, we download the artifacts.
 Remember that the test is not always running on the same system that it was invoked on.
 
 
-If the <code>tests.yml</code> file doesn't yet exist, create it.
-Lets continue with our above example and create a <code>tests.yml</code> with the following content:
+If the `tests.yml` file doesn't yet exist, create it.
+Lets continue with our above example and create a `tests.yml` with the following content:
 
- - import_playbook: test_simple.yml
+[source,ansible]
+----
+- import_playbook: test_simple.yml
+----
 
-Try running this example test against an [[CI/Standard_Test_Roles#Atomic|Atomic Host]] or [[CI/Standard_Test_Roles#Container|Docker Container]].
+Try running this example test against an xref:standard-test-roles.adoc#_atomic[Atomic Host] or xref:standard-test-roles.adoc#_container[Docker Container].
 It should pass.
 
-See [[CI/Standard_Test_Roles|Standard Test Roles]] documentation for instructions how to wrap a [[CI/Standard_Test_Roles#BeakerLib|BeakerLib]] and [[CI/Standard_Test_Roles#RHTS|RHTS]] tests.
+See xref:standard-test-roles.adoc[Standard Test Roles] documentation for instructions how to wrap a xref:standard-test-roles.adoc#_beakerlib[BeakerLib] and xref:standard-test-roles.adoc#_rhts[RHTS] tests.
 
-See the [[CI/Quick_Start_Guide#Contributing|Quick Start Guide]] to get recommendations for contributing new tests.
+See the xref:quick-start-guide.adoc#_contributing[Quick Start Guide] to get recommendations for contributing new tests.
 
 [[Category:FedoraAtomicCi]]