From c7aaa1c3bef66dd59d23a340e6fbf5d4b6b08031 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 12 2020 00:59:12 +0000 Subject: [PATCH 1/2] Adapt older win-build docs Fixes https://pagure.io/koji/issue/1836 --- diff --git a/docs/source/index.rst b/docs/source/index.rst index f7eadbb..cde4049 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -26,6 +26,7 @@ Contents defining_hub_policies external_repo_server_bootstrap image_build + windows tag_inheritance misc release_notes/release_notes diff --git a/docs/source/windows.rst b/docs/source/windows.rst new file mode 100644 index 0000000..bf3b872 --- /dev/null +++ b/docs/source/windows.rst @@ -0,0 +1,258 @@ +==================== +Building for Windows +==================== + +Koji provides a fairly basic mechanism to perform builds for Windows. + + +Introduction +============ + +Koji supports building components in a Windows-based environment. This is +accomplished by cloning and starting a Windows VM, checking out sources and +building from within it. The VM would have compile tools and utility already +installed, and the results are then passed back up the hub. Windows builds are +treated similarly to RPM builds in that they have a unique name, version, and +release field. The resulting builds are built using targets and tagged into tags +just as normal builds are. When you build a Windows component, you can +reference other components and they will be downloaded into the build +environment to satisfy build-time dependencies. + + +Initiating a build +================== + +Note: access to performing Windows builds is governed by the ``vm`` policy on the hub. +The default policy requires the user to have the ``win-admin`` permission. +Different Koji instances may have different policies. +If the policy denies you access, you will see an ``ActionNotAllowed`` error. +In that case, you will need file a request with your Koji administators. + + +Windows builds are initiated with the ``win-build`` subcommand. + +.. + + $ koji win-build + +Where ``target`` is the build target to use for the build, +``scm-url`` is the SCM URL for the sources to build, +and ``vm-name`` is the name of the vm image to use for the build. + +The ``target`` works like targets for other types of builds. +The build tag for the target determines where Koji will pull build dependencies from, +and the destination tag for the target determines where the resulting build is tagged +after building is completed. + +The ``scm-url`` follows the same syntax used for other Koji builds + +.. + + scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier + +In order for Koji to perform the build, a "Windows spec file" is required (see below). +This spec file can either be included in the top directory of the sources or specified +separately with the ``--specfile`` option. + +The ``vm-name`` argument is the name of the vm image that will be used for the build. +This name must be one of the know images available on the builders. + +An option you may be interested in is ``--specfile``, which allows you to specify a second +remote repository that contains the windows spec file using the same SCM URL syntax as +described below. That way you are not forced to keep the spec in with the rest of your +sources if you don't want to. There is also a ``--patches`` option which is just +like ``--specfile``, except meant for a repository of separate patches, which will +be applied to the sources before the build is launched. ``--specfile`` and +--patches may reference the same repo. + + +The Windows "Spec-File" +======================= + +This is the file that controls the build process, and is modeled very loosely +on RPM's spec files. The Windows spec file is in .ini format. + +It's probably easiest to start by looking at this +`example `__ +spec file. + +All values in the [naming] section must be defined. +The name, version, and release are used to determine the NVR of the build. +As with all NVRs in Koji, this must be a unique value. + +The [building] section defines preconditions that must be satisfied before the +build will be launched, and the commands used to perform the build. platform +indicates to VM type the build is running in, and should be in the format of +osname-arch. + + +The ``preinstalled`` value under [building] lists the files and directories +that must be present for the build +to run. Any tools that were installed in the VM when it was setup should be +listed here. If any of the files or directories listed here are missing when +the build runs, it will fail immediately. Files and directories may be listed +in Windows format (C:\Program Files\...), full Cygwin paths (/bin/...), or as +command names (tar, unzip, etc). If they are listed as command names, the +Cygwin PATH will be checked for the command. + + +The ``buildrequires`` value under [building] lists other packages that +the build depends on. As with rpm +BuildRequires, the buildrequires entries should be package names only (no +version information). The latest build of that package will be looked up in +the -build tag associated with the build target that was used when launching +the build, and files from that build will be downloaded into the VM. Each +package listed in buildrequires can have a colon-delimited list of options +associated with it, which determine which files from the build will be +downloaded. By default, all Windows files associated with the dependent build +will be downloaded (this is the same as specifying type=win). In this case, +comma-separated lists of platforms= and flags can also be specified, in which +case the files downloaded into the VM will be limited to those associated with +one or more of the platform and flag values (more about this in the [files] +section below. If type=rpm is specified, then all rpms associated with the +dependent build will be installed. In this case a comma-separated list of +arches= may be specified, and only rpms matching one of those arches will be +downloaded. If type=maven is specified, then comma-separated lists of +group_ids=, artifact_ids=, and/or versions= may be specified, and only Maven +artifacts matching at least one value in each list will be downloaded. In all +cases, a patterns= option may be specified, which is a comma-separated list of +globs to match against the filenames. Only files matching at least one of the +patterns will be downloaded. + + +The files downloaded to satisfy the buildrequires are placed in a directory +based on their type, and variables are set which point to their location. The +variable names are constructed by converting the dependency name to variable +format (a leading number is converted to and underscore, and any character that +is not a letter, number, or underscore is converted to an underscore). If +type= is specified, then _dir is appended to complete the variable name. +Otherwise, just _dir is appended. In the example above, the directory +containing the boost-win files would be indicated by $boost_win_dir. The +directory containing the .src.rpm from the qpid-cpp-mrg build would be +$qpid_cpp_mrg_rpm_dir. Note the extra _rpm there, because it specified +type=rpm. There is also a corresponding _files variable which is a +newline-delimited list of all files downloaded for each dependency. In the +example, both $boost_win_files and $qpid_cpp_mrg_rpm_files would be defined. + + +The files are downloaded to the VM unmodified. It is up to the build process +to extract them if necessary, and move/copy them to whatever location is +required by the current build. + + +The ``provides`` value under [building] is optional, and can be used to indicate what the build is +producing. It is freeform, but a structure like - is +encouraged. + + +The ``shell`` value under [building] indicates how the build script should be run. +Valid values are +bash, which will cause the script to be run in bash, and cmd.exe, which will +cause the script to be run in Windows cmd.exe. cmd is also an alias for +cmd.exe. bash is the default if no shell value is present. + + +The ``execute`` value under [building] is required, and this is what drives the build process. +This value is multiline, and each line is treated as a separate command to be execute in the +shell specified above. In addition to the variables defined for the buildrequires, the +following variables will be available when the script is executed: + + * name: name from the [naming] section + * version: version from the [naming] section + * release: release from the [naming] section + * source_dir: the directory the sources were checked out into + * spec_dir: the directory the .ini was checked out into. If there was no separate --winspec + option passed on the command-line, this will be the same as source_dir + * patches_dir: the directory the patches were checked out into. If there was no --patches + option passed on the command-line, this will be undefined. + + +If using bash, the build script will be executed with -x and -e, which will +cause all commands to be echoed, and will cause the script to fail if any +commands have a non-zero exit status. There is no equivalent fail-fast option +for Windows cmd.exe. If executing the script using cmd.exe, it is recommend +that you frequently check the return value of commands with this: + +.. + + if %ERRORLEVEL% neq 0 exit %ERRORLEVEL% + + +The script will start in the root directory of the sources that were checked +out ($source_dir). Extra scripts or supplementary files required by the build +may be checked in along with the .ini file, and will be available under +$spec_dir (assuming a separate --specfile option was used). + + +The postbuild value is optional, and specifies a list of files and directories +(relative to $source_dir) that must be present for the build to be considered +successful. If any of them are missing, the build will fail. This is useful +for verifying that all expected output was generated correctly, in the case +that some commands may have failed silently. + + +The [files] section describes what output should be collected after the build +completes. The logfiles value is optional, but it should be set to whatever +build logs are produced from the build. The syntax for the output variable is +as follows: + +.. + + output = qpid-cpp-x86-$version.zip:i386:chk,fre + +Note the colon-separated nature. The first token is the path to the file you +want collected as part of your build output. This path is rooted at the +checkout from the SCM ($source_dir). The file path relative to $source_dir is +retained when output is uploaded to Koji and when it is downloaded as a +buildrequires by future builds. If you don't want a long, confusing file path, +it may be desirable to copy the build output to $source_dir at the end of your +build script. File globs are not allowed, but the $name, $version, and +$release variables will be expanded in the file paths and names, using the +values from the [naming] section. The second token is a comma-separated list of +platforms (which we haven't really standardized on yet, but i386 and/or x86_64 +are logical choices). The last is a comma-separated list of build flags. These +fields are purely informational, they do not influence future builds at this +time, but they do make for good housekeeping in the future. Common flags are +chk (indicating a debug build) and fre (indicating an optimized build). If an +output file contains both kinds of builds, both may be specified. + + +The logs value indicates extra log files generated during the build that should +be tracked. The contents of these log files will be streamed to the hub during +the build and may be watched in realtime using the "Watch logs" +feature of the web interface, or the "koji watch-logs" cli command. +They will also be included in the build logs stored on in Koji. + + +Source Repository +================= + +As noted earlier, the sources are checked out from within a Windows VM that +Koji clones and starts when you submit your task with the client. Koji supports +checking out sources from CVS, SVN, and git. +As with other types of builds, the ``allowed_scms`` setting limits which sources +can be built from. + + +Administration +============== + +Windows Build Hosts +------------------- + +By default, all ``winbuild`` tasks go to the ``vm`` channel. +The hosts in this channel require special setup. + + * They run the ``kojivmd`` daemon instead of the regular ``kojid`` daemon + * VM images for builds must be stored in the local image directory + + +Managing VM Images +------------------ + +The directory where ``kojivmd`` looks for vm images can be controlled by setting ``imagedir`` +in ``/etc/kojivmd/kojivmd.conf``. The default value is ``/var/lib/libvirt/images``. + +These images must be qcow2 images named with a ``.qcow2`` extension. The basename of the +image file is the same name that users refer to in the ``vm-name`` parameter to the +``win-build`` command. From a44e36fb37b258da491e845cc8704711b94e98b4 Mon Sep 17 00:00:00 2001 From: Mike McLean Date: Mar 13 2020 19:30:38 +0000 Subject: [PATCH 2/2] include example spec --- diff --git a/docs/source/index.rst b/docs/source/index.rst index cde4049..a7b3acb 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -26,7 +26,7 @@ Contents defining_hub_policies external_repo_server_bootstrap image_build - windows + winbuild tag_inheritance misc release_notes/release_notes diff --git a/docs/source/win_spec_example.rst b/docs/source/win_spec_example.rst new file mode 100644 index 0000000..21a28cc --- /dev/null +++ b/docs/source/win_spec_example.rst @@ -0,0 +1,100 @@ +========================= +Example Windows Spec File +========================= + +The following is an example ini-format spec file for a :doc:`Windows build ` in Koji. + +.. code-block:: ini + + [naming] + ; naming and versioning of the component + name = qpid-cpp-win + version = 2.0.0.1 + release = 1 + description = Windows build of qpid-cpp-mrg + + [building] + ; use os-arch + platform = w2k8r2-x64 + + ; file, directories, and commands that must be available in the build environment + preinstalled = /cygdrive/c/Program Files/7-Zip/7z.exe + /cygdrive/c/Program Files (x86)/CMake 2.8/bin/cmake.exe + /cygdrive/c/Python26/python.exe + /cygdrive/c/Ruby186/bin/ruby.exe + C:\Ruby186\bin\msvcrt-ruby18.dll + /cygdrive/c/Program Files (x86)/doxygen/bin/doxygen.exe + /cygdrive/c/Program Files (x86)/Microsoft Visual Studio 9.0/Common7/IDE/devenv.exe + cpio + tar + patch + powershell + + ; To specify other components you need fetched to be able to build this, fill in a white-space + ; delimited list of the other components you need by their names. Specific versions are not + ; yet supported, the latest tagged will always be fetched. + buildrequires = boost-win + qpid-cpp-mrg:type=rpm:arches=src + + ; what does this package provide? + provides = qpid-cpp-win + + ; what shell are we running the commands below in? + shell = bash + + ; what should we execute to build it? + execute = read MAJOR MINOR REVISION BUILD < <(echo $version | tr . " ") + pushd $boost_win_dir + mkdir dist + cd dist + for z in ../boost-win-*.tar.bz2; do + tar xjf $z + done + popd + PATH="/cygdrive/c/Program Files/7-Zip:$PATH" + PATH="/cygdrive/c/Program Files (x86)/CMake 2.8/bin:$PATH" + PATH="/cygdrive/c/Python26:$PATH" + PATH="/cygdrive/c/Ruby186/bin:$PATH" + PATH="/cygdrive/c/Program Files (x86)/doxygen/bin:$PATH" + PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 9.0/Common7/IDE:$PATH" + export PATH + # extract the tarball from the qpid-cpp-mrg rpm + pushd $qpid_cpp_mrg_rpm_dir/src + 7z x qpid-cpp-*.src.rpm + cpio -idmv < qpid-cpp-*.cpio + popd + mkdir source + cd source + tar xzf $qpid_cpp_mrg_rpm_dir/src/qpid-cpp-*.tar.gz + cd qpid-cpp-* + # apply patches + for p in ../../*.patch; do + patch -p1 --fuzz=0 < $p + done + cd cpp + cat <> src/CMakeWinVersions.cmake + set("winver_FILE_VERSION_N1" "$MAJOR") + set("winver_FILE_VERSION_N2" "$MINOR") + set("winver_FILE_VERSION_N3" "$REVISION") + set("winver_FILE_VERSION_N4" "$BUILD") + set("winver_PRODUCT_VERSION_N1" "$MAJOR") + set("winver_PRODUCT_VERSION_N2" "$MINOR") + set("winver_PRODUCT_VERSION_N3" "$REVISION") + set("winver_PRODUCT_VERSION_N4" "$BUILD") + EOF + powershell -ExecutionPolicy unrestricted -File bld-winsdk.ps1 $(basename $(dirname $PWD)) $(cygpath -wa $boost_win_dir/dist/boost-win-*-32bit) $(cygpath -wa $boost_win_dir/dist/boost-win-*-64bit) $version + mv ../../x86/qpid-cpp-x86-$version.zip ../../x64/qpid-cpp-x64-$version.zip ../../../ + + ; list of files that must be present after the build for the build to be + ; considered successful, but are not included in the list of build output + postbuild = + + [files] + ; all values in this section may be multi-line + ; output files we're concerned with (specify paths relative to scm root) + output = qpid-cpp-x86-$version.zip:i386:chk,fre + qpid-cpp-x64-$version.zip:x86_64:chk,fre + + ; logs we should report + logs = + diff --git a/docs/source/winbuild.rst b/docs/source/winbuild.rst new file mode 100644 index 0000000..f33d4d1 --- /dev/null +++ b/docs/source/winbuild.rst @@ -0,0 +1,262 @@ +==================== +Building for Windows +==================== + +Koji provides a fairly basic mechanism to perform builds for Windows. + +.. toctree:: + :hidden: + + win_spec_example + +Introduction +============ + +Koji supports building components in a Windows-based environment. This is +accomplished by cloning and starting a Windows VM, checking out sources and +building from within it. The VM would have compile tools and utility already +installed, and the results are then passed back up the hub. Windows builds are +treated similarly to RPM builds in that they have a unique name, version, and +release field. The resulting builds are built using targets and tagged into tags +just as normal builds are. When you build a Windows component, you can +reference other components and they will be downloaded into the build +environment to satisfy build-time dependencies. + + +Initiating a build +================== + +Note: access to performing Windows builds is governed by the ``vm`` policy on the hub. +The default policy requires the user to have the ``win-admin`` permission. +Different Koji instances may have different policies. +If the policy denies you access, you will see an ``ActionNotAllowed`` error. +In that case, you will need file a request with your Koji administators. + + +Windows builds are initiated with the ``win-build`` subcommand. + +.. + + $ koji win-build + +Where ``target`` is the build target to use for the build, +``scm-url`` is the SCM URL for the sources to build, +and ``vm-name`` is the name of the vm image to use for the build. + +The ``target`` works like targets for other types of builds. +The build tag for the target determines where Koji will pull build dependencies from, +and the destination tag for the target determines where the resulting build is tagged +after building is completed. + +The ``scm-url`` follows the same syntax used for other Koji builds + +.. + + scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier + +In order for Koji to perform the build, a "Windows spec file" is required (see below). +This spec file can either be included in the top directory of the sources or specified +separately with the ``--specfile`` option. + +The ``vm-name`` argument is the name of the vm image that will be used for the build. +This name must be one of the know images available on the builders. + +An option you may be interested in is ``--specfile``, which allows you to specify a second +remote repository that contains the windows spec file using the same SCM URL syntax as +described below. That way you are not forced to keep the spec in with the rest of your +sources if you don't want to. There is also a ``--patches`` option which is just +like ``--specfile``, except meant for a repository of separate patches, which will +be applied to the sources before the build is launched. ``--specfile`` and +--patches may reference the same repo. + + +The Windows "Spec-File" +======================= + +This is the file that controls the build process, and is modeled very loosely +on RPM's spec files. The Windows spec file is in .ini format. + +It's probably easiest to start by looking at this +:doc:`example ` +spec file. + +All values in the [naming] section must be defined. +The name, version, and release are used to determine the NVR of the build. +As with all NVRs in Koji, this must be a unique value. + +The [building] section defines preconditions that must be satisfied before the +build will be launched, and the commands used to perform the build. platform +indicates to VM type the build is running in, and should be in the format of +osname-arch. + + +The ``preinstalled`` value under [building] lists the files and directories +that must be present for the build +to run. Any tools that were installed in the VM when it was setup should be +listed here. If any of the files or directories listed here are missing when +the build runs, it will fail immediately. Files and directories may be listed +in Windows format (C:\Program Files\...), full Cygwin paths (/bin/...), or as +command names (tar, unzip, etc). If they are listed as command names, the +Cygwin PATH will be checked for the command. + + +The ``buildrequires`` value under [building] lists other packages that +the build depends on. As with rpm +BuildRequires, the buildrequires entries should be package names only (no +version information). The latest build of that package will be looked up in +the -build tag associated with the build target that was used when launching +the build, and files from that build will be downloaded into the VM. Each +package listed in buildrequires can have a colon-delimited list of options +associated with it, which determine which files from the build will be +downloaded. By default, all Windows files associated with the dependent build +will be downloaded (this is the same as specifying type=win). In this case, +comma-separated lists of platforms= and flags can also be specified, in which +case the files downloaded into the VM will be limited to those associated with +one or more of the platform and flag values (more about this in the [files] +section below. If type=rpm is specified, then all rpms associated with the +dependent build will be installed. In this case a comma-separated list of +arches= may be specified, and only rpms matching one of those arches will be +downloaded. If type=maven is specified, then comma-separated lists of +group_ids=, artifact_ids=, and/or versions= may be specified, and only Maven +artifacts matching at least one value in each list will be downloaded. In all +cases, a patterns= option may be specified, which is a comma-separated list of +globs to match against the filenames. Only files matching at least one of the +patterns will be downloaded. + + +The files downloaded to satisfy the buildrequires are placed in a directory +based on their type, and variables are set which point to their location. The +variable names are constructed by converting the dependency name to variable +format (a leading number is converted to and underscore, and any character that +is not a letter, number, or underscore is converted to an underscore). If +type= is specified, then _dir is appended to complete the variable name. +Otherwise, just _dir is appended. In the example above, the directory +containing the boost-win files would be indicated by $boost_win_dir. The +directory containing the .src.rpm from the qpid-cpp-mrg build would be +$qpid_cpp_mrg_rpm_dir. Note the extra _rpm there, because it specified +type=rpm. There is also a corresponding _files variable which is a +newline-delimited list of all files downloaded for each dependency. In the +example, both $boost_win_files and $qpid_cpp_mrg_rpm_files would be defined. + + +The files are downloaded to the VM unmodified. It is up to the build process +to extract them if necessary, and move/copy them to whatever location is +required by the current build. + + +The ``provides`` value under [building] is optional, and can be used to indicate what the build is +producing. It is freeform, but a structure like - is +encouraged. + + +The ``shell`` value under [building] indicates how the build script should be run. +Valid values are +bash, which will cause the script to be run in bash, and cmd.exe, which will +cause the script to be run in Windows cmd.exe. cmd is also an alias for +cmd.exe. bash is the default if no shell value is present. + + +The ``execute`` value under [building] is required, and this is what drives the build process. +This value is multiline, and each line is treated as a separate command to be execute in the +shell specified above. In addition to the variables defined for the buildrequires, the +following variables will be available when the script is executed: + + * name: name from the [naming] section + * version: version from the [naming] section + * release: release from the [naming] section + * source_dir: the directory the sources were checked out into + * spec_dir: the directory the .ini was checked out into. If there was no separate --winspec + option passed on the command-line, this will be the same as source_dir + * patches_dir: the directory the patches were checked out into. If there was no --patches + option passed on the command-line, this will be undefined. + + +If using bash, the build script will be executed with -x and -e, which will +cause all commands to be echoed, and will cause the script to fail if any +commands have a non-zero exit status. There is no equivalent fail-fast option +for Windows cmd.exe. If executing the script using cmd.exe, it is recommend +that you frequently check the return value of commands with this: + +.. + + if %ERRORLEVEL% neq 0 exit %ERRORLEVEL% + + +The script will start in the root directory of the sources that were checked +out ($source_dir). Extra scripts or supplementary files required by the build +may be checked in along with the .ini file, and will be available under +$spec_dir (assuming a separate --specfile option was used). + + +The postbuild value is optional, and specifies a list of files and directories +(relative to $source_dir) that must be present for the build to be considered +successful. If any of them are missing, the build will fail. This is useful +for verifying that all expected output was generated correctly, in the case +that some commands may have failed silently. + + +The [files] section describes what output should be collected after the build +completes. The logfiles value is optional, but it should be set to whatever +build logs are produced from the build. The syntax for the output variable is +as follows: + +.. + + output = qpid-cpp-x86-$version.zip:i386:chk,fre + +Note the colon-separated nature. The first token is the path to the file you +want collected as part of your build output. This path is rooted at the +checkout from the SCM ($source_dir). The file path relative to $source_dir is +retained when output is uploaded to Koji and when it is downloaded as a +buildrequires by future builds. If you don't want a long, confusing file path, +it may be desirable to copy the build output to $source_dir at the end of your +build script. File globs are not allowed, but the $name, $version, and +$release variables will be expanded in the file paths and names, using the +values from the [naming] section. The second token is a comma-separated list of +platforms (which we haven't really standardized on yet, but i386 and/or x86_64 +are logical choices). The last is a comma-separated list of build flags. These +fields are purely informational, they do not influence future builds at this +time, but they do make for good housekeeping in the future. Common flags are +chk (indicating a debug build) and fre (indicating an optimized build). If an +output file contains both kinds of builds, both may be specified. + + +The logs value indicates extra log files generated during the build that should +be tracked. The contents of these log files will be streamed to the hub during +the build and may be watched in realtime using the "Watch logs" +feature of the web interface, or the "koji watch-logs" cli command. +They will also be included in the build logs stored on in Koji. + + +Source Repository +================= + +As noted earlier, the sources are checked out from within a Windows VM that +Koji clones and starts when you submit your task with the client. Koji supports +checking out sources from CVS, SVN, and git. +As with other types of builds, the ``allowed_scms`` setting limits which sources +can be built from. + + +Administration +============== + +Windows Build Hosts +------------------- + +By default, all ``winbuild`` tasks go to the ``vm`` channel. +The hosts in this channel require special setup. + + * They run the ``kojivmd`` daemon instead of the regular ``kojid`` daemon + * VM images for builds must be stored in the local image directory + + +Managing VM Images +------------------ + +The directory where ``kojivmd`` looks for vm images can be controlled by setting ``imagedir`` +in ``/etc/kojivmd/kojivmd.conf``. The default value is ``/var/lib/libvirt/images``. + +These images must be qcow2 images named with a ``.qcow2`` extension. The basename of the +image file is the same name that users refer to in the ``vm-name`` parameter to the +``win-build`` command. diff --git a/docs/source/windows.rst b/docs/source/windows.rst deleted file mode 100644 index bf3b872..0000000 --- a/docs/source/windows.rst +++ /dev/null @@ -1,258 +0,0 @@ -==================== -Building for Windows -==================== - -Koji provides a fairly basic mechanism to perform builds for Windows. - - -Introduction -============ - -Koji supports building components in a Windows-based environment. This is -accomplished by cloning and starting a Windows VM, checking out sources and -building from within it. The VM would have compile tools and utility already -installed, and the results are then passed back up the hub. Windows builds are -treated similarly to RPM builds in that they have a unique name, version, and -release field. The resulting builds are built using targets and tagged into tags -just as normal builds are. When you build a Windows component, you can -reference other components and they will be downloaded into the build -environment to satisfy build-time dependencies. - - -Initiating a build -================== - -Note: access to performing Windows builds is governed by the ``vm`` policy on the hub. -The default policy requires the user to have the ``win-admin`` permission. -Different Koji instances may have different policies. -If the policy denies you access, you will see an ``ActionNotAllowed`` error. -In that case, you will need file a request with your Koji administators. - - -Windows builds are initiated with the ``win-build`` subcommand. - -.. - - $ koji win-build - -Where ``target`` is the build target to use for the build, -``scm-url`` is the SCM URL for the sources to build, -and ``vm-name`` is the name of the vm image to use for the build. - -The ``target`` works like targets for other types of builds. -The build tag for the target determines where Koji will pull build dependencies from, -and the destination tag for the target determines where the resulting build is tagged -after building is completed. - -The ``scm-url`` follows the same syntax used for other Koji builds - -.. - - scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier - -In order for Koji to perform the build, a "Windows spec file" is required (see below). -This spec file can either be included in the top directory of the sources or specified -separately with the ``--specfile`` option. - -The ``vm-name`` argument is the name of the vm image that will be used for the build. -This name must be one of the know images available on the builders. - -An option you may be interested in is ``--specfile``, which allows you to specify a second -remote repository that contains the windows spec file using the same SCM URL syntax as -described below. That way you are not forced to keep the spec in with the rest of your -sources if you don't want to. There is also a ``--patches`` option which is just -like ``--specfile``, except meant for a repository of separate patches, which will -be applied to the sources before the build is launched. ``--specfile`` and ---patches may reference the same repo. - - -The Windows "Spec-File" -======================= - -This is the file that controls the build process, and is modeled very loosely -on RPM's spec files. The Windows spec file is in .ini format. - -It's probably easiest to start by looking at this -`example `__ -spec file. - -All values in the [naming] section must be defined. -The name, version, and release are used to determine the NVR of the build. -As with all NVRs in Koji, this must be a unique value. - -The [building] section defines preconditions that must be satisfied before the -build will be launched, and the commands used to perform the build. platform -indicates to VM type the build is running in, and should be in the format of -osname-arch. - - -The ``preinstalled`` value under [building] lists the files and directories -that must be present for the build -to run. Any tools that were installed in the VM when it was setup should be -listed here. If any of the files or directories listed here are missing when -the build runs, it will fail immediately. Files and directories may be listed -in Windows format (C:\Program Files\...), full Cygwin paths (/bin/...), or as -command names (tar, unzip, etc). If they are listed as command names, the -Cygwin PATH will be checked for the command. - - -The ``buildrequires`` value under [building] lists other packages that -the build depends on. As with rpm -BuildRequires, the buildrequires entries should be package names only (no -version information). The latest build of that package will be looked up in -the -build tag associated with the build target that was used when launching -the build, and files from that build will be downloaded into the VM. Each -package listed in buildrequires can have a colon-delimited list of options -associated with it, which determine which files from the build will be -downloaded. By default, all Windows files associated with the dependent build -will be downloaded (this is the same as specifying type=win). In this case, -comma-separated lists of platforms= and flags can also be specified, in which -case the files downloaded into the VM will be limited to those associated with -one or more of the platform and flag values (more about this in the [files] -section below. If type=rpm is specified, then all rpms associated with the -dependent build will be installed. In this case a comma-separated list of -arches= may be specified, and only rpms matching one of those arches will be -downloaded. If type=maven is specified, then comma-separated lists of -group_ids=, artifact_ids=, and/or versions= may be specified, and only Maven -artifacts matching at least one value in each list will be downloaded. In all -cases, a patterns= option may be specified, which is a comma-separated list of -globs to match against the filenames. Only files matching at least one of the -patterns will be downloaded. - - -The files downloaded to satisfy the buildrequires are placed in a directory -based on their type, and variables are set which point to their location. The -variable names are constructed by converting the dependency name to variable -format (a leading number is converted to and underscore, and any character that -is not a letter, number, or underscore is converted to an underscore). If -type= is specified, then _dir is appended to complete the variable name. -Otherwise, just _dir is appended. In the example above, the directory -containing the boost-win files would be indicated by $boost_win_dir. The -directory containing the .src.rpm from the qpid-cpp-mrg build would be -$qpid_cpp_mrg_rpm_dir. Note the extra _rpm there, because it specified -type=rpm. There is also a corresponding _files variable which is a -newline-delimited list of all files downloaded for each dependency. In the -example, both $boost_win_files and $qpid_cpp_mrg_rpm_files would be defined. - - -The files are downloaded to the VM unmodified. It is up to the build process -to extract them if necessary, and move/copy them to whatever location is -required by the current build. - - -The ``provides`` value under [building] is optional, and can be used to indicate what the build is -producing. It is freeform, but a structure like - is -encouraged. - - -The ``shell`` value under [building] indicates how the build script should be run. -Valid values are -bash, which will cause the script to be run in bash, and cmd.exe, which will -cause the script to be run in Windows cmd.exe. cmd is also an alias for -cmd.exe. bash is the default if no shell value is present. - - -The ``execute`` value under [building] is required, and this is what drives the build process. -This value is multiline, and each line is treated as a separate command to be execute in the -shell specified above. In addition to the variables defined for the buildrequires, the -following variables will be available when the script is executed: - - * name: name from the [naming] section - * version: version from the [naming] section - * release: release from the [naming] section - * source_dir: the directory the sources were checked out into - * spec_dir: the directory the .ini was checked out into. If there was no separate --winspec - option passed on the command-line, this will be the same as source_dir - * patches_dir: the directory the patches were checked out into. If there was no --patches - option passed on the command-line, this will be undefined. - - -If using bash, the build script will be executed with -x and -e, which will -cause all commands to be echoed, and will cause the script to fail if any -commands have a non-zero exit status. There is no equivalent fail-fast option -for Windows cmd.exe. If executing the script using cmd.exe, it is recommend -that you frequently check the return value of commands with this: - -.. - - if %ERRORLEVEL% neq 0 exit %ERRORLEVEL% - - -The script will start in the root directory of the sources that were checked -out ($source_dir). Extra scripts or supplementary files required by the build -may be checked in along with the .ini file, and will be available under -$spec_dir (assuming a separate --specfile option was used). - - -The postbuild value is optional, and specifies a list of files and directories -(relative to $source_dir) that must be present for the build to be considered -successful. If any of them are missing, the build will fail. This is useful -for verifying that all expected output was generated correctly, in the case -that some commands may have failed silently. - - -The [files] section describes what output should be collected after the build -completes. The logfiles value is optional, but it should be set to whatever -build logs are produced from the build. The syntax for the output variable is -as follows: - -.. - - output = qpid-cpp-x86-$version.zip:i386:chk,fre - -Note the colon-separated nature. The first token is the path to the file you -want collected as part of your build output. This path is rooted at the -checkout from the SCM ($source_dir). The file path relative to $source_dir is -retained when output is uploaded to Koji and when it is downloaded as a -buildrequires by future builds. If you don't want a long, confusing file path, -it may be desirable to copy the build output to $source_dir at the end of your -build script. File globs are not allowed, but the $name, $version, and -$release variables will be expanded in the file paths and names, using the -values from the [naming] section. The second token is a comma-separated list of -platforms (which we haven't really standardized on yet, but i386 and/or x86_64 -are logical choices). The last is a comma-separated list of build flags. These -fields are purely informational, they do not influence future builds at this -time, but they do make for good housekeeping in the future. Common flags are -chk (indicating a debug build) and fre (indicating an optimized build). If an -output file contains both kinds of builds, both may be specified. - - -The logs value indicates extra log files generated during the build that should -be tracked. The contents of these log files will be streamed to the hub during -the build and may be watched in realtime using the "Watch logs" -feature of the web interface, or the "koji watch-logs" cli command. -They will also be included in the build logs stored on in Koji. - - -Source Repository -================= - -As noted earlier, the sources are checked out from within a Windows VM that -Koji clones and starts when you submit your task with the client. Koji supports -checking out sources from CVS, SVN, and git. -As with other types of builds, the ``allowed_scms`` setting limits which sources -can be built from. - - -Administration -============== - -Windows Build Hosts -------------------- - -By default, all ``winbuild`` tasks go to the ``vm`` channel. -The hosts in this channel require special setup. - - * They run the ``kojivmd`` daemon instead of the regular ``kojid`` daemon - * VM images for builds must be stored in the local image directory - - -Managing VM Images ------------------- - -The directory where ``kojivmd`` looks for vm images can be controlled by setting ``imagedir`` -in ``/etc/kojivmd/kojivmd.conf``. The default value is ``/var/lib/libvirt/images``. - -These images must be qcow2 images named with a ``.qcow2`` extension. The basename of the -image file is the same name that users refer to in the ``vm-name`` parameter to the -``win-build`` command.