From 01ad55388440231e7a59043e6111a447c2b4b960 Mon Sep 17 00:00:00 2001 From: Adam Samalik Date: Sep 04 2017 10:34:16 +0000 Subject: [PATCH 1/2] move 'get involved' under 'project' --- diff --git a/source/development/get-involved.rst b/source/development/get-involved.rst deleted file mode 100644 index d7f690b..0000000 --- a/source/development/get-involved.rst +++ /dev/null @@ -1,76 +0,0 @@ -Get Involved -============ - -Steps to get involved ---------------------- - -.. raw:: mediawiki - - {{Team contact|Modularity WG|devel|#fedora-modularity}} - -To get involved with the Modularity Working Group, please follow the -following steps for becoming a part of the team. - -#. Read this wiki page and understand the development methods and - technical details. -#. Get in touch with us on the `devel mailing list - `__ - and our IRC channel, `#fedora-modularity - `__. -#. Attend one of our `weekly - meetings `__ and - introduce yourself. -#. Create an account on `Pagure `__ and file an - issue in the `Fedora @modularity - group `__ requesting membership. - -Get in touch ------------- - -There's no dedicated mailing list for this effort, everything regarding this -topic should be discussed on the general `Fedora Development list -`__. -Most of us also hang out on the #fedora-modularity channel on Freenode. - -Planning --------- - -Formal meetings are held once every two weeks by the Modularity Working Group. -See the -`Fedocal `__ to -find out when the group will meet. - -Technical details ------------------ - -Project repositories -~~~~~~~~~~~~~~~~~~~~ - -We host our individual projects either on `Pagure `__, -`GitHub `__ or in the `Fedora Packages repository -`__. - -Tooling and infrastructure projects can be found in the `Pagure group for -Modularity `__ or the |gh-fedmod-org|_. -Modules themselves are in the `Fedora Packages repository -`__ under the ``modules`` namespace, they have -counterparts in the |gh-modmod-org|_ which deal with their individual package -components. - -.. |gh-fedmod-org| replace:: ``fedora-modularity`` GitHub organization -.. _gh-fedmod-org: https://github.com/fedora-modularity/ -.. |gh-modmod-org| replace:: ``modularity-modules`` GitHub organization -.. _gh-modmod-org: https://github.com/modularity-modules/ - -Integration of changes -~~~~~~~~~~~~~~~~~~~~~~ - -In order so that our code base is always clean and maintainable, we have to -enforce certain rules on how code is written or formatted, how changes are -broken up into commits and how pull requests are handled. - -.. toctree:: - :maxdepth: 1 - - integration/coding-style-python - integration/grooming diff --git a/source/docs.rst b/source/docs.rst index 2e2a9f2..d6efbc3 100644 --- a/source/docs.rst +++ b/source/docs.rst @@ -27,7 +27,6 @@ Documentation Index :caption: Development :maxdepth: 2 - development/get-involved development/resources development/building-modules @@ -45,4 +44,10 @@ Documentation Index usability/personas +.. toctree:: + :caption: Project + :maxdepth: 2 + + project/get-involved + diff --git a/source/project/get-involved.rst b/source/project/get-involved.rst new file mode 100644 index 0000000..d7f690b --- /dev/null +++ b/source/project/get-involved.rst @@ -0,0 +1,76 @@ +Get Involved +============ + +Steps to get involved +--------------------- + +.. raw:: mediawiki + + {{Team contact|Modularity WG|devel|#fedora-modularity}} + +To get involved with the Modularity Working Group, please follow the +following steps for becoming a part of the team. + +#. Read this wiki page and understand the development methods and + technical details. +#. Get in touch with us on the `devel mailing list + `__ + and our IRC channel, `#fedora-modularity + `__. +#. Attend one of our `weekly + meetings `__ and + introduce yourself. +#. Create an account on `Pagure `__ and file an + issue in the `Fedora @modularity + group `__ requesting membership. + +Get in touch +------------ + +There's no dedicated mailing list for this effort, everything regarding this +topic should be discussed on the general `Fedora Development list +`__. +Most of us also hang out on the #fedora-modularity channel on Freenode. + +Planning +-------- + +Formal meetings are held once every two weeks by the Modularity Working Group. +See the +`Fedocal `__ to +find out when the group will meet. + +Technical details +----------------- + +Project repositories +~~~~~~~~~~~~~~~~~~~~ + +We host our individual projects either on `Pagure `__, +`GitHub `__ or in the `Fedora Packages repository +`__. + +Tooling and infrastructure projects can be found in the `Pagure group for +Modularity `__ or the |gh-fedmod-org|_. +Modules themselves are in the `Fedora Packages repository +`__ under the ``modules`` namespace, they have +counterparts in the |gh-modmod-org|_ which deal with their individual package +components. + +.. |gh-fedmod-org| replace:: ``fedora-modularity`` GitHub organization +.. _gh-fedmod-org: https://github.com/fedora-modularity/ +.. |gh-modmod-org| replace:: ``modularity-modules`` GitHub organization +.. _gh-modmod-org: https://github.com/modularity-modules/ + +Integration of changes +~~~~~~~~~~~~~~~~~~~~~~ + +In order so that our code base is always clean and maintainable, we have to +enforce certain rules on how code is written or formatted, how changes are +broken up into commits and how pull requests are handled. + +.. toctree:: + :maxdepth: 1 + + integration/coding-style-python + integration/grooming From 6358bf22b541337618f70d0fc348939a849eed3a Mon Sep 17 00:00:00 2001 From: Adam Samalik Date: Sep 04 2017 10:42:01 +0000 Subject: [PATCH 2/2] move container development under 'building containers' --- diff --git a/source/development/building-containers.rst b/source/development/building-containers.rst new file mode 100644 index 0000000..e6ac8d1 --- /dev/null +++ b/source/development/building-containers.rst @@ -0,0 +1,12 @@ +Building Containers +================= + +.. rubric:: TOC + +.. toctree:: + :maxdepth: 1 + + + building-containers/containers-local + building-containers/standalone-containers + building-containers/containers-infra diff --git a/source/development/building-containers/containers-infra.rst b/source/development/building-containers/containers-infra.rst new file mode 100644 index 0000000..3cf58f7 --- /dev/null +++ b/source/development/building-containers/containers-infra.rst @@ -0,0 +1,53 @@ +Building containers in Fedora +============================= + +Your Dockerfile needs to be formally reviewed before becoming a part of Fedora. +The review process is described in the `Fedora wiki +`__. + + +Builds in production +-------------------- + +Production builds of container images are documented in `this section +`__ of +the review process document mentioned above. Please follow the process to +perform an infrastructure build of your container image. + +In order to get the correct content (modules, RPMs) in your container image, +you should make sure that those artifacts are released and available: + +* For modules, this means to have a valid compose with your module. The module + needs to be incorporated inside `this pungi configuration file + `__, if + you want the module to be part of a compose. + +* For RPMs, the package should pass the Bodhi update process and be available + in repositories. + + +Inspecting registry +------------------- + +At the time of writing (Aug 2017), the Fedora docker image registry +doesn't have any frontend. You can access its API to get a list of available +images: + +:: + + $ curl -s https://registry.fedoraproject.org/v2/_catalog | \ + python -m json.tool + { + "repositories": [ + "cockpit", + "f24/cockpit", + "f25/cockpit", + "f25/kube-apiserver", + "f25/kubernetes-master", + "f25/owncloud", + "f26/cockpit", + "fedora", + "fedora/cockpit", + "openshift/origin-pod" + ] + } diff --git a/source/development/building-containers/containers-local.rst b/source/development/building-containers/containers-local.rst new file mode 100644 index 0000000..abcc368 --- /dev/null +++ b/source/development/building-containers/containers-local.rst @@ -0,0 +1,62 @@ +Building containers locally +=========================== + +When you have your module built, let's put it in a container, so we can +use it. As an example, we will use the `perl image +`__. + +Module RPM repository +--------------------- + +First, upload your RPM repository from the previous step somewhere publicly +accessible. For example, you can use Fedorapeople to host your packages: + +:: + + $ mkdir module + $ cp $RPMS module/ + $ createrepo_c module/ + $ rsync ./module/ $USER@fedorapeople.org:public_html/ + +Dockerfile +---------- + +Next step will be writing a Dockerfile to build a container image with your +module. + +The Base Runtime image is currently only available from Docker Hub and you can +use it as your base image by specifying ``FROM /baseruntime/baseruntime``. + +You also need to write a repo file for your module and add it to your +container, so you can install the module in it. See the example of the repo +file in ``__. The +following snippet then shows how you copy the repo and install the files from +it in your container: + +:: + + COPY repos/* /etc/yum.repos.d/ + # Perl and build tools install + user addition + RUN BUILD_TOOlS="bsdtar \ + findutils \ + gcc \ + make \ + gettext \ + tar \ + wget \ + python " && \ + microdnf --nodocs --enablerepo perl install perl perl-devel && \ + microdnf --nodocs --enablerepo fedora install -y mod_perl cpan cpanminus httpd \ + $BUILD_TOOlS && \ + microdnf clean all + RUN mkdir -p /opt/app-root/src/ && \ + useradd -u 1002 -r -g 0 -d /opt/app-root/src -s /sbin/nologin \ + -c "Default Application User" default && \ + chown -R 1002:0 /opt/app-root + + +Building the image +------------------- + +Finally, when you have your repository and Dockerfile ready, use the ``docker +build --tag=`` command to build the container image. diff --git a/source/development/building-containers/standalone-containers.rst b/source/development/building-containers/standalone-containers.rst new file mode 100644 index 0000000..93e17ce --- /dev/null +++ b/source/development/building-containers/standalone-containers.rst @@ -0,0 +1,120 @@ +Standalone containers +===================== + + +About +----- + +Standalone containers enable you to use containers in a similar way as you use +traditional system services. The goal is to preserve muscle memory so you don't +have to learn new commands while still being able to work with containers. + + +Requirements +------------ + +- Only a single instance of a standalone container is supported to run on a + host (installation of the same container image twice won't be successful). +- Services are managed via systemd unit files. +- Containers are installed to a system. +- There is no dedicated network namespace. +- Configuration is stored in the same locations on the host as a + non-containerized service. +- Persistent data can be found in the same locations on the host as for + non-containerized service. +- Service presence, configuration and persistent data are tracked by the RPM + database. +- Application lifecycle is close to services deployed as RPMs. +- Some commands may need to be executed inside the container since the + service is not running directly on host. + + +Quick how-to +------------ + +We utilize the atomic command to install a container image on a host. + +In order to get files from a container image to the host, you should have a +specific directory structure inside your container image. Here's an example +of nginx container image: + +:: + + / + └── exports + └── hostfs + ├── etc + │   └── nginx + │   ├── conf.d + │   ├── default.d + │   ├── fastcgi.conf + │   ├── fastcgi.conf.default + │   ├── fastcgi_params + │   ├── fastcgi_params.default + │   ├── koi-utf + │   ├── koi-win + │   ├── mime.types + │   ├── mime.types.default + │   ├── nginx.conf + │   ├── nginx.conf.default + │   ├── scgi_params + │   ├── scgi_params.default + │   ├── uwsgi_params + │   ├── uwsgi_params.default + │   └── win-utf + └── usr + ├── lib + │   └── systemd + │   └── system + │   └── nginx-container.service + └── share + └── nginx + └── html + ├── 404.html + ├── 50x.html + ├── index.html + ├── nginx-logo.png + └── poweredby.png + + +- **exports** directory is what atomic uses, it is in a root of the container + image +- **hostfs** is a tree of directories and files which will land on the host, + tracked by a generated RPM +- a systemd unit :code:`nginx-container.service`, which controls the + containerized nginx, is placed in + :code:`/exports/hostfs/usr/lib/systemd/system` + +Here's the mentioned :code:`nginx-container.service`: + +:: + + [Unit] + Description="Standalone container version of NGINX webserver." + + [Service] + ExecStartPre=/usr/bin/docker create -t -i -v /etc/nginx:/etc/nginx/:ro --net=host -v /usr/share/nginx:/usr/share/nginx/:ro --name nginx-container modularitycontainers/nginx + ExecStart=/usr/bin/docker start -a nginx-container + ExecStop=/usr/bin/docker stop nginx-container + ExecStopPost=/usr/bin/docker rm -f nginx-container + + [Install] + WantedBy=multi-user.target + + +Once the image is built, you can install it like this: + +:: + + $ atomic install docker:modularitycontainers/nginx + + +There is no released version of atomic with this functionality, yet. The +version of atomic command built `in this copr repo +`__ contains the +functionality to install standalone container image. + +Most of the principles of standalone containers are based on the model & +technology of system containers. If you would like to know more about system +containers read `the blog post +`__. diff --git a/source/development/building-modules.rst b/source/development/building-modules.rst index 7ca35ac..e758dc1 100644 --- a/source/development/building-modules.rst +++ b/source/development/building-modules.rst @@ -9,9 +9,6 @@ Building Modules building-modules/developing building-modules/building-local - building-modules/containers-local - building-modules/standalone-containers building-modules/adding-modules-to-fedora building-modules/building-infra - building-modules/containers-infra building-modules/testing diff --git a/source/development/building-modules/containers-infra.rst b/source/development/building-modules/containers-infra.rst deleted file mode 100644 index 3cf58f7..0000000 --- a/source/development/building-modules/containers-infra.rst +++ /dev/null @@ -1,53 +0,0 @@ -Building containers in Fedora -============================= - -Your Dockerfile needs to be formally reviewed before becoming a part of Fedora. -The review process is described in the `Fedora wiki -`__. - - -Builds in production --------------------- - -Production builds of container images are documented in `this section -`__ of -the review process document mentioned above. Please follow the process to -perform an infrastructure build of your container image. - -In order to get the correct content (modules, RPMs) in your container image, -you should make sure that those artifacts are released and available: - -* For modules, this means to have a valid compose with your module. The module - needs to be incorporated inside `this pungi configuration file - `__, if - you want the module to be part of a compose. - -* For RPMs, the package should pass the Bodhi update process and be available - in repositories. - - -Inspecting registry -------------------- - -At the time of writing (Aug 2017), the Fedora docker image registry -doesn't have any frontend. You can access its API to get a list of available -images: - -:: - - $ curl -s https://registry.fedoraproject.org/v2/_catalog | \ - python -m json.tool - { - "repositories": [ - "cockpit", - "f24/cockpit", - "f25/cockpit", - "f25/kube-apiserver", - "f25/kubernetes-master", - "f25/owncloud", - "f26/cockpit", - "fedora", - "fedora/cockpit", - "openshift/origin-pod" - ] - } diff --git a/source/development/building-modules/containers-local.rst b/source/development/building-modules/containers-local.rst deleted file mode 100644 index abcc368..0000000 --- a/source/development/building-modules/containers-local.rst +++ /dev/null @@ -1,62 +0,0 @@ -Building containers locally -=========================== - -When you have your module built, let's put it in a container, so we can -use it. As an example, we will use the `perl image -`__. - -Module RPM repository ---------------------- - -First, upload your RPM repository from the previous step somewhere publicly -accessible. For example, you can use Fedorapeople to host your packages: - -:: - - $ mkdir module - $ cp $RPMS module/ - $ createrepo_c module/ - $ rsync ./module/ $USER@fedorapeople.org:public_html/ - -Dockerfile ----------- - -Next step will be writing a Dockerfile to build a container image with your -module. - -The Base Runtime image is currently only available from Docker Hub and you can -use it as your base image by specifying ``FROM /baseruntime/baseruntime``. - -You also need to write a repo file for your module and add it to your -container, so you can install the module in it. See the example of the repo -file in ``__. The -following snippet then shows how you copy the repo and install the files from -it in your container: - -:: - - COPY repos/* /etc/yum.repos.d/ - # Perl and build tools install + user addition - RUN BUILD_TOOlS="bsdtar \ - findutils \ - gcc \ - make \ - gettext \ - tar \ - wget \ - python " && \ - microdnf --nodocs --enablerepo perl install perl perl-devel && \ - microdnf --nodocs --enablerepo fedora install -y mod_perl cpan cpanminus httpd \ - $BUILD_TOOlS && \ - microdnf clean all - RUN mkdir -p /opt/app-root/src/ && \ - useradd -u 1002 -r -g 0 -d /opt/app-root/src -s /sbin/nologin \ - -c "Default Application User" default && \ - chown -R 1002:0 /opt/app-root - - -Building the image -------------------- - -Finally, when you have your repository and Dockerfile ready, use the ``docker -build --tag=`` command to build the container image. diff --git a/source/development/building-modules/standalone-containers.rst b/source/development/building-modules/standalone-containers.rst deleted file mode 100644 index 93e17ce..0000000 --- a/source/development/building-modules/standalone-containers.rst +++ /dev/null @@ -1,120 +0,0 @@ -Standalone containers -===================== - - -About ------ - -Standalone containers enable you to use containers in a similar way as you use -traditional system services. The goal is to preserve muscle memory so you don't -have to learn new commands while still being able to work with containers. - - -Requirements ------------- - -- Only a single instance of a standalone container is supported to run on a - host (installation of the same container image twice won't be successful). -- Services are managed via systemd unit files. -- Containers are installed to a system. -- There is no dedicated network namespace. -- Configuration is stored in the same locations on the host as a - non-containerized service. -- Persistent data can be found in the same locations on the host as for - non-containerized service. -- Service presence, configuration and persistent data are tracked by the RPM - database. -- Application lifecycle is close to services deployed as RPMs. -- Some commands may need to be executed inside the container since the - service is not running directly on host. - - -Quick how-to ------------- - -We utilize the atomic command to install a container image on a host. - -In order to get files from a container image to the host, you should have a -specific directory structure inside your container image. Here's an example -of nginx container image: - -:: - - / - └── exports - └── hostfs - ├── etc - │   └── nginx - │   ├── conf.d - │   ├── default.d - │   ├── fastcgi.conf - │   ├── fastcgi.conf.default - │   ├── fastcgi_params - │   ├── fastcgi_params.default - │   ├── koi-utf - │   ├── koi-win - │   ├── mime.types - │   ├── mime.types.default - │   ├── nginx.conf - │   ├── nginx.conf.default - │   ├── scgi_params - │   ├── scgi_params.default - │   ├── uwsgi_params - │   ├── uwsgi_params.default - │   └── win-utf - └── usr - ├── lib - │   └── systemd - │   └── system - │   └── nginx-container.service - └── share - └── nginx - └── html - ├── 404.html - ├── 50x.html - ├── index.html - ├── nginx-logo.png - └── poweredby.png - - -- **exports** directory is what atomic uses, it is in a root of the container - image -- **hostfs** is a tree of directories and files which will land on the host, - tracked by a generated RPM -- a systemd unit :code:`nginx-container.service`, which controls the - containerized nginx, is placed in - :code:`/exports/hostfs/usr/lib/systemd/system` - -Here's the mentioned :code:`nginx-container.service`: - -:: - - [Unit] - Description="Standalone container version of NGINX webserver." - - [Service] - ExecStartPre=/usr/bin/docker create -t -i -v /etc/nginx:/etc/nginx/:ro --net=host -v /usr/share/nginx:/usr/share/nginx/:ro --name nginx-container modularitycontainers/nginx - ExecStart=/usr/bin/docker start -a nginx-container - ExecStop=/usr/bin/docker stop nginx-container - ExecStopPost=/usr/bin/docker rm -f nginx-container - - [Install] - WantedBy=multi-user.target - - -Once the image is built, you can install it like this: - -:: - - $ atomic install docker:modularitycontainers/nginx - - -There is no released version of atomic with this functionality, yet. The -version of atomic command built `in this copr repo -`__ contains the -functionality to install standalone container image. - -Most of the principles of standalone containers are based on the model & -technology of system containers. If you would like to know more about system -containers read `the blog post -`__.