Merge pull request #866 from usrpro/bors-ng

Automatic creation of review images

.mergify.yml

@@ -0,0 +1,51 @@
+pull_request_rules:
+
+  - name: Build testing images; trigger bors try
+    conditions:
+      - -title~=(WIP|wip)
+      - -label~=^(status/wip|status/blocked)$
+    actions:
+      comment:
+        message: |
+          Thanks for submitting this pull request.
+          Bors-ng will now build test images. When it succeeds, we will continue to review and test your PR.
+          
+          bors try
+          
+          Note: if this build fails, [read this](http://mailu.io/master/contributors/environment.html#when-bors-try-fails).
+
+  - name: 2 approved reviews; trigger bors r+
+    conditions:
+      - -title~=(WIP|wip)
+      - -label~=^(status/wip|status/blocked)$
+      - "#approved-reviews-by>=2"
+    actions:
+      comment:
+        message: bors r+
+
+  - name: Trusted author and 1 approved review; trigger bors r+
+    conditions:
+      - author~=^(kaiyou|muhlemmer|mildred|HorayNarea|adi90x|hoellen|ofthesun9|Nebukadneza|ionutfilip)$
+      - -title~=(WIP|wip)
+      - -label~=^(status/wip|status/blocked|review/need2)$
+      - "#approved-reviews-by>=1"
+    actions:
+      comment:
+        message: bors r+
+
+  - name: Backport to 1.6 branch
+    conditions:
+      - base=master
+      - label=type/backport
+    actions:
+      backport:
+        branches:
+          - '1.6'
+
+  - name: remove outdated reviews
+    conditions:
+      - base~=^(master|1.6)$
+    actions:
+      dismiss_reviews:
+        approved: True
+

.travis.yml

@@ -1,3 +1,11 @@
+branches:
+  only:
+    - staging
+    - testing
+    - '1.5'
+    - '1.6'
+    - master
+
 sudo: required
 services: docker
 addons:
@@ -6,7 +14,8 @@ addons:
       - docker-ce
 
 env:
-  - MAILU_VERSION=$TRAVIS_BRANCH
+  - MAILU_VERSION=${TRAVIS_BRANCH////-}
+
 language: python
 python:
   - "3.6"
@@ -20,6 +29,7 @@ before_script:
   - docker-compose -f tests/build.yml build
   - sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*'
 
+
 script:
 # test.py, test name and timeout between start and tests.
   - python tests/compose/test.py core 1

bors.toml

@@ -0,0 +1,3 @@
+status = [
+  "continuous-integration/travis-ci/push"
+]

docs/contributors/environment.rst

@@ -173,10 +173,20 @@ Finally, if you need to install packages inside the containers for debugging:
 Reviewing
 ---------
 
+Members of the **Mailu/contributors** team leave reviews to open PR's.
+In the case of a PR from a fellow team member, a single review is enough
+to initiate merging. In all other cases, two approving reviews are required.
+There is also a possibility to set the ``review/need2`` to require a second review.
+
+After Travis successfully tests the PR and the required amount of reviews are acquired,
+Mergify will trigger with a ``bors r+`` command. Bors will batch any approved PR's,
+merges them with master in a staging branch where Travis builds and tests the result.
+After a successful test, the actual master gets fast-forwarded to that point.
+
 System requirements
 ```````````````````
 
-Reviewing pull requests requires some additional git setup. First, for 90% of the review jobs,
+Reviewing pull requests sometimes requires some additional git setup. First, for 90% of the review jobs,
 you will need a PC or server that can expose all Mailu ports to the outside world. Also, a valid
 domain name would be required. This can be a simple free DynDNS account. Do not use a production
 server, as there are cases where data corruption occurs and you need to delete the ``/mailu``
@@ -188,6 +198,67 @@ He can provide access to a testing server, if a trust relation can be establishe
 
 .. _`muhlemmer on Matrix`: https://matrix.to/#/@muhlemmer:matrix.org
 
+.. _testing:
+
+Test images
+```````````
+
+All PR's automatically get build by Travis, controlled by `bors-ng`_.
+Some primitive auto testing is done.
+The resulting images get uploaded to Docker hub, under the
+tag name ``mailutest/<name>:pr-<no>``.
+
+For example, to test PR #500 against master, reviewers can use:
+
+.. code-block:: bash
+
+  export DOCKER_ORG="mailutest"
+  export MAILU_VERSION="pr-500"
+  docker-compose pull
+  docker-compose up -d
+
+You can now test the PR. Play around. See if (external) mails work. Check for whatever functionality the PR is
+trying to fix. When happy, you can approve the PR. When running into failures, mark the review as
+"request changes" and try to provide as much as possible details on the failure.
+(Logs, error codes from clients etc).
+
+.. _`bors-ng`: https://bors.tech/documentation/
+
+Additional commits
+``````````````````
+
+Sometimes users add new commits after ``bors try`` was  run automatically.
+In such cases, a reviewer will have to re-issue a ``bors try`` manually in order
+to get the latest changes in the test image. The reviewer will have to be sure the
+build finished successful before pulling the new images.
+
+Any previous reviews get dismissed automatically, whenever a new commit is done afterwards.
+
+When bors try fails
+```````````````````
+
+Sometimes Travis fails when another PR triggers a ``bors try`` command,
+before Travis cloned the git repository.
+Inspect the build log in the link provided by *bors-ng* to find out the cause.
+If you see something like the following error on top of the logs,
+feel free to write a comment with ``bors retry``.
+
+.. code-block:: bash
+
+  The command "git checkout -qf <hash>" failed and exited with 128 during .
+
+Please wait a few minutes to do so, not to interfere with other builds.
+Also, don't abuse this command if anything else went wrong,
+the author needs to try to fix it instead!
+
+Reviewing by git
+----------------
+
+Sometimes it might not be possible or enough to pull the test images from Docker hub.
+In those cases, it will be necessary to do a local git merge and perhaps manually building
+of the relevant images.
+
+
 Preparations
 ````````````
 
@@ -243,18 +314,6 @@ The following must be done on every PR or after every new commit to an existing
 If git opens a editor for a commit message just save and exit as-is. If you have a merge conflict,
 see above and do the complete procedure from ``git fetch`` onward again.
 
-Test
-````
-
-You can now build and run the containers for testing. See the "`Docker containers`_" section for
-instructions. Play around. See if (external) mails work. Check for whatever functionality the PR is
-trying to fix. When happy, you can approve the PR. When running into failures, mark the review as
-"request changes" and try to provide as much as possible details on the failure.
-(Logs, error codes form clients etc).
-
-.. note:: Github marks positive reviews as obsolete when a new commit is added to a PR.
-  This requires a new review from your side.
-
 Web administration
 ------------------
 

docs/contributors/guide.rst

@@ -52,15 +52,19 @@ master directly if you find this appropriate. Still, keep in mind that:
   that you use branch names prefixed with ``feat-`` or ``fix-`` and followed
   either by the name of the Github issue or a short and meaningful name.
 
-Workflow
-````````
+PR Workflow
+````````````
 
-All commits will be merged to the main ``master`` branch for testing. New
-images are built by Docker Hub with the ``testing`` tag for each new commit on
-the ``master`` branch.
+All pull requests have to be against the main ``master`` branch.
+The PR gets build by Travis and some primitive auto-testing is done.
+Test images get uploaded to a separate section in Docker hub.
+Reviewers will check the PR and test the resulting images.
+See the :ref:`testing` section for more info.
 
-After some brief testing, urgent fixes will be cherry-picked to the current stable
-branch and new stable builds will be released.
+Urgent fixes can be backported to the stable branch.
+For this a member of **Mailu/contributors** has to set the ``type/backport`` label.
+Upon merge of the original PR, a copy of the PR is created against the stable branch.
+After some testing on master, we will approve and merge this new PR as well.
 
 At the end of every milestone, a new stable branch will be created from ``master``
 or any previous commit that matches the completion of the milestone.

tests/deploy.sh

@@ -1,4 +1,16 @@
 #!/bin/bash
 
+# Skip deploy for staging branch
+[ "$TRAVIS_BRANCH" = "staging" ] && exit 0
+
+# Retag in case of `bors try`
+if [ "$TRAVIS_BRANCH" = "testing" ]; then
+    export DOCKER_ORG="mailutest"
+    # Commit message is like "Try #99".
+    # This sets the version tag to "pr-99"
+    export MAILU_VERSION="pr-${TRAVIS_COMMIT_MESSAGE//[!0-9]/}"
+    docker-compose -f tests/build.yml build
+fi
+
 docker login -u $DOCKER_UN -p $DOCKER_PW
 docker-compose -f tests/build.yml push