- Build images & build cache are pushed to ghcr.io.
- Tests will make use of the images pushed to ghcr.io.
- Deploy step only copies images from ghcr.io to docker.io.
- Resolves strange build errors tied to buildx+intermediate builds
- Results in quicker build times.
Build base image before the other images.
Change cache key to make it is re-used for all builds. This is not
dangerous. The docker build process can determine itself whether
a cache can be safely re-used or not.
The previous method of using gha cache via buildx proved to be
unreliable. Using local cache via actions/cache@v3 is much more
reliable.
The build job will re-use cache from previous workflow runs.
The total workflow time is still similar ~12 minutes.
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it.
- Arm images can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images.
This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
2058: Implement versioning for CI/CD workflow. r=mergify[bot] a=Diman0
## What type of PR?
Feature!
## What does this PR do?
This PR introduces 3 things
- Add versioning (tagging) for branch x.y (1.8). E.g. 1.8.0, 1.8.1 etc.
- docker repo will contain x.y (latest) and x.y.z (pinned version) images.
- The X.Y.Z tag is incremented automatically. E.g. if 1.8.0 already exists, then the next merge on 1.8 will result in the new tag 1.8.1 being used.
- Make the version available in the image.
- For X.Y and X.Y.Z write the version (X.Y.Z) into /version on the image and add a label with version=X.Y.Z
- This means that the latest X.Y image shows the pinned version (X.Y.Z e.g. 1.8.1) it was based on. Via the tag X.Y.Z you can see the commit hash that triggered the built.
- For master write the commit hash into /version on the image and add a label with version={commit hash}
- Automatic releases. For x.y triggered builts (e.g. merge on 1.9) do a new github release for the pinned x.y.z (e.g. 1.9.2).
- Release shows a static message (see RELEASE_TEMPLATE.md) that explains how to reach the newsfragments folder and change the branch to the tag (x.y.z) mentioned in the release. Now you can get the changelog by reading all newsfragment files in this folder.
This PR does not change anything to our workflow (what we (human persons) do). Our processes are still exactly the same. The above introduced logic is automatic. When we backport to X.Y all the magic for creating the pinned version X.Y.Z is handled by the CI/CD workflow.
### Related issue(s)
- closes#1182
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
## Testing
Suggested testing steps. This should cover all situations including BORS. It does require that you use your own docker repo or temporarily create a new one.
Suggested testing steps.
1. Create new github repo.
2. Add the required docker secrets to the project (see beginning of CI.yml for the secret names), DOCKER_UN, DOCKER_PW, DOCKER_ORG, DOCKER_ORG_TESTS.
3. Clone the project.
4. Copy the contents of the PR to the cloned project.
5. Push to your new github repo.
6. Now master images are built. Check that images with tag master are pushed to your docker repo
7. Check with docker inspect nginx:master that it has the label version={commit hash}.
8. Run an image, run `docker-compose exec <name> cat /version`. Note that /version also contains the pinned version. For master the pinned version is the commit hash.
9. Create branch 1.8.
10. Push branch 1.8 to repo.
11. Note that tags 1.8 and 1.8.0 are built and pushed to docker repo
12. Inspect label and /version. Note that 1.8 and 1.8.0 both show version 1.8.0.
13. Push another commit to branch 1.8.
14. Note that tags 1.8 and 1.8.1 are built and pushed to docker repo
15. Inspect label and /version. Note that 1.8 and 1.8.1 both show version 1.8.1.
16. Let's check BORS stuff.
17. Create branch testing.
18. Push the commit with the exact commit text (IMPORTANT!!): `Try #1234:`'.
19. Note that images are built and pushed for tag `pr-1234`.
20. Inspect label and /version. Note that the version is `pr-1234`.
20. Create branch staging.
21. Push the commit with commit text: `Merge #1234`.
22. Note that this image is not pushed to docker (as expected).
but you could also check the GH repo and docker repo I used:
https://github.com/Diman0/Mailu_Forkhttps://hub.docker.com/r/diman/rainloop/tags
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
- Moved run to bottom of Dockerfile to allow using unmodified / cached states.
- Simplified bash code in deploy.sh.
- Improved the large bash one-liner in CI.yml. It could not handle >9 for 1.x.
Workflow file only triggers if it resides in the branch. If an old PR based on the mailu repo without CI.yml is tested, then the workflow run will not trigger. The merged commit on TESTING/STAGING branch does not contain the required CI.yml workflow file after all. In these cases simply run the workflow manually on the TESTING or STAGING branch,
Dimitri Huisman