70 Commits (802ab533d25cb16dda1188a078b216199fab345a)

Author SHA1 Message Date
Florent Daigniere 3e45a791cf Implement oletools to filter out bad macros 2 years ago
Florent Daigniere d3d7916b58 Merge remote-tracking branch 'upstream/master' into upgrade-alpine 2 years ago
Florent Daigniere e79d7fed55 Reduce the number of warnings on the CI 2 years ago
Florent Daigniere dc9e2a3e70 Upgrade Snappymail to 2.21 and merge the webmail containers 2 years ago
Dimitri Huisman 2a3266b6b8
Forgot to update both deploy jobs 2 years ago
Dimitri Huisman b2e47642f7
Tag the images with latest tag as well. 2 years ago
Florent Daigniere 3e9def6cd9 Use the new notation: arm64/v8 instead of arm64 2 years ago
Dimitri Huisman db7ce8c83e
Login docker.io to prevent rate limiting for pulling images 2 years ago
Dimitri Huisman b3151e9904
Actually push the build arm images to ghcr.io 2 years ago
Dimitri Huisman 6549dbf247
Sigh. needs.* context is only available if you include it in needs: 2 years ago
Dimitri Huisman c7cba1b075
Finishing touches for fixing arm builds
- Use self-hosted runners for arm base image
- Use seperate docker image cache for arm build
- Remove unneeded needs items.
2 years ago
Dimitri Huisman e915e444e9
Remove superfluous cache export entry for arm 2 years ago
Dimitri Huisman 4be0cbf4da
Switch workflow to ghcr.io
- Build images & build cache are pushed to ghcr.io.
- Tests will make use of the images pushed to ghcr.io.
- Deploy step only copies images from ghcr.io to docker.io.
- Resolves strange build errors tied to buildx+intermediate builds
- Results in quicker build times.
2 years ago
Dimitri Huisman 451738e32b We want the function result. Not the function statement. 2 years ago
Dimitri Huisman f9ba0e688f Removed syntax error 2 years ago
Dimitri Huisman 92cb8c146b Refine build_test_deploy.yml:
Build base image before the other images.
Change cache key to make it is re-used for all builds. This is not
dangerous. The docker build process can determine itself whether
a cache can be safely re-used or not.
2 years ago
Florent Daigniere 254277a829
runner.temp is what should be used 2 years ago
Dimitri Huisman 22fe65e4d8 Use RUNNER_TEMP for storing cache files. 2 years ago
Dimitri Huisman 56d0e795eb
Push arm images via arm self-hosted runner 2 years ago
Dimitri Huisman d76d3b4959 Switch to ARM64 self-hosted for ARM build 2 years ago
Dimitri Huisman 5732b2316f Switch to local build cache, cached via actions/cache@v3
The previous method of using gha cache via buildx proved to be
unreliable. Using local cache via actions/cache@v3 is much more
reliable.
The build job will re-use cache from previous workflow runs.
The total workflow time is still similar ~12 minutes.
2 years ago
Dimitri Huisman 312a733ae3 prefix the cache key with a hash (to help with sharding) 2 years ago
Dimitri Huisman f35d82b3a0 Also ${{ github.run_id }} for cache key.
This makes rue that the cache key is unique across workflow runs
2 years ago
Dimitri Huisman d56eb16f52 Hotfix for workflow. For build step do not build from cache.
Make sure cache layers are unique by using ${{ github.ref }}
2 years ago
Dimitri Huisman dd3f1a3376 Switch to mode=min for GHA cache for docker buildx to prevent ratelimiting in GHA workflow 2 years ago
Dimitri Huisman ae18217e07 Fix adding -arm tag correctly to PINNED_MAILU_VERSION in arm.yml. 2 years ago
Dimitri Huisman 7e21ab4007 Merge master cont'd 2 years ago
Dimitri Huisman f6de2b2938 Switch from docker build to buildx for CI/CD.
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it.
- Arm images can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images.
  This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
2 years ago
Dimitri Huisman 413ecea479 Switched from "$(/usr/bin/git log -1 --format='%H')" to "$(/usr/bin/git rev-parse HEAD)" 3 years ago
Dimitri Huisman 5cef8925dc Remove superfluous parentheses 3 years ago
Dimitri Huisman cf328cdf33 Tag for release is correctly tagged to branch x.y. 3 years ago
Dimitri Huisman b4d3d4b3c9 Preparations for 1.9 release. 3 years ago
Dimitri Huisman 2efad07c0b Merge branch 'master' of github.com:Diman0/Mailu into remove-mailu-postgresql 3 years ago
bors[bot] 08be233607
Merge #2058
2058: Implement versioning for CI/CD workflow. r=mergify[bot] a=Diman0

## What type of PR?

Feature!

## What does this PR do?
This PR introduces 3 things
- Add versioning (tagging) for branch x.y (1.8). E.g. 1.8.0, 1.8.1 etc.
  - docker repo will contain x.y (latest) and x.y.z (pinned version) images.
  - The X.Y.Z tag is incremented automatically. E.g. if 1.8.0 already exists, then the next merge on 1.8 will result in the new tag 1.8.1 being used.
- Make the version available in the image.
  -  For X.Y and X.Y.Z write the version (X.Y.Z) into /version on the image and add a label with version=X.Y.Z
	  -  This means that the latest X.Y image shows the pinned version (X.Y.Z e.g. 1.8.1) it was based on. Via the tag X.Y.Z you can see the commit hash that triggered the built.
  -  For master write the commit hash into /version on the image and add a label with version={commit hash}
-  Automatic releases. For x.y triggered builts (e.g. merge on 1.9) do a new github release for the pinned x.y.z (e.g. 1.9.2). 
  -  Release shows a static message (see RELEASE_TEMPLATE.md) that explains how to reach the newsfragments folder and change the branch to the tag (x.y.z) mentioned in the release. Now you can get the changelog by reading all newsfragment files in this folder.

This PR does not change anything to our workflow (what we (human persons) do). Our processes are still exactly the same. The above introduced logic is automatic. When we backport to X.Y all the magic for creating the pinned version X.Y.Z is handled by the CI/CD workflow.

### Related issue(s)
- closes #1182

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

## Testing
Suggested testing steps. This should cover all situations including BORS. It does require that you use your own docker repo or temporarily create a new one.
Suggested testing steps.
1. Create new github repo.
2. Add the required docker secrets to the project (see beginning of CI.yml for the secret names), DOCKER_UN, DOCKER_PW, DOCKER_ORG, DOCKER_ORG_TESTS.
3. Clone the project.
4. Copy the contents of the PR to the cloned project.
5. Push to your new github repo.
6. Now master images are built. Check that images with tag master are pushed to your docker repo
7. Check with docker inspect nginx:master that it has the label version={commit hash}.
8. Run an image, run `docker-compose exec <name> cat /version`. Note that /version also contains the pinned version. For master the pinned version is the commit hash.
9. Create branch 1.8. 
10. Push branch 1.8 to repo.
11. Note that tags 1.8 and 1.8.0 are built and pushed to docker repo
12. Inspect label and /version. Note that 1.8 and 1.8.0 both show version 1.8.0.
13. Push another commit to branch 1.8.
14. Note that tags 1.8 and 1.8.1 are built and pushed to docker repo
15. Inspect label and /version. Note that 1.8 and 1.8.1 both show version 1.8.1.
16. Let's check BORS stuff.
17. Create branch testing.
18. Push the commit with the exact commit text (IMPORTANT!!): `Try #1234:`'.
19. Note that images are built and pushed for tag `pr-1234`.
20. Inspect label and /version. Note that the version is `pr-1234`.
20. Create branch staging.
21. Push the commit with commit text: `Merge #1234`.
22. Note that this image is not pushed to docker (as expected).

but you could also check the GH repo and docker repo I used:
https://github.com/Diman0/Mailu_Fork
https://hub.docker.com/r/diman/rainloop/tags

Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
3 years ago
Dimitri Huisman 15e64e8e50 Add concurrency to ensure that only a single workflow can run for a branch. 3 years ago
Dimitri Huisman 9eba2feae2 Remove Mailu PostgreSQL from CI.yml. 3 years ago
Dimitri Huisman 6cb8f101d9 Update stale bot with clearer message why an issue is marked stale. 3 years ago
Dimitri Huisman f7677543c6 Process code review remarks
- Moved run to bottom of Dockerfile to allow using unmodified / cached states.
- Simplified bash code in deploy.sh.
- Improved the large bash one-liner in CI.yml. It could not handle >9 for 1.x.
3 years ago
Dimitri Huisman 56dd70cf4a Implement versioning for CI/CD workflow (see #1182). 3 years ago
Dimitri Huisman b20d0a83d5 Doh! 3 years ago
Dimitri Huisman b391692698 It is handy to close strings. 3 years ago
Dimitri Huisman e2512c7cdc Testing images are pushed to DOCKER_ORG_TESTS again. 3 years ago
Diman0 3157fc3623 Give docker containers in each test one more minute for starting. 3 years ago
Dimitri Huisman 609e0f9f7c
Env vars are not shared between jobs 3 years ago
Dimitri Huisman c3f47f1ca0 Forgot that env var set in job 1 is not shared with job 2. Added logic for deriving MAILU_VERSION to deploy job. 3 years ago
Dimitri Huisman 58e751415c Yet another small typo. One day computers will understand I meant env.MAILU_BRANCH instead of MAILU_BRANCH. 3 years ago
Dimitri Huisman 7aa9b496fa Moved logic for building testing images from deploy job to build job. This should save time. Test number for ci workflow #1234 3 years ago
Dimitri Huisman 1566dfb077
Forgot to add condition to load docker images step 3 years ago
Dimitri Huisman b2840fed26
Update CI.yml 3 years ago
Dimitri Huisman cf894dae03
Merge branch 'Mailu:master' into test-ci-parallel 3 years ago