keep key during certificate renewal

2719: Introduce connection string (database url) for roundcube & simplify setup r=mergify[bot] a=Diman0

## What type of PR?

enhancement

## What does this PR do?
As discussed in earlier Mailu meetings (#1582), we want to simplify configuring Mailu and make it more user-friendly. Especially the last part is an important mission statement of the Mailu project. 

This PR will remove the choice of what DB to use from setup. New users are guided now to make the correct choice of using SQLite.

For simplifying the configuration, all the database environment variables have been removed and replaced with a single connection string environment variable. 

For backwards compatibility, the old *DB_* setting can still be used. This is to make sure that master does not immediately break for all users. After X months after the next Mailu release, we can remove the old settings from the software. This provides a transition period. 

### Related issue(s)
- #2533

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>

.github/workflows/arm.yml

@@ -1,101 +0,0 @@
-name: start-linux-arm
-on:
-  push:
-    branches:
-      - '1.9'
-      - master
-
-concurrency: ci-arm-${{ github.ref }}
-
-###############################################
-# REQUIRED secrets
-# ${{ secrets.Docker_Login }}
-#    Username of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
-# ${{ secrets.Docker_Password }}
-#    Password of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
-# Add the above secrets to your github repo to determine where the images will be pushed.
-################################################
-# REQUIRED global variables
-# DOCKER_ORG, docker org used for pushing release images (branch x.y and master)
-# DOCKER_ORG_TEST, docker org used for pushing images for testing (branch testing).
-env:
-  DOCKER_ORG: mailu
-  DOCKER_ORG_TEST: mailuci
-
-jobs:
-# This job calculates all global job variables that are required by all the subsequent jobs.
-# All subsequent jobs will retrieve and use these variables. This way the variables only have to be derived once.
-  derive-variables:
-    name: derive variables
-    runs-on: ubuntu-latest
-    outputs:
-        MAILU_VERSION: ${{ env.MAILU_VERSION }}
-        PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        DOCKER_ORG: ${{ env.DOCKER_ORG_DERIVED }}
-        BRANCH: ${{ env.BRANCH }}
-        DEPLOY: ${{ env.DEPLOY }}
-        RELEASE: ${{ env.RELEASE }}
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          # fetch-depth 0 is required to also retrieve all tags.
-          fetch-depth: 0
-      - name: Extract branch name
-        shell: bash
-        run: |
-          echo "BRANCH=${GITHUB_REF#refs/heads/}" >> $GITHUB_ENV
-        #For branch TESTING, we set the image tag to pr-xxxx
-      - name: Derive MAILU_VERSION and DEPLOY/RELEASE for other branches than testing
-        if: env.BRANCH != 'testing'
-        shell: bash
-        run: |
-          echo "MAILU_VERSION=${{ env.BRANCH }}" >> $GITHUB_ENV
-          echo "DOCKER_ORG_DERIVED=${{ env.DOCKER_ORG }}" >> $GITHUB_ENV
-          echo "DEPLOY=true" >> $GITHUB_ENV
-          echo "RELEASE=false" >> $GITHUB_ENV
-      - name: Derive PINNED_MAILU_VERSION and DEPLOY/RELEASE for normal release x.y
-        if: env.BRANCH != 'testing' && env.BRANCH != 'staging' && env.BRANCH != 'master'
-        shell: bash
-        run: |
-          version=$( git tag --sort=version:refname --list "${{ env.MAILU_VERSION }}.*" | tail -1  );root_version=${version%.*};patch_version=${version##*.};if [ "$patch_version" == "" ]; then pinned_version=${{ env.MAILU_VERSION }}.0; else pinned_version=$root_version.$(expr $patch_version + 1); fi;echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV
-          echo "RELEASE=true" >> $GITHUB_ENV
-          echo "DEPLOY=true" >> $GITHUB_ENV
-          echo "RELEASE=true" >> $GITHUB_ENV
-      - name: Derive PINNED_MAILU_VERSION for staging for master
-        if: env.BRANCH == 'master'
-        shell: bash
-        env:
-          GITHUB_SHA: ${{ env.GITHUB_SHA }}
-        run: |
-          echo "PINNED_MAILU_VERSION=$GITHUB_SHA" >> $GITHUB_ENV
-          echo "DEPLOY=true" >> $GITHUB_ENV
-          echo "RELEASE=false" >> $GITHUB_ENV
-
-  build-test-deploy:
-    needs:
-      - derive-variables
-    uses: ./.github/workflows/build_test_deploy.yml
-    with:
-      architecture: 'linux/arm64,linux/arm/v7'
-      mailu_version: ${{needs.derive-variables.outputs.MAILU_VERSION}}-arm
-      pinned_mailu_version: ${{needs.derive-variables.outputs.PINNED_MAILU_VERSION}}-arm
-      docker_org: ${{needs.derive-variables.outputs.DOCKER_ORG}}
-      branch: ${{needs.derive-variables.outputs.BRANCH}}
-      deploy: ${{needs.derive-variables.outputs.DEPLOY}}
-      release: ${{needs.derive-variables.outputs.RELEASE}}
-    secrets: inherit
-
-################################################
-# Code block that is used as one liner for the step:
-# Derive PINNED_MAILU_VERSION and DEPLOY/RELEASE for normal release x.y
-##!/bin/bash
-#version=$( git tag --sort=version:refname --list "{{ env.MAILU_VERSION }}.*" | tail -1  )
-#root_version=${version%.*}
-#patch_version=${version##*.}
-#if [ "$patch_version" == "" ]
-#then
-#  pinned_version={{ env.MAILU_VERSION }}.0
-#else
-#  pinned_version=$root_version.$(expr $patch_version + 1)
-#fi
-#echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV

.github/workflows/build_test_deploy.yml

@@ -1,11 +1,23 @@
+###############################################
+# REQUIRED secrets
+# ${{ secrets.Docker_Login }}
+#    Username of docker login for logging in docker for pulling images (higher pull rate limit)
+# ${{ secrets.Docker_Password }}
+#    Password of docker login for logging in docker for pulling images (higher pull rate limit)
+# ${{ secrets.Docker_Login2 }}
+#    Second Username of docker login for logging in docker for pulling images (higher pull rate limit)
+# ${{ secrets.Docker_Password2 }}
+#    Second Password of docker login for logging in docker for pulling images (higher pull rate limit)
+################################################
+
 name: build-test-deploy
 on:
   workflow_call:
     inputs:
       architecture:
-        description: 'The architecture of the images that will be build.'
+        description: 'The architecture(s) of the images that will be build. linux/amd64 or linux/arm64/v8,linux/arm/v7 or linux/amd64,linux/arm64/v8,linux/arm/v7'
         required: false
-        default: 'linux/amd64'
+        default: 'linux/amd64,linux/arm64/v8,linux/arm/v7'
         type: string
       mailu_version:
         description: 'The main version that is build. E.g. master or x.y.'
@@ -16,7 +28,7 @@ on:
         required: true
         type: string
       docker_org:
-        description: 'The docker organisation where the images are pushed to.'
+        description: 'The docker organisation where the images are pushed to. E.g. ghcr.io/mailu'
         required: true
         type: string
       branch:
@@ -24,7 +36,7 @@ on:
         required: true
         type: string
       deploy:
-        description: Deploy to docker hub. Happens for all branches but staging. Use string true or false.
+        description: Deploy to container registry. Happens for all branches but staging. Use string true or false.
         default: true
         required: false
         type: string
@@ -37,9 +49,9 @@ on:
   workflow_dispatch:
     inputs:
       architecture:
-        description: 'The architecture of the images that will be build.'
+        description: 'The architecture(s) of the images that will be build. linux/amd64 or linux/arm64/v8,linux/arm/v7 or linux/amd64,linux/arm64/v8,linux/arm/v7'
         required: false
-        default: 'linux/amd64'
+        default: 'linux/amd64,linux/arm64/v8,linux/arm/v7'
         type: string
       mailu_version:
         description: 'The main version that is build. E.g. master or x.y.'
@@ -50,7 +62,7 @@ on:
         required: true
         type: string
       docker_org:
-        description: 'The docker organisation where the images are pushed to.'
+        description: 'The docker organisation where the images are pushed to. E.g. ghcr.io/mailu'
         required: true
         type: string
       branch:
@@ -58,15 +70,15 @@ on:
         required: true
         type: string
       deploy:
-        description: Deploy to docker hub. Happens for all branches but staging
+        description: Deploy to container registry. Happens for all branches but staging. Use string true or false.
         default: true
         required: false
-        type: boolean
+        type: string
       release:
-        description: 'Tag and create the github release. Only happens for branch x.y (release branch)'
+        description: Tag and create the github release. Use string true or false.
         default: false
         required: false
-        type: boolean
+        type: string
 env:
   HCL_FILE: ./tests/build.hcl
 
@@ -84,18 +96,156 @@ jobs:
       - name: Create matrix
         id: targets
         run: |
-          echo ::set-output name=matrix::$(docker buildx bake -f ${{env.HCL_FILE}} --print | jq -cr '.group.default.targets')
+          echo matrix=$(docker buildx bake -f ${{env.HCL_FILE}} --print | jq -cr '.group.default.targets') >> $GITHUB_OUTPUT
       - name: Show matrix
         run: |
           echo ${{ steps.targets.outputs.matrix }}
 
+## This job builds the base image. The base image is used by all other images.
+  build-base-image-x64:
+    name: Build base image x64
+    if: contains(inputs.architecture, 'linux/amd64')
+    needs:
+      - targets
+    runs-on: ubuntu-latest
+    permissions:
+     contents: read
+     packages: write
+    steps:
+      - uses: actions/checkout@v3
+      - name: Retrieve global variables
+        shell: bash
+        run: |
+          echo "BRANCH=${{ inputs.branch }}" >> $GITHUB_ENV
+          echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
+          echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
+          echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - uses: crazy-max/ghaction-github-runtime@v2
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v2
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
+        with:
+          string: ${{ github.repository_owner }}
+      - name: Get uuid
+        id: uuid
+        run: |
+            echo uuid=$RANDOM >> $GITHUB_OUTPUT
+      - name: Build docker base image with retry
+        env:
+          DOCKER_ORG: ghcr.io/${{ steps.string.outputs.lowercase }}
+          MAILU_VERSION: ${{ env.MAILU_VERSION }}
+          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          LABEL_VERSION: ${{ env.MAILU_VERSION }}
+          PINNED_LABEL_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          ARCH: 'linux/amd64'
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+          DOCKER_LOGIN: ${{ secrets.Docker_Login }}
+          DOCKER_PASSW: ${{ secrets.Docker_Password }}
+          BUILDX_NO_DEFAULT_ATTESTATIONS: 1
+        uses: nick-fields/retry@v2
+        with:
+          timeout_minutes: 20
+          retry_wait_seconds: 30
+          max_attempts: 3
+          shell: bash
+          command: |
+            set -euxo pipefail \
+              ; /usr/bin/docker info \
+              ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
+              ; echo "$DOCKER_PASSW" |    docker login --username "$DOCKER_LOGIN" --password-stdin \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
+              || echo "builder does not exist" \
+              ; /usr/bin/docker buildx create --name builder-${{ env.BUILDER }} --driver docker-container --use \
+              ; /usr/bin/docker buildx bake --file ./tests/build.hcl --set *.cache-from=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }} --set *.cache-to=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }},mode=max --set *.platform=${{ env.ARCH }} base \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+      - name: cleanup docker buildx instance after failure of build step
+        if: ${{ failure() }}
+        shell: bash
+        env:
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+        run: |
+          /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+
+## This job builds the base image. The base image is used by all other images.
+  build-base-image-arm:
+    name: Build base image arm
+    if: contains(inputs.architecture, 'linux/arm64/v8,linux/arm/v7')
+    needs:
+      - targets
+    runs-on: self-hosted
+    permissions:
+     contents: read
+     packages: write
+    steps:
+      - uses: actions/checkout@v3
+      - name: Retrieve global variables
+        shell: bash
+        run: |
+          echo "BRANCH=${{ inputs.branch }}" >> $GITHUB_ENV
+          echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
+          echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
+          echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v2
+      - uses: crazy-max/ghaction-github-runtime@v2
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
+        with:
+          string: ${{ github.repository_owner }}
+      - name: Get uuid
+        id: uuid
+        run: |
+            echo uuid=$RANDOM >> $GITHUB_OUTPUT
+      - name: Build docker base image with retry
+        env:
+          DOCKER_ORG: ghcr.io/${{ steps.string.outputs.lowercase }}
+          MAILU_VERSION: ${{ env.MAILU_VERSION }}-arm
+          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}-arm
+          LABEL_VERSION: ${{ env.MAILU_VERSION }}
+          PINNED_LABEL_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          ARCH: linux/arm64/v8,linux/arm/v7
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+          DOCKER_LOGIN2: ${{ secrets.Docker_Login2 }}
+          DOCKER_PASSW2: ${{ secrets.Docker_Password2 }}
+          BUILDX_NO_DEFAULT_ATTESTATIONS: 1
+        uses: nick-fields/retry@v2
+        with:
+          timeout_minutes: 30
+          retry_wait_seconds: 30
+          max_attempts: 10
+          shell: bash
+          command: |
+            set -euxo pipefail \
+              ; /usr/bin/docker info \
+              ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
+              ; echo "$DOCKER_PASSW2" |    docker login --username "$DOCKER_LOGIN2" --password-stdin \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
+              || echo "builder does not exist" \
+              ; /usr/bin/docker buildx create --name builder-${{ env.BUILDER }} --driver docker-container --use \
+              ; /usr/bin/docker buildx bake --file ./tests/build.hcl --set *.cache-from=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }}-arm --set *.cache-to=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }}-arm,mode=max --set *.platform=${{ env.ARCH }} base \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+      - name: cleanup docker buildx instance after failure of build step
+        if: ${{ failure() }}
+        shell: bash
+        env:
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+        run: |
+          /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+
+
 # This job builds all the images. The build cache is stored in the github actions cache.
 # In further jobs, this cache is used to quickly rebuild the images.
   build:
     name: Build images for linux/amd64
-    if: inputs.architecture == 'linux/amd64'
+    if: contains(inputs.architecture, 'linux/amd64')
     needs:
       - targets
+      - build-base-image-x64
     strategy:
       fail-fast: false
       matrix:
@@ -113,46 +263,64 @@ jobs:
           echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
           echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
-      - name: Configure actions/cache@v3 action for storing build cache in the ${{ runner.temp }}/cache folder
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/${{ matrix.target }}
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-${{ matrix.target }}-${{ github.run_id }}
-          restore-keys: |
-            ${{ github.ref }}-${{ inputs.mailu_version }}-${{ matrix.target }}
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - uses: crazy-max/ghaction-github-runtime@v2
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
-      - name: Login to Docker Hub
-        uses: docker/login-action@v2
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
         with:
-          username: ${{ secrets.Docker_Login }}
-          password: ${{ secrets.Docker_Password }}
-      - name: Build all docker images
+          string: ${{ github.repository_owner }}
+      - name: Get uuid
+        id: uuid
+        run: |
+            echo uuid=$RANDOM >> $GITHUB_OUTPUT
+      - name: Build docker image with retry
         env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        uses: docker/bake-action@v2
+          DOCKER_ORG: ghcr.io/${{ steps.string.outputs.lowercase }}
+          MAILU_VERSION: ${{ env.MAILU_VERSION }}-build
+          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}-build
+          LABEL_VERSION: ${{ env.MAILU_VERSION }}
+          PINNED_LABEL_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          ARCH: 'linux/amd64'
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+          DOCKER_LOGIN: ${{ secrets.Docker_Login }}
+          DOCKER_PASSW: ${{ secrets.Docker_Password }}
+          BUILDX_NO_DEFAULT_ATTESTATIONS: 1
+        uses: nick-fields/retry@v2
         with:
-          files: ${{env.HCL_FILE}}
-          targets: ${{ matrix.target }}
-          load: false
-          push: false
-          set: |
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/${{ matrix.target }}
-            *.cache-to=type=local,dest=${{ runner.temp }}/cache/${{ matrix.target }},mode=max
-            *.platform=${{ inputs.architecture }}
+          timeout_minutes: 20
+          retry_wait_seconds: 30
+          max_attempts: 3
+          shell: bash
+          command: |
+            set -euxo pipefail \
+              ; /usr/bin/docker info \
+              ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
+              ; echo "$DOCKER_PASSW" |   docker login --username "$DOCKER_LOGIN" --password-stdin \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
+              || echo "builder does not exist" \
+              ; /usr/bin/docker buildx create --name builder-${{ env.BUILDER }} --driver docker-container --use \
+              ; /usr/bin/docker buildx bake --push --file ./tests/build.hcl --set *.cache-from=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:buildcache --set *.cache-to=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:buildcache,mode=max --set *.cache-from=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }} --set *.platform=${{ env.ARCH }} ${{ matrix.target }} \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+      - name: cleanup docker buildx instance after failure of build step
+        if: ${{ failure() }}
+        shell: bash
+        env:
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+        run: |
+          /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
 
 # This job builds all the images. The build cache is stored in the github actions cache.
 # In further jobs, this cache is used to quickly rebuild the images.
   build-arm:
     name: Build images for ARM64 & ARM/V7
-    if: inputs.architecture != 'linux/amd64'
+    if: contains(inputs.architecture, 'linux/arm64/v8,linux/arm/v7')
     needs:
       - targets
+      - build-base-image-arm
     strategy:
       fail-fast: false
       matrix:
@@ -170,54 +338,69 @@ jobs:
           echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
           echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
-      - name: Configure actions/cache@v3 action for storing build cache in the ${{ runner.temp }}/cache folder
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/${{ matrix.target }}
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-${{ matrix.target }}-${{ github.run_id }}
-          restore-keys: |
-            ${{ github.ref }}-${{ inputs.mailu_version }}-${{ matrix.target }}
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - uses: crazy-max/ghaction-github-runtime@v2
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-      - name: Login to Docker Hub
-        uses: docker/login-action@v2
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
         with:
-          username: ${{ secrets.Docker_Login }}
-          password: ${{ secrets.Docker_Password }}
-      - name: Build all docker images
+          string: ${{ github.repository_owner }}
+      #This is to prevent to shared runners from generating the same uuid
+      - name: Get unique random number
+        id: uuid
+        run: |
+            echo uuid=$RANDOM >> $GITHUB_OUTPUT
+      - name: Build docker image with retry
         env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        uses: docker/bake-action@v2
+          DOCKER_ORG: ghcr.io/${{ steps.string.outputs.lowercase }}
+          MAILU_VERSION: ${{ env.MAILU_VERSION }}-build-arm
+          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}-build-arm
+          LABEL_VERSION: ${{ env.MAILU_VERSION }}
+          PINNED_LABEL_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          ARCH: linux/arm64/v8,linux/arm/v7
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+          DOCKER_LOGIN2: ${{ secrets.Docker_Login2 }}
+          DOCKER_PASSW2: ${{ secrets.Docker_Password2 }}
+          BUILDX_NO_DEFAULT_ATTESTATIONS: 1
+        uses: nick-fields/retry@v2
         with:
-          files: ${{env.HCL_FILE}}
-          targets: ${{ matrix.target }}
-          load: false
-          push: false
-          set: |
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/${{ matrix.target }}
-            *.cache-to=type=local,dest=${{ runner.temp }}/cache/${{ matrix.target }},mode=max
-            *.platform=${{ inputs.architecture }}
+          timeout_minutes: 30
+          retry_wait_seconds: 30
+          max_attempts: 10
+          shell: bash
+          command: |
+            set -euxo pipefail \
+              ; /usr/bin/docker info \
+              ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
+              ; echo "$DOCKER_PASSW2" |    docker login --username "$DOCKER_LOGIN2" --password-stdin \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
+              || echo "builder does not exist" \
+              ; /usr/bin/docker buildx create --name builder-${{ env.BUILDER }} --driver docker-container --use \
+              ; /usr/bin/docker buildx bake --push --file ./tests/build.hcl --set *.cache-from=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }}-arm --set *.cache-to=type=registry,ref=ghcr.io/${{ steps.string.outputs.lowercase }}/base:${{ hashFiles('core/base/Dockerfile','core/base/requirements-prod.txt') }}-arm,mode=max --set *.platform=${{ env.ARCH }} ${{ matrix.target }} \
+              ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
+      - name: cleanup docker buildx instance after failure of build step
+        if: ${{ failure() }}
+        shell: bash
+        env:
+          BUILDER: ${{ steps.uuid.outputs.uuid }}
+        run: |
+          /usr/bin/docker buildx rm builder-${{ env.BUILDER }}
 
 # This job runs all the tests.
   tests:
     name: tests
-    if: inputs.architecture == 'linux/amd64'
+    if: contains(inputs.architecture, 'linux/amd64')
     runs-on: ubuntu-latest
     permissions:
      contents: read
      packages: read
     needs:
-      - targets
       - build
     strategy:
       fail-fast: false
       matrix:
-        target: ["core", "fetchmail", "filters", "snappymail", "roundcube", "webdav"]
+        target: ["core", "fetchmail", "filters", "webmail", "webdav"]
         time: ["2"]
         include:
           - target: "filters"
@@ -234,112 +417,22 @@ jobs:
           echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
           echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
-      - name: Configure /cache for image docs
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/docs
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-docs-${{ github.run_id }}
-      - name: Configure /cache for image setup
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/setup
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-setup-${{ github.run_id }}
-      - name: Configure /cache for image admin
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/admin
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-admin-${{ github.run_id }}
-      - name: Configure /cache for image antispam
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antispam
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antispam-${{ github.run_id }}
-      - name: Configure /cache for image front
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/front
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-front-${{ github.run_id }}
-      - name: Configure /cache for image imap
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/imap
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-imap-${{ github.run_id }}
-      - name: Configure /cache for image smtp
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/smtp
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-smtp-${{ github.run_id }}
-      - name: Configure /cache for image snappymail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/snappymail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-snappymail-${{ github.run_id }}
-      - name: Configure /cache for image roundcube
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/roundcube
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-roundcube-${{ github.run_id }}
-      - name: Configure /cache for image antivirus
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antivirus
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antivirus-${{ github.run_id }}
-      - name: Configure /cache for image fetchmail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/fetchmail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-fetchmail-${{ github.run_id }}
-      - name: Configure /cache for image resolver
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/resolver
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-resolver-${{ github.run_id }}
-      - name: Configure /cache for image traefik-certdumper
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/traefik-certdumper
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-traefik-certdumper-${{ github.run_id }}
-      - name: Configure /cache for image webdav
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/webdav
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-webdav-${{ github.run_id }}
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - uses: crazy-max/ghaction-github-runtime@v2
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
-      - name: Login to Docker Hub
+      - name: Login to GitHub Container Registry
         uses: docker/login-action@v2
         with:
-          username: ${{ secrets.Docker_Login }}
-          password: ${{ secrets.Docker_Password }}
-      - name: Build docker images for testing from cache
-        env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        uses: docker/bake-action@v2
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
         with:
-          files: ${{env.HCL_FILE}}
-          load: true
-          push: false
-          set: |
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/docs
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/setup
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/admin
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antispam
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/front
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/imap
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/smtp
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/snappymail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/roundcube
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antivirus
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/fetchmail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/resolver
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/traefik-certdumper
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/webdav
-            *.platform=${{ inputs.architecture }}
+          string: ${{ github.repository_owner }}
       - name: Install python packages
         run: python3 -m pip install -r tests/requirements.txt
       - name: Copy all certs
@@ -347,12 +440,10 @@ jobs:
       - name: Test ${{ matrix.target }}
         run: python tests/compose/test.py ${{ matrix.target }} ${{ matrix.time }}
         env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
+          DOCKER_ORG: ghcr.io/${{ steps.string.outputs.lowercase }}
+          MAILU_VERSION: ${{ env.MAILU_VERSION }}-build
+          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}-build
 
-# This job deploys the docker images to the docker repository. The build.hcl file contains logic that determines what tags are pushed.
-# E.g. for master only the :master and :latest tags are pushed.
   deploy:
     name: Deploy images
     # Deploying is not required for staging
@@ -360,129 +451,12 @@ jobs:
     runs-on: ubuntu-latest
     needs:
       - build
-      - tests
-    steps:
-      - uses: actions/checkout@v3
-      - name: Retrieve global variables
-        shell: bash
-        run: |
-          echo "BRANCH=${{ inputs.branch }}" >> $GITHUB_ENV
-          echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
-          echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
-          echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
-      - name: Configure /cache for image docs
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/docs
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-docs-${{ github.run_id }}
-      - name: Configure /cache for image setup
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/setup
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-setup-${{ github.run_id }}
-      - name: Configure /cache for image admin
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/admin
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-admin-${{ github.run_id }}
-      - name: Configure /cache for image antispam
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antispam
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antispam-${{ github.run_id }}
-      - name: Configure /cache for image front
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/front
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-front-${{ github.run_id }}
-      - name: Configure /cache for image imap
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/imap
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-imap-${{ github.run_id }}
-      - name: Configure /cache for image smtp
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/smtp
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-smtp-${{ github.run_id }}
-      - name: Configure /cache for image snappymail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/snappymail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-snappymail-${{ github.run_id }}
-      - name: Configure /cache for image roundcube
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/roundcube
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-roundcube-${{ github.run_id }}
-      - name: Configure /cache for image antivirus
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antivirus
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antivirus-${{ github.run_id }}
-      - name: Configure /cache for image fetchmail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/fetchmail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-fetchmail-${{ github.run_id }}
-      - name: Configure /cache for image resolver
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/resolver
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-resolver-${{ github.run_id }}
-      - name: Configure /cache for image traefik-certdumper
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/traefik-certdumper
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-traefik-certdumper-${{ github.run_id }}
-      - name: Configure /cache for image webdav
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/webdav
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-webdav-${{ github.run_id }}
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v2
-      - uses: crazy-max/ghaction-github-runtime@v2
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v2
-      - name: Login to Docker Hub
-        uses: docker/login-action@v2
-        with:
-          username: ${{ secrets.Docker_Login }}
-          password: ${{ secrets.Docker_Password }}
-      - name: Deploy images to docker hub. Build.hcl contains the logic for the tags that are pushed.
-        env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        uses: docker/bake-action@v2
-        with:
-          files: ${{env.HCL_FILE}}
-          push: true
-          set: |
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/docs
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/setup
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/admin
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antispam
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/front
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/imap
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/smtp
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/snappymail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/roundcube
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antivirus
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/fetchmail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/resolver
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/traefik-certdumper
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/webdav
-            *.platform=${{ inputs.architecture }}
-
-  deploy-arm:
-    name: Deploy images for arm
-    # Deploying is not required for staging
-    if: inputs.deploy == 'true' && inputs.architecture != 'linux/amd64'
-    runs-on: self-hosted
-    needs:
       - build-arm
+      - tests
+    strategy:
+      fail-fast: false
+      matrix:
+        target: ["setup", "docs", "fetchmail", "webmail", "admin", "traefik-certdumper", "radicale", "clamav", "rspamd", "oletools", "postfix", "dovecot", "unbound", "nginx"]
     steps:
       - uses: actions/checkout@v3
       - name: Retrieve global variables
@@ -492,111 +466,53 @@ jobs:
           echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
           echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
-      - name: Configure /cache for image docs
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/docs
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-docs-${{ github.run_id }}
-      - name: Configure /cache for image setup
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/setup
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-setup-${{ github.run_id }}
-      - name: Configure /cache for image admin
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/admin
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-admin-${{ github.run_id }}
-      - name: Configure /cache for image antispam
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antispam
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antispam-${{ github.run_id }}
-      - name: Configure /cache for image front
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/front
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-front-${{ github.run_id }}
-      - name: Configure /cache for image imap
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/imap
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-imap-${{ github.run_id }}
-      - name: Configure /cache for image smtp
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/smtp
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-smtp-${{ github.run_id }}
-      - name: Configure /cache for image snappymail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/snappymail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-snappymail-${{ github.run_id }}
-      - name: Configure /cache for image roundcube
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/roundcube
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-roundcube-${{ github.run_id }}
-      - name: Configure /cache for image antivirus
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/antivirus
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-antivirus-${{ github.run_id }}
-      - name: Configure /cache for image fetchmail
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/fetchmail
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-fetchmail-${{ github.run_id }}
-      - name: Configure /cache for image resolver
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/resolver
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-resolver-${{ github.run_id }}
-      - name: Configure /cache for image traefik-certdumper
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/traefik-certdumper
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-traefik-certdumper-${{ github.run_id }}
-      - name: Configure /cache for image webdav
-        uses: actions/cache@v3
-        with:
-          path: ${{ runner.temp }}/cache/webdav
-          key: ${{ github.ref }}-${{ inputs.mailu_version }}-webdav-${{ github.run_id }}
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
       - uses: crazy-max/ghaction-github-runtime@v2
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v2
-      - name: Login to Docker Hub
+      - name: Login to GitHub Container Registry
         uses: docker/login-action@v2
         with:
-          username: ${{ secrets.Docker_Login }}
-          password: ${{ secrets.Docker_Password }}
-      - name: Deploy images to docker hub. Build.hcl contains the logic for the tags that are pushed.
-        env:
-          DOCKER_ORG: ${{ env.DOCKER_ORG }}
-          MAILU_VERSION: ${{ env.MAILU_VERSION }}
-          PINNED_MAILU_VERSION: ${{ env.PINNED_MAILU_VERSION }}
-        uses: docker/bake-action@v2
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Helper to convert docker org to lowercase
+        id: string
+        uses: ASzc/change-string-case-action@v5
         with:
-          files: ${{env.HCL_FILE}}
-          push: true
-          set: |
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/docs
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/setup
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/admin
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antispam
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/front
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/imap
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/smtp
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/snappymail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/roundcube
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/antivirus
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/fetchmail
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/resolver
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/traefik-certdumper
-            *.cache-from=type=local,src=${{ runner.temp }}/cache/webdav
-            *.platform=${{ inputs.architecture }}
+          string: ${{ github.repository_owner }}
+      - name: Push multiarch image to Github (ghcr.io)
+        if: contains(inputs.architecture, 'linux/amd64') && contains(inputs.architecture, 'linux/arm64/v8,linux/arm/v7')
+        shell: bash
+        run: |
+          if [ '${{ env.MAILU_VERSION }}' == 'master' ]; then pinned_mailu_version='master'; else pinned_mailu_version=${{ env.PINNED_MAILU_VERSION}}; fi;
+          docker buildx imagetools create \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:${{ env.MAILU_VERSION }} \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:$pinned_mailu_version \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:latest \
+            ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:${{ env.MAILU_VERSION }}-build \
+            ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:${{ env.MAILU_VERSION }}-build-arm
+      - name: Push x64 image to Github (ghcr.io)
+        if: contains(inputs.architecture, 'linux/amd64') && !contains(inputs.architecture, 'linux/arm64/v8,linux/arm/v7')
+        shell: bash
+        run: |
+          if [ '${{ env.MAILU_VERSION }}' == 'master' ]; then pinned_mailu_version='master'; else pinned_mailu_version=${{ env.PINNED_MAILU_VERSION}}; fi;
+          docker buildx imagetools create \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:${{ env.MAILU_VERSION }} \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:$pinned_mailu_version \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:latest \
+            ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:${{ env.MAILU_VERSION }}-build
+      - name: Push arm image to Github (ghcr.io)
+        if: contains(inputs.architecture, 'linux/arm64/v8,linux/arm/v7') && !contains(inputs.architecture, 'linux/amd64')
+        shell: bash
+        run: |
+          if [ '${{ env.MAILU_VERSION }}' == 'master' ]; then pinned_mailu_version='master'; else pinned_mailu_version=${{ env.PINNED_MAILU_VERSION}}; fi;
+          docker buildx imagetools create \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:${{ env.MAILU_VERSION }} \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:$pinned_mailu_version \
+            --tag ${{ inputs.docker_org }}/${{ matrix.target }}:latest \
+            ghcr.io/${{ steps.string.outputs.lowercase }}/${{ matrix.target }}:${{ env.MAILU_VERSION }}-build-arm
 
 #This job creates a tagged release. A tag is created for the pinned version x.y.z. The GH release refers to this tag.
   tag-release:
@@ -609,22 +525,57 @@ jobs:
         with:
           # fetch-depth 0 is required to also retrieve all tags.
           fetch-depth: 0
+        # A bug in actions/checkout@v3 results in all files having mtime of the job running.
+      - name: Restore Timestamps
+        uses: chetan/git-restore-mtime-action@v1
       - name: Retrieve global variables
         shell: bash
         run: |
-          echo "BRANCH=${{ inputs.branch }}" >> $GITHUB_ENV
           echo "MAILU_VERSION=${{ inputs.mailu_version }}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=${{ inputs.pinned_mailu_version }}" >> $GITHUB_ENV
-          echo "DOCKER_ORG=${{ inputs.docker_org }}" >> $GITHUB_ENV
       - name: Create tag for branch x.y.
         shell: bash
         run: |
           echo git tag ${{ env.PINNED_MAILU_VERSION }} $(/usr/bin/git rev-parse HEAD)
           git tag ${{ env.PINNED_MAILU_VERSION }} $(/usr/bin/git rev-parse HEAD)
           git push origin ${{ env.PINNED_MAILU_VERSION }}
+      - name: Show list of changelog files (we pick the newest)
+        shell: bash
+        run: |
+          ls -Artl towncrier/newsfragments
+      - name: Get latest changelog
+        id: changelog
+        shell: bash
+        run: |
+            pushd . && cd towncrier/newsfragments && ls -Art | tail -n 1 | cut -d. -f1 | xargs -0I % echo "issue=%" >> $GITHUB_OUTPUT && popd
+            pushd . && cd towncrier/newsfragments && ls -Art | tail -n 1 | xargs cat | xargs -0I % echo "content=%" >> $GITHUB_OUTPUT && popd
+      - name: Construct message for release
+        shell: bash
+        env:
+          issue: "${{ steps.changelog.outputs.issue }}"
+          changelog: "${{ steps.changelog.outputs.content }}"
+        run: |
+            message="Changelog :mailbox:
+            ---------
+            + ${{ env.changelog }}
+
+            + This release was triggered by PR/Issue [${{ env.issue }}](https://github.com/Mailu/Mailu/issues/${{ env.issue }}).
+
+            + The release notes of the original main release can be accessed via menu item 'Release notes' on [mailu.io](https://mailu.io/).
+
+            Update
+            ------
+            The main version X.Y (e.g. 1.9) will always reflect the latest version of the branch. To update your Mailu installation simply pull the latest images \`docker compose pull && docker compose up -d\`.
+
+            The pinned version X.Y.Z (e.g. 1.9.1) is not updated. It is pinned to the commit that was used for creating this release. You can use a pinned version to make sure your Mailu installation is not suddenly updated when recreating containers. The pinned version allows the user to manually update. It also allows to go back to a previous pinned version.
+            " && echo "$message" >> release_note.md
+      - name: Show release note
+        shell: bash
+        run: |
+            cat release_note.md
       - name: Create release for tag x.y.z.
         uses: ncipollo/release-action@v1
         with:
-          bodyFile: "RELEASE_TEMPLATE.md"
+          bodyFile: "release_note.md"
           tag: ${{ env.PINNED_MAILU_VERSION }}
           token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/multiarch.yml

@@ -1,4 +1,4 @@
-name: start-linux-amd64
+name: start-linux-multiarch
 on:
   push:
     branches:
@@ -6,23 +6,14 @@ on:
       - staging
       - '1.9'
       - master
+      - test-*
 
-concurrency: ci-x64-${{ github.ref }}
+concurrency: ci-multiarch-${{ github.ref }}
 
-###############################################
-# REQUIRED secrets
-# ${{ secrets.Docker_Login }}
-#    Username of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
-# ${{ secrets.Docker_Password }}
-#    Password of docker login for pushing the images to repo env.DOCKER_ORG and env.DOCKER_ORG_TESTS
-# Add the above secrets to your github repo to determine where the images will be pushed.
-################################################
 # REQUIRED global variables
-# DOCKER_ORG, docker org used for pushing release images (branch x.y and master)
-# DOCKER_ORG_TEST, docker org used for pushing images for testing (branch testing).
+# DOCKER_ORG, docker org used for pushing images.
 env:
-  DOCKER_ORG: mailu
-  DOCKER_ORG_TEST: mailuci
+  DOCKER_ORG: ghcr.io/mailu
 
 jobs:
 # This job calculates all global job variables that are required by all the subsequent jobs.
@@ -55,7 +46,7 @@ jobs:
         run: |
           echo "MAILU_VERSION=pr-${COMMIT_MESSAGE//[!0-9]/}" >> $GITHUB_ENV
           echo "PINNED_MAILU_VERSION=pr-${COMMIT_MESSAGE//[!0-9]/}" >> $GITHUB_ENV
-          echo "DOCKER_ORG_DERIVED=${{ env.DOCKER_ORG_TEST }}" >> $GITHUB_ENV
+          echo "DOCKER_ORG_DERIVED=${{ env.DOCKER_ORG }}" >> $GITHUB_ENV
           echo "DEPLOY=true" >> $GITHUB_ENV
           echo "RELEASE=false" >> $GITHUB_ENV
       - name: Derive MAILU_VERSION and DEPLOY/RELEASE for other branches than testing
@@ -67,7 +58,7 @@ jobs:
           echo "DEPLOY=true" >> $GITHUB_ENV
           echo "RELEASE=false" >> $GITHUB_ENV
       - name: Derive PINNED_MAILU_VERSION and DEPLOY/RELEASE for normal release x.y
-        if: env.BRANCH != 'testing' && env.BRANCH != 'staging' && env.BRANCH != 'master'
+        if: env.BRANCH != 'testing' && env.BRANCH != 'staging' && env.BRANCH != 'master' && !(contains(env.BRANCH, 'test-'))
         shell: bash
         run: |
           version=$( git tag --sort=version:refname --list "${{ env.MAILU_VERSION }}.*" | tail -1  );root_version=${version%.*};patch_version=${version##*.};if [ "$patch_version" == "" ]; then pinned_version=${{ env.MAILU_VERSION }}.0; else pinned_version=$root_version.$(expr $patch_version + 1); fi;echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV
@@ -82,7 +73,7 @@ jobs:
           echo "PINNED_MAILU_VERSION=staging" >> $GITHUB_ENV
           echo "DEPLOY=false" >> $GITHUB_ENV
           echo "RELEASE=false" >> $GITHUB_ENV
-      - name: Derive PINNED_MAILU_VERSION for staging for master
+      - name: Derive PINNED_MAILU_VERSION for master
         if: env.BRANCH == 'master'
         shell: bash
         env:
@@ -91,13 +82,21 @@ jobs:
           echo "PINNED_MAILU_VERSION=$GITHUB_SHA" >> $GITHUB_ENV
           echo "DEPLOY=true" >> $GITHUB_ENV
           echo "RELEASE=false" >> $GITHUB_ENV
+      - name: Derive for branch test-*
+        if: contains(env.BRANCH, 'test-')
+        run: |
+          echo "MAILU_VERSION=${{ env.BRANCH }}" >> $GITHUB_ENV
+          echo "PINNED_MAILU_VERSION=${{ env.BRANCH }}" >> $GITHUB_ENV
+          echo "DOCKER_ORG_DERIVED=${{ env.DOCKER_ORG }}" >> $GITHUB_ENV
+          echo "DEPLOY=true" >> $GITHUB_ENV
+          echo "RELEASE=false" >> $GITHUB_ENV
 
   build-test-deploy:
     needs:
       - derive-variables
     uses: ./.github/workflows/build_test_deploy.yml
     with:
-      architecture: 'linux/amd64'
+      architecture: 'linux/amd64,linux/arm64/v8,linux/arm/v7'
       mailu_version: ${{needs.derive-variables.outputs.MAILU_VERSION}}
       pinned_mailu_version: ${{needs.derive-variables.outputs.PINNED_MAILU_VERSION}}
       docker_org: ${{needs.derive-variables.outputs.DOCKER_ORG}}
@@ -131,4 +130,4 @@ jobs:
 #else
 #  pinned_version=$root_version.$(expr $patch_version + 1)
 #fi
-#echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV
+#echo "PINNED_MAILU_VERSION=$pinned_version" >> $GITHUB_ENV

.gitignore

@@ -1,5 +1,5 @@
-*.pyc
-*.mo
+**/*.pyc
+**/*.mo
 __pycache__
 pip-selfcheck.json
 /core/admin/lib*
@@ -10,6 +10,7 @@ pip-selfcheck.json
 /docs/include
 /docs/_build
 /.env
+/.venv
 /docker-compose.yml
 /.idea
 /.vscode

CHANGELOG.md

@@ -10,7 +10,7 @@ These settings tell Mailu that the HTTP header with the remote client IP address
 For more information see the [configuration reference](https://mailu.io/1.9/configuration.html#advanced-settings).
 
 One major change for the docker compose file is that the antispam container needs a fixed hostname [#1837](https://github.com/Mailu/Mailu/issues/1837).
-This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. 
+This is handled when you regenerate the docker compose file. A fixed hostname is required to retain rspamd history.
 
 After changing mailu.env, it is required to recreate all containers for the changes to be propagated.
 
@@ -314,8 +314,8 @@ v1.6.0 - 2019-01-18
 - Enhancement: Reverse proxy - Real ip header and mail-letsencrypt ([#358](https://github.com/Mailu/Mailu/issues/358))
 - Enhancement: Parametrize hosts ([#373](https://github.com/Mailu/Mailu/issues/373))
 - Enhancement: Expose ports in dockerfiles ([#392](https://github.com/Mailu/Mailu/issues/392))
-- Enhancement: Added webmail-imap dependency in docker-compose ([#403](https://github.com/Mailu/Mailu/issues/403))
-- Enhancement: Add environment variables to allow running outside of docker-compose ([#429](https://github.com/Mailu/Mailu/issues/429))
+- Enhancement: Added webmail-imap dependency in docker compose ([#403](https://github.com/Mailu/Mailu/issues/403))
+- Enhancement: Add environment variables to allow running outside of docker compose ([#429](https://github.com/Mailu/Mailu/issues/429))
 - Enhancement: Add original Delivered-To header to received messages ([#433](https://github.com/Mailu/Mailu/issues/433))
 - Enhancement: Use HOST_ADMIN in "Forwarding authentication server" ([#436](https://github.com/Mailu/Mailu/issues/436), [#437](https://github.com/Mailu/Mailu/issues/437))
 - Enhancement: Use POD_ADDRESS_RANGE for Dovecot ([#448](https://github.com/Mailu/Mailu/issues/448))

CONTRIBUTING.md

@@ -4,4 +4,4 @@ This project is open source, and your contributions are all welcome. There are m
 2. contribute code and/or configuration to the repository (see [the development guidelines](https://mailu.io/master/contributors/workflow.html) for details);
 3. contribute localization to your native language (see [the localization docs](https://mailu.io/master/contributors/localization.html) for details);
 
-Either way, keep in mind that the code you write or the translation you produce muts be licensed under the same conditions as the project itself. Additionally, all contributors are considered equal co-authors of the project.
+Either way, keep in mind that the code you write or the translation you produce must be licensed under the same conditions as the project itself. Additionally, all contributors are considered equal co-authors of the project.

ISSUE_TEMPLATE.md

@@ -1,44 +1,71 @@
+<!--
+
 Thank you for opening an issue with Mailu. Please understand that issues are meant for bugs and enhancement-requests.
 For **user-support questions**, reach out to us  on [matrix](https://matrix.to/#/#mailu:tedomum.net).
 
 To be able to help you best, we need some more information.
 
-## Before you open your issue
-- [ ] Check if no issue or pull-request for this already exists.
-- [ ] Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
-- [ ] You understand `Mailu` is made by volunteers in their **free time** — be conscise, civil and accept that delays can occur.
-- [ ] The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.
+Before you open your issue
+- Check if no issue or pull-request for this already exists.
+- Check [documentation](https://mailu.io/master/) and [FAQ](https://mailu.io/master/faq.html). (Tip, use the search function on the documentation page)
+- You understand `Mailu` is made by volunteers in their **free time** — be concise, civil and accept that delays can occur.
+- The title of the issue should be short and simple. It should contain specific terms related to the actual issue. Be specific while writing the title.
+
+Please put your text outside of the comment blocks to be visible. You can use the button "Preview" above to check.
+
+-->
+
+## Environment & Version
 
-## Environment & Versions
 ### Environment
- - [ ] docker-compose
- - [ ] kubernetes
- - [ ] docker swarm
 
-### Versions
+- [ ] docker compose
+- [ ] kubernetes
+- [ ] docker swarm
+
+### Version
+
+- Version: `master`
+
+<!--
 To find your version, get the image name of a mailu container and read  the version from the tag (example for version 1.7).
-```
+
 $> docker ps -a | grep mailu
 140b09d4b09c    mailu/roundcube:1.7    "docker-php-entrypoi…"    2 weeks ago    Up 2 days (healthy)    80/tcp
 $> grep MAILU_VERSION docker-compose.yml mailu.env
-```
+-->
 
 ## Description
+<!--
 Further explain the bug in a few words. It should be clear what the unexpected behaviour is.  Share it in an easy-to-understand language.
+-->
 
 ## Replication Steps
+<!--
 Steps for replicating your issue
+-->
+
+## Observed behaviour
+<!--
+Explain or paste the result you received.
+-->
 
 ## Expected behaviour
-Explain what results you expected - be as specific as possible. Just saying "it doesn’t work as expected" is not useful. It's also helpful to describe what you actually experienced.
+<!--
+Explain what results you expected - be as specific as possible.
+Just saying "it doesn’t work as expected" is not useful. It's also helpful to describe what you actually experienced.
+-->
 
 ## Logs
-Often it is very useful to include log fragments of the involved component. You can get the logs via `docker logs <container name> --tail 1000`. For example for the admin container:
-`docker logs mailu_admin_1 --tail 1000`
-or using docker-compose `docker-compose -f /mailu/docker-compose.yml logs --tail 1000 admin`
+<!--
+Often it is very useful to include log fragments of the involved component.
+You can get the logs via `docker logs <container name> --tail 1000`.
+For example for the admin container: `docker logs mailu_admin_1 --tail 1000`
+or using docker compose `docker compose -f /mailu/docker-compose.yml logs --tail 1000 admin`
+
 If you can find the relevant section, please share only the parts that seem relevant. If you have any logs, please enclose them in code tags, like so:
-````markdown
+
 ```
 Your logs here!
 ```
-````
+-->

README.md

@@ -22,8 +22,8 @@ Main features include:
 - **Web access**, multiple Webmails and administration interface
 - **User features**, aliases, auto-reply, auto-forward, fetched accounts
 - **Admin features**, global admins, announcements, per-domain delegation, quotas
-- **Security**, enforced TLS, DANE, MTA-STS, Letsencrypt!, outgoing DKIM, anti-virus scanner
-- **Antispam**, auto-learn, greylisting, DMARC and SPF
+- **Security**, enforced TLS, DANE, MTA-STS, Letsencrypt!, outgoing DKIM, anti-virus scanner, [Snuffleupagus](https://github.com/jvoisin/snuffleupagus/), block malicious attachments
+- **Antispam**, auto-learn, greylisting, DMARC and SPF, anti-spoofing
 - **Freedom**, all FOSS components, no tracker included
 
 ![Domains](docs/assets/screenshots/domains.png)

RELEASE_TEMPLATE.md

@@ -1,5 +1,5 @@
 This is a new automatic release of Mailu. The new version can be seen in the tag name.
-The main version X.Y (e.g. 1.9) will always reflect the latest version of the branch. To update your Mailu installation simply pull the latest images `docker-compose pull && docker-compose up -d`.
+The main version X.Y (e.g. 1.9) will always reflect the latest version of the branch. To update your Mailu installation simply pull the latest images `docker compose pull && docker compose up -d`.
 The pinned version X.Y.Z (e.g. 1.9.1) is not updated. It is pinned to the commit that was used for creating this release. You can use a pinned version to make sure your Mailu installation is not suddenly updated when recreating containers. The pinned version allows the user to manually update. It also allows to go back to a previous pinned version.
 
 To check what was changed:

core/admin/.gitignore

@@ -2,3 +2,4 @@
 lib64
 .vscode
 tags
+dev

core/admin/Dockerfile

@@ -1,61 +1,31 @@
-# First stage to build assets
-ARG DISTRO=alpine:3.14.5
+# syntax=docker/dockerfile-upstream:1.4.3
 
-FROM node:16-alpine3.16 as assets
-
-COPY package.json ./
-RUN set -eu \
- && npm config set update-notifier false \
- && npm install --no-fund
-
-COPY webpack.config.js ./
-COPY assets ./assets
-RUN set -eu \
- && sed -i 's/#007bff/#55a5d9/' node_modules/admin-lte/build/scss/_bootstrap-variables.scss \
- && for l in ca da de:de-DE en:en-GB es:es-ES eu fr:fr-FR he hu is it:it-IT ja nb_NO:no-NB nl:nl-NL pl pt:pt-PT ru sv:sv-SE zh; do \
-      cp node_modules/datatables.net-plugins/i18n/${l#*:}.json assets/${l%:*}.json; \
-    done \
- && node_modules/.bin/webpack-cli --color
-
-
-# Actual application
-FROM $DISTRO
-ARG VERSION
-
-ENV TZ Etc/UTC
+# admin image
+FROM base
 
+ARG VERSION=local
 LABEL version=$VERSION
 
-# python3 shared with most images
-RUN set -eu \
- && apk add --no-cache python3 py3-pip py3-wheel git bash tzdata \
- && pip3 install --upgrade pip
+RUN set -euxo pipefail \
+  ; apk add --no-cache libressl mariadb-connector-c postgresql-libs
 
-RUN mkdir -p /app
-WORKDIR /app
+COPY --from=assets /work/static/ ./mailu/static/
 
-COPY requirements-prod.txt requirements.txt
-RUN set -eu \
- && apk add --no-cache libressl curl postgresql-libs mariadb-connector-c \
- && pip install --no-cache-dir -r requirements.txt --only-binary=:all: --no-binary=Flask-bootstrap,PyYAML,SQLAlchemy \
- || ( apk add --no-cache --virtual build-dep libressl-dev libffi-dev python3-dev build-base postgresql-dev mariadb-connector-c-dev cargo \
- && pip install --upgrade pip \
- && pip install -r requirements.txt \
- && apk del --no-cache build-dep )
+COPY audit.py /
+COPY start.py /
 
-COPY --from=assets static ./mailu/static
-COPY mailu ./mailu
-COPY migrations ./migrations
-COPY start.py /start.py
-COPY audit.py /audit.py
+COPY migrations/ ./migrations/
 
-RUN pybabel compile -d mailu/translations
+COPY mailu/ ./mailu/
+RUN set -euxo pipefail \
+  ; venv/bin/pybabel compile -d mailu/translations
+
+RUN echo $VERSION >/version
+
+#EXPOSE 80/tcp
+HEALTHCHECK CMD curl -skfLo /dev/null http://localhost/ping
 
-EXPOSE 80/tcp
 VOLUME ["/data","/dkim"]
-ENV FLASK_APP mailu
 
+ENV FLASK_APP=mailu
 CMD /start.py
-
-HEALTHCHECK CMD curl -f -L http://localhost/sso/login?next=ui.index || exit 1
-RUN echo $VERSION >> /version

core/admin/assets/Dockerfile

@@ -0,0 +1,22 @@
+# syntax=docker/dockerfile-upstream:1.4.3
+
+FROM node:16-alpine3.16
+
+WORKDIR /work
+
+COPY package.json ./
+
+RUN set -euxo pipefail \
+  ; npm config set update-notifier false \
+  ; npm install --no-audit --no-fund \
+  ; sed -i 's/#007bff/#55a5d9/' node_modules/admin-lte/build/scss/_bootstrap-variables.scss \
+  ; mkdir assets \
+  ; for l in ca da de:de-DE en:en-GB es:es-ES eu fr:fr-FR he hu is it:it-IT ja nb_NO:no-NB nl:nl-NL pl pt:pt-PT ru sv:sv-SE zh; do \
+      cp node_modules/datatables.net-plugins/i18n/${l#*:}.json assets/${l%:*}.json; \
+    done
+
+COPY assets/ ./assets/
+COPY webpack.config.js ./
+
+RUN set -euxo pipefail \
+  ; node_modules/.bin/webpack-cli --color

core/admin/assets/app.js

@@ -1,79 +0,0 @@
-require('./app.css');
-
-import logo from './mailu.png';
-import modules from "./*.json";
-
-// TODO: conditionally (or lazy) load select2 and dataTable
-$('document').ready(function() {
-
-    // intercept anchors with data-clicked attribute and open alternate location instead
-    $('[data-clicked]').click(function(e) {
-        e.preventDefault();
-        window.location.href = $(this).data('clicked');
-    });
-
-    // use post for language selection
-    $('#mailu-languages > a').click(function(e) {
-        e.preventDefault();
-        $.post({
-            url: $(this).attr('href'),
-            success: function() {
-                window.location = window.location.href;
-            },
-        });
-    });
-
-    // allow en-/disabling of inputs in fieldset with checkbox in legend
-    $('fieldset legend input[type=checkbox]').change(function() {
-        var fieldset = $(this).parents('fieldset');
-        if (this.checked) {
-            fieldset.removeAttr('disabled');
-            fieldset.find('input,textarea').not(this).removeAttr('disabled');
-        } else {
-            fieldset.attr('disabled', '');
-            fieldset.find('input,textarea').not(this).attr('disabled', '');
-        }
-    });
-
-    // display of range input value
-    $('input[type=range]').each(function() {
-        var value_element = $('#'+this.id+'_value');
-        if (value_element.length) {
-            value_element = $(value_element[0]);
-            var infinity = $(this).data('infinity');
-            var step = $(this).attr('step');
-            $(this).on('input', function() {
-                var num = (infinity && this.value == 0) ? '∞' : (this.value/step).toFixed(2);
-                if (num.endsWith('.00')) num = num.substr(0, num.length - 3);
-                value_element.text(num);
-            }).trigger('input');
-        }
-    });
-
-    // init select2
-    $('.mailselect').select2({
-        tags: true,
-        tokenSeparators: [',', ' '],
-    });
-
-    // init dataTable
-    var d = $(document.documentElement);
-    $('.dataTable').DataTable({
-        'responsive': true,
-        language: {
-            url: d.data('static') + d.attr('lang') + '.json',
-        },
-    });
-
-    // init clipboard.js
-    new ClipboardJS('.btn-clip');
-
-    // disable login if not possible
-    var l = $('#login_needs_https');
-    if (l.length && window.location.protocol != 'https:') {
-        l.removeClass("d-none");
-        $('form :input').prop('disabled', true);
-    }
-
-});
-

core/admin/assets/assets/app.css

@@ -57,3 +57,9 @@ fieldset:disabled .form-control:disabled {
 .input-group-text {
   margin-right: 1em;
 }
+
+/* version string */
+.mailu-version {
+  font-size: 60%;
+  line-height: 0;
+}

core/admin/assets/assets/app.js

@@ -0,0 +1,139 @@
+// Inspired from https://github.com/mehdibo/hibp-js/blob/master/hibp.js
+function sha1(string) {
+    var buffer = new TextEncoder("utf-8").encode(string);
+    return crypto.subtle.digest("SHA-1", buffer).then(function (buffer) {
+        // Get the hex code
+        var hexCodes = [];
+        var view = new DataView(buffer);
+        for (var i = 0; i < view.byteLength; i += 4) {
+            // Using getUint32 reduces the number of iterations needed (we process 4 bytes each time)
+            var value = view.getUint32(i);
+            // toString(16) will give the hex representation of the number without padding
+            var stringValue = value.toString(16);
+            // We use concatenation and slice for padding
+            var padding = '00000000';
+            var paddedValue = (padding + stringValue).slice(-padding.length);
+            hexCodes.push(paddedValue);
+        }
+        // Join all the hex strings into one
+        return hexCodes.join("");
+    });
+}
+
+function hibpCheck(pwd) {
+    // We hash the pwd first
+    sha1(pwd).then(function(hash){
+        // We send the first 5 chars of the hash to hibp's API
+        const req = new XMLHttpRequest();
+        req.open('GET', 'https://api.pwnedpasswords.com/range/'+hash.substr(0, 5));
+        req.setRequestHeader('Add-Padding', 'true');
+        req.addEventListener("load", function(){
+            // When we get back a response from the server
+            // We create an array of lines and loop through them
+            const lines = this.responseText.split("\n");
+            const hashSub = hash.slice(5).toUpperCase();
+            for (var i in lines){
+                // Check if the line matches the rest of the hash
+                if (lines[i].substring(0, 35) == hashSub){
+                    const val = parseInt(lines[i].trimEnd("\r").split(":")[1]);
+                    if (val > 0) {
+                        $("#pwned").val(val);
+                    }
+                    return; // If found no need to continue the loop
+                }
+            }
+            $("#pwned").val(0);
+        });
+        req.send();
+    });
+}
+
+// TODO: conditionally (or lazy) load select2 and dataTable
+$('document').ready(function() {
+
+    // intercept anchors with data-clicked attribute and open alternate location instead
+    $('[data-clicked]').click(function(e) {
+        e.preventDefault();
+        window.location.href = $(this).data('clicked');
+    });
+
+    // use post for language selection
+    $('#mailu-languages > a').click(function(e) {
+        e.preventDefault();
+        $.post({
+            url: $(this).attr('href'),
+            success: function() {
+                window.location = window.location.href;
+            },
+        });
+    });
+
+    // allow en-/disabling of inputs in fieldset with checkbox in legend
+    $('fieldset legend input[type=checkbox]').change(function() {
+        var fieldset = $(this).parents('fieldset');
+        if (this.checked) {
+            fieldset.removeAttr('disabled');
+            fieldset.find('input,textarea').not(this).removeAttr('disabled');
+        } else {
+            fieldset.attr('disabled', '');
+            fieldset.find('input,textarea').not(this).attr('disabled', '');
+        }
+    });
+
+    // display of range input value
+    $('input[type=range]').each(function() {
+        var value_element = $('#'+this.id+'_value');
+        if (value_element.length) {
+            value_element = $(value_element[0]);
+            var infinity = $(this).data('infinity');
+            var unit = $(this).data('unit');
+            if (typeof unit === 'undefined' || unit === false) {
+                unit=1;
+            }
+            $(this).on('input', function() {
+                var num = (infinity && this.value == 0) ? '∞' : (this.value/unit).toFixed(2);
+                if (num.endsWith('.00')) num = num.substr(0, num.length - 3);
+                value_element.text(num);
+            }).trigger('input');
+        }
+    });
+
+    // init select2
+    $('.mailselect').select2({
+        tags: true,
+        tokenSeparators: [',', ' '],
+    });
+
+    // init dataTable
+    var d = $(document.documentElement);
+    $('.dataTable').DataTable({
+        'responsive': true,
+        language: {
+            url: d.data('static') + d.attr('lang') + '.json',
+        },
+    });
+
+    // init clipboard.js
+    new ClipboardJS('.btn-clip');
+
+    // disable login if not possible
+    var l = $('#login_needs_https');
+    if (l.length && window.location.protocol != 'https:') {
+        l.removeClass("d-none");
+        $('form :input').prop('disabled', true);
+    }
+
+    if (window.isSecureContext) {
+        $("#pw").on("change paste", function(){
+            hibpCheck($(this).val());
+            return true;
+        });
+        $("#pw").closest("form").submit(function(event){
+            if (parseInt($("#pwned").val()) < 0) {
+                hibpCheck($("#pw").val());
+            }
+        });
+    }
+
+});
+

core/admin/assets/assets/vendor.js

@@ -1,5 +1,5 @@
 // AdminLTE
-import 'admin-lte/plugins/jquery/jquery.min.js';
+window.$ = window.jQuery = require('admin-lte/plugins/jquery/jquery.min.js');
 import 'admin-lte/plugins/bootstrap/js/bootstrap.bundle.min.js';
 import 'admin-lte/build/scss/adminlte.scss';
 import 'admin-lte/build/js/AdminLTE.js';
@@ -18,7 +18,7 @@ import 'admin-lte/plugins/datatables/jquery.dataTables.min.js';
 import 'admin-lte/plugins/datatables-bs4/js/dataTables.bootstrap4.min.js';
 import 'admin-lte/plugins/datatables-responsive/js/dataTables.responsive.min.js';
 import 'admin-lte/plugins/datatables-responsive/js/responsive.bootstrap4.min.js';
+import modules from "./*.json";
 
 // clipboard.js
-import 'clipboard/dist/clipboard.min.js';
-
+window.ClipboardJS = require('clipboard/dist/clipboard.min.js');

core/admin/assets/webpack.config.js

@@ -9,7 +9,7 @@ module.exports = {
     mode: 'production',
     entry: {
         app: {
-            import: './assets/app.js',
+            import: ['./assets/app.css', './assets/mailu.png', './assets/app.js'],
             dependOn: 'vendor',
         },
         vendor: './assets/vendor.js',

core/admin/audit.py

@@ -1,4 +1,4 @@
-#!/usr/bin/python3
+#!/usr/bin/env python3
 
 import sys
 import tabulate

core/admin/mailu/__init__.py

@@ -5,9 +5,24 @@ import flask
 import flask_bootstrap
 
 from mailu import utils, debug, models, manage, configuration
+from gunicorn import glogging
+import logging
 
 import hmac
 
+class NoPingFilter(logging.Filter):
+    def filter(self, record):
+        if not (record.args['{host}i'] == 'localhost' and record.args['r'] == 'GET /ping HTTP/1.1'):
+            return True
+
+class Logger(glogging.Logger):
+    def setup(self, cfg):
+        super().setup(cfg)
+
+        # Add filters to Gunicorn logger
+        logger = logging.getLogger("gunicorn.access")
+        logger.addFilter(NoPingFilter())
+
 def create_app_from_config(config):
     """ Create a new application based on the given configuration
     """
@@ -44,8 +59,10 @@ def create_app_from_config(config):
     # Initialize debugging tools
     if app.config.get("DEBUG"):
         debug.toolbar.init_app(app)
-        # TODO: add a specific configuration variable for profiling
-        # debug.profiler.init_app(app)
+    if app.config.get("DEBUG_PROFILER"):
+        debug.profiler.init_app(app)
+    if assets := app.config.get('DEBUG_ASSETS'):
+        app.static_folder = assets
 
     # Inject the default variables in the Jinja parser
     # TODO: move this to blueprints when needed
@@ -55,6 +72,7 @@ def create_app_from_config(config):
         return dict(
             signup_domains= signup_domains,
             config        = app.config,
+            get_locale    = utils.get_locale,
         )
 
     # Jinja filters
@@ -66,11 +84,16 @@ def create_app_from_config(config):
     def format_datetime(value):
         return utils.flask_babel.format_datetime(value) if value else ''
 
+    def ping():
+        return ''
+    app.route('/ping')(ping)
+
     # Import views
-    from mailu import ui, internal, sso
+    from mailu import ui, internal, sso, api
     app.register_blueprint(ui.ui, url_prefix=app.config['WEB_ADMIN'])
     app.register_blueprint(internal.internal, url_prefix='/internal')
     app.register_blueprint(sso.sso, url_prefix='/sso')
+    api.register(app, web_api_root=app.config.get('WEB_API'))
     return app
 
 

core/admin/mailu/api/__init__.py

@@ -0,0 +1,32 @@
+from flask import redirect, url_for, Blueprint
+from flask_restx import apidoc
+from . import v1 as APIv1
+
+def register(app, web_api_root):
+
+    APIv1.app = app
+    # register api bluprint(s)
+    apidoc.apidoc.url_prefix = f'{web_api_root}/v{int(APIv1.VERSION)}'
+    APIv1.api_token = app.config['API_TOKEN']
+    if app.config['API_TOKEN'] != '':
+        app.register_blueprint(APIv1.blueprint, url_prefix=f'{web_api_root}/v{int(APIv1.VERSION)}')
+
+        # add redirect to current api version
+        redirect_api = Blueprint('redirect_api', __name__)
+        @redirect_api.route('/')
+        def redir():
+            return redirect(url_for(f'{APIv1.blueprint.name}.root'))
+        app.register_blueprint(redirect_api, url_prefix=f'{web_api_root}')
+
+        # swagger ui config
+        app.config.SWAGGER_UI_DOC_EXPANSION = 'list'
+        app.config.SWAGGER_UI_OPERATION_ID = True
+        app.config.SWAGGER_UI_REQUEST_DURATION = True
+        app.config.RESTX_MASK_SWAGGER = False
+    else:
+        api = Blueprint('api', __name__)
+        @api.route('/', defaults={'path': ''})
+        @api.route('/<path:path>')
+        def api_token_missing(path):
+            return "<p>Error: API_TOKEN is not configured</p>", 500
+        app.register_blueprint(api, url_prefix=f'{web_api_root}')

core/admin/mailu/api/common.py

@@ -0,0 +1,42 @@
+from .. import models, utils
+from . import v1
+from flask import request
+import flask
+import hmac
+from functools import wraps
+from flask_restx import abort
+from sqlalchemy.sql.expression import label
+
+def fqdn_in_use(name):
+    d = models.db.session.query(label('name', models.Domain.name))
+    a = models.db.session.query(label('name', models.Alternative.name))
+    r = models.db.session.query(label('name', models.Relay.name))
+    u = d.union_all(a).union_all(r).filter_by(name=name)
+    if models.db.session.query(u.exists()).scalar():
+        return True
+    return False
+
+""" Decorator for validating api token for authentication """
+def api_token_authorization(func):
+    @wraps(func)
+    def decorated_function(*args, **kwds):
+        client_ip = flask.request.headers.get('X-Real-IP', flask.request.remote_addr)
+        if utils.limiter.should_rate_limit_ip(client_ip):
+            abort(429, 'Too many attempts from your IP (rate-limit)' )
+        if not request.headers.get('Authorization'):
+            abort(401, 'A valid Bearer token is expected which is provided as request header')
+        #Client provides 'Authentication: Bearer <token>'
+        if (' ' in request.headers.get('Authorization')
+                and not hmac.compare_digest(request.headers.get('Authorization'), 'Bearer ' + v1.api_token)):
+            utils.limiter.rate_limit_ip(client_ip)
+            flask.current_app.logger.warn(f'Invalid API token provided by {client_ip}.')
+            abort(403, 'A valid Bearer token is expected which is provided as request header')
+        #Client provides 'Authentication: <token>'
+        elif (' ' not in request.headers.get('Authorization')
+                and not hmac.compare_digest(request.headers.get('Authorization'), v1.api_token)):
+            utils.limiter.rate_limit_ip(client_ip)
+            flask.current_app.logger.warn(f'Invalid API token provided by {client_ip}.')
+            abort(403, 'A valid Bearer token is expected which is provided as request header')
+        flask.current_app.logger.info(f'Valid API token provided by {client_ip}.')
+        return func(*args, **kwds)
+    return decorated_function

core/admin/mailu/api/v1/__init__.py

@@ -0,0 +1,43 @@
+from flask import Blueprint
+from flask_restx import Api, fields
+
+
+VERSION = 1.0
+api_token = None
+
+blueprint = Blueprint(f'api_v{int(VERSION)}', __name__)
+
+authorization = {
+    'Bearer': {
+        'type': 'apiKey',
+        'in': 'header',
+        'name': 'Authorization'
+    }
+}
+
+api = Api(
+    blueprint, version=f'{VERSION:.1f}',
+    title='Mailu API', default_label='Mailu',
+    validate=True,
+    authorizations=authorization,
+    security='Bearer',
+    doc='/'
+)
+
+response_fields = api.model('Response', {
+    'code': fields.Integer,
+    'message': fields.String,
+})
+
+error_fields = api.model('Error', {
+    'errors': fields.Nested(api.model('Error_Key', {
+        'key': fields.String,
+        'message':fields.String
+    })),
+    'message': fields.String,
+})
+
+from . import domains
+from . import alias
+from . import relay
+from . import user

core/admin/mailu/api/v1/alias.py

@@ -0,0 +1,126 @@
+from flask_restx import Resource, fields, marshal
+from . import api, response_fields
+from .. import common
+from ... import models
+
+db = models.db
+
+alias = api.namespace('alias', description='Alias operations')
+
+alias_fields_update = alias.model('AliasUpdate', {
+    'comment': fields.String(description='a comment'),
+    'destination': fields.List(fields.String(description='alias email address', example='user@example.com')),
+    'wildcard': fields.Boolean(description='enable SQL Like wildcard syntax')
+})
+
+alias_fields = alias.inherit('Alias',alias_fields_update, {
+  'email': fields.String(description='the alias email address', example='user@example.com', required=True),
+  'destination': fields.List(fields.String(description='alias email address', example='user@example.com', required=True)),
+
+})
+
+
+@alias.route('')
+class Aliases(Resource):
+    @alias.doc('list_alias')
+    @alias.marshal_with(alias_fields, as_list=True, skip_none=True, mask=None)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self):
+        """ List aliases """
+        return models.Alias.query.all()
+
+    @alias.doc('create_alias')
+    @alias.expect(alias_fields)
+    @alias.response(200, 'Success', response_fields)
+    @alias.response(400, 'Input validation exception', response_fields)
+    @alias.response(409, 'Duplicate alias', response_fields)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self):
+        """ Create a new alias """
+        data = api.payload
+
+        alias_found = models.Alias.query.filter_by(email = data['email']).first()
+        if alias_found:
+          return { 'code': 409, 'message': f'Duplicate alias {data["email"]}'}, 409
+
+        alias_model = models.Alias(email=data["email"],destination=data['destination'])
+        if 'comment' in data:
+          alias_model.comment = data['comment']
+        if 'wildcard' in data:
+          alias_model.wildcard = data['wildcard']
+        db.session.add(alias_model)
+        db.session.commit()
+
+        return {'code': 200, 'message': f'Alias {data["email"]} to destination {data["destination"]} has been created'}, 200
+
+@alias.route('/<string:alias>')
+class Alias(Resource):
+    @alias.doc('find_alias')
+    @alias.response(200, 'Success', alias_fields)
+    @alias.response(404, 'Alias not found', response_fields)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, alias):
+        """ Find alias """
+        alias_found = models.Alias.query.filter_by(email = alias).first()
+        if alias_found is None:
+          return { 'code': 404, 'message': f'Alias {alias} cannot be found'}, 404
+        else:
+          return marshal(alias_found,alias_fields), 200
+
+    @alias.doc('update_alias')
+    @alias.expect(alias_fields_update)
+    @alias.response(200, 'Success', response_fields)
+    @alias.response(404, 'Alias not found', response_fields)
+    @alias.response(400, 'Input validation exception', response_fields)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def patch(self, alias):
+      """ Update alias """
+      data = api.payload
+      alias_found = models.Alias.query.filter_by(email = alias).first()
+      if alias_found is None:
+        return { 'code': 404, 'message': f'Alias {alias} cannot be found'}, 404
+      if 'comment' in data:
+        alias_found.comment = data['comment']
+      if 'destination' in data:
+        alias_found.destination = data['destination']
+      if 'wildcard' in data:
+        alias_found.wildcard = data['wildcard']
+      db.session.add(alias_found)
+      db.session.commit()
+      return {'code': 200, 'message': f'Alias {alias} has been updated'}
+
+    @alias.doc('delete_alias')
+    @alias.response(200, 'Success', response_fields)
+    @alias.response(404, 'Alias not found', response_fields)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, alias):
+      """ Delete alias """
+      alias_found = models.Alias.query.filter_by(email = alias).first()
+      if alias_found is None:
+        return { 'code': 404, 'message': f'Alias {alias} cannot be found'}, 404
+      db.session.delete(alias_found)
+      db.session.commit()
+      return {'code': 200, 'message': f'Alias {alias} has been deleted'}, 200
+
+@alias.route('/destination/<string:domain>')
+class AliasWithDest(Resource):
+    @alias.doc('find_alias_filter_domain')
+    @alias.response(200, 'Success', alias_fields)
+    @alias.response(404, 'Alias or domain not found', response_fields)
+    @alias.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, domain):
+        """ Find aliases of domain """
+        domain_found = models.Domain.query.filter_by(name=domain).first()
+        if domain_found is None:
+          return { 'code': 404, 'message': f'Domain {domain} cannot be found'}, 404
+        aliases_found = domain_found.aliases
+        if aliases_found.count == 0:
+          return { 'code': 404, 'message': f'No alias can be found for domain {domain}'}, 404
+        else:
+          return marshal(aliases_found, alias_fields), 200

core/admin/mailu/api/v1/domains.py

@@ -0,0 +1,410 @@
+import validators
+from flask_restx import Resource, fields, marshal
+from . import api, response_fields, user
+from .. import common
+from ... import models
+
+db = models.db
+
+dom = api.namespace('domain', description='Domain operations')
+alt = api.namespace('alternative', description='Alternative operations')
+
+domain_fields = api.model('Domain', {
+    'name': fields.String(description='FQDN (e.g. example.com)', example='example.com', required=True),
+    'comment': fields.String(description='a comment'),
+    'max_users': fields.Integer(description='maximum number of users', min=-1, default=-1),
+    'max_aliases': fields.Integer(description='maximum number of aliases', min=-1, default=-1),
+    'max_quota_bytes': fields.Integer(description='maximum quota for mailbox', min=0),
+    'signup_enabled': fields.Boolean(description='allow signup'),
+    'alternatives': fields.List(fields.String(attribute='name', description='FQDN', example='example2.com')),
+})
+
+domain_fields_update = api.model('DomainUpdate', {
+    'comment': fields.String(description='a comment'),
+    'max_users': fields.Integer(description='maximum number of users', min=-1, default=-1),
+    'max_aliases': fields.Integer(description='maximum number of aliases', min=-1, default=-1),
+    'max_quota_bytes': fields.Integer(description='maximum quota for mailbox', min=0),
+    'signup_enabled': fields.Boolean(description='allow signup'),
+    'alternatives': fields.List(fields.String(attribute='name', description='FQDN', example='example2.com')),
+})
+
+domain_fields_get = api.model('DomainGet', {
+    'name': fields.String(description='FQDN (e.g. example.com)', example='example.com', required=True),
+    'comment': fields.String(description='a comment'),
+    'max_users': fields.Integer(description='maximum number of users', min=-1, default=-1),
+    'max_aliases': fields.Integer(description='maximum number of aliases', min=-1, default=-1),
+    'max_quota_bytes': fields.Integer(description='maximum quota for mailbox', min=0),
+    'signup_enabled': fields.Boolean(description='allow signup'),
+    'alternatives': fields.List(fields.String(attribute='name', description='FQDN', example='example2.com')),
+    'dns_autoconfig': fields.List(fields.String(description='DNS client auto-configuration entry')),
+    'dns_mx': fields.String(Description='MX record for domain'),
+    'dns_spf': fields.String(Description='SPF record for domain'),
+    'dns_dkim': fields.String(Description='DKIM record for domain'),
+    'dns_dmarc': fields.String(Description='DMARC record for domain'),
+    'dns_dmarc_report': fields.String(Description='DMARC report record for domain'),
+    'dns_tlsa': fields.String(Description='TLSA record for domain'),
+})
+
+domain_fields_dns = api.model('DomainDNS', {
+    'dns_autoconfig': fields.List(fields.String(description='DNS client auto-configuration entry')),
+    'dns_mx': fields.String(Description='MX record for domain'),
+    'dns_spf': fields.String(Description='SPF record for domain'),
+    'dns_dkim': fields.String(Description='DKIM record for domain'),
+    'dns_dmarc': fields.String(Description='DMARC record for domain'),
+    'dns_dmarc_report': fields.String(Description='DMARC report record for domain'),
+    'dns_tlsa': fields.String(Description='TLSA record for domain'),
+})
+
+manager_fields = api.model('Manager', {
+    'domain_name': fields.String(description='domain managed by manager'),
+    'user_email': fields.String(description='email address of manager'),
+})
+
+manager_fields_create = api.model('ManagerCreate', {
+    'user_email': fields.String(description='email address of manager', required=True),
+})
+
+alternative_fields_update = api.model('AlternativeDomainUpdate', {
+    'domain': fields.String(description='domain FQDN', example='example.com', required=False),
+})
+
+alternative_fields = api.model('AlternativeDomain', {
+    'name': fields.String(description='alternative FQDN', example='example2.com', required=True),
+    'domain': fields.String(description='domain FQDN', example='example.com', required=True),
+})
+
+
+@dom.route('')
+class Domains(Resource):
+    @dom.doc('list_domain')
+    @dom.marshal_with(domain_fields_get, as_list=True, skip_none=True, mask=None)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self):
+        """ List domains """
+        return models.Domain.query.all()
+
+    @dom.doc('create_domain')
+    @dom.expect(domain_fields)
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(409, 'Duplicate domain/alternative name', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self):
+        """ Create a new domain """
+        data = api.payload
+        if not validators.domain(data['name']):
+            return { 'code': 400, 'message': f'Domain {data["name"]} is not a valid domain'}, 400
+
+        if common.fqdn_in_use(data['name']):
+            return { 'code': 409, 'message': f'Duplicate domain name {data["name"]}'}, 409
+        if 'alternatives' in data:
+            #check if duplicate alternatives are supplied
+            if [x for x in data['alternatives'] if data['alternatives'].count(x) >= 2]:
+                return { 'code': 409, 'message': f'Duplicate alternative domain names in request' }, 409
+            for item in data['alternatives']:
+                if common.fqdn_in_use(item):
+                    return { 'code': 409, 'message': f'Duplicate alternative domain name {item}' }, 409
+                if not validators.domain(item):
+                    return { 'code': 400, 'message': f'Alternative domain {item} is not a valid domain'}, 400
+            for item in data['alternatives']:
+                alternative = models.Alternative(name=item, domain_name=data['name'])
+                models.db.session.add(alternative)
+        domain_new = models.Domain(name=data['name'])
+        if 'comment' in data:
+            domain_new.comment = data['comment']
+        if 'max_users' in data:
+            domain_new.comment = data['max_users']
+        if 'max_aliases' in data:
+            domain_new.comment = data['max_aliases']
+        if 'max_quota_bytes' in data:
+            domain_new.comment = data['max_quota_bytes']
+        if 'signup_enabled' in data:
+            domain_new.comment = data['signup_enabled']
+        models.db.session.add(domain_new)
+        #apply the changes
+        db.session.commit()
+        return  {'code': 200, 'message': f'Domain {data["name"]} has been created'}, 200
+
+@dom.route('/<domain>')
+class Domain(Resource):
+
+    @dom.doc('find_domain')
+    @dom.response(200, 'Success', domain_fields)
+    @dom.response(404, 'Domain not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, domain):
+        """ Find domain by name """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain_found = models.Domain.query.get(domain)
+        if not domain_found:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        return marshal(domain_found, domain_fields_get), 200
+
+    @dom.doc('update_domain')
+    @dom.expect(domain_fields_update)
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'Domain not found', response_fields)
+    @dom.response(409, 'Duplicate domain/alternative name', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def patch(self, domain):
+        """ Update an existing domain """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain_found = models.Domain.query.get(domain)
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {data["name"]} does not exist'}, 404
+        data = api.payload
+
+        if 'alternatives' in data:
+            #check if duplicate alternatives are supplied
+            if [x for x in data['alternatives'] if data['alternatives'].count(x) >= 2]:
+                return { 'code': 409, 'message': f'Duplicate alternative domain names in request' }, 409
+            for item in data['alternatives']:
+                if common.fqdn_in_use(item):
+                    return { 'code': 409, 'message': f'Duplicate alternative domain name {item}' }, 409
+                if not validators.domain(item):
+                    return { 'code': 400, 'message': f'Alternative domain {item} is not a valid domain'}, 400
+            for item in data['alternatives']:
+                alternative = models.Alternative(name=item, domain_name=data['name'])
+                models.db.session.add(alternative)
+
+        if 'comment' in data:
+            domain_found.comment = data['comment']
+        if 'max_users' in data:
+            domain_found.comment = data['max_users']
+        if 'max_aliases' in data:
+            domain_found.comment = data['max_aliases']
+        if 'max_quota_bytes' in data:
+            domain_found.comment = data['max_quota_bytes']
+        if 'signup_enabled' in data:
+            domain_found.comment = data['signup_enabled']
+        models.db.session.add(domain_found)
+
+        #apply the changes
+        db.session.commit()
+        return  {'code': 200, 'message': f'Domain {domain} has been updated'}, 200
+
+    @dom.doc('delete_domain')
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'Domain not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, domain):
+        """ Delete domain """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain_found = models.Domain.query.get(domain)
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        db.session.delete(domain_found)
+        db.session.commit()
+        return {'code': 200, 'message': f'Domain {domain} has been deleted'}, 200
+
+@dom.route('/<domain>/dkim')
+class Domain(Resource):
+    @dom.doc('generate_dkim')
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'Domain not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self, domain):
+        """ Generate new DKIM/DMARC keys for domain """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain_found = models.Domain.query.get(domain)
+        if not domain_found:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        domain_found.generate_dkim_key()
+        domain_found.save_dkim_key()
+        return {'code': 200, 'message': f'DKIM/DMARC keys have been generated for domain {domain}'}, 200
+
+@dom.route('/<domain>/manager')
+class Manager(Resource):
+    @dom.doc('list_managers')
+    @dom.marshal_with(manager_fields, as_list=True, skip_none=True, mask=None)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'domain not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, domain):
+        """ List managers of domain """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        domain = models.Domain.query.filter_by(name=domain)
+        return domain.managers
+
+    @dom.doc('create_manager')
+    @dom.expect(manager_fields_create)
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'User or domain not found', response_fields)
+    @dom.response(409, 'Duplicate domain manager', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self, domain):
+        """ Create a new domain manager """
+        data = api.payload
+        if not validators.email(data['user_email']):
+            return {'code': 400, 'message': f'Invalid email address {data["user_email"]}'}, 400
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain = models.Domain.query.get(domain)
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        user = models.User.query.get(data['user_email'])
+        if not user:
+            return { 'code': 404, 'message': f'User {data["user_email"]} does not exist'}, 404
+        if user in domain.managers:
+            return {'code': 409, 'message': f'User {data["user_email"]} is already a manager of the domain {domain} '}, 409
+        domain.managers.append(user)
+        models.db.session.commit()
+        return {'code': 200, 'message': f'User {data["user_email"]} has been added as manager of the domain {domain} '},200
+
+@dom.route('/<domain>/manager/<email>')
+class Domain(Resource):
+    @dom.doc('find_manager')
+    @dom.response(200, 'Success', manager_fields)
+    @dom.response(404, 'Manager not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, domain, email):
+        """ Find manager by email address """
+        if not validators.email(email):
+            return {'code': 400, 'message': f'Invalid email address {email}'}, 400
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain = models.Domain.query.get(domain)
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        user = models.User.query.get(email)
+        if not user:
+            return { 'code': 404, 'message': f'User {email} does not exist'}, 404
+        if user in domain.managers:
+            for manager in domain.managers:
+                if manager.email == email:
+                    return marshal(manager, manager_fields),200
+        else:
+            return { 'code': 404, 'message': f'User {email} is not a manager of the domain {domain}'}, 404
+
+
+    @dom.doc('delete_manager')
+    @dom.response(200, 'Success', response_fields)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'Manager not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, domain, email):
+        if not validators.email(email):
+            return {'code': 400, 'message': f'Invalid email address {email}'}, 400
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain = models.Domain.query.get(domain)
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        user = models.User.query.get(email)
+        if not user:
+            return { 'code': 404, 'message': f'User {email} does not exist'}, 404
+        if user in domain.managers:
+            domain.managers.remove(user)
+            models.db.session.commit()
+            return {'code': 200, 'message': f'User {email} has been removed as a manager of the domain {domain} '},200
+        else:
+            return { 'code': 404, 'message': f'User {email} is not a manager of the domain {domain}'}, 404
+
+@dom.route('/<domain>/users')
+class User(Resource):
+    @dom.doc('list_user_domain')
+    @dom.marshal_with(user.user_fields_get, as_list=True, skip_none=True, mask=None)
+    @dom.response(400, 'Input validation exception', response_fields)
+    @dom.response(404, 'Domain not found', response_fields)
+    @dom.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, domain):
+        """ List users from domain """
+        if not validators.domain(domain):
+            return { 'code': 400, 'message': f'Domain {domain} is not a valid domain'}, 400
+        domain_found = models.Domain.query.get(domain)
+        if not domain_found:
+            return { 'code': 404, 'message': f'Domain {domain} does not exist'}, 404
+        return  models.User.query.filter_by(domain=domain_found).all()
+
+@alt.route('')
+class Alternatives(Resource):
+
+    @alt.doc('list_alternative')
+    @alt.marshal_with(alternative_fields, as_list=True, skip_none=True, mask=None)
+    @alt.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self):
+      """ List alternatives """
+      return models.Alternative.query.all()
+
+
+    @alt.doc('create_alternative')
+    @alt.expect(alternative_fields)
+    @alt.response(200, 'Success', response_fields)
+    @alt.response(400, 'Input validation exception', response_fields)
+    @alt.response(404, 'Domain not found or missing', response_fields)
+    @alt.response(409, 'Duplicate alternative domain name', response_fields)
+    @alt.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self):
+        """ Create new alternative (for domain) """
+        data = api.payload
+        if not validators.domain(data['name']):
+            return { 'code': 400, 'message': f'Alternative domain {data["name"]} is not a valid domain'}, 400
+        if not validators.domain(data['domain']):
+            return { 'code': 400, 'message': f'Domain {data["domain"]} is not a valid domain'}, 400
+        domain = models.Domain.query.get(data['domain'])
+        if not domain:
+            return { 'code': 404, 'message': f'Domain {data["domain"]} does not exist'}, 404
+        if common.fqdn_in_use(data['name']):
+            return { 'code': 409, 'message': f'Duplicate alternative domain name {data["name"]}'}, 409
+
+        alternative = models.Alternative(name=data['name'], domain_name=data['domain'])
+        models.db.session.add(alternative)
+        db.session.commit()
+        return {'code': 200, 'message': f'Alternative {data["name"]} for domain {data["domain"]} has been created'}, 200
+
+@alt.route('/<string:alt>')
+class Alternative(Resource):
+    @alt.doc('find_alternative')
+    @alt.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, alt):
+        """ Find alternative (of domain) """
+        if not validators.domain(alt):
+            return { 'code': 400, 'message': f'Alternative domain {alt} is not a valid domain'}, 400
+        alternative = models.Alternative.query.filter_by(name=alt).first()
+        if not alternative:
+            return{ 'code': 404, 'message': f'Alternative domain {alt} does not exist'}, 404
+        return marshal(alternative, alternative_fields), 200
+
+    @alt.doc('delete_alternative')
+    @alt.response(200, 'Success', response_fields)
+    @alt.response(400, 'Input validation exception', response_fields)
+    @alt.response(404, 'Alternative/Domain not found or missing', response_fields)
+    @alt.response(409, 'Duplicate domain name', response_fields)
+    @alt.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, alt):
+        """ Delete alternative (for domain) """
+        if not validators.domain(alt):
+            return { 'code': 400, 'message': f'Alternative domain {alt} is not a valid domain'}, 400
+        alternative = models.Alternative.query.filter_by(name=alt).scalar()
+        if not alternative:
+            return { 'code': 404, 'message': f'Alternative domain {alt} does not exist'}, 404
+        domain = alternative.domain_name
+        db.session.delete(alternative)
+        db.session.commit()
+        return {'code': 200, 'message': f'Alternative {alt} for domain {domain} has been deleted'}, 200

core/admin/mailu/api/v1/relay.py

@@ -0,0 +1,118 @@
+from flask_restx import Resource, fields, marshal
+import validators
+
+from . import api, response_fields
+from .. import common
+from ... import models
+
+db = models.db
+
+relay = api.namespace('relay', description='Relay operations')
+
+relay_fields = api.model('Relay', {
+    'name': fields.String(description='relayed domain name', example='example.com', required=True),
+    'smtp': fields.String(description='remote host', example='example.com', required=False),
+    'comment': fields.String(description='a comment', required=False)
+})
+
+relay_fields_update = api.model('RelayUpdate', {
+    'smtp': fields.String(description='remote host', example='example.com', required=False),
+    'comment': fields.String(description='a comment', required=False)
+})
+
+@relay.route('')
+class Relays(Resource):
+    @relay.doc('list_relays')
+    @relay.marshal_with(relay_fields, as_list=True, skip_none=True, mask=None)
+    @relay.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self):
+        "List relays"
+        return models.Relay.query.all()
+
+    @relay.doc('create_relay')
+    @relay.expect(relay_fields)
+    @relay.response(200, 'Success', response_fields)
+    @relay.response(400, 'Input validation exception')
+    @relay.response(409, 'Duplicate relay', response_fields)
+    @relay.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self):
+        """ Create relay """
+        data = api.payload
+
+        if not validators.domain(name):
+            return { 'code': 400, 'message': f'Relayed domain {name} is not a valid domain'}, 400
+
+        if common.fqdn_in_use(data['name']):
+            return { 'code': 409, 'message': f'Duplicate domain {data["name"]}'}, 409
+        relay_model = models.Relay(name=data['name'])
+        if 'smtp' in data:
+            relay_model.smtp = data['smtp']
+        if 'comment' in data:
+            relay_model.comment = data['comment']
+        db.session.add(relay_model)
+        db.session.commit()
+        return {'code': 200, 'message': f'Relayed domain {data["name"]} has been created'}, 200
+
+@relay.route('/<string:name>')
+class Relay(Resource):
+    @relay.doc('find_relay')
+    @relay.response(400, 'Input validation exception', response_fields)
+    @relay.response(404, 'Relay not found', response_fields)
+    @relay.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, name):
+        """ Find relay """
+        if not validators.domain(name):
+            return { 'code': 400, 'message': f'Relayed domain {name} is not a valid domain'}, 400
+
+        relay_found = models.Relay.query.filter_by(name=name).first()
+        if relay_found is None:
+            return { 'code': 404, 'message': f'Relayed domain {name} cannot be found'}, 404
+        return  marshal(relay_found, relay_fields), 200
+
+    @relay.doc('update_relay')
+    @relay.expect(relay_fields_update)
+    @relay.response(200, 'Success', response_fields)
+    @relay.response(400, 'Input validation exception', response_fields)
+    @relay.response(404, 'Relay not found', response_fields)
+    @relay.response(409, 'Duplicate relay', response_fields)
+    @relay.doc(security='Bearer')
+    @common.api_token_authorization
+    def patch(self, name):
+        """ Update relay """
+        data = api.payload
+
+        if not validators.domain(name):
+            return { 'code': 400, 'message': f'Relayed domain {name} is not a valid domain'}, 400
+
+        relay_found = models.Relay.query.filter_by(name=name).first()
+        if relay_found is None:
+            return { 'code': 404, 'message': f'Relayed domain {name} cannot be found'}, 404
+
+        if 'smtp' in data:
+            relay_found.smtp = data['smtp']
+        if 'comment' in data:
+            relay_found.comment = data['comment']
+        db.session.add(relay_found)
+        db.session.commit()
+        return { 'code': 200, 'message': f'Relayed domain {name} has been updated'}, 200
+
+
+    @relay.doc('delete_relay')
+    @relay.response(200, 'Success', response_fields)
+    @relay.response(400, 'Input validation exception', response_fields)
+    @relay.response(404, 'Relay not found', response_fields)
+    @relay.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, name):
+        """ Delete relay """
+        if not validators.domain(name):
+            return { 'code': 400, 'message': f'Relayed domain {name} is not a valid domain'}, 400
+        relay_found = models.Relay.query.filter_by(name=name).first()
+        if relay_found is None:
+            return { 'code': 404, 'message': f'Relayed domain {name} cannot be found'}, 404
+        db.session.delete(relay_found)
+        db.session.commit()
+        return { 'code': 200, 'message': f'Relayed domain {name} has been deleted'}, 200

core/admin/mailu/api/v1/user.py

@@ -0,0 +1,262 @@
+from flask_restx import Resource, fields, marshal
+import validators, datetime
+
+from . import api, response_fields
+from .. import common
+from ... import models
+
+db = models.db
+
+user = api.namespace('user', description='User operations')
+
+user_fields_get = api.model('UserGet', {
+    'email': fields.String(description='The email address of the user', example='John.Doe@example.com', attribute='_email'),
+    'password': fields.String(description="Hash of the user's password; Example='$bcrypt-sha256$v=2,t=2b,r=12$fmsAdJbYAD1gGQIE5nfJq.$zLkQUEs2XZfTpAEpcix/1k5UTNPm0jO'"),
+    'comment': fields.String(description='A description for the user. This description is shown on the Users page', example='my comment'),
+    'quota_bytes': fields.Integer(description='The maximum quota for the user’s email box in bytes', example='1000000000'),
+    'global_admin': fields.Boolean(description='Make the user a global administrator'),
+    'enabled': fields.Boolean(description='Enable the user. When an user is disabled, the user is unable to login to the Admin GUI or webmail or access his email via IMAP/POP3 or send mail'),
+    'enable_imap': fields.Boolean(description='Allow email retrieval via IMAP'),
+    'enable_pop': fields.Boolean(description='Allow email retrieval via POP3'),
+    'allow_spoofing': fields.Boolean(description='Allow the user to spoof the sender (send email as anyone)'),
+    'forward_enabled': fields.Boolean(description='Enable auto forwarding'),
+    'forward_destination': fields.List(fields.String(description='Email address to forward emails to'), example='Other@example.com'),
+    'forward_keep': fields.Boolean(description='Keep a copy of the forwarded email in the inbox'),
+    'reply_enabled': fields.Boolean(description='Enable automatic replies. This is also known as out of office (ooo) or out of facility (oof) replies'),
+    'reply_subject': fields.String(description='Optional subject for the automatic reply', example='Out of office'),
+    'reply_body': fields.String(description='The body of the automatic reply email', example='Hello, I am out of office. I will respond when I am back.'),
+    'reply_startdate': fields.Date(description='Start date for automatic replies in YYYY-MM-DD format.', example='2022-02-10'),
+    'reply_enddate': fields.Date(description='End date for automatic replies in YYYY-MM-DD format.', example='2022-02-22'),
+    'displayed_name': fields.String(description='The display name of the user within the Admin GUI', example='John Doe'),
+    'spam_enabled': fields.Boolean(description='Enable the spam filter'),
+    'spam_mark_as_read': fields.Boolean(description='Enable marking spam mails as read'),
+    'spam_threshold': fields.Integer(description='The user defined spam filter tolerance', example='80'),
+})
+
+user_fields_post = api.model('UserCreate', {
+    'email': fields.String(description='The email address of the user', example='John.Doe@example.com', attribute='_email', required=True),
+    'raw_password': fields.String(description='The raw (plain text) password of the user. Mailu will hash the password using BCRYPT-SHA256', example='secret', required=True),
+    'comment': fields.String(description='A description for the user. This description is shown on the Users page', example='my comment'),
+    'quota_bytes': fields.Integer(description='The maximum quota for the user’s email box in bytes', example='1000000000'),
+    'global_admin': fields.Boolean(description='Make the user a global administrator'),
+    'enabled': fields.Boolean(description='Enable the user. When an user is disabled, the user is unable to login to the Admin GUI or webmail or access his email via IMAP/POP3 or send mail'),
+    'enable_imap': fields.Boolean(description='Allow email retrieval via IMAP'),
+    'enable_pop': fields.Boolean(description='Allow email retrieval via POP3'),
+    'allow_spoofing': fields.Boolean(description='Allow the user to spoof the sender (send email as anyone)'),
+    'forward_enabled': fields.Boolean(description='Enable auto forwarding'),
+    'forward_destination': fields.List(fields.String(description='Email address to forward emails to'), example='Other@example.com'),
+    'forward_keep': fields.Boolean(description='Keep a copy of the forwarded email in the inbox'),
+    'reply_enabled': fields.Boolean(description='Enable automatic replies. This is also known as out of office (ooo) or out of facility (oof) replies'),
+    'reply_subject': fields.String(description='Optional subject for the automatic reply', example='Out of office'),
+    'reply_body': fields.String(description='The body of the automatic reply email', example='Hello, I am out of office. I will respond when I am back.'),
+    'reply_startdate': fields.Date(description='Start date for automatic replies in YYYY-MM-DD format.', example='2022-02-10'),
+    'reply_enddate': fields.Date(description='End date for automatic replies in YYYY-MM-DD format.', example='2022-02-22'),
+    'displayed_name': fields.String(description='The display name of the user within the Admin GUI', example='John Doe'),
+    'spam_enabled': fields.Boolean(description='Enable the spam filter'),
+    'spam_mark_as_read': fields.Boolean(description='Enable marking spam mails as read'),
+    'spam_threshold': fields.Integer(description='The user defined spam filter tolerance', example='80'),
+})
+
+user_fields_put = api.model('UserUpdate', {
+    'raw_password': fields.String(description='The raw (plain text) password of the user. Mailu will hash the password using BCRYPT-SHA256', example='secret'),
+    'comment': fields.String(description='A description for the user. This description is shown on the Users page', example='my comment'),
+    'quota_bytes': fields.Integer(description='The maximum quota for the user’s email box in bytes', example='1000000000'),
+    'global_admin': fields.Boolean(description='Make the user a global administrator'),
+    'enabled': fields.Boolean(description='Enable the user. When an user is disabled, the user is unable to login to the Admin GUI or webmail or access his email via IMAP/POP3 or send mail'),
+    'enable_imap': fields.Boolean(description='Allow email retrieval via IMAP'),
+    'enable_pop': fields.Boolean(description='Allow email retrieval via POP3'),
+    'allow_spoofing': fields.Boolean(description='Allow the user to spoof the sender (send email as anyone)'),
+    'forward_enabled': fields.Boolean(description='Enable auto forwarding'),
+    'forward_destination': fields.List(fields.String(description='Email address to forward emails to'), example='Other@example.com'),
+    'forward_keep': fields.Boolean(description='Keep a copy of the forwarded email in the inbox'),
+    'reply_enabled': fields.Boolean(description='Enable automatic replies. This is also known as out of office (ooo) or out of facility (oof) replies'),
+    'reply_subject': fields.String(description='Optional subject for the automatic reply', example='Out of office'),
+    'reply_body': fields.String(description='The body of the automatic reply email', example='Hello, I am out of office. I will respond when I am back.'),
+    'reply_startdate': fields.Date(description='Start date for automatic replies in YYYY-MM-DD format.', example='2022-02-10'),
+    'reply_enddate': fields.Date(description='End date for automatic replies in YYYY-MM-DD format.', example='2022-02-22'),
+    'displayed_name': fields.String(description='The display name of the user within the Admin GUI', example='John Doe'),
+    'spam_enabled': fields.Boolean(description='Enable the spam filter'),
+    'spam_mark_as_read': fields.Boolean(description='Enable marking spam mails as read'),
+    'spam_threshold': fields.Integer(description='The user defined spam filter tolerance', example='80'),
+})
+
+
+@user.route('')
+class Users(Resource):
+    @user.doc('list_users')
+    @user.marshal_with(user_fields_get, as_list=True, skip_none=True, mask=None)
+    @user.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self):
+        "List users"
+        return models.User.query.all()
+
+    @user.doc('create_user')
+    @user.expect(user_fields_post)
+    @user.response(200, 'Success', response_fields)
+    @user.response(400, 'Input validation exception')
+    @user.response(409, 'Duplicate user', response_fields)
+    @user.doc(security='Bearer')
+    @common.api_token_authorization
+    def post(self):
+        """ Create user """
+        data = api.payload
+        if not validators.email(data['email']):
+            return { 'code': 400, 'message': f'Provided email address {data["email"]} is not a valid email address'}, 400
+        localpart, domain_name = data['email'].lower().rsplit('@', 1)
+        domain_found = models.Domain.query.get(domain_name)
+        if not domain_found:
+            return { 'code': 404, 'message': f'Domain {domain_name} does not exist'}, 404
+
+        user_new = models.User(email=data['email'])
+        if 'raw_password' in data:
+            user_new.set_password(data['raw_password'])
+        if 'comment' in data:
+            user_new.comment = data['comment']
+        if 'quota_bytes' in data:
+            user_new.quota_bytes = data['quota_bytes']
+        if 'global_admin' in data:
+            user_new.global_admin = data['global_admin']
+        if 'enabled' in data:
+            user_new.enabled = data['enabled']
+        if 'enable_imap' in data:
+            user_new.enable_imap = data['enable_imap']
+        if 'enable_pop' in data:
+            user_new.enable_pop = data['enable_pop']
+        if 'allow_spoofing' in data:
+            user_new.allow_spoofing = data['allow_spoofing']
+        if 'forward_enabled' in data:
+            user_new.forward_enabled = data['forward_enabled']
+        if 'forward_destination' in data:
+            user_new.forward_destination = data['forward_destination']
+        if 'forward_keep' in data:
+            user_new.forward_keep = data['forward_keep']
+        if 'reply_enabled' in data:
+            user_new.reply_enabled = data['reply_enabled']
+        if 'reply_subject' in data:
+            user_new.reply_subject = data['reply_subject']
+        if 'reply_body' in data:
+            user_new.reply_body = data['reply_body']
+        if 'reply_startdate' in data:
+            year, month, day = data['reply_startdate'].split('-')
+            date = datetime.datetime(int(year), int(month), int(day))
+            user_new.reply_startdate = date
+        if 'reply_enddate' in data:
+            year, month, day = data['reply_enddate'].split('-')
+            date = datetime.datetime(int(year), int(month), int(day))
+            user_new.reply_enddate = date
+        if 'displayed_name' in data:
+            user_new.displayed_name = data['displayed_name']
+        if 'spam_enabled' in data:
+            user_new.spam_enabled = data['spam_enabled']
+        if 'spam_mark_as_read' in data:
+            user_new.spam_mark_as_read = data['spam_mark_as_read']
+        if 'spam_threshold' in data:
+            user_new.spam_threshold = data['spam_threshold']
+        db.session.add(user_new)
+        db.session.commit()
+
+        return {'code': 200,'message': f'User {data["email"]} has been created'}, 200
+
+
+@user.route('/<string:email>')
+class User(Resource):
+    @user.doc('find_user')
+    @user.response(400, 'Input validation exception', response_fields)
+    @user.response(404, 'User not found', response_fields)
+    @user.doc(security='Bearer')
+    @common.api_token_authorization
+    def get(self, email):
+        """ Find user """
+        if not validators.email(email):
+            return { 'code': 400, 'message': f'Provided email address {email} is not a valid email address'}, 400
+
+        email_found = models.User.query.filter_by(email=email).first()
+        if email_found is None:
+            return { 'code': 404, 'message': f'User {email} cannot be found'}, 404
+        return  marshal(email_found, user_fields_get), 200
+
+    @user.doc('update_user')
+    @user.expect(user_fields_put)
+    @user.response(200, 'Success', response_fields)
+    @user.response(400, 'Input validation exception', response_fields)
+    @user.response(404, 'User not found', response_fields)
+    @user.response(409, 'Duplicate user', response_fields)
+    @user.doc(security='Bearer')
+    @common.api_token_authorization
+    def patch(self, email):
+        """ Update user """
+        data = api.payload
+        if not validators.email(email):
+            return { 'code': 400, 'message': f'Provided email address {data["email"]} is not a valid email address'}, 400
+        user_found = models.User.query.get(email)
+        if not user_found:
+            return {'code': 404, 'message': f'User {email} cannot be found'}, 404
+
+        if 'raw_password' in data:
+            user_found.set_password(data['raw_password'])
+        if 'comment' in data:
+            user_found.comment = data['comment']
+        if 'quota_bytes' in data:
+            user_found.quota_bytes = data['quota_bytes']
+        if 'global_admin' in data:
+            user_found.global_admin = data['global_admin']
+        if 'enabled' in data:
+            user_found.enabled = data['enabled']
+        if 'enable_imap' in data:
+            user_found.enable_imap = data['enable_imap']
+        if 'enable_pop' in data:
+            user_found.enable_pop = data['enable_pop']
+        if 'allow_spoofing' in data:
+            user_found.allow_spoofing = data['allow_spoofing']
+        if 'forward_enabled' in data:
+            user_found.forward_enabled = data['forward_enabled']
+        if 'forward_destination' in data:
+            user_found.forward_destination = data['forward_destination']
+        if 'forward_keep' in data:
+            user_found.forward_keep = data['forward_keep']
+        if 'reply_enabled' in data:
+            user_found.reply_enabled = data['reply_enabled']
+        if 'reply_subject' in data:
+            user_found.reply_subject = data['reply_subject']
+        if 'reply_body' in data:
+            user_found.reply_body = data['reply_body']
+        if 'reply_startdate' in data:
+            year, month, day = data['reply_startdate'].split('-')
+            date = datetime.datetime(int(year), int(month), int(day))
+            user_found.reply_startdate = date
+        if 'reply_enddate' in data:
+            year, month, day = data['reply_enddate'].split('-')
+            date = datetime.datetime(int(year), int(month), int(day))
+            user_found.reply_enddate = date
+        if 'displayed_name' in data:
+            user_found.displayed_name = data['displayed_name']
+        if 'spam_enabled' in data:
+            user_found.spam_enabled = data['spam_enabled']
+        if 'spam_mark_as_read' in data:
+            user_found.spam_mark_as_read = data['spam_mark_as_read']
+        if 'spam_threshold' in data:
+            user_found.spam_threshold = data['spam_threshold']
+        db.session.add(user_found)
+        db.session.commit()
+
+        return {'code': 200,'message': f'User {email} has been updated'}, 200
+
+
+    @user.doc('delete_user')
+    @user.response(200, 'Success', response_fields)
+    @user.response(400, 'Input validation exception', response_fields)
+    @user.response(404, 'User not found', response_fields)
+    @user.doc(security='Bearer')
+    @common.api_token_authorization
+    def delete(self, email):
+        """ Delete user """
+        if not validators.email(email):
+            return { 'code': 400, 'message': f'Provided email address {email} is not a valid email address'}, 400
+
+        email_found = models.User.query.filter_by(email=email).first()
+        if email_found is None:
+            return { 'code': 404, 'message': f'User {email} cannot be found'}, 404
+        db.session.delete(email_found)
+        db.session.commit()
+        return { 'code': 200, 'message': f'User {email} has been deleted'}, 200

core/admin/mailu/configuration.py

@@ -1,7 +1,6 @@
 import os
 
 from datetime import timedelta
-from socrate import system
 import ipaddress
 
 DEFAULT_CONFIG = {
@@ -11,19 +10,24 @@ DEFAULT_CONFIG = {
     'BABEL_DEFAULT_TIMEZONE': 'UTC',
     'BOOTSTRAP_SERVE_LOCAL': True,
     'RATELIMIT_STORAGE_URL': '',
-    'QUOTA_STORAGE_URL': '',
     'DEBUG': False,
+    'DEBUG_PROFILER': False,
+    'DEBUG_TB_INTERCEPT_REDIRECTS': False,
+    'DEBUG_ASSETS': '',
     'DOMAIN_REGISTRATION': False,
     'TEMPLATES_AUTO_RELOAD': True,
     'MEMORY_SESSIONS': False,
+    'FETCHMAIL_ENABLED': True,
+    'MAILU_VERSION': 'unknown',
     # Database settings
     'DB_FLAVOR': None,
     'DB_USER': 'mailu',
     'DB_PW': None,
     'DB_HOST': 'database',
     'DB_NAME': 'mailu',
-    'SQLITE_DATABASE_FILE':'data/main.db',
+    'SQLITE_DATABASE_FILE': 'data/main.db',
     'SQLALCHEMY_DATABASE_URI': 'sqlite:////data/main.db',
+    'SQLALCHEMY_DATABASE_URI_ROUNDCUBE': 'sqlite:////data/roundcube.db',
     'SQLALCHEMY_TRACK_MODIFICATIONS': False,
     # Statistics management
     'INSTANCE_ID_PATH': '/data/instance',
@@ -37,9 +41,9 @@ DEFAULT_CONFIG = {
     'TLS_FLAVOR': 'cert',
     'INBOUND_TLS_ENFORCE': False,
     'DEFER_ON_TLS_ERROR': True,
-    'AUTH_RATELIMIT_IP': '60/hour',
+    'AUTH_RATELIMIT_IP': '5/hour',
     'AUTH_RATELIMIT_IP_V4_MASK': 24,
-    'AUTH_RATELIMIT_IP_V6_MASK': 56,
+    'AUTH_RATELIMIT_IP_V6_MASK': 48,
     'AUTH_RATELIMIT_USER': '100/day',
     'AUTH_RATELIMIT_EXEMPTION': '',
     'AUTH_RATELIMIT_EXEMPTION_LENGTH': 86400,
@@ -59,7 +63,7 @@ DEFAULT_CONFIG = {
     # Web settings
     'SITENAME': 'Mailu',
     'WEBSITE': 'https://mailu.io',
-    'ADMIN' : 'none',
+    'ADMIN': 'none',
     'WEB_ADMIN': '/admin',
     'WEB_WEBMAIL': '/webmail',
     'WEBMAIL': 'none',
@@ -68,27 +72,24 @@ DEFAULT_CONFIG = {
     'LOGO_URL': None,
     'LOGO_BACKGROUND': None,
     # Advanced settings
+    'API': False,
+    'WEB_API': '/api',
+    'API_TOKEN': None,
     'LOG_LEVEL': 'WARNING',
     'SESSION_KEY_BITS': 128,
     'SESSION_TIMEOUT': 3600,
     'PERMANENT_SESSION_LIFETIME': 30*24*3600,
-    'SESSION_COOKIE_SECURE': True,
+    'SESSION_COOKIE_SECURE': None,
     'CREDENTIAL_ROUNDS': 12,
     'TLS_PERMISSIVE': True,
     'TZ': 'Etc/UTC',
-    # Host settings
-    'HOST_IMAP': 'imap',
-    'HOST_LMTP': 'imap:2525',
-    'HOST_POP3': 'imap',
-    'HOST_SMTP': 'smtp',
-    'HOST_AUTHSMTP': 'smtp',
-    'HOST_ADMIN': 'admin',
-    'HOST_WEBMAIL': 'webmail',
-    'HOST_WEBDAV': 'webdav:5232',
-    'HOST_REDIS': 'redis',
-    'HOST_FRONT': 'front',
+    'DEFAULT_SPAM_THRESHOLD': 80,
+    'PROXY_AUTH_WHITELIST': '',
+    'PROXY_AUTH_HEADER': 'X-Auth-Email',
+    'PROXY_AUTH_CREATE': False,
+    'PROXY_AUTH_LOGOUT_URL': None,
     'SUBNET': '192.168.203.0/24',
-    'SUBNET6': None
+    'SUBNET6': None,
 }
 
 class ConfigManager:
@@ -98,25 +99,12 @@ class ConfigManager:
     DB_TEMPLATES = {
         'sqlite': 'sqlite:////{SQLITE_DATABASE_FILE}',
         'postgresql': 'postgresql://{DB_USER}:{DB_PW}@{DB_HOST}/{DB_NAME}',
-        'mysql': 'mysql+mysqlconnector://{DB_USER}:{DB_PW}@{DB_HOST}/{DB_NAME}'
+        'mysql': 'mysql+mysqlconnector://{DB_USER}:{DB_PW}@{DB_HOST}/{DB_NAME}',
     }
 
     def __init__(self):
         self.config = dict()
 
-    def get_host_address(self, name):
-        # if MYSERVICE_ADDRESS is defined, use this
-        if f'{name}_ADDRESS' in os.environ:
-            return os.environ.get(f'{name}_ADDRESS')
-        # otherwise use the host name and resolve it
-        return system.resolve_address(self.config[f'HOST_{name}'])
-
-    def resolve_hosts(self):
-        for key in ['IMAP', 'POP3', 'AUTHSMTP', 'SMTP', 'REDIS']:
-            self.config[f'{key}_ADDRESS'] = self.get_host_address(key)
-        if self.config['WEBMAIL'] != 'none':
-            self.config['WEBMAIL_ADDRESS'] = self.get_host_address('WEBMAIL')
-
     def __get_env(self, key, value):
         key_file = key + "_FILE"
         if key_file in os.environ:
@@ -137,33 +125,45 @@ class ConfigManager:
         # get current app config
         self.config.update(app.config)
         # get environment variables
+        for key in os.environ:
+            if key.endswith('_ADDRESS'):
+                self.config[key] = os.environ[key]
+
         self.config.update({
             key: self.__coerce_value(self.__get_env(key, value))
             for key, value in DEFAULT_CONFIG.items()
         })
-        self.resolve_hosts()
 
         # automatically set the sqlalchemy string
         if self.config['DB_FLAVOR']:
             template = self.DB_TEMPLATES[self.config['DB_FLAVOR']]
             self.config['SQLALCHEMY_DATABASE_URI'] = template.format(**self.config)
 
-        self.config['RATELIMIT_STORAGE_URL'] = f'redis://{self.config["REDIS_ADDRESS"]}/2'
-        self.config['QUOTA_STORAGE_URL'] = f'redis://{self.config["REDIS_ADDRESS"]}/1'
+        if not self.config.get('RATELIMIT_STORAGE_URL'):
+            self.config['RATELIMIT_STORAGE_URL'] = f'redis://{self.config["REDIS_ADDRESS"]}/2'
+
         self.config['SESSION_STORAGE_URL'] = f'redis://{self.config["REDIS_ADDRESS"]}/3'
         self.config['SESSION_COOKIE_SAMESITE'] = 'Strict'
         self.config['SESSION_COOKIE_HTTPONLY'] = True
+        if self.config['SESSION_COOKIE_SECURE'] is None:
+            self.config['SESSION_COOKIE_SECURE'] = self.config['TLS_FLAVOR'] != 'notls'
         self.config['SESSION_PERMANENT'] = True
         self.config['SESSION_TIMEOUT'] = int(self.config['SESSION_TIMEOUT'])
+        self.config['SESSION_KEY_BITS'] = int(self.config['SESSION_KEY_BITS'])
         self.config['PERMANENT_SESSION_LIFETIME'] = int(self.config['PERMANENT_SESSION_LIFETIME'])
         self.config['AUTH_RATELIMIT_IP_V4_MASK'] = int(self.config['AUTH_RATELIMIT_IP_V4_MASK'])
         self.config['AUTH_RATELIMIT_IP_V6_MASK'] = int(self.config['AUTH_RATELIMIT_IP_V6_MASK'])
-        hostnames = [host.strip() for host in self.config['HOSTNAMES'].split(',')]
         self.config['AUTH_RATELIMIT_EXEMPTION'] = set(ipaddress.ip_network(cidr, False) for cidr in (cidr.strip() for cidr in self.config['AUTH_RATELIMIT_EXEMPTION'].split(',')) if cidr)
         self.config['MESSAGE_RATELIMIT_EXEMPTION'] = set([s for s in self.config['MESSAGE_RATELIMIT_EXEMPTION'].lower().replace(' ', '').split(',') if s])
+        hostnames = [host.strip() for host in self.config['HOSTNAMES'].split(',')]
         self.config['HOSTNAMES'] = ','.join(hostnames)
         self.config['HOSTNAME'] = hostnames[0]
+        self.config['DEFAULT_SPAM_THRESHOLD'] = int(self.config['DEFAULT_SPAM_THRESHOLD'])
+        self.config['PROXY_AUTH_WHITELIST'] = set(ipaddress.ip_network(cidr, False) for cidr in (cidr.strip() for cidr in self.config['PROXY_AUTH_WHITELIST'].split(',')) if cidr)
+        try:
+            self.config['MAILU_VERSION'] = open('/version', 'r').read()
+        except FileNotFoundError:
+            pass
 
         # update the app config
         app.config.update(self.config)
-

core/admin/mailu/dkim.py

@@ -2,20 +2,20 @@
 They are thus represented as ASCII armored PEM.
 """
 
-from OpenSSL import crypto
+from cryptography.hazmat.primitives import serialization
+from cryptography.hazmat.primitives.asymmetric import rsa
 
 
-def gen_key(key_type=crypto.TYPE_RSA, bits=2048):
+def gen_key(bits=2048):
     """ Generate and return a new RSA key.
     """
-    key = crypto.PKey()
-    key.generate_key(key_type, bits)
-    return crypto.dump_privatekey(crypto.FILETYPE_PEM, key)
+    k = rsa.generate_private_key(public_exponent=65537, key_size=bits)
+    return k.private_bytes(encoding=serialization.Encoding.PEM,format=serialization.PrivateFormat.PKCS8,encryption_algorithm=serialization.NoEncryption())
 
 
 def strip_key(pem):
     """ Return only the b64 part of the ASCII armored PEM.
     """
-    key = crypto.load_privatekey(crypto.FILETYPE_PEM, pem)
-    public_pem = crypto.dump_publickey(crypto.FILETYPE_PEM, key)
+    priv_key = serialization.load_pem_private_key(pem, password=None)
+    public_pem = priv_key.public_key().public_bytes(encoding=serialization.Encoding.PEM,format=serialization.PublicFormat.SubjectPublicKeyInfo)
     return public_pem.replace(b"\n", b"").split(b"-----")[2]

core/admin/mailu/internal/nginx.py

@@ -1,12 +1,10 @@
 from mailu import models, utils
 from flask import current_app as app
+from socrate import system
 
-import re
 import urllib
 import ipaddress
-import socket
 import sqlalchemy.exc
-import tenacity
 
 SUPPORTED_AUTH_METHODS = ["none", "plain"]
 
@@ -27,12 +25,14 @@ STATUSES = {
     }),
 }
 
+WEBMAIL_PORTS = ['10143', '10025']
+
 def check_credentials(user, password, ip, protocol=None, auth_port=None):
-    if not user or not user.enabled or (protocol == "imap" and not user.enable_imap) or (protocol == "pop3" and not user.enable_pop):
+    if not user or not user.enabled or (protocol == "imap" and not user.enable_imap and not auth_port in WEBMAIL_PORTS) or (protocol == "pop3" and not user.enable_pop):
         return False
     is_ok = False
     # webmails
-    if auth_port in ['10143', '10025'] and password.startswith('token-'):
+    if auth_port in WEBMAIL_PORTS and password.startswith('token-'):
         if utils.verify_temp_token(user.get_id(), password):
             is_ok = True
     # All tokens are 32 characters hex lowercase
@@ -127,32 +127,20 @@ def get_status(protocol, status):
     status, codes = STATUSES[status]
     return status, codes[protocol]
 
-def extract_host_port(host_and_port, default_port):
-    host, _, port = re.match('^(.*?)(:([0-9]*))?$', host_and_port).groups()
-    return host, int(port) if port else default_port
-
 def get_server(protocol, authenticated=False):
     if protocol == "imap":
-        hostname, port = extract_host_port(app.config['IMAP_ADDRESS'], 143)
+        hostname, port = app.config['IMAP_ADDRESS'], 143
     elif protocol == "pop3":
-        hostname, port = extract_host_port(app.config['POP3_ADDRESS'], 110)
+        hostname, port = app.config['IMAP_ADDRESS'], 110
     elif protocol == "smtp":
         if authenticated:
-            hostname, port = extract_host_port(app.config['AUTHSMTP_ADDRESS'], 10025)
+            hostname, port = app.config['SMTP_ADDRESS'], 10025
         else:
-            hostname, port = extract_host_port(app.config['SMTP_ADDRESS'], 25)
+            hostname, port = app.config['SMTP_ADDRESS'], 25
     try:
-        # test if hostname is already resolved to an ip adddress
+        # test if hostname is already resolved to an ip address
         ipaddress.ip_address(hostname)
     except:
         # hostname is not an ip address - so we need to resolve it
-        hostname = resolve_hostname(hostname)
+        hostname = system.resolve_hostname(hostname)
     return hostname, port
-
-@tenacity.retry(stop=tenacity.stop_after_attempt(100),
-                wait=tenacity.wait_random(min=2, max=5))
-def resolve_hostname(hostname):
-    """ This function uses system DNS to resolve a hostname.
-    It is capable of retrying in case the host is not immediately available
-    """
-    return socket.gethostbyname(hostname)

core/admin/mailu/internal/views/auth.py

@@ -33,8 +33,8 @@ def nginx_authentication():
     for key, value in headers.items():
         response.headers[key] = str(value)
     is_valid_user = False
+    username = response.headers.get('Auth-User', None)
     if response.headers.get("Auth-User-Exists") == "True":
-        username = response.headers["Auth-User"]
         if utils.limiter.should_rate_limit_user(username, client_ip):
             # FIXME could be done before handle_authentication()
             status, code = nginx.get_status(flask.request.headers['Auth-Protocol'], 'ratelimit')
@@ -50,7 +50,7 @@ def nginx_authentication():
     elif is_valid_user:
         utils.limiter.rate_limit_user(username, client_ip)
     elif not is_from_webmail:
-        utils.limiter.rate_limit_ip(client_ip)
+        utils.limiter.rate_limit_ip(client_ip, username)
     return response
 
 @internal.route("/auth/admin")
@@ -109,7 +109,7 @@ def basic_authentication():
                 utils.limiter.exempt_ip_from_ratelimits(client_ip)
                 return response
             # We failed check_credentials
-            utils.limiter.rate_limit_user(user_email, client_ip) if user else utils.limiter.rate_limit_ip(client_ip)
+            utils.limiter.rate_limit_user(user_email, client_ip) if user else utils.limiter.rate_limit_ip(client_ip, user_email)
     response = flask.Response(status=401)
     response.headers["WWW-Authenticate"] = 'Basic realm="Login Required"'
     return response

core/admin/mailu/internal/views/dovecot.py

@@ -5,6 +5,7 @@ from flask import current_app as app
 import flask
 import socket
 import os
+import sqlalchemy.exc
 
 @internal.route("/dovecot/passdb/<path:user_email>")
 def dovecot_passdb_dict(user_email):
@@ -16,15 +17,23 @@ def dovecot_passdb_dict(user_email):
     return flask.jsonify({
         "password": None,
         "nopassword": "Y",
-        "allow_nets": ",".join(allow_nets)
+        "allow_real_nets": ",".join(allow_nets)
     })
 
+@internal.route("/dovecot/userdb/")
+def dovecot_userdb_dict_list():
+    return flask.jsonify([
+        user[0] for user in models.User.query.filter(models.User.enabled.is_(True)).with_entities(models.User.email).all()
+    ])
 
 @internal.route("/dovecot/userdb/<path:user_email>")
 def dovecot_userdb_dict(user_email):
-    user = models.User.query.get(user_email) or flask.abort(404)
+    try:
+        quota = models.User.query.filter(models.User.email==user_email).with_entities(models.User.quota_bytes).one_or_none() or flask.abort(404)
+    except sqlalchemy.exc.StatementError as exc:
+        flask.abort(404)
     return flask.jsonify({
-        "quota_rule": "*:bytes={}".format(user.quota_bytes)
+        "quota_rule": f"*:bytes={quota[0]}"
     })
 
 
@@ -33,6 +42,7 @@ def dovecot_quota(ns, user_email):
     user = models.User.query.get(user_email) or flask.abort(404)
     if ns == "storage":
         user.quota_bytes_used = flask.request.get_json()
+        user.dont_change_updated_at()
         models.db.session.commit()
     return flask.jsonify(None)
 

core/admin/mailu/internal/views/fetch.py

@@ -12,10 +12,12 @@ def fetch_list():
             "id": fetch.id,
             "tls": fetch.tls,
             "keep": fetch.keep,
+            "scan": fetch.scan,
             "user_email": fetch.user_email,
             "protocol": fetch.protocol,
             "host": fetch.host,
             "port": fetch.port,
+            "folders": fetch.folders,
             "username": fetch.username,
             "password": fetch.password
         } for fetch in models.Fetch.query.all()
@@ -27,6 +29,7 @@ def fetch_done(fetch_id):
     fetch = models.Fetch.query.get(fetch_id) or flask.abort(404)
     fetch.last_check = datetime.datetime.now()
     fetch.error_message = str(flask.request.get_json())
+    fetch.dont_change_updated_at()
     models.db.session.add(fetch)
     models.db.session.commit()
     return ""

core/admin/mailu/internal/views/postfix.py

@@ -143,8 +143,9 @@ def postfix_sender_login(sender):
     if localpart is None:
         return flask.jsonify(",".join(wildcard_senders)) if wildcard_senders else flask.abort(404)
     localpart = localpart[:next((i for i, ch in enumerate(localpart) if ch in flask.current_app.config.get('RECIPIENT_DELIMITER')), None)]
-    destinations = models.Email.resolve_destination(localpart, domain_name, True) or []
-    destinations.extend(wildcard_senders)
+    destinations = set(models.Email.resolve_destination(localpart, domain_name, True) or [])
+    destinations.update(wildcard_senders)
+    destinations.update(i[0] for i in models.User.query.filter_by(allow_spoofing=True).with_entities(models.User.email).all())
     if destinations:
         return flask.jsonify(",".join(idna_encode(destinations)))
     return flask.abort(404)
@@ -158,21 +159,6 @@ def postfix_sender_rate(sender):
     user = models.User.get(sender) or flask.abort(404)
     return flask.abort(404) if user.sender_limiter.hit() else flask.jsonify("450 4.2.1 You are sending too many emails too fast.")
 
-@internal.route("/postfix/sender/access/<path:sender>")
-def postfix_sender_access(sender):
-    """ Simply reject any sender that pretends to be from a local domain
-    """
-    if '@' in sender:
-        if sender.startswith('<') and sender.endswith('>'):
-            sender = sender[1:-1]
-        try:
-            localpart, domain_name = models.Email.resolve_domain(sender)
-            if models.Domain.query.get(domain_name):
-                return flask.jsonify("REJECT")
-        except sqlalchemy.exc.StatementError:
-            pass
-    return flask.abort(404)
-
 # idna encode domain part of each address in list of addresses
 def idna_encode(addresses):
     return [

core/admin/mailu/internal/views/rspamd.py

@@ -25,3 +25,7 @@ def rspamd_dkim_key(domain_name):
                 }
             )
     return flask.jsonify({'data': {'selectors': selectors}})
+
+@internal.route("/rspamd/local_domains", methods=['GET'])
+def rspamd_local_domains():
+    return '\n'.join(domain[0] for domain in models.Domain.query.with_entities(models.Domain.name).all() + models.Alternative.query.with_entities(models.Alternative.name).all())

core/admin/mailu/limiter.py

@@ -52,10 +52,13 @@ class LimitWraperFactory(object):
             app.logger.warn(f'Authentication attempt from {ip} has been rate-limited.')
         return is_rate_limited
 
-    def rate_limit_ip(self, ip):
-        limiter = self.get_limiter(app.config["AUTH_RATELIMIT_IP"], 'auth-ip')
+    def rate_limit_ip(self, ip, username=None):
+        limiter = self.get_limiter(app.config['AUTH_RATELIMIT_IP'], 'auth-ip')
         client_network = utils.extract_network_from_ip(ip)
         if self.is_subject_to_rate_limits(ip):
+            if username and self.storage.get(f'dedup-{client_network}-{username}') > 0:
+                return
+            self.storage.incr(f'dedup-{client_network}-{username}', limits.parse(app.config['AUTH_RATELIMIT_IP']).GRANULARITY.seconds, True)
             limiter.hit(client_network)
 
     def should_rate_limit_user(self, username, ip, device_cookie=None, device_cookie_name=None):

core/admin/mailu/manage.py

@@ -304,6 +304,7 @@ def config_update(verbose=False, delete_objects=False):
                 if verbose:
                     print(f'Deleting domain: {domain.name}')
                 db.session.delete(domain)
+
     db.session.commit()
 
 
@@ -351,7 +352,7 @@ def config_import(verbose=0, secrets=False, debug=False, quiet=False, color=Fals
             raise click.ClickException(msg) from exc
         raise
 
-    # don't commit when running dry
+    # do not commit when running dry
     if dry_run:
         log.changes('Dry run. Not committing changes.')
         db.session.rollback()
@@ -385,6 +386,7 @@ def config_export(full=False, secrets=False, color=False, dns=False, output=None
         'dns': dns,
     }
 
+    old_umask = os.umask(0o077)
     try:
         schema = MailuSchema(only=only, context=context)
         if as_json:
@@ -396,17 +398,22 @@ def config_export(full=False, secrets=False, color=False, dns=False, output=None
         if msg := log.format_exception(exc):
             raise click.ClickException(msg) from exc
         raise
+    finally:
+        os.umask(old_umask)
 
 
 @mailu.command()
 @click.argument('email')
+@click.option('-r', '--really', is_flag=True)
 @with_appcontext
-def user_delete(email):
-    """delete user"""
-    user = models.User.query.get(email)
-    if user:
-        db.session.delete(user)
-    db.session.commit()
+def user_delete(email, really=False):
+    """disable or delete user"""
+    if user := models.User.query.get(email):
+        if really:
+            db.session.delete(user)
+        else:
+            user.enabled = False
+        db.session.commit()
 
 
 @mailu.command()
@@ -414,10 +421,9 @@ def user_delete(email):
 @with_appcontext
 def alias_delete(email):
     """delete alias"""
-    alias = models.Alias.query.get(email)
-    if alias:
+    if alias := models.Alias.query.get(email):
         db.session.delete(alias)
-    db.session.commit()
+        db.session.commit()
 
 
 @mailu.command()

core/admin/mailu/models.py

@@ -2,7 +2,6 @@
 """
 
 import os
-import smtplib
 import json
 
 from datetime import date
@@ -25,6 +24,7 @@ from flask import current_app as app
 from sqlalchemy.ext import declarative
 from sqlalchemy.ext.hybrid import hybrid_property
 from sqlalchemy.inspection import inspect
+from sqlalchemy.orm.attributes import flag_modified
 from werkzeug.utils import cached_property
 
 from mailu import dkim, utils
@@ -75,7 +75,7 @@ class CommaSeparatedList(db.TypeDecorator):
     """ Stores a list as a comma-separated string, compatible with Postfix.
     """
 
-    impl = db.String
+    impl = db.String(255)
     cache_ok = True
     python_type = list
 
@@ -96,7 +96,7 @@ class JSONEncoded(db.TypeDecorator):
     """ Represents an immutable structure as a json-encoded string.
     """
 
-    impl = db.String
+    impl = db.String(255)
     cache_ok = True
     python_type = str
 
@@ -154,6 +154,10 @@ class Base(db.Model):
             self.__hashed = id(self) if primary is None else hash(primary)
         return self.__hashed
 
+    def dont_change_updated_at(self):
+        """ Mark updated_at as modified, but keep the old date when updating the model"""
+        flag_modified(self, 'updated_at')
+
 
 # Many-to-many association table for domain managers
 managers = db.Table('manager', Base.metadata,
@@ -415,14 +419,18 @@ class Email(object):
 
     def sendmail(self, subject, body):
         """ send an email to the address """
-        f_addr = f'{app.config["POSTMASTER"]}@{idna.encode(app.config["DOMAIN"]).decode("ascii")}'
-        with smtplib.SMTP(app.config['HOST_AUTHSMTP'], port=10025) as smtp:
-            to_address = f'{self.localpart}@{idna.encode(self.domain_name).decode("ascii")}'
-            msg = text.MIMEText(body)
-            msg['Subject'] = subject
-            msg['From'] = f_addr
-            msg['To'] = to_address
-            smtp.sendmail(f_addr, [to_address], msg.as_string())
+        try:
+            f_addr = f'{app.config["POSTMASTER"]}@{idna.encode(app.config["DOMAIN"]).decode("ascii")}'
+            with smtplib.LMTP(host=app.config['IMAP_ADDRESS'], port=2525) as lmtp:
+                to_address = f'{self.localpart}@{idna.encode(self.domain_name).decode("ascii")}'
+                msg = text.MIMEText(body)
+                msg['Subject'] = subject
+                msg['From'] = f_addr
+                msg['To'] = to_address
+                lmtp.sendmail(f_addr, [to_address], msg.as_string())
+            return True
+        except smtplib.SMTPException:
+            return False
 
     @classmethod
     def resolve_domain(cls, email):
@@ -496,6 +504,7 @@ class User(Base, Email):
     # Features
     enable_imap = db.Column(db.Boolean, nullable=False, default=True)
     enable_pop = db.Column(db.Boolean, nullable=False, default=True)
+    allow_spoofing = db.Column(db.Boolean, nullable=False, default=False)
 
     # Filters
     forward_enabled = db.Column(db.Boolean, nullable=False, default=False)
@@ -513,7 +522,7 @@ class User(Base, Email):
     displayed_name = db.Column(db.String(160), nullable=False, default='')
     spam_enabled = db.Column(db.Boolean, nullable=False, default=True)
     spam_mark_as_read = db.Column(db.Boolean, nullable=False, default=True)
-    spam_threshold = db.Column(db.Integer, nullable=False, default=80)
+    spam_threshold = db.Column(db.Integer, nullable=False, default=lambda:int(app.config.get("DEFAULT_SPAM_THRESHOLD", 80)))
 
     # Flask-login attributes
     is_authenticated = True
@@ -541,8 +550,8 @@ class User(Base, Email):
         now = date.today()
         return (
             self.reply_enabled and
-            self.reply_startdate < now and
-            self.reply_enddate > now
+            self.reply_startdate <= now and
+            self.reply_enddate >= now
         )
 
     @property
@@ -766,6 +775,8 @@ class Fetch(Base):
     username = db.Column(db.String(255), nullable=False)
     password = db.Column(db.String(255), nullable=False)
     keep = db.Column(db.Boolean, nullable=False, default=False)
+    scan = db.Column(db.Boolean, nullable=False, default=False)
+    folders = db.Column(CommaSeparatedList, nullable=True, default=list)
     last_check = db.Column(db.DateTime, nullable=True)
     error = db.Column(db.String(1023), nullable=True)
 

core/admin/mailu/schemas.py

@@ -5,6 +5,7 @@ from copy import deepcopy
 from collections import Counter
 from datetime import timezone
 
+import inspect
 import json
 import logging
 import yaml
@@ -19,7 +20,7 @@ from marshmallow_sqlalchemy.fields import RelatedList
 
 from flask_marshmallow import Marshmallow
 
-from OpenSSL import crypto
+from cryptography.hazmat.primitives import serialization
 
 from pygments import highlight
 from pygments.token import Token
@@ -609,8 +610,8 @@ class DkimKeyField(fields.String):
 
         # check key validity
         try:
-            crypto.load_privatekey(crypto.FILETYPE_PEM, value)
-        except crypto.Error as exc:
+            serialization.load_pem_private_key(bytes(value, "ascii"), password=None)
+        except (UnicodeEncodeError, ValueError) as exc:
             raise ValidationError(f'invalid dkim key {bad_key!r}') from exc
         else:
             return value
@@ -669,20 +670,15 @@ class Storage:
 
     context = {}
 
-    def _bind(self, key, bind):
-        if bind is True:
-            return (self.__class__, key)
-        if isinstance(bind, str):
-            return (get_schema(self.recall(bind).__class__), key)
-        return (bind, key)
-
-    def store(self, key, value, bind=None):
+    def store(self, key, value):
         """ store value under key """
-        self.context.setdefault('_track', {})[self._bind(key, bind)]= value
+        key = f'{self.__class__.__name__}.{key}'
+        self.context.setdefault('_track', {})[key] = value
 
-    def recall(self, key, bind=None):
+    def recall(self, key):
         """ recall value from key """
-        return self.context['_track'][self._bind(key, bind)]
+        key = f'{self.__class__.__name__}.{key}'
+        return self.context['_track'][key]
 
 class BaseOpts(SQLAlchemyAutoSchemaOpts):
     """ Option class with sqla session
@@ -790,10 +786,16 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
             for key, value in data.items()
         }
 
-    def _call_and_store(self, *args, **kwargs):
-        """ track current parent field for pruning """
-        self.store('field', kwargs['field_name'], True)
-        return super()._call_and_store(*args, **kwargs)
+    def get_parent(self):
+        """ helper to determine parent of current object """
+        for x in inspect.stack():
+            loc = x[0].f_locals
+            if 'ret_d' in loc:
+                if isinstance(loc['self'], MailuSchema):
+                    return self.context.get('config'), loc['attr_name']
+                else:
+                    return loc['self'].get_instance(loc['ret_d']), loc['attr_name']
+        return None, None
 
     # this is only needed to work around the declared attr "email" primary key in model
     def get_instance(self, data):
@@ -803,9 +805,13 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         if keys := getattr(self.Meta, 'primary_keys', None):
             filters = {key: data.get(key) for key in keys}
             if None not in filters.values():
-                res= self.session.query(self.opts.model).filter_by(**filters).first()
-                return res
-        res= super().get_instance(data)
+                try:
+                    res = self.session.query(self.opts.model).filter_by(**filters).first()
+                except sqlalchemy.exc.StatementError as exc:
+                    raise ValidationError(f'Invalid {keys[0]}: {data.get(keys[0])!r}', data.get(keys[0])) from exc
+                else:
+                    return res
+        res = super().get_instance(data)
         return res
 
     @pre_load(pass_many=True)
@@ -829,6 +835,10 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         want_prune = []
         def patch(count, data):
 
+            # we only process objects here
+            if type(data) is not dict:
+                raise ValidationError(f'Invalid item. {self.Meta.model.__tablename__.title()} needs to be an object.', f'{data!r}')
+
             # don't allow __delete__ coming from input
             if '__delete__' in data:
                 raise ValidationError('Unknown field.', f'{count}.__delete__')
@@ -882,10 +892,10 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         ]
 
         # remember if prune was requested for _prune_items@post_load
-        self.store('prune', bool(want_prune), True)
+        self.store('prune', bool(want_prune))
 
         # remember original items to stabilize password-changes in _add_instance@post_load
-        self.store('original', items, True)
+        self.store('original', items)
 
         return items
 
@@ -909,13 +919,15 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         # stabilize import of auto-increment primary keys (not required),
         # by matching import data to existing items and setting primary key
         if not self._primary in data:
-            for item in getattr(self.recall('parent'), self.recall('field', 'parent')):
-                existing = self.dump(item, many=False)
-                this = existing.pop(self._primary)
-                if data == existing:
-                    instance = item
-                    data[self._primary] = this
-                    break
+            parent, field = self.get_parent()
+            if parent is not None:
+                for item in getattr(parent, field):
+                    existing = self.dump(item, many=False)
+                    this = existing.pop(self._primary)
+                    if data == existing:
+                        self.instance = item
+                        data[self._primary] = this
+                        break
 
         # try to load instance
         instance = self.instance or self.get_instance(data)
@@ -931,9 +943,6 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         else:
 
             if self.context.get('update'):
-                # remember instance as parent for pruning siblings
-                if not self.Meta.sibling:
-                    self.store('parent', instance)
                 # delete instance from session when marked
                 if '__delete__' in data:
                     self.opts.sqla_session.delete(instance)
@@ -968,7 +977,7 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
                                 ) from exc
                     # sort list of new values
                     data[key] = sorted(new_value)
-                    # log backref modification not catched by modify hook
+                    # log backref modification not caught by modify hook
                     if isinstance(self.fields[key], RelatedList):
                         if callback := self.context.get('callback'):
                             before = {str(v) for v in getattr(instance, key)}
@@ -997,7 +1006,7 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
             return items
 
         # get prune flag from _patch_many@pre_load
-        want_prune = self.recall('prune', True)
+        want_prune = self.recall('prune')
 
         # prune: determine if existing items in db need to be added or marked for deletion
         add_items = False
@@ -1014,14 +1023,17 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
                 del_items = True
 
         if add_items or del_items:
-            existing = {item[self._primary] for item in items if self._primary in item}
-            for item in getattr(self.recall('parent'), self.recall('field', 'parent')):
-                key = getattr(item, self._primary)
-                if key not in existing:
-                    if add_items:
-                        items.append({self._primary: key})
-                    else:
-                        items.append({self._primary: key, '__delete__': '?'})
+            parent, field = self.get_parent()
+            if parent is not None:
+                existing = {item[self._primary] for item in items if self._primary in item}
+                for item in getattr(parent, field):
+                    key = getattr(item, self._primary)
+                    if key not in existing:
+                        if add_items:
+                            items.append({self._primary: key})
+                        else:
+                            if self.context.get('update'):
+                                self.opts.sqla_session.delete(self.instance or self.get_instance({self._primary: key}))
 
         return items
 
@@ -1042,7 +1054,7 @@ class BaseSchema(ma.SQLAlchemyAutoSchema, Storage):
         # did we hash a new plaintext password?
         original = None
         pkey = getattr(item, self._primary)
-        for data in self.recall('original', True):
+        for data in self.recall('original'):
             if 'hash_password' in data and data.get(self._primary) == pkey:
                 original = data['password']
                 break
@@ -1238,12 +1250,6 @@ class MailuSchema(Schema, Storage):
                 if field in fieldlist:
                     fieldlist[field] = fieldlist.pop(field)
 
-    def _call_and_store(self, *args, **kwargs):
-        """ track current parent and field for pruning """
-        self.store('field', kwargs['field_name'], True)
-        self.store('parent', self.context.get('config'))
-        return super()._call_and_store(*args, **kwargs)
-
     @pre_load
     def _clear_config(self, data, many, **kwargs): # pylint: disable=unused-argument
         """ create config object in context if missing

core/admin/mailu/sso/forms.py

@@ -5,7 +5,8 @@ import flask_wtf
 class LoginForm(flask_wtf.FlaskForm):
     class Meta:
         csrf = False
-    email = fields.StringField(_('E-mail'), [validators.Email(), validators.DataRequired()])
+    email = fields.StringField(_('E-mail'), [validators.Email(), validators.DataRequired()], render_kw={'autofocus': True})
     pw = fields.PasswordField(_('Password'), [validators.DataRequired()])
+    pwned = fields.HiddenField(label='', default=-1)
     submitWebmail = fields.SubmitField(_('Sign in'))
     submitAdmin = fields.SubmitField(_('Sign in'))

core/admin/mailu/sso/templates/base_sso.html

@@ -1,7 +1,7 @@
 {%- import "macros.html" as macros %}
 {%- import "bootstrap/utils.html" as utils %}
 <!doctype html>
-<html lang="{{ session['language'] }}" data-static="/static/">
+<html lang="{{ get_locale() }}" data-static="/static/">
   <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1">
@@ -34,8 +34,8 @@
         <ul class="navbar-nav ml-auto">
           <li class="nav-item dropdown">
             <a class="nav-link" data-toggle="dropdown" href="#" aria-expanded="false">
-              <i class="fas fa-language text-xl" aria-hidden="true" title="{% trans %}change language{% endtrans %}"></i><span class="sr-only">Language</span>
-              <span class="badge badge-primary navbar-badge">{{ session['language'] }}</span></a>
+              <i class="fas fa-language text-xl" aria-hidden="true" title="{% trans %}change language{% endtrans %}"></i><span class="sr-only">{% trans %}change language{% endtrans %}</span>
+              <span class="badge badge-primary navbar-badge">{{ get_locale() }}</span></a>
             <div class="dropdown-menu dropdown-menu-right p-0" id="mailu-languages">
             {%- for locale in config.translations.values() %}
               <a class="dropdown-item{% if locale|string() == session['language'] %} active{% endif %}" href="{{ url_for('sso.set_language', language=locale) }}">{{ locale.get_language_name().title() }}</a>

core/admin/mailu/sso/templates/form_sso.html

@@ -3,6 +3,7 @@
 {%- block content %}
 {%- call macros.card() %}
 <form class="form" method="post" role="form">
+  {{ form.hidden_tag() }}
   {{ macros.form_field(form.email) }}
   {{ macros.form_field(form.pw) }}
   {{ macros.form_fields(fields, label=False, class="btn btn-default") }}

core/admin/mailu/sso/templates/sidebar_sso.html

@@ -36,6 +36,12 @@
         </a>
       </li>
     {%- endif %}
+      <li class="nav-item" role="none">
+        <a href="{{ url_for('sso.login') }}" class="nav-link" role="menuitem">
+          <i class="nav-icon fas fa-sign-in-alt"></i>
+          <p>{% trans %}Sign in{% endtrans %}</p>
+        </a>
+      </li>
     {#-
       User self-registration is only available when
        - Admin is available

core/admin/mailu/sso/views/base.py

@@ -6,26 +6,40 @@ from mailu.ui import access
 from flask import current_app as app
 import flask
 import flask_login
+import secrets
+import ipaddress
+from urllib.parse import urlparse, urljoin
+from werkzeug.urls import url_unquote
 
 @sso.route('/login', methods=['GET', 'POST'])
 def login():
+    if flask.request.headers.get(app.config['PROXY_AUTH_HEADER']) and not 'noproxyauth' in flask.request.url:
+        return _proxy()
+
     client_ip = flask.request.headers.get('X-Real-IP', flask.request.remote_addr)
     form = forms.LoginForm()
-    form.submitAdmin.label.text = form.submitAdmin.label.text + ' Admin'
-    form.submitWebmail.label.text = form.submitWebmail.label.text + ' Webmail'
 
     fields = []
-    if str(app.config["WEBMAIL"]).upper() != "NONE":
-        fields.append(form.submitWebmail)
-    if str(app.config["ADMIN"]).upper() != "FALSE":
+
+    if 'url' in flask.request.args and not 'homepage' in flask.request.url:
         fields.append(form.submitAdmin)
+    else:
+        form.submitAdmin.label.text = form.submitAdmin.label.text + ' Admin'
+        form.submitWebmail.label.text = form.submitWebmail.label.text + ' Webmail'
+        if str(app.config["WEBMAIL"]).upper() != "NONE":
+            fields.append(form.submitWebmail)
+        if str(app.config["ADMIN"]).upper() != "FALSE":
+            fields.append(form.submitAdmin)
     fields = [fields]
 
     if form.validate_on_submit():
-        if form.submitAdmin.data:
-            destination = app.config['WEB_ADMIN']
-        elif form.submitWebmail.data:
-            destination = app.config['WEB_WEBMAIL']
+        if destination := _has_usable_redirect():
+            pass
+        else:
+            if form.submitAdmin.data:
+                destination = app.config['WEB_ADMIN']
+            elif form.submitWebmail.data:
+                destination = app.config['WEB_WEBMAIL']
         device_cookie, device_cookie_username = utils.limiter.parse_device_cookie(flask.request.cookies.get('rate_limit'))
         username = form.email.data
         if username != device_cookie_username and utils.limiter.should_rate_limit_ip(client_ip):
@@ -40,10 +54,12 @@ def login():
             flask_login.login_user(user)
             response = flask.redirect(destination)
             response.set_cookie('rate_limit', utils.limiter.device_cookie(username), max_age=31536000, path=flask.url_for('sso.login'), secure=app.config['SESSION_COOKIE_SECURE'], httponly=True)
-            flask.current_app.logger.info(f'Login succeeded for {username} from {client_ip}.')
+            flask.current_app.logger.info(f'Login succeeded for {username} from {client_ip} pwned={form.pwned.data}.')
+            if msg := utils.isBadOrPwned(form):
+                flask.flash(msg, "error")
             return response
         else:
-            utils.limiter.rate_limit_user(username, client_ip, device_cookie, device_cookie_username) if models.User.get(username) else utils.limiter.rate_limit_ip(client_ip)
+            utils.limiter.rate_limit_user(username, client_ip, device_cookie, device_cookie_username) if models.User.get(username) else utils.limiter.rate_limit_ip(client_ip, username)
             flask.current_app.logger.warn(f'Login failed for {username} from {client_ip}.')
             flask.flash('Wrong e-mail or password', 'error')
     return flask.render_template('login.html', form=form, fields=fields)
@@ -53,5 +69,65 @@ def login():
 def logout():
     flask_login.logout_user()
     flask.session.destroy()
-    return flask.redirect(flask.url_for('.login'))
+    response = flask.redirect(app.config['PROXY_AUTH_LOGOUT_URL'] or flask.url_for('.login'))
+    for cookie in ['roundcube_sessauth', 'roundcube_sessid', 'smsession']:
+        response.set_cookie(cookie, 'empty', expires=0)
+    return response
 
+"""
+Redirect to the url passed in parameter if any; Ensure that this is not an open-redirect too...
+https://cheatsheetseries.owasp.org/cheatsheets/Unvalidated_Redirects_and_Forwards_Cheat_Sheet.html
+"""
+def _has_usable_redirect(is_proxied=False):
+    if 'homepage' in flask.request.url and not is_proxied:
+        return None
+    if url := flask.request.args.get('url'):
+        url = url_unquote(url)
+        target = urlparse(urljoin(flask.request.url, url))
+        if target.netloc == urlparse(flask.request.url).netloc:
+            return target.geturl()
+    return None
+
+"""
+https://mailu.io/master/configuration.html#header-authentication-using-an-external-proxy
+"""
+def _proxy():
+    proxy_ip = flask.request.headers.get('X-Forwarded-By', flask.request.remote_addr)
+    ip = ipaddress.ip_address(proxy_ip)
+    if not any(ip in cidr for cidr in app.config['PROXY_AUTH_WHITELIST']):
+        return flask.abort(500, '%s is not on PROXY_AUTH_WHITELIST' % proxy_ip)
+
+    email = flask.request.headers.get(app.config['PROXY_AUTH_HEADER'])
+    if not email:
+        return flask.abort(500, 'No %s header' % app.config['PROXY_AUTH_HEADER'])
+
+    url = _has_usable_redirect(True) or app.config['WEB_ADMIN']
+
+    user = models.User.get(email)
+    if user:
+        flask.session.regenerate()
+        flask_login.login_user(user)
+        return flask.redirect(url)
+
+    if not app.config['PROXY_AUTH_CREATE']:
+        return flask.abort(500, 'You don\'t exist. Go away! (%s)' % email)
+
+    client_ip = flask.request.headers.get('X-Real-IP', flask.request.remote_addr)
+    try:
+        localpart, desireddomain = email.rsplit('@')
+    except Exception as e:
+        flask.current_app.logger.error('Error creating a new user via proxy for %s from %s: %s' % (email, client_ip, str(e)), e)
+        return flask.abort(500, 'You don\'t exist. Go away! (%s)' % email)
+    domain = models.Domain.query.get(desireddomain) or flask.abort(500, 'You don\'t exist. Go away! (domain=%s)' % desireddomain)
+    if not domain.max_users == -1 and len(domain.users) >= domain.max_users:
+        flask.current_app.logger.warning('Too many users for domain %s' % domain)
+        return flask.abort(500, 'Too many users in (domain=%s)' % domain)
+    user = models.User(localpart=localpart, domain=domain)
+    user.set_password(secrets.token_urlsafe())
+    models.db.session.add(user)
+    models.db.session.commit()
+    flask.session.regenerate()
+    flask_login.login_user(user)
+    user.send_welcome()
+    flask.current_app.logger.info(f'Login succeeded by proxy created user: {user} from {client_ip} through {flask.request.remote_addr}.')
+    return flask.redirect(url)

core/admin/mailu/sso/views/languages.py

@@ -1,7 +1,7 @@
 from mailu.sso import sso
 import flask
 
-@sso.route('/language/<language>', methods=['POST'])
+@sso.route('/language/<language>', methods=['GET','POST'])
 def set_language(language=None):
     if language:
         flask.session['language'] = language

core/admin/mailu/translations/ca/LC_MESSAGES/messages.po

@@ -660,7 +660,7 @@ msgid "New relay domain"
 msgstr "Nou domini llegat (relayed)"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Editeu domini llegat (relayed)"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/cs/LC_MESSAGES/messages.po

@@ -0,0 +1,733 @@
+# Czech translations for Mailu.io.
+# Copyright (C) 2023 S474N
+# This file is distributed under the same license as the PROJECT project.
+# S474N <translate@s474n.com>, 2023.
+#
+msgid ""
+msgstr ""
+"Project-Id-Version: PROJECT VERSION\n"
+"Report-Msgid-Bugs-To: translate@s474n.com\n"
+"POT-Creation-Date: 2022-05-22 18:47+0200\n"
+"PO-Revision-Date: 2023-02-21 16:14+0100\n"
+"Last-Translator: S474N <translate@s474n.com>\n"
+"Language-Team: Czech\n"
+"Language: cs_CZ\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Plural-Forms: nplurals=3; plural=(n==1 ? 0 : n>=2 && n<=4 ? 1 : 2);\n"
+"Generated-By: Babel 2.3.4\n"
+"X-Generator: Poedit 3.2.2\n"
+
+#: mailu/sso/forms.py:8 mailu/ui/forms.py:79
+msgid "E-mail"
+msgstr "E-mail"
+
+#: mailu/sso/forms.py:9 mailu/ui/forms.py:80 mailu/ui/forms.py:93
+#: mailu/ui/forms.py:112 mailu/ui/forms.py:166
+#: mailu/ui/templates/client.html:32 mailu/ui/templates/client.html:57
+msgid "Password"
+msgstr "Heslo"
+
+#: mailu/sso/forms.py:10 mailu/sso/forms.py:11 mailu/sso/templates/login.html:4
+#: mailu/ui/templates/sidebar.html:142
+msgid "Sign in"
+msgstr "Přihlásit se"
+
+#: mailu/sso/templates/base_sso.html:8 mailu/ui/templates/base.html:8
+msgid "Admin page for"
+msgstr "Admin stránka pro"
+
+#: mailu/sso/templates/base_sso.html:19 mailu/ui/templates/base.html:19
+msgid "toggle sidebar"
+msgstr "přepnout postranní panel"
+
+#: mailu/sso/templates/base_sso.html:37 mailu/ui/templates/base.html:37
+msgid "change language"
+msgstr "změnit jazyk"
+
+#: mailu/sso/templates/sidebar_sso.html:4 mailu/ui/templates/sidebar.html:94
+msgid "Go to"
+msgstr "Jít"
+
+#: mailu/sso/templates/sidebar_sso.html:9 mailu/ui/templates/client.html:4
+#: mailu/ui/templates/sidebar.html:50 mailu/ui/templates/sidebar.html:107
+msgid "Client setup"
+msgstr "Nastavení klienta"
+
+#: mailu/sso/templates/sidebar_sso.html:16 mailu/ui/templates/sidebar.html:114
+msgid "Website"
+msgstr "Webová stránka"
+
+#: mailu/sso/templates/sidebar_sso.html:22 mailu/ui/templates/sidebar.html:120
+msgid "Help"
+msgstr "Pomoc"
+
+#: mailu/sso/templates/sidebar_sso.html:35
+#: mailu/ui/templates/domain/signup.html:4 mailu/ui/templates/sidebar.html:127
+msgid "Register a domain"
+msgstr "Registrovat doménu"
+
+#: mailu/sso/templates/sidebar_sso.html:49 mailu/ui/forms.py:95
+#: mailu/ui/templates/sidebar.html:149 mailu/ui/templates/user/signup.html:4
+#: mailu/ui/templates/user/signup_domain.html:4
+msgid "Sign up"
+msgstr "Registrovat se"
+
+#: mailu/ui/forms.py:33 mailu/ui/forms.py:36
+msgid "Invalid email address."
+msgstr "Špatná mailová adresa."
+
+#: mailu/ui/forms.py:45
+msgid "Confirm"
+msgstr "Potvrdit"
+
+#: mailu/ui/forms.py:48 mailu/ui/forms.py:58
+#: mailu/ui/templates/domain/details.html:26
+#: mailu/ui/templates/domain/list.html:19 mailu/ui/templates/relay/list.html:18
+msgid "Domain name"
+msgstr "Název domény"
+
+#: mailu/ui/forms.py:49
+msgid "Maximum user count"
+msgstr "Maximální počet uživatelů"
+
+#: mailu/ui/forms.py:50
+msgid "Maximum alias count"
+msgstr "Maximální počet aliasů"
+
+#: mailu/ui/forms.py:51
+msgid "Maximum user quota"
+msgstr "Maximální uživatelská kvóta"
+
+#: mailu/ui/forms.py:52
+msgid "Enable sign-up"
+msgstr "Povolit registraci"
+
+#: mailu/ui/forms.py:53 mailu/ui/forms.py:74 mailu/ui/forms.py:86
+#: mailu/ui/forms.py:132 mailu/ui/forms.py:144
+#: mailu/ui/templates/alias/list.html:22 mailu/ui/templates/domain/list.html:22
+#: mailu/ui/templates/relay/list.html:20 mailu/ui/templates/token/list.html:20
+#: mailu/ui/templates/user/list.html:24
+msgid "Comment"
+msgstr "Komentář"
+
+#: mailu/ui/forms.py:54 mailu/ui/forms.py:68 mailu/ui/forms.py:75
+#: mailu/ui/forms.py:88 mailu/ui/forms.py:136 mailu/ui/forms.py:145
+msgid "Save"
+msgstr "Uložit"
+
+#: mailu/ui/forms.py:59
+msgid "Initial admin"
+msgstr "Hlavní admin"
+
+#: mailu/ui/forms.py:60
+msgid "Admin password"
+msgstr "Heslo admina"
+
+#: mailu/ui/forms.py:61 mailu/ui/forms.py:81 mailu/ui/forms.py:94
+msgid "Confirm password"
+msgstr "Potvrdit heslo"
+
+#: mailu/ui/forms.py:63
+msgid "Create"
+msgstr "Vytvořit"
+
+#: mailu/ui/forms.py:67
+msgid "Alternative name"
+msgstr "Alternativní jméno"
+
+#: mailu/ui/forms.py:72
+msgid "Relayed domain name"
+msgstr "Seznam předávaných domén"
+
+#: mailu/ui/forms.py:73 mailu/ui/templates/relay/list.html:19
+msgid "Remote host"
+msgstr "Vzdálený hostitel"
+
+#: mailu/ui/forms.py:82 mailu/ui/templates/user/list.html:23
+#: mailu/ui/templates/user/signup_domain.html:16
+msgid "Quota"
+msgstr "Kvóta"
+
+#: mailu/ui/forms.py:83
+msgid "Allow IMAP access"
+msgstr "Povolit přístup IMAP"
+
+#: mailu/ui/forms.py:84
+msgid "Allow POP3 access"
+msgstr "Povolit přístup POP3"
+
+#: mailu/ui/forms.py:85 mailu/ui/forms.py:101
+#: mailu/ui/templates/user/settings.html:15
+msgid "Displayed name"
+msgstr "Zobrazené jméno"
+
+#: mailu/ui/forms.py:87
+msgid "Enabled"
+msgstr "Povoleno"
+
+#: mailu/ui/forms.py:92
+msgid "Email address"
+msgstr "Emailová adresa"
+
+#: mailu/ui/forms.py:102
+msgid "Enable spam filter"
+msgstr "Povolit filtr spamu"
+
+#: mailu/ui/forms.py:103
+msgid "Enable marking spam mails as read"
+msgstr "Povolit označování spamových e-mailů jako přečtených"
+
+#: mailu/ui/forms.py:104
+msgid "Spam filter tolerance"
+msgstr "Tolerance spamového filtru"
+
+#: mailu/ui/forms.py:105
+msgid "Enable forwarding"
+msgstr "Povolit přeposílání"
+
+#: mailu/ui/forms.py:106
+msgid "Keep a copy of the emails"
+msgstr "Zachovat kopii e-mailů"
+
+#: mailu/ui/forms.py:107 mailu/ui/forms.py:143
+#: mailu/ui/templates/alias/list.html:21
+msgid "Destination"
+msgstr "Cíl"
+
+#: mailu/ui/forms.py:108
+msgid "Save settings"
+msgstr "Uložit nastavení"
+
+#: mailu/ui/forms.py:113
+msgid "Password check"
+msgstr "Kontrola hesla"
+
+#: mailu/ui/forms.py:114 mailu/ui/templates/sidebar.html:25
+msgid "Update password"
+msgstr "Aktualizovat heslo"
+
+#: mailu/ui/forms.py:118
+msgid "Enable automatic reply"
+msgstr "Povolit automatickou odpověď"
+
+#: mailu/ui/forms.py:119
+msgid "Reply subject"
+msgstr "Předmět odpovědi"
+
+#: mailu/ui/forms.py:120
+msgid "Reply body"
+msgstr "Tělo odpovědi"
+
+#: mailu/ui/forms.py:122
+msgid "Start of vacation"
+msgstr "Začátek dovolené"
+
+#: mailu/ui/forms.py:123
+msgid "End of vacation"
+msgstr "Konec dovolené"
+
+#: mailu/ui/forms.py:124
+msgid "Update"
+msgstr "Aktualizovat"
+
+#: mailu/ui/forms.py:129
+msgid "Your token (write it down, as it will never be displayed again)"
+msgstr "Váš token (zapište si ho, protože se již nikdy nezobrazí)"
+
+#: mailu/ui/forms.py:134 mailu/ui/templates/token/list.html:21
+msgid "Authorized IP"
+msgstr "Autorizovaná IP"
+
+#: mailu/ui/forms.py:140
+msgid "Alias"
+msgstr "Alias"
+
+#: mailu/ui/forms.py:142
+msgid "Use SQL LIKE Syntax (e.g. for catch-all aliases)"
+msgstr "Použít syntaxi jako SQL (např. pro doménové koše)"
+
+#: mailu/ui/forms.py:149
+msgid "Admin email"
+msgstr "Email admina"
+
+#: mailu/ui/forms.py:150 mailu/ui/forms.py:155 mailu/ui/forms.py:168
+msgid "Submit"
+msgstr "Poslat"
+
+#: mailu/ui/forms.py:154
+msgid "Manager email"
+msgstr "E-mail manažera"
+
+#: mailu/ui/forms.py:159
+msgid "Protocol"
+msgstr "Protokol"
+
+#: mailu/ui/forms.py:162
+msgid "Hostname or IP"
+msgstr "Hostitel nebo IP"
+
+#: mailu/ui/forms.py:163 mailu/ui/templates/client.html:20
+#: mailu/ui/templates/client.html:45
+msgid "TCP port"
+msgstr "TCP port"
+
+#: mailu/ui/forms.py:164
+msgid "Enable TLS"
+msgstr "Povolit TLS"
+
+#: mailu/ui/forms.py:165 mailu/ui/templates/client.html:28
+#: mailu/ui/templates/client.html:53 mailu/ui/templates/fetch/list.html:21
+msgid "Username"
+msgstr "Uživatelské jméno"
+
+#: mailu/ui/forms.py:167
+msgid "Keep emails on the server"
+msgstr "Zachovat e-maily na serveru"
+
+#: mailu/ui/forms.py:172
+msgid "Announcement subject"
+msgstr "Předmět oznámení"
+
+#: mailu/ui/forms.py:174
+msgid "Announcement body"
+msgstr "Tělo oznámení"
+
+#: mailu/ui/forms.py:176
+msgid "Send"
+msgstr "Poslat"
+
+#: mailu/ui/templates/announcement.html:4
+msgid "Public announcement"
+msgstr "Veřejné oznámení"
+
+#: mailu/ui/templates/antispam.html:4 mailu/ui/templates/sidebar.html:80
+#: mailu/ui/templates/user/settings.html:19
+msgid "Antispam"
+msgstr "Antispam"
+
+#: mailu/ui/templates/antispam.html:8
+msgid "RSPAMD status page"
+msgstr "Stavová stránka RSPAMD"
+
+#: mailu/ui/templates/client.html:8
+msgid "configure your email client"
+msgstr "nakonfigurovat e-mailového klienta"
+
+#: mailu/ui/templates/client.html:13
+msgid "Incoming mail"
+msgstr "Příchozí mail"
+
+#: mailu/ui/templates/client.html:16 mailu/ui/templates/client.html:41
+msgid "Mail protocol"
+msgstr "Poštovní protokol"
+
+#: mailu/ui/templates/client.html:24 mailu/ui/templates/client.html:49
+msgid "Server name"
+msgstr "Název serveru"
+
+#: mailu/ui/templates/client.html:38
+msgid "Outgoing mail"
+msgstr "Odchozí pošta"
+
+#: mailu/ui/templates/confirm.html:4
+msgid "Confirm action"
+msgstr "Potvrdit akci"
+
+#: mailu/ui/templates/confirm.html:13
+#, python-format
+msgid "You are about to %(action)s. Please confirm your action."
+msgstr "Chystáte se %(action)s. Potvrďte prosím vaši akci."
+
+#: mailu/ui/templates/docker-error.html:4
+msgid "Docker error"
+msgstr "Chyba Dockeru"
+
+#: mailu/ui/templates/docker-error.html:12
+msgid "An error occurred while talking to the Docker server."
+msgstr "Při komunikaci se serverem Docker došlo k chybě."
+
+#: mailu/ui/templates/macros.html:129
+msgid "copy to clipboard"
+msgstr "zkopírovat do schránky"
+
+#: mailu/ui/templates/sidebar.html:15
+msgid "My account"
+msgstr "Můj účet"
+
+#: mailu/ui/templates/sidebar.html:19 mailu/ui/templates/user/list.html:37
+msgid "Settings"
+msgstr "Nastavení"
+
+#: mailu/ui/templates/sidebar.html:31 mailu/ui/templates/user/list.html:38
+msgid "Auto-reply"
+msgstr "Automatická odpověď"
+
+#: mailu/ui/templates/fetch/list.html:4 mailu/ui/templates/sidebar.html:37
+#: mailu/ui/templates/user/list.html:39
+msgid "Fetched accounts"
+msgstr "Fetched účty"
+
+#: mailu/ui/templates/sidebar.html:43 mailu/ui/templates/token/list.html:4
+msgid "Authentication tokens"
+msgstr "Autentizační tokeny"
+
+#: mailu/ui/templates/sidebar.html:56
+msgid "Administration"
+msgstr "Administrace"
+
+#: mailu/ui/templates/sidebar.html:62
+msgid "Announcement"
+msgstr "Oznámení"
+
+#: mailu/ui/templates/sidebar.html:68
+msgid "Administrators"
+msgstr "Administrátoři"
+
+#: mailu/ui/templates/sidebar.html:74
+msgid "Relayed domains"
+msgstr "Relayované domény"
+
+#: mailu/ui/templates/sidebar.html:88
+msgid "Mail domains"
+msgstr "Poštovní domény"
+
+#: mailu/ui/templates/sidebar.html:99
+msgid "Webmail"
+msgstr "Webmail"
+
+#: mailu/ui/templates/sidebar.html:135
+msgid "Sign out"
+msgstr "Odhlásit se"
+
+#: mailu/ui/templates/working.html:4
+msgid "We are still working on this feature!"
+msgstr "Na této funkci stále pracujeme!"
+
+#: mailu/ui/templates/admin/create.html:4
+msgid "Add a global administrator"
+msgstr "Přidat globálního administrátora"
+
+#: mailu/ui/templates/admin/list.html:4
+msgid "Global administrators"
+msgstr "Globální administrátor"
+
+#: mailu/ui/templates/admin/list.html:9
+msgid "Add administrator"
+msgstr "Přidat administrátora"
+
+#: mailu/ui/templates/admin/list.html:17 mailu/ui/templates/alias/list.html:19
+#: mailu/ui/templates/alternative/list.html:19
+#: mailu/ui/templates/domain/list.html:17 mailu/ui/templates/fetch/list.html:19
+#: mailu/ui/templates/manager/list.html:19
+#: mailu/ui/templates/relay/list.html:17 mailu/ui/templates/token/list.html:19
+#: mailu/ui/templates/user/list.html:19
+msgid "Actions"
+msgstr "Akce"
+
+#: mailu/ui/templates/admin/list.html:18 mailu/ui/templates/alias/list.html:20
+#: mailu/ui/templates/manager/list.html:20 mailu/ui/templates/user/list.html:21
+msgid "Email"
+msgstr "Email"
+
+#: mailu/ui/templates/admin/list.html:25 mailu/ui/templates/alias/list.html:32
+#: mailu/ui/templates/alternative/list.html:29
+#: mailu/ui/templates/domain/list.html:34 mailu/ui/templates/fetch/list.html:34
+#: mailu/ui/templates/manager/list.html:27
+#: mailu/ui/templates/relay/list.html:30 mailu/ui/templates/token/list.html:30
+#: mailu/ui/templates/user/list.html:34
+msgid "Delete"
+msgstr "Vymazat"
+
+#: mailu/ui/templates/alias/create.html:4
+msgid "Create alias"
+msgstr "Vytvořit alias"
+
+#: mailu/ui/templates/alias/edit.html:4
+msgid "Edit alias"
+msgstr "Upravit alias"
+
+#: mailu/ui/templates/alias/list.html:4
+msgid "Alias list"
+msgstr "Seznam aliasů"
+
+#: mailu/ui/templates/alias/list.html:12
+msgid "Add alias"
+msgstr "Přidat alias"
+
+#: mailu/ui/templates/alias/list.html:23
+#: mailu/ui/templates/alternative/list.html:21
+#: mailu/ui/templates/domain/list.html:23 mailu/ui/templates/fetch/list.html:25
+#: mailu/ui/templates/relay/list.html:21 mailu/ui/templates/token/list.html:22
+#: mailu/ui/templates/user/list.html:25
+msgid "Created"
+msgstr "Vytvořeno"
+
+#: mailu/ui/templates/alias/list.html:24
+#: mailu/ui/templates/alternative/list.html:22
+#: mailu/ui/templates/domain/list.html:24 mailu/ui/templates/fetch/list.html:26
+#: mailu/ui/templates/relay/list.html:22 mailu/ui/templates/token/list.html:23
+#: mailu/ui/templates/user/list.html:26
+msgid "Last edit"
+msgstr "Poslední úprava"
+
+#: mailu/ui/templates/alias/list.html:31 mailu/ui/templates/domain/list.html:33
+#: mailu/ui/templates/fetch/list.html:33 mailu/ui/templates/relay/list.html:29
+#: mailu/ui/templates/user/list.html:33
+msgid "Edit"
+msgstr "Upravit"
+
+#: mailu/ui/templates/alternative/create.html:4
+msgid "Create alternative domain"
+msgstr "Vytvořit alternativní doménu"
+
+#: mailu/ui/templates/alternative/list.html:4
+msgid "Alternative domain list"
+msgstr "Seznam alternativních domén"
+
+#: mailu/ui/templates/alternative/list.html:12
+msgid "Add alternative"
+msgstr "Přidat alternativu"
+
+#: mailu/ui/templates/alternative/list.html:20
+msgid "Name"
+msgstr "Jméno"
+
+#: mailu/ui/templates/domain/create.html:4
+#: mailu/ui/templates/domain/list.html:9
+msgid "New domain"
+msgstr "Nová doména"
+
+#: mailu/ui/templates/domain/details.html:4
+msgid "Domain details"
+msgstr "Podrobnosti o doméně"
+
+#: mailu/ui/templates/domain/details.html:15
+msgid "Regenerate keys"
+msgstr "Obnovit klíče"
+
+#: mailu/ui/templates/domain/details.html:17
+msgid "Generate keys"
+msgstr "Generovat klíče"
+
+#: mailu/ui/templates/domain/details.html:30
+msgid "DNS MX entry"
+msgstr "Záznam DNS MX"
+
+#: mailu/ui/templates/domain/details.html:34
+msgid "DNS SPF entries"
+msgstr "Záznamy DNS SPF"
+
+#: mailu/ui/templates/domain/details.html:40
+msgid "DKIM public key"
+msgstr "Veřejný klíč DKIM"
+
+#: mailu/ui/templates/domain/details.html:44
+msgid "DNS DKIM entry"
+msgstr "Záznam DNS DKIM"
+
+#: mailu/ui/templates/domain/details.html:48
+msgid "DNS DMARC entry"
+msgstr "Záznam DNS DMARC"
+
+#: mailu/ui/templates/domain/details.html:58
+msgid "DNS TLSA entry"
+msgstr "Záznam DNS TLSA"
+
+#: mailu/ui/templates/domain/details.html:63
+msgid "DNS client auto-configuration entries"
+msgstr "Položky automatické konfigurace klienta DNS"
+
+#: mailu/ui/templates/domain/edit.html:4
+msgid "Edit domain"
+msgstr "Upravit doménu"
+
+#: mailu/ui/templates/domain/list.html:4
+msgid "Domain list"
+msgstr "Seznam domén"
+
+#: mailu/ui/templates/domain/list.html:18
+msgid "Manage"
+msgstr "Spravovat"
+
+#: mailu/ui/templates/domain/list.html:20
+msgid "Mailbox count"
+msgstr "Počet poštovních schránek"
+
+#: mailu/ui/templates/domain/list.html:21
+msgid "Alias count"
+msgstr "Počet aliasů"
+
+#: mailu/ui/templates/domain/list.html:31
+msgid "Details"
+msgstr "Podrobnosti"
+
+#: mailu/ui/templates/domain/list.html:38
+msgid "Users"
+msgstr "Uživatelů"
+
+#: mailu/ui/templates/domain/list.html:39
+msgid "Aliases"
+msgstr "Aliasů"
+
+#: mailu/ui/templates/domain/list.html:40
+msgid "Managers"
+msgstr "Manažerů"
+
+#: mailu/ui/templates/domain/list.html:42
+msgid "Alternatives"
+msgstr "Alternativ"
+
+#: mailu/ui/templates/domain/signup.html:13
+msgid ""
+"In order to register a new domain, you must first setup the\n"
+"    domain zone so that the domain <code>MX</code> points to this server"
+msgstr ""
+"Chcete-li zaregistrovat novou doménu, musíte nejprve nastavit\n"
+"     zónu domény tak, aby doménový <code>MX</code> záznam ukazovala na tento "
+"server"
+
+#: mailu/ui/templates/domain/signup.html:18
+msgid ""
+"If you do not know how to setup an <code>MX</code> record for your DNS "
+"zone,\n"
+"    please contact your DNS provider or administrator. Also, please wait a\n"
+"    couple minutes after the <code>MX</code> is set so the local server "
+"cache\n"
+"    expires."
+msgstr ""
+"Pokud nevíte, jak nastavit <code>MX</code> záznam pro zónu DNS,\n"
+"     kontaktujte svého poskytovatele DNS nebo správce. Také prosím počkejte "
+"a\n"
+"     několik minut po <code>MX</code> tak, aby vypršela  v mezipaměti "
+"místního\n"
+"     serveru."
+
+#: mailu/ui/templates/fetch/create.html:4
+msgid "Add a fetched account"
+msgstr "Přidejte fetched účet"
+
+#: mailu/ui/templates/fetch/edit.html:4
+msgid "Update a fetched account"
+msgstr "Aktualizujte fetched účet"
+
+#: mailu/ui/templates/fetch/list.html:12
+msgid "Add an account"
+msgstr "Přidat účet"
+
+#: mailu/ui/templates/fetch/list.html:20
+msgid "Endpoint"
+msgstr "Koncový bod"
+
+#: mailu/ui/templates/fetch/list.html:22
+msgid "Keep emails"
+msgstr "Zachovat emaily"
+
+#: mailu/ui/templates/fetch/list.html:23
+msgid "Last check"
+msgstr "Poslední kontrola"
+
+#: mailu/ui/templates/fetch/list.html:24
+msgid "Status"
+msgstr "Status"
+
+#: mailu/ui/templates/fetch/list.html:38
+msgid "yes"
+msgstr "ano"
+
+#: mailu/ui/templates/fetch/list.html:38
+msgid "no"
+msgstr "ne"
+
+#: mailu/ui/templates/manager/create.html:4
+msgid "Add a manager"
+msgstr "Přidat manažera"
+
+#: mailu/ui/templates/manager/list.html:4
+msgid "Manager list"
+msgstr "Seznam manažerů"
+
+#: mailu/ui/templates/manager/list.html:12
+msgid "Add manager"
+msgstr "Přidat manažera"
+
+#: mailu/ui/templates/relay/create.html:4
+msgid "New relay domain"
+msgstr "Nová relay doména"
+
+#: mailu/ui/templates/relay/edit.html:4
+msgid "Edit relayed domain"
+msgstr "Upravit relay doménu"
+
+#: mailu/ui/templates/relay/list.html:4
+msgid "Relayed domain list"
+msgstr "Seznam relay domén"
+
+#: mailu/ui/templates/relay/list.html:9
+msgid "New relayed domain"
+msgstr "Nová relay doména"
+
+#: mailu/ui/templates/token/create.html:4
+msgid "Create an authentication token"
+msgstr "Vytvořit ověřovací token"
+
+#: mailu/ui/templates/token/list.html:12
+msgid "New token"
+msgstr "Nový token"
+
+#: mailu/ui/templates/user/create.html:4
+msgid "New user"
+msgstr "Nový uživatel"
+
+#: mailu/ui/templates/user/create.html:15
+msgid "General"
+msgstr "Všeobecné"
+
+#: mailu/ui/templates/user/create.html:23
+msgid "Features and quotas"
+msgstr "Funkce a kvóty"
+
+#: mailu/ui/templates/user/edit.html:4
+msgid "Edit user"
+msgstr "Upravit uživatele"
+
+#: mailu/ui/templates/user/list.html:4
+msgid "User list"
+msgstr "Seznam uživatelů"
+
+#: mailu/ui/templates/user/list.html:12
+msgid "Add user"
+msgstr "Přidat uživatele"
+
+#: mailu/ui/templates/user/list.html:20 mailu/ui/templates/user/settings.html:4
+msgid "User settings"
+msgstr "Uživatelské nastavení"
+
+#: mailu/ui/templates/user/list.html:22
+msgid "Features"
+msgstr "Funkce"
+
+#: mailu/ui/templates/user/password.html:4
+msgid "Password update"
+msgstr "Aktualizace hesla"
+
+#: mailu/ui/templates/user/reply.html:4
+msgid "Automatic reply"
+msgstr "Automatická odpověď"
+
+#: mailu/ui/templates/user/settings.html:27
+msgid "Auto-forward"
+msgstr "Automatické přeposlání"
+
+#: mailu/ui/templates/user/signup_domain.html:8
+msgid "pick a domain for the new account"
+msgstr "vybrat doménu pro nový účet"
+
+#: mailu/ui/templates/user/signup_domain.html:14
+msgid "Domain"
+msgstr "Doména"
+
+#: mailu/ui/templates/user/signup_domain.html:15
+msgid "Available slots"
+msgstr "Dostupných slotů"

core/admin/mailu/translations/da/LC_MESSAGES/messages.po

@@ -649,7 +649,7 @@ msgid "New relay domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/de/LC_MESSAGES/messages.po

@@ -655,7 +655,7 @@ msgid "New relay domain"
 msgstr "Neue Relay-Domain"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Relay-Domain bearbeiten"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/en/LC_MESSAGES/messages.po

@@ -649,7 +649,7 @@ msgid "New relay domain"
 msgstr "New relay domain"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/es/LC_MESSAGES/messages.po

@@ -651,7 +651,7 @@ msgid "New relay domain"
 msgstr "Nuevo dominio externo (relay)"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Editar dominio externo (relay)"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/eu/LC_MESSAGES/messages.po

@@ -649,7 +649,7 @@ msgid "New relay domain"
 msgstr "Igorritako domeinu berria"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Editatu igorritako domeinua"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/fr/LC_MESSAGES/messages.po

@@ -653,7 +653,7 @@ msgid "New relay domain"
 msgstr "Nouveau domaine relayé"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Modifier le domaine relayé"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/he/LC_MESSAGES/messages.po

@@ -658,7 +658,7 @@ msgid "New relay domain"
 msgstr "שם תחום מועבר"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "עריכת שמות תחום מועברים"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/hu/LC_MESSAGES/messages.po

@@ -652,7 +652,7 @@ msgid "New relay domain"
 msgstr "Új továbbító tartomány"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Továbbított tartomány szerkesztése"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/is/LC_MESSAGES/messages.po

@@ -648,7 +648,7 @@ msgid "New relay domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/it/LC_MESSAGES/messages.po

@@ -655,7 +655,7 @@ msgid "New relay domain"
 msgstr "Nuovo dominio affidato"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Editare dominio affidato"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/ja/LC_MESSAGES/messages.po

@@ -649,7 +649,7 @@ msgid "New relay domain"
 msgstr "リレードメイン名を追加"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "リレードメイン名を編集"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/nb_NO/LC_MESSAGES/messages.po

@@ -657,7 +657,7 @@ msgid "New relay domain"
 msgstr "Nytt videresendt domene"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Endre videresendt domene"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/nl/LC_MESSAGES/messages.po

@@ -650,7 +650,7 @@ msgid "New relay domain"
 msgstr "Nieuw relay domein"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Bewerk relay domein"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/pl/LC_MESSAGES/messages.po

@@ -659,7 +659,7 @@ msgstr "Nowa domena do przekierowania"
 
 #: mailu/ui/templates/relay/edit.html:4
 #, fuzzy
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Edycja domeny"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/pt/LC_MESSAGES/messages.po

@@ -650,7 +650,7 @@ msgid "New relay domain"
 msgstr "Novo domínio para encaminhamento"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Editar domínio de encaminhamento"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/ru/LC_MESSAGES/messages.po

@@ -656,7 +656,7 @@ msgid "New relay domain"
 msgstr "Новый релейный домен"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Изменить релейный домен"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/sv/LC_MESSAGES/messages.po

@@ -645,7 +645,7 @@ msgid "New relay domain"
 msgstr "Ny relä-domän"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "Redigera reläade domäner"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/translations/zh/LC_MESSAGES/messages.po

@@ -646,7 +646,7 @@ msgid "New relay domain"
 msgstr "新的中继域"
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr "编辑中继域"
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/mailu/ui/forms.py

@@ -37,7 +37,17 @@ class MultipleEmailAddressesVerify(object):
         self.message = message
 
     def __call__(self, form, field):
-        pattern = re.compile(r'^([_a-z0-9\-]+)(\.[_a-z0-9\-]+)*@([a-z0-9\-]{1,}\.)*([a-z]{1,})(,([_a-z0-9\-]+)(\.[_a-z0-9\-]+)*@([a-z0-9\-]{1,}\.)*([a-z]{2,}))*$')
+        pattern = re.compile(r'^([_a-z0-9\-\+]+)(\.[_a-z0-9\-\+]+)*@([a-z0-9\-]{1,}\.)*([a-z]{1,})(,([_a-z0-9\-\+]+)(\.[_a-z0-9\-\+]+)*@([a-z0-9\-]{1,}\.)*([a-z]{2,}))*$')
+        if not pattern.match(field.data.replace(" ", "")):
+            raise validators.ValidationError(self.message)
+
+class MultipleFoldersVerify(object):
+    """ Ensure that we have CSV formated data """
+    def __init__(self,message=_('Invalid list of folders.')):
+        self.message = message
+
+    def __call__(self, form, field):
+        pattern = re.compile(r'^[^,]+(,[^,]+)*$')
         if not pattern.match(field.data.replace(" ", "")):
             raise validators.ValidationError(self.message)
 
@@ -59,6 +69,7 @@ class DomainSignupForm(flask_wtf.FlaskForm):
     localpart = fields.StringField(_('Initial admin'), [validators.DataRequired()])
     pw = fields.PasswordField(_('Admin password'), [validators.DataRequired()])
     pw2 = fields.PasswordField(_('Confirm password'), [validators.EqualTo('pw')])
+    pwned = fields.HiddenField(label='', default=-1)
     captcha = flask_wtf.RecaptchaField()
     submit = fields.SubmitField(_('Create'))
 
@@ -79,9 +90,11 @@ class UserForm(flask_wtf.FlaskForm):
     localpart = fields.StringField(_('E-mail'), [validators.DataRequired(), validators.Regexp(LOCALPART_REGEX)])
     pw = fields.PasswordField(_('Password'))
     pw2 = fields.PasswordField(_('Confirm password'), [validators.EqualTo('pw')])
+    pwned = fields.HiddenField(label='', default=-1)
     quota_bytes = fields_.IntegerSliderField(_('Quota'), default=10**9)
     enable_imap = fields.BooleanField(_('Allow IMAP access'), default=True)
     enable_pop = fields.BooleanField(_('Allow POP3 access'), default=True)
+    allow_spoofing = fields.BooleanField(_('Allow the user to spoof the sender (send email as anyone)'), default=False)
     displayed_name = fields.StringField(_('Displayed name'))
     comment = fields.StringField(_('Comment'))
     enabled = fields.BooleanField(_('Enabled'), default=True)
@@ -92,6 +105,7 @@ class UserSignupForm(flask_wtf.FlaskForm):
     localpart = fields.StringField(_('Email address'), [validators.DataRequired(), validators.Regexp(LOCALPART_REGEX)])
     pw = fields.PasswordField(_('Password'), [validators.DataRequired()])
     pw2 = fields.PasswordField(_('Confirm password'), [validators.EqualTo('pw')])
+    pwned = fields.HiddenField(label='', default=-1)
     submit = fields.SubmitField(_('Sign up'))
 
 class UserSignupFormCaptcha(UserSignupForm):
@@ -111,6 +125,7 @@ class UserSettingsForm(flask_wtf.FlaskForm):
 class UserPasswordForm(flask_wtf.FlaskForm):
     pw = fields.PasswordField(_('Password'), [validators.DataRequired()])
     pw2 = fields.PasswordField(_('Password check'), [validators.DataRequired()])
+    pwned = fields.HiddenField(label='', default=-1)
     submit = fields.SubmitField(_('Update password'))
 
 
@@ -119,8 +134,8 @@ class UserReplyForm(flask_wtf.FlaskForm):
     reply_subject = fields.StringField(_('Reply subject'))
     reply_body = fields.StringField(_('Reply body'),
         widget=widgets.TextArea())
-    reply_startdate = fields.html5.DateField(_('Start of vacation'))
-    reply_enddate = fields.html5.DateField(_('End of vacation'))
+    reply_startdate = fields.DateField(_('Start of vacation'))
+    reply_enddate = fields.DateField(_('End of vacation'))
     submit = fields.SubmitField(_('Update'))
 
 
@@ -160,11 +175,13 @@ class FetchForm(flask_wtf.FlaskForm):
         ('imap', 'IMAP'), ('pop3', 'POP3')
     ])
     host = fields.StringField(_('Hostname or IP'), [validators.DataRequired()])
-    port = fields.IntegerField(_('TCP port'), [validators.DataRequired(), validators.NumberRange(min=0, max=65535)])
-    tls = fields.BooleanField(_('Enable TLS'))
+    port = fields.IntegerField(_('TCP port'), [validators.DataRequired(), validators.NumberRange(min=0, max=65535)], default=993)
+    tls = fields.BooleanField(_('Enable TLS'), default=True)
     username = fields.StringField(_('Username'), [validators.DataRequired()])
     password = fields.PasswordField(_('Password'))
     keep = fields.BooleanField(_('Keep emails on the server'))
+    scan = fields.BooleanField(_('Rescan emails locally'))
+    folders = fields.StringField(_('Folders to fetch on the server'), [validators.Optional(), MultipleFoldersVerify()], default='INBOX,Junk')
     submit = fields.SubmitField(_('Submit'))
 
 

core/admin/mailu/ui/templates/admin/list.html

@@ -11,10 +11,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Email{% endtrans %}</th>
   </tr>
 </thead>
@@ -22,6 +22,7 @@
   {%- for admin in admins %}
   <tr>
     <td>
+      <a href="{{ url_for('.user_edit', user_email=admin.email) }}" title="{% trans %}Edit{% endtrans %}"><i class="fas fa-pencil-alt"></i></a>&nbsp;
       <a href="{{ url_for('.admin_delete', admin=admin.email) }}" title="{% trans %}Delete{% endtrans %}"><i class="fa fa-trash"></i></a>
     </td>
     <td>{{ admin }}</td>

core/admin/mailu/ui/templates/alias/list.html

@@ -13,10 +13,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Email{% endtrans %}</th>
     <th>{% trans %}Destination{% endtrans %}</th>
     <th>{% trans %}Comment{% endtrans %}</th>
@@ -34,8 +34,8 @@
     <td>{{ alias }}</td>
     <td>{{ alias.destination|join(', ') or '-' }}</td>
     <td>{{ alias.comment or '' }}</td>
-    <td>{{ alias.created_at | format_date }}</td>
-    <td>{{ alias.updated_at | format_date }}</td>
+    <td data-sort="{{ alias.created_at or '0000-00-00' }}">{{ alias.created_at | format_date }}</td>
+    <td data-sort="{{ alias.updated_at or '0000-00-00' }}">{{ alias.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/alternative/list.html

@@ -13,10 +13,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Name{% endtrans %}</th>
     <th>{% trans %}Created{% endtrans %}</th>
     <th>{% trans %}Last edit{% endtrans %}</th>
@@ -29,8 +29,8 @@
       <a href="{{ url_for('.alternative_delete', alternative=alternative.name) }}" title="{% trans %}Delete{% endtrans %}"><i class="fa fa-trash"></i></a>
     </td>
     <td>{{ alternative }}</td>
-    <td>{{ alternative.created_at | format_date }}</td>
-    <td>{{ alternative.updated_at | format_date }}</td>
+    <td data-sort="{{ alternative.created_at or '0000-00-00' }}">{{ alternative.created_at | format_date }}</td>
+    <td data-sort="{{ alternative.updated_at or '0000-00-00' }}">{{ alternative.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/base.html

@@ -1,7 +1,7 @@
 {%- import "macros.html" as macros %}
 {%- import "bootstrap/utils.html" as utils %}
 <!doctype html>
-<html lang="{{ session['language'] }}" data-static="/static/">
+<html lang="{{ get_locale() }}" data-static="/static/">
   <head>
     <meta charset="utf-8">
     <meta name="viewport" content="width=device-width, initial-scale=1">
@@ -34,8 +34,8 @@
         <ul class="navbar-nav ml-auto">
           <li class="nav-item dropdown">
             <a class="nav-link" data-toggle="dropdown" href="#" aria-expanded="false">
-              <i class="fas fa-language text-xl" aria-hidden="true" title="{% trans %}change language{% endtrans %}"></i><span class="sr-only">Language</span>
-              <span class="badge badge-primary navbar-badge">{{ session['language'] }}</span></a>
+              <i class="fas fa-language text-xl" aria-hidden="true" title="{% trans %}change language{% endtrans %}"></i><span class="sr-only">{% trans %}change language{% endtrans %}</span>
+              <span class="badge badge-primary navbar-badge">{{ get_locale() }}</span></a>
             <div class="dropdown-menu dropdown-menu-right p-0" id="mailu-languages">
             {%- for locale in config.translations.values() %}
               <a class="dropdown-item{% if locale|string() == session['language'] %} active{% endif %}" href="{{ url_for('.set_language', language=locale) }}">{{ locale.get_language_name().title() }}</a>
@@ -78,6 +78,11 @@
           <i class="fa fa-code-branch" aria-hidden="true"></i><span class="sr-only">fork</span>
           on <a href="https://github.com/Mailu/Mailu">Github</a>
         </span>
+        <div class="mailu-version">
+          <span class="fa-pull-right">
+            {{ config["MAILU_VERSION"] }}
+          </span>
+        </div>
       </footer>
     </div>
     <script src="{{ url_for('static', filename='vendor.js') }}"></script>

core/admin/mailu/ui/templates/client.html

@@ -9,7 +9,6 @@
 {%- endblock %}
 
 {%- block content %}
-<div>If you use an Apple device, <a href="/apple.mobileconfig">click here to autoconfigure it.</a></div>
 {%- call macros.table(title=_("Incoming mail"), datatable=False) %}
   <tbody>
     <tr>
@@ -22,7 +21,7 @@
     </tr>
     <tr>
       <th>{% trans %}Server name{% endtrans %}</th>
-      <td><pre class="pre-config border bg-light">{{ config["HOSTNAMES"] }}</pre></td>
+      <td><pre class="pre-config border bg-light">{{ config["HOSTNAME"] }}</pre></td>
     </tr>
     <tr>
       <th>{% trans %}Username{% endtrans %}</th>
@@ -47,7 +46,7 @@
     </tr>
     <tr>
       <th>{% trans %}Server name{% endtrans %}</th>
-      <td><pre class="pre-config border bg-light">{{ config["HOSTNAMES"] }}</pre></td>
+      <td><pre class="pre-config border bg-light">{{ config["HOSTNAME"] }}</pre></td>
     </tr>
     <tr>
       <th>{% trans %}Username{% endtrans %}</th>
@@ -59,4 +58,8 @@
     </tr>
   </tbody>
 {%- endcall %}
+<blockquote>
+  {% trans %}If you use an Apple device,{% endtrans %}
+  <a href="/apple.mobileconfig">{% trans %}click here to autoconfigure it.{% endtrans %}</a>
+</blockquote>
 {%- endblock %}

core/admin/mailu/ui/templates/domain/create.html

@@ -10,7 +10,7 @@
   {{ form.hidden_tag() }}
   {{ macros.form_field(form.name) }}
   {{ macros.form_fields((form.max_users, form.max_aliases)) }}
-  {{ macros.form_field(form.max_quota_bytes, step=10**9, max=50*10**9, data_infinity="true",
+  {{ macros.form_field(form.max_quota_bytes, step=50*10**6, max=50*10**9, data_infinity="true", data_unit=10**9,
       prepend='<span class="input-group-text"><span id="max_quota_bytes_value"></span>&nbsp;GB</span>') }}
   {{ macros.form_field(form.signup_enabled) }}
   {{ macros.form_field(form.comment) }}

core/admin/mailu/ui/templates/domain/list.html

@@ -11,15 +11,17 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[2,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
-    <th>{% trans %}Manage{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Manage{% endtrans %}</th>
     <th>{% trans %}Domain name{% endtrans %}</th>
     <th>{% trans %}Mailbox count{% endtrans %}</th>
     <th>{% trans %}Alias count{% endtrans %}</th>
+    <th>{% trans %}Quota{% endtrans %}</th>
     <th>{% trans %}Comment{% endtrans %}</th>
+    <th>{% trans %}Enable sign-up{% endtrans %}</th>
     <th>{% trans %}Created{% endtrans %}</th>
     <th>{% trans %}Last edit{% endtrans %}</th>
   </tr>
@@ -43,11 +45,13 @@
       {%- endif %}
     </td>
     <td>{{ domain.name }}</td>
-    <td>{{ domain.users | count }} / {{ '∞' if domain.max_users == -1 else domain.max_users }}</td>
-    <td>{{ domain.aliases | count }} / {{ '∞' if domain.max_aliases == -1 else domain.max_aliases }}</td>
+    <td data-order="{{ domain.users | count }}">{{ domain.users | count }} / {{ '∞' if domain.max_users == -1 else domain.max_users }}</td>
+    <td data-order="{{ domain.aliases | count }}">{{ domain.aliases | count }} / {{ '∞' if domain.max_aliases == -1 else domain.max_aliases }}</td>
+    <td data-sort="{{ domain.max_quota_bytes }}">{{ (domain.max_quota_bytes | filesizeformat) if domain.max_quota_bytes else '∞' }}</td>
     <td>{{ domain.comment or '' }}</td>
-    <td>{{ domain.created_at | format_date }}</td>
-    <td>{{ domain.updated_at | format_date }}</td>
+    <td data-sort="{{ domain.signup_enabled }}">{% if domain.signup_enabled %}{% trans %}yes{% endtrans %}{% else %}{% trans %}no{% endtrans %}{% endif %}</td>
+    <td data-order="{{ domain.created_at or '0000-00-00' }}">{{ domain.created_at | format_date }}</td>
+    <td data-order="{{ domain.updated_at or '0000-00-00' }}">{{ domain.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/fetch/create.html

@@ -24,6 +24,8 @@
 
   {%- call macros.card(title="Settings") %}
   {{ macros.form_field(form.keep) }}
+  {{ macros.form_field(form.scan) }}
+  {{ macros.form_field(form.folders) }}
   {%- endcall %}
 
   {{ macros.form_field(form.submit) }}

core/admin/mailu/ui/templates/fetch/list.html

@@ -13,13 +13,15 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Endpoint{% endtrans %}</th>
     <th>{% trans %}Username{% endtrans %}</th>
     <th>{% trans %}Keep emails{% endtrans %}</th>
+    <th>{% trans %}Rescan emails{% endtrans %}</th>
+    <th>{% trans %}Folders{% endtrans %}</th>
     <th>{% trans %}Last check{% endtrans %}</th>
     <th>{% trans %}Status{% endtrans %}</th>
     <th>{% trans %}Created{% endtrans %}</th>
@@ -35,11 +37,13 @@
     </td>
     <td>{{ fetch.protocol }}{{ 's' if fetch.tls else '' }}://{{ fetch.host }}:{{ fetch.port }}</td>
     <td>{{ fetch.username }}</td>
-    <td>{% if fetch.keep %}{% trans %}yes{% endtrans %}{% else %}{% trans %}no{% endtrans %}{% endif %}</td>
+    <td data-sort="{{ fetch.keep }}">{% if fetch.keep %}{% trans %}yes{% endtrans %}{% else %}{% trans %}no{% endtrans %}{% endif %}</td>
+    <td data-sort="{{ fetch.scan }}">{% if fetch.scan %}{% trans %}yes{% endtrans %}{% else %}{% trans %}no{% endtrans %}{% endif %}</td>
+    <td>{{ fetch.folders | join(',') }}</td>
     <td>{{ fetch.last_check | format_datetime or '-' }}</td>
     <td>{{ fetch.error or '-' }}</td>
-    <td>{{ fetch.created_at | format_date }}</td>
-    <td>{{ fetch.updated_at | format_date }}</td>
+    <td data-sort="{{ fetch.created_at or '0000-00-00' }}">{{ fetch.created_at | format_date }}</td>
+    <td data-sort="{{ fetch.updated_at or '0000-00-00' }}">{{ fetch.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/macros.html

@@ -3,7 +3,7 @@
     {%- for fieldname, errors in form.errors.items() %}
       {%- if bootstrap_is_hidden_field(form[fieldname]) %}
         {%- for error in errors %}
-          <p class="error">{{error}}</p>
+          <p class="form-text text-danger">{{error}}</p>
         {%- endfor %}
       {%- endif %}
     {%- endfor %}
@@ -13,7 +13,7 @@
 {%- macro form_field_errors(field) %}
   {%- if field.errors %}
     {%- for error in field.errors %}
-      <p class="help-block inline">{{ error }}</p>
+      <p class="form-text text-danger">{{ error }}</p>
     {%- endfor %}
   {%- endif %}
 {%- endmacro %}
@@ -23,7 +23,7 @@
   <div class="form-group">
     <div class="row">
       {%- for field in fields %}
-      <div class="col-lg-{{ width }} col-xs-12 {{ 'has-error' if field.errors else '' }}">
+      <div class="col-lg-{{ width }} col-xs-12">
         {%- if field.__class__.__name__ == 'list' %}
           {%- for subfield in field %}
             {{ form_individual_field(subfield, prepend=prepend, append=append, label=label, **kwargs) }}
@@ -38,12 +38,13 @@
 {%- endmacro %}
 
 {%- macro form_individual_field(field, prepend='', append='', label=True, class_="") %}
+  {%- set fieldclass=" ".join(["form-control"] + ([class_] if class_ else []) + (["is-invalid"] if field.errors else [])) %}
   {%- if field.type == "BooleanField" %}
     {{ field(**kwargs) }}<span>&nbsp;&nbsp;</span>{{ field.label if label else '' }}
   {%- else %}
     {{ field.label if label else '' }}{{ form_field_errors(field) }}
     {%- if prepend %}<div class="input-group-prepend">{%- elif append %}<div class="input-group-append">{%- endif %}
-      {{ prepend|safe }}{{ field(class_=("form-control " + class_) if class_ else "form-control", **kwargs) }}{{ append|safe }}
+      {{ prepend|safe }}{{ field(class_=fieldclass, **kwargs) }}{{ append|safe }}
     {%- if prepend or append %}</div>{%- endif %}
   {%- endif %}
 {%- endmacro %}
@@ -60,9 +61,7 @@
 <form class="form" method="post" role="form">
   {{ form.hidden_tag() }}
   {%- for field in form %}
-    {%- if bootstrap_is_hidden_field(field) %}
-      {{ field() }}
-    {%- else %}
+    {%- if not bootstrap_is_hidden_field(field) %}
       {{ form_field(field) }}
     {%- endif %}
   {%- endfor %}
@@ -88,7 +87,7 @@
 </div>
 {%- endmacro %}
 
-{%- macro table(title=None, theme="primary", datatable=True) %}
+{%- macro table(title=None, theme="primary", datatable=True, order=None) %}
 <div class="row">
   <div class="col-lg-12">
     <div class="card card-outline card-{{ theme }}">
@@ -98,7 +97,7 @@
       </div>
       {%- endif %}
       <div class="card-body">
-        <table class="table table-bordered{% if datatable %} dataTable{% endif %}">
+        <table class="table table-bordered{% if datatable %} dataTable{% endif %}" data-order="{{ order or '[]' | e }}">
           {{- caller() }}
         </table>
       </div>

core/admin/mailu/ui/templates/manager/list.html

@@ -13,10 +13,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Email{% endtrans %}</th>
   </tr>
 </thead>
@@ -24,6 +24,7 @@
 {%- for manager in domain.managers %}
 <tr>
   <td>
+    <a href="{{ url_for('.user_edit', user_email=manager.email) }}" title="{% trans %}Edit{% endtrans %}"><i class="fas fa-pencil-alt"></i></a>&nbsp;
     <a href="{{ url_for('.manager_delete', domain_name=domain.name, user_email=manager.email) }}" title="{% trans %}Delete{% endtrans %}"><i class="fa fa-trash"></i></a>
   </td>
   <td>{{ manager }}</td>

core/admin/mailu/ui/templates/relay/edit.html

@@ -1,7 +1,7 @@
 {%- extends "form.html" %}
 
 {%- block title %}
-{% trans %}Edit relayd domain{% endtrans %}
+{% trans %}Edit relayed domain{% endtrans %}
 {%- endblock %}
 
 {%- block subtitle %}

core/admin/mailu/ui/templates/relay/list.html

@@ -11,10 +11,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Domain name{% endtrans %}</th>
     <th>{% trans %}Remote host{% endtrans %}</th>
     <th>{% trans %}Comment{% endtrans %}</th>
@@ -32,8 +32,8 @@
     <td>{{ relay.name }}</td>
     <td>{{ relay.smtp or '-' }}</td>
     <td>{{ relay.comment or '' }}</td>
-    <td>{{ relay.created_at | format_date }}</td>
-    <td>{{ relay.updated_at | format_date }}</td>
+    <td data-sort="{{ relay.created_at or '0000-00-00' }}">{{ relay.created_at | format_date }}</td>
+    <td data-sort="{{ relay.updated_at or '0000-00-00' }}">{{ relay.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/sidebar.html

@@ -31,12 +31,14 @@
           <p>{% trans %}Auto-reply{% endtrans %}</p>
         </a>
       </li>
+      {%- if config["FETCHMAIL_ENABLED"] %}
       <li class="nav-item" role="none">
         <a href="{{ url_for('.fetch_list') }}" class="nav-link" role="menuitem">
           <i class="nav-icon fas fa-download"></i>
           <p>{% trans %}Fetched accounts{% endtrans %}</p>
         </a>
       </li>
+      {%- endif %}
       <li class="nav-item" role="none">
         <a href="{{ url_for('.token_list') }}" class="nav-link" role="menuitem">
           <i class="nav-icon fas fa-ticket-alt"></i>
@@ -94,9 +96,9 @@
       <li class="nav-header text-uppercase text-primary" role="none">{% trans %}Go to{% endtrans %}</li>
       {%- if config["WEBMAIL"] != "none" and current_user.is_authenticated %}
       <li class="nav-item" role="none">
-        <a href="{{ config["WEB_WEBMAIL"] }}" target="_blank" class="nav-link" role="menuitem">
+        <a href="{{ config["WEB_WEBMAIL"] }}" class="nav-link" role="menuitem">
         <i class="nav-icon far fa-envelope"></i>
-        <p>{% trans %}Webmail{% endtrans %} <i class="fas fa-external-link-alt text-xs"></i></p>
+        <p>{% trans %}Webmail{% endtrans %}</p>
         </a>
       </li>
       {%- endif %}

core/admin/mailu/ui/templates/token/list.html

@@ -13,10 +13,10 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[1,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
     <th>{% trans %}Comment{% endtrans %}</th>
     <th>{% trans %}Authorized IP{% endtrans %}</th>
     <th>{% trans %}Created{% endtrans %}</th>
@@ -31,8 +31,8 @@
     </td>
     <td>{{ token.comment }}</td>
     <td>{{ token.ip or "any" }}</td>
-    <td>{{ token.created_at | format_date }}</td>
-    <td>{{ token.updated_at | format_date }}</td>
+    <td data-sort="{{ token.created_at or '0000-00-00' }}">{{ token.created_at | format_date }}</td>
+    <td data-sort="{{ token.updated_at or '0000-00-00' }}">{{ token.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/user/create.html

@@ -21,10 +21,11 @@
   {%- endcall %}
 
   {%- call macros.card(_("Features and quotas"), theme="success") %}
-  {{ macros.form_field(form.quota_bytes, step=1000000000, max=(max_quota_bytes or domain.max_quota_bytes or 50*10**9), data_infinity="true",
+  {{ macros.form_field(form.quota_bytes, step=50*10**6, max=(max_quota_bytes or domain.max_quota_bytes or 50*10**9), data_infinity="true", data_unit=10**9,
       prepend='<span class="input-group-text"><span id="quota_bytes_value"></span>&nbsp;GB</span>') }}
   {{ macros.form_field(form.enable_imap) }}
   {{ macros.form_field(form.enable_pop) }}
+  {{ macros.form_field(form.allow_spoofing) }}
   {%- endcall %}
 
   {{ macros.form_field(form.submit) }}

core/admin/mailu/ui/templates/user/list.html

@@ -13,11 +13,11 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
+{%- call macros.table(order='[[2,"asc"]]') %}
 <thead>
   <tr>
-    <th>{% trans %}Actions{% endtrans %}</th>
-    <th>{% trans %}User settings{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}Actions{% endtrans %}</th>
+    <th data-orderable="false">{% trans %}User settings{% endtrans %}</th>
     <th>{% trans %}Email{% endtrans %}</th>
     <th>{% trans %}Features{% endtrans %}</th>
     <th>{% trans %}Quota{% endtrans %}</th>
@@ -31,22 +31,24 @@
   <tr{% if not user.enabled %} class="warning"{% endif %}>
     <td>
       <a href="{{ url_for('.user_edit', user_email=user.email) }}" title="{% trans %}Edit{% endtrans %}"><i class="fas fa-pencil-alt"></i></a>&nbsp;
-      <a href="{{ url_for('.user_delete', user_email=user.email) }}" title="{% trans %}Delete{% endtrans %}"><i class="fa fa-trash"></i></a>
     </td>
     <td>
       <a href="{{ url_for('.user_settings', user_email=user.email) }}" title="{% trans %}Settings{% endtrans %}"><i class="fa fa-wrench"></i></a>&nbsp;
       <a href="{{ url_for('.user_reply', user_email=user.email) }}" title="{% trans %}Auto-reply{% endtrans %}"><i class="fa fa-plane"></i></a>&nbsp;
+      {%- if config["FETCHMAIL_ENABLED"] -%}
       <a href="{{ url_for('.fetch_list', user_email=user.email) }}" title="{% trans %}Fetched accounts{% endtrans %}"><i class="fa fa-download"></i></a>&nbsp;
+      {%- endif -%}
     </td>
     <td>{{ user }}</td>
-    <td>
-      {% if user.enable_imap %}<span class="badge bg-info">imap</span>{% endif %}
-      {% if user.enable_pop %}<span class="badge bg-info">pop3</span>{% endif %}
+    <td data-sort="{{ user.allow_spoofing*4 + user.enable_imap*2 + user.enable_pop }}">
+      {% if user.enable_imap %}<span class="badge bg-primary">imap</span>{% endif %}
+      {% if user.enable_pop %}<span class="badge bg-secondary">pop3</span>{% endif %}
+      {% if user.allow_spoofing %}<span class="badge bg-danger">allow-spoofing</span>{% endif %}
     </td>
-    <td>{{ user.quota_bytes_used | filesizeformat }} / {{ (user.quota_bytes | filesizeformat) if user.quota_bytes else '∞' }}</td>
+    <td data-sort="{{ user.quota_bytes_used }}">{{ user.quota_bytes_used | filesizeformat }} / {{ (user.quota_bytes | filesizeformat) if user.quota_bytes else '∞' }}</td>
     <td>{{ user.comment or '-' }}</td>
-    <td>{{ user.created_at | format_date }}</td>
-    <td>{{ user.updated_at | format_date }}</td>
+    <td data-sort="{{ user.created_at or '0000-00-00' }}">{{ user.created_at | format_date }}</td>
+    <td data-sort="{{ user.updated_at or '0000-00-00' }}">{{ user.updated_at | format_date }}</td>
   </tr>
   {%- endfor %}
 </tbody>

core/admin/mailu/ui/templates/user/signup_domain.html

@@ -9,18 +9,22 @@
 {%- endblock %}
 
 {%- block content %}
-{%- call macros.table() %}
-<tr>
-  <th>{% trans %}Domain{% endtrans %}</th>
-  <th>{% trans %}Available slots{% endtrans %}</th>
-  <th>{% trans %}Quota{% endtrans %}</th>
-</tr>
+{%- call macros.table(order='[[1,"asc"]]') %}
+<thead>
+  <tr>
+    <th>{% trans %}Domain{% endtrans %}</th>
+    <th>{% trans %}Available slots{% endtrans %}</th>
+    <th>{% trans %}Quota{% endtrans %}</th>
+  </tr>
+</thead>
+<tbody>
 {%- for domain_name, domain in available_domains.items() %}
 <tr>
   <td><a href="{{ url_for('.user_signup', domain_name=domain_name) }}">{{ domain_name }}</a></td>
-  <td>{{ '∞' if domain.max_users == -1 else domain.max_users - (domain.users | count)}}</td>
-  <td>{{ domain.max_quota_bytes or config['DEFAULT_QUOTA'] | filesizeformat }}</td>
+  <td data-sort="{{ -1 if domain.max_users == -1 else domain.max_users - (domain.users | count)}}">{{ '∞' if domain.max_users == -1 else domain.max_users - (domain.users | count)}}</td>
+  <td data-sort="{{ domain.max_quota_bytes or config['DEFAULT_QUOTA'] }}">{{ domain.max_quota_bytes or config['DEFAULT_QUOTA'] | filesizeformat }}</td>
 </tr>
 {%- endfor %}
+</tbody>
 {%- endcall %}
 {%- endblock %}

core/admin/mailu/ui/views/base.py

@@ -21,8 +21,9 @@ def announcement():
     form = forms.AnnouncementForm()
     if form.validate_on_submit():
         for user in models.User.query.all():
-            user.sendmail(form.announcement_subject.data,
-                form.announcement_body.data)
+            if not user.sendmail(form.announcement_subject.data,
+                    form.announcement_body.data):
+                flask.flash('Failed to send to %s' % user.email, 'error')
         # Force-empty the form
         form.announcement_subject.data = ''
         form.announcement_body.data = ''

core/admin/mailu/ui/views/domains.py

@@ -1,4 +1,4 @@
-from mailu import models
+from mailu import models, utils
 from mailu.ui import ui, forms, access
 from flask import current_app as app
 
@@ -93,6 +93,9 @@ def domain_signup(domain_name=None):
         del form.pw
         del form.pw2
     if form.validate_on_submit():
+        if msg := utils.isBadOrPwned(form):
+            flask.flash(msg, "error")
+            return flask.render_template('domain/signup.html', form=form)
         conflicting_domain = models.Domain.query.get(form.name.data)
         conflicting_alternative = models.Alternative.query.get(form.name.data)
         conflicting_relay = models.Relay.query.get(form.name.data)

core/admin/mailu/ui/views/fetches.py

@@ -1,5 +1,6 @@
-from mailu import models
+from mailu import models, utils
 from mailu.ui import ui, forms, access
+from flask import current_app as app
 
 import flask
 import flask_login
@@ -10,6 +11,8 @@ import wtforms
 @ui.route('/fetch/list/<path:user_email>', methods=['GET'])
 @access.owner(models.User, 'user_email')
 def fetch_list(user_email):
+    if not app.config['FETCHMAIL_ENABLED']:
+        flask.abort(404)
     user_email = user_email or flask_login.current_user.email
     user = models.User.query.get(user_email) or flask.abort(404)
     return flask.render_template('fetch/list.html', user=user)
@@ -19,13 +22,18 @@ def fetch_list(user_email):
 @ui.route('/fetch/create/<path:user_email>', methods=['GET', 'POST'])
 @access.owner(models.User, 'user_email')
 def fetch_create(user_email):
+    if not app.config['FETCHMAIL_ENABLED']:
+        flask.abort(404)
     user_email = user_email or flask_login.current_user.email
     user = models.User.query.get(user_email) or flask.abort(404)
     form = forms.FetchForm()
     form.password.validators = [wtforms.validators.DataRequired()]
+    utils.formatCSVField(form.folders)
     if form.validate_on_submit():
         fetch = models.Fetch(user=user)
         form.populate_obj(fetch)
+        if form.folders.data:
+            fetch.folders = form.folders.data.replace(' ','').split(',')
         models.db.session.add(fetch)
         models.db.session.commit()
         flask.flash('Fetch configuration created')
@@ -37,12 +45,17 @@ def fetch_create(user_email):
 @ui.route('/fetch/edit/<fetch_id>', methods=['GET', 'POST'])
 @access.owner(models.Fetch, 'fetch_id')
 def fetch_edit(fetch_id):
+    if not app.config['FETCHMAIL_ENABLED']:
+        flask.abort(404)
     fetch = models.Fetch.query.get(fetch_id) or flask.abort(404)
     form = forms.FetchForm(obj=fetch)
+    utils.formatCSVField(form.folders)
     if form.validate_on_submit():
         if not form.password.data:
             form.password.data = fetch.password
         form.populate_obj(fetch)
+        if form.folders.data:
+            fetch.folders = form.folders.data.replace(' ','').split(',')
         models.db.session.commit()
         flask.flash('Fetch configuration updated')
         return flask.redirect(
@@ -55,6 +68,8 @@ def fetch_edit(fetch_id):
 @access.confirmation_required("delete a fetched account")
 @access.owner(models.Fetch, 'fetch_id')
 def fetch_delete(fetch_id):
+    if not app.config['FETCHMAIL_ENABLED']:
+        flask.abort(404)
     fetch = models.Fetch.query.get(fetch_id) or flask.abort(404)
     user = fetch.user
     models.db.session.delete(fetch)

core/admin/mailu/ui/views/users.py

@@ -1,4 +1,4 @@
-from mailu import models
+from mailu import models, utils
 from mailu.ui import ui, access, forms
 from flask import current_app as app
 
@@ -28,6 +28,10 @@ def user_create(domain_name):
         form.quota_bytes.validators = [
             wtforms.validators.NumberRange(max=domain.max_quota_bytes)]
     if form.validate_on_submit():
+        if msg := utils.isBadOrPwned(form):
+            flask.flash(msg, "error")
+            return flask.render_template('user/create.html',
+                domain=domain, form=form)
         if domain.has_email(form.localpart.data):
             flask.flash('Email is already used', 'error')
         else:
@@ -60,6 +64,11 @@ def user_edit(user_email):
         form.quota_bytes.validators = [
             wtforms.validators.NumberRange(max=max_quota_bytes)]
     if form.validate_on_submit():
+        if form.pw.data:
+            if msg := utils.isBadOrPwned(form):
+                flask.flash(msg, "error")
+                return flask.render_template('user/edit.html', form=form, user=user,
+                    domain=user.domain, max_quota_bytes=max_quota_bytes)
         form.populate_obj(user)
         if form.pw.data:
             user.set_password(form.pw.data)
@@ -71,19 +80,6 @@ def user_edit(user_email):
         domain=user.domain, max_quota_bytes=max_quota_bytes)
 
 
-@ui.route('/user/delete/<path:user_email>', methods=['GET', 'POST'])
-@access.domain_admin(models.User, 'user_email')
-@access.confirmation_required("delete {user_email}")
-def user_delete(user_email):
-    user = models.User.query.get(user_email) or flask.abort(404)
-    domain = user.domain
-    models.db.session.delete(user)
-    models.db.session.commit()
-    flask.flash('User %s deleted' % user)
-    return flask.redirect(
-        flask.url_for('.user_list', domain_name=domain.name))
-
-
 @ui.route('/user/settings', methods=['GET', 'POST'], defaults={'user_email': None})
 @ui.route('/user/usersettings/<path:user_email>', methods=['GET', 'POST'])
 @access.owner(models.User, 'user_email')
@@ -91,11 +87,7 @@ def user_settings(user_email):
     user_email_or_current = user_email or flask_login.current_user.email
     user = models.User.query.get(user_email_or_current) or flask.abort(404)
     form = forms.UserSettingsForm(obj=user)
-    if isinstance(form.forward_destination.data,str):
-        data = form.forward_destination.data.replace(" ","").split(",")
-    else:
-        data = form.forward_destination.data
-    form.forward_destination.data = ", ".join(data)
+    utils.formatCSVField(form.forward_destination)
     if form.validate_on_submit():
         form.forward_destination.data = form.forward_destination.data.replace(" ","").split(",")
         form.populate_obj(user)
@@ -119,6 +111,9 @@ def user_password(user_email):
         if form.pw.data != form.pw2.data:
             flask.flash('Passwords do not match', 'error')
         else:
+            if msg := utils.isBadOrPwned(form):
+                flask.flash(msg, "error")
+                return flask.render_template('user/password.html', form=form, user=user)
             flask.session.regenerate()
             user.set_password(form.pw.data)
             models.db.session.commit()
@@ -170,6 +165,9 @@ def user_signup(domain_name=None):
         if domain.has_email(form.localpart.data) or models.Alias.resolve(form.localpart.data, domain_name):
             flask.flash('Email is already used', 'error')
         else:
+            if msg := utils.isBadOrPwned(form):
+                flask.flash(msg, "error")
+                return flask.render_template('user/signup.html', domain=domain, form=form)
             flask.session.regenerate()
             user = models.User(domain=domain)
             form.populate_obj(user)

core/admin/mailu/utils.py

@@ -42,7 +42,7 @@ login.login_view = "sso.login"
 def handle_needs_login():
     """ redirect unauthorized requests to login page """
     return flask.redirect(
-        flask.url_for('sso.login')
+        flask.url_for('sso.login', url=flask.request.url)
     )
 
 # DNS stub configured to do DNSSEC enabled queries
@@ -472,7 +472,7 @@ class MailuSessionExtension:
                 redis.StrictRedis().from_url(app.config['SESSION_STORAGE_URL'])
             )
 
-            # clean expired sessions oonce on first use in case lifetime was changed
+            # clean expired sessions once on first use in case lifetime was changed
             def cleaner():
                 with cleaned.get_lock():
                     if not cleaned.value:
@@ -507,3 +507,21 @@ def gen_temp_token(email, session):
             app.config['PERMANENT_SESSION_LIFETIME'],
     )
     return token
+
+def isBadOrPwned(form):
+    try:
+        if len(form.pw.data) < 8:
+            return "This password is too short."
+        breaches = int(form.pwned.data)
+    except ValueError:
+        breaches = -1
+    if breaches > 0:
+        return f"This password appears in {breaches} data breaches! It is not unique; please change it."
+    return None
+
+def formatCSVField(field):
+    if isinstance(field.data,str):
+        data = field.data.replace(" ","").split(",")
+    else:
+        data = field.data
+    field.data = ", ".join(data)

core/admin/messages.pot

@@ -647,7 +647,7 @@ msgid "New relay domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/edit.html:4
-msgid "Edit relayd domain"
+msgid "Edit relayed domain"
 msgstr ""
 
 #: mailu/ui/templates/relay/list.html:4

core/admin/migrations/versions/7ac252f2bbbf_.py

@@ -0,0 +1,22 @@
+""" Add user.allow_spoofing
+
+Revision ID: 7ac252f2bbbf
+Revises: 8f9ea78776f4
+Create Date: 2022-11-20 08:57:16.879152
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = '7ac252f2bbbf'
+down_revision = 'f4f0f89e0047'
+
+from alembic import op
+import sqlalchemy as sa
+
+
+def upgrade():
+    op.add_column('user', sa.Column('allow_spoofing', sa.Boolean(), nullable=False, server_default=sa.sql.expression.false()))
+
+
+def downgrade():
+    op.drop_column('user', 'allow_spoofing')

core/admin/migrations/versions/f4f0f89e0047_.py

@@ -0,0 +1,25 @@
+""" Add fetch.scan and fetch.folders
+
+Revision ID: f4f0f89e0047
+Revises: 8f9ea78776f4
+Create Date: 2022-11-13 16:29:01.246509
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = 'f4f0f89e0047'
+down_revision = '8f9ea78776f4'
+
+from alembic import op
+import sqlalchemy as sa
+import mailu
+
+def upgrade():
+    with op.batch_alter_table('fetch') as batch:
+        batch.add_column(sa.Column('scan', sa.Boolean(), nullable=False, server_default=sa.sql.expression.false()))
+        batch.add_column(sa.Column('folders', mailu.models.CommaSeparatedList(), nullable=True))
+
+def downgrade():
+    with op.batch_alter_table('fetch') as batch:
+        batch.drop_column('folders')
+        batch.drop_column('scan')

core/admin/requirements-prod.txt

@@ -1,78 +0,0 @@
-alembic==1.7.4
-appdirs==1.4.4
-Babel==2.9.1
-bcrypt==3.2.0
-blinker==1.4
-CacheControl==0.12.9
-certifi==2021.10.8
-# cffi==1.15.0
-chardet==4.0.0
-click==8.0.3
-colorama==0.4.4
-contextlib2==21.6.0
-cryptography==35.0.0
-decorator==5.1.0
-# distlib==0.3.1
-# distro==1.5.0
-dnspython==2.1.0
-dominate==2.6.0
-email-validator==1.1.3
-Flask==2.0.2
-Flask-Babel==2.0.0
-Flask-Bootstrap==3.3.7.1
-Flask-DebugToolbar==0.11.0
-Flask-Limiter==1.4
-Flask-Login==0.5.0
-flask-marshmallow==0.14.0
-Flask-Migrate==3.1.0
-Flask-Script==2.0.6
-Flask-SQLAlchemy==2.5.1
-Flask-WTF==0.15.1
-greenlet==1.1.2
-gunicorn==20.1.0
-html5lib==1.1
-idna==3.3
-infinity==1.5
-intervals==0.9.2
-itsdangerous==2.0.1
-Jinja2==3.0.2
-limits==1.5.1
-lockfile==0.12.2
-Mako==1.1.5
-MarkupSafe==2.0.1
-marshmallow==3.14.0
-marshmallow-sqlalchemy==0.26.1
-msgpack==1.0.2
-# mysqlclient==2.0.3
-mysql-connector-python==8.0.25
-ordered-set==4.0.2
-# packaging==20.9
-passlib==1.7.4
-# pep517==0.10.0
-progress==1.6
-#psycopg2==2.9.1
-psycopg2-binary==2.9.3
-pycparser==2.20
-Pygments==2.10.0
-pyOpenSSL==21.0.0
-pyparsing==3.0.4
-pytz==2021.3
-PyYAML==6.0
-redis==3.5.3
-requests==2.26.0
-retrying==1.3.3
-# six==1.15.0
-socrate==0.2.0
-SQLAlchemy==1.4.26
-srslib==0.1.4
-tabulate==0.8.9
-tenacity==8.0.1
-toml==0.10.2
-urllib3==1.26.7
-validators==0.18.2
-visitor==0.1.3
-webencodings==0.5.1
-Werkzeug==2.0.2
-WTForms==2.3.3
-WTForms-Components==0.10.5
-xmltodict==0.12.0

core/admin/requirements.txt

@@ -1,28 +0,0 @@
-Flask
-Flask-Login
-Flask-SQLAlchemy
-Flask-bootstrap
-Flask-Babel
-Flask-migrate
-Flask-script
-Flask-wtf
-Flask-debugtoolbar
-limits
-redis
-WTForms-Components
-socrate
-passlib
-gunicorn
-tabulate
-PyYAML
-PyOpenSSL
-Pygments
-dnspython
-tenacity
-mysql-connector-python
-idna
-srslib
-marshmallow
-flask-marshmallow
-marshmallow-sqlalchemy
-xmltodict

core/admin/run_dev.sh

@@ -0,0 +1,159 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+### CONFIG
+
+DEV_NAME="${DEV_NAME:-mailu-dev}"
+DEV_DB="${DEV_DB:-}"
+DEV_PROFILER="${DEV_PROFILER:-false}"
+DEV_LISTEN="${DEV_LISTEN:-127.0.0.1:8080}"
+[[ "${DEV_LISTEN}" == *:* ]] || DEV_LISTEN="127.0.0.1:${DEV_LISTEN}"
+DEV_ADMIN="${DEV_ADMIN:-admin@example.com}"
+DEV_PASSWORD="${DEV_PASSWORD:-letmein}"
+DEV_ARGS=( "$@" )
+
+### MAIN
+
+[[ -n "${DEV_DB}" ]] && {
+    [[ -f "${DEV_DB}" ]] || {
+        echo "Sorry, can't find DEV_DB: '${DEV_DB}'"
+        exit 1
+    }
+    DEV_DB="$(realpath "${DEV_DB}")"
+}
+
+docker="$(command -v podman || command -v docker || echo false)"
+[[ "${docker}" == "false" ]] && {
+    echo "Sorry, you'll need podman or docker to run this."
+    exit 1
+}
+
+tmp="$(mktemp -d)"
+[[ -n "${tmp}" && -d "${tmp}" ]] || {
+    echo "Sorry, can't create temporary folder."
+    exit 1
+}
+trap "rm -rf '${tmp}'" INT TERM EXIT
+
+admin="$(realpath "$(pwd)/${0%/*}")"
+base="${admin}/../base"
+assets="${admin}/assets"
+
+cd "${tmp}"
+
+# base
+cp "${base}"/requirements-* .
+cp -r "${base}"/libs .
+sed -E '/^#/d;s:^FROM system$:FROM system AS base:' "${base}/Dockerfile" >Dockerfile
+
+# assets
+cp "${assets}/package.json" .
+cp -r "${assets}/assets" ./assets
+awk '/new compress/{f=1}!f{print}/}),/{f=0}' <"${assets}/webpack.config.js" >webpack.config.js
+sed -E '/^#/d;s:^(FROM [^ ]+$):\1 AS assets:' "${assets}/Dockerfile" >>Dockerfile
+
+# admin
+sed -E '/^#/d;/^(COPY|EXPOSE|HEALTHCHECK|VOLUME|CMD) /d; s:^(.* )[^ ]*pybabel[^\\]*(.*):\1true \2:' "${admin}/Dockerfile" >>Dockerfile
+
+# development
+cat >>Dockerfile <<EOF
+COPY --from=assets /work/static/ ./static/
+
+RUN set -euxo pipefail \
+  ; mkdir /data \
+  ; ln -s /app/audit.py / \
+  ; ln -s /app/start.py /
+
+ENV \
+    FLASK_DEBUG="true" \
+    MEMORY_SESSIONS="true" \
+    RATELIMIT_STORAGE_URL="memory://" \
+    SESSION_COOKIE_SECURE="false" \
+    \
+    DEBUG="true" \
+    DEBUG_PROFILER="${DEV_PROFILER}" \
+    DEBUG_ASSETS="/app/static" \
+    DEBUG_TB_INTERCEPT_REDIRECTS=False \
+    \
+    ADMIN_ADDRESS="127.0.0.1" \
+    FRONT_ADDRESS="127.0.0.1" \
+    SMTP_ADDRESS="127.0.0.1" \
+    IMAP_ADDRESS="127.0.0.1" \
+    REDIS_ADDRESS="127.0.0.1" \
+    ANTIVIRUS_ADDRESS="127.0.0.1" \
+    ANTISPAM_ADDRESS="127.0.0.1" \
+    WEBMAIL_ADDRESS="127.0.0.1" \
+    WEBDAV_ADDRESS="127.0.0.1"
+
+CMD ["/bin/bash", "-c", "flask db upgrade &>/dev/null && flask mailu admin '${DEV_ADMIN/@*}' '${DEV_ADMIN#*@}' '${DEV_PASSWORD}' --mode ifmissing >/dev/null; flask --debug run --host=0.0.0.0 --port=8080"]
+EOF
+
+# build
+chmod -R u+rwX,go+rX .
+echo Running: "${docker/*\/}" build --tag "${DEV_NAME}:latest" "${DEV_ARGS[@]}" .
+"${docker}" build --tag "${DEV_NAME}:latest" "${DEV_ARGS[@]}" .
+
+# gather volumes to map into container
+volumes=()
+
+[[ -n "${DEV_DB}" ]] && volumes+=( --volume "${DEV_DB}:/data/main.db" )
+
+for vol in audit.py start.py mailu/ migrations/; do
+    volumes+=( --volume "${admin}/${vol}:/app/${vol}" )
+done
+
+for file in "${assets}/assets"/*; do
+    [[ ! -f "${file}" || "${file}" == */vendor.js ]] && continue
+    volumes+=( --volume "${file}:/app/static/${file/*\//}" )
+done
+
+# show configuration
+cat <<EOF
+
+=============================================================================
+
+The "${DEV_NAME}" container was built using this configuration:
+
+DEV_NAME="${DEV_NAME}"
+DEV_DB="${DEV_DB}"
+DEV_PROFILER="${DEV_PROFILER}"
+DEV_LISTEN="${DEV_LISTEN}"
+DEV_ADMIN="${DEV_ADMIN}"
+DEV_PASSWORD="${DEV_PASSWORD}"
+DEV_ARGS=( ${DEV_ARGS[*]} )
+
+=============================================================================
+
+You can start the container later using this command:
+
+${docker/*\/} run --rm -it --name "${DEV_NAME}" --publish ${DEV_LISTEN}:8080$(printf " %q" "${volumes[@]}") "${DEV_NAME}"
+
+=============================================================================
+
+Enter the running container using this command:
+${docker/*\/} exec -it "${DEV_NAME}" /bin/bash
+
+=============================================================================
+
+To update requirements-prod.txt you can build (and test) using:
+${docker/*\/} build --tag "${DEV_NAME}:latest" --build-arg MAILU_DEPS=dev .
+
+And then fetch the new dependencies with:
+${docker/*\/} exec "${DEV_NAME}" pip freeze >$(realpath "${base}")/requirements-new.txt
+
+=============================================================================
+
+The Mailu UI can be found here: http://${DEV_LISTEN}/sso/login
+EOF
+[[ -z "${DEV_DB}" ]] && echo "You can log in with user ${DEV_ADMIN} and password ${DEV_PASSWORD}"
+cat <<EOF
+
+=============================================================================
+
+Starting mailu dev environment...
+EOF
+
+# run
+"${docker}" run --rm -it --name "${DEV_NAME}" --publish "${DEV_LISTEN}:8080" "${volumes[@]}" "${DEV_NAME}"
+

core/admin/start.py

@@ -1,10 +1,16 @@
-#!/usr/bin/python3
+#!/usr/bin/env python3
 
 import os
 import logging as log
 import sys
+from socrate import system
+
+os.system("chown mailu:mailu -R /dkim")
+os.system("find /data | grep -v /fetchmail | xargs -n1 chown mailu:mailu")
+system.drop_privs_to('mailu')
 
 log.basicConfig(stream=sys.stderr, level=os.environ.get("LOG_LEVEL", "INFO"))
+system.set_env(['SECRET'])
 
 os.system("flask mailu advertise")
 os.system("flask db upgrade")
@@ -15,7 +21,7 @@ password = os.environ.get("INITIAL_ADMIN_PW")
 
 if account is not None and domain is not None and password is not None:
     mode = os.environ.get("INITIAL_ADMIN_MODE", default="ifmissing")
-    log.info("Creating initial admin accout %s@%s with mode %s",account,domain,mode)
+    log.info("Creating initial admin account %s@%s with mode %s", account, domain, mode)
     os.system("flask mailu admin %s %s '%s' --mode %s" % (account, domain, password, mode))
 
 def test_DNS():
@@ -37,7 +43,7 @@ def test_DNS():
             try:
                 result = resolver.resolve('example.org', dns.rdatatype.A, dns.rdataclass.IN, lifetime=10)
             except Exception as e:
-                log.critical("Your DNS resolver at %s is not working (%s). Please see https://mailu.io/master/faq.html#the-admin-container-won-t-start-and-its-log-says-critical-your-dns-resolver-isn-t-doing-dnssec-validation", ns, e);
+                log.critical("Your DNS resolver at %s is not working (%s). Please see https://mailu.io/master/faq.html#the-admin-container-won-t-start-and-its-log-says-critical-your-dns-resolver-isn-t-doing-dnssec-validation", ns, e)
             else:
                 if result.response.flags & dns.flags.AD:
                     break
@@ -46,12 +52,21 @@ def test_DNS():
 
 test_DNS()
 
-start_command="".join([
-    "gunicorn --threads ", str(os.cpu_count()),
-    " -b :80 ",
-    "--access-logfile - " if (log.root.level<=log.INFO) else "",
-    "--error-logfile - ",
-    "--preload ",
-    "'mailu:create_app()'"])
+cmdline = [
+	"gunicorn",
+	"--threads", f"{os.cpu_count()}",
+	# If SUBNET6 is defined, gunicorn must listen on IPv6 as well as IPv4
+	"-b", f"{'[::]' if os.environ.get('SUBNET6') else ''}:80",
+    "--logger-class mailu.Logger",
+    "--worker-tmp-dir /dev/shm",
+	"--error-logfile", "-",
+	"--preload"
+]
 
-os.system(start_command)
+# logging
+if log.root.level <= log.INFO:
+	cmdline.extend(["--access-logfile", "-"])
+
+cmdline.append("'mailu:create_app()'")
+
+os.system(" ".join(cmdline))

core/base/Dockerfile

@@ -0,0 +1,91 @@
+# syntax=docker/dockerfile-upstream:1.4.3
+
+# base system image (intermediate)
+ARG DISTRO=alpine:3.17.2
+FROM $DISTRO as system
+
+ENV TZ=Etc/UTC LANG=C.UTF-8
+
+ARG MAILU_UID=1000
+ARG MAILU_GID=1000
+
+RUN set -euxo pipefail \
+  ; addgroup -Sg ${MAILU_GID} mailu \
+  ; adduser -Sg ${MAILU_UID} -G mailu -h /app -g "mailu app" -s /bin/bash mailu \
+  ; apk add --no-cache bash ca-certificates curl python3 tzdata libcap \
+  ; ! [[ "$(uname -m)" == x86_64 ]] \
+    || apk add --no-cache --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing hardened-malloc==11-r0
+
+WORKDIR /app
+
+CMD /bin/bash
+
+
+# build virtual env (intermediate)
+FROM system as build
+
+ARG MAILU_DEPS=prod
+ARG SNUFFLEUPAGUS_VERSION=0.9.0
+
+ENV VIRTUAL_ENV=/app/venv
+
+COPY requirements-build.txt ./
+
+RUN set -euxo pipefail \
+  ; apk add --no-cache py3-pip \
+  ; python3 -m venv ${VIRTUAL_ENV} \
+  ; ${VIRTUAL_ENV}/bin/pip install --no-cache-dir -r requirements-build.txt \
+  ; apk del -r py3-pip \
+  ; rm -f /tmp/*.pem
+
+COPY requirements-${MAILU_DEPS}.txt ./
+COPY libs/ libs/
+
+ENV \
+  PATH="${VIRTUAL_ENV}/bin:${PATH}" \
+  CXXFLAGS="-g -O2 -fdebug-prefix-map=/app=. -fstack-protector-strong -Wformat -Werror=format-security -fstack-clash-protection -fexceptions" \
+  CFLAGS="-g -O2 -fdebug-prefix-map=/app=. -fstack-protector-strong -Wformat -Werror=format-security -fstack-clash-protection -fexceptions" \
+  CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" \
+  LDFLAGS="-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now" \
+  SNUFFLEUPAGUS_URL="https://github.com/jvoisin/snuffleupagus/archive/refs/tags/v${SNUFFLEUPAGUS_VERSION}.tar.gz"
+
+RUN set -euxo pipefail \
+  ;  machine="$(uname -m)" \
+  ; deps="build-base gcc libffi-dev python3-dev" \
+  ; [[ "${machine}" != x86_64 ]] && \
+      deps="${deps} cargo git libretls-dev mariadb-connector-c-dev postgresql-dev" \
+  ; apk add --virtual .build-deps ${deps} \
+  ; [[ "${machine}" == armv7* ]] && \
+      mkdir -p /root/.cargo/registry/index && \
+      git clone --bare https://github.com/rust-lang/crates.io-index.git /root/.cargo/registry/index/github.com-1285ae84e5963aae \
+  ; pip install -r requirements-${MAILU_DEPS}.txt \
+  ; curl -sL ${SNUFFLEUPAGUS_URL} | tar xz \
+  ; cd snuffleupagus-${SNUFFLEUPAGUS_VERSION} \
+  ; rm -rf src/tests/*php7*/ src/tests/*session*/ src/tests/broken_configuration/ src/tests/*cookie* src/tests/upload_validation/ \
+  ; apk add --virtual .build-deps php81-dev php81-cgi php81-simplexml php81-xml pcre-dev build-base php81-pear php81-openssl re2c \
+  ; pecl install vld-beta \
+  ; make -j $(grep -c processor /proc/cpuinfo) release \
+  ; cp src/.libs/snuffleupagus.so /app \
+  ; rm -rf /root/.cargo /tmp/*.pem /root/.cache
+
+# base mailu image
+FROM system
+
+COPY --from=build /app/venv/ /app/venv/
+COPY --chown=root:root --from=build /app/snuffleupagus.so /usr/lib/php81/modules/
+RUN setcap 'cap_net_bind_service=+ep' /app/venv/bin/gunicorn 'cap_net_bind_service=+ep' /usr/bin/python3.10
+
+ENV \
+  VIRTUAL_ENV=/app/venv \
+  PATH="/app/venv/bin:${PATH}" \
+  LD_PRELOAD="/usr/lib/libhardened_malloc.so" \
+  ADMIN_ADDRESS="admin" \
+  FRONT_ADDRESS="front" \
+  SMTP_ADDRESS="smtp" \
+  IMAP_ADDRESS="imap" \
+  OLETOOLS_ADDRESS="oletools" \
+  REDIS_ADDRESS="redis" \
+  ANTIVIRUS_ADDRESS="antivirus" \
+  ANTISPAM_ADDRESS="antispam" \
+  WEBMAIL_ADDRESS="webmail" \
+  WEBDAV_ADDRESS="webdav"

core/base/libs/podop/.gitignore

@@ -0,0 +1,20 @@
+.DS_Store
+.idea
+tmp
+
+*.bak
+*~
+.*.swp
+
+__pycache__/
+*.pyc
+*.pyo
+*.egg-info/
+
+.build
+.env*
+.venv
+
+*.code-workspace
+
+build/

core/base/libs/podop/CONTRIBUTING.md

@@ -0,0 +1,7 @@
+This project is open source, and your contributions are all welcome. There are mostly three different ways one can contribute to the project:
+
+1. use Podop, either on test or on production servers, and report meaningful bugs when you find some;
+2. write and publish, or contribute to mail distributions based on Podop, like Mailu;
+2. contribute code and/or configuration to the repository (see [the development guidelines](https://mailu.io/contributors/guide.html) for details);
+
+Either way, keep in mind that the code you write must be licensed under the same conditions as the project itself. Additionally, all contributors are considered equal co-authors of the project.