From e7caff9811512dc47bddcd9aa11ce983254e7c40 Mon Sep 17 00:00:00 2001 From: David Fairbrother Date: Mon, 5 Oct 2020 15:13:07 +0100 Subject: [PATCH 01/44] Add ability to set no WEBROOT_REDIRECT to Nginx Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /` configuration is written to nginx.conf. This is useful for setting up Mailu and Mailman where we override the root to proxy to the mailing list server instead. Without this change the nginx container will not start, or for 1.7 users can set their WEBMAIL_PATH to / with no webmail to get the same results. This fix means that future users don't have to choose between webmail and a root override and makes the configuration intention clear. --- core/nginx/conf/nginx.conf | 2 +- docs/configuration.rst | 15 ++++++++++----- docs/faq.rst | 2 ++ 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/core/nginx/conf/nginx.conf b/core/nginx/conf/nginx.conf index 8f6eaa0d..f672c7a3 100644 --- a/core/nginx/conf/nginx.conf +++ b/core/nginx/conf/nginx.conf @@ -115,7 +115,7 @@ http { include /overrides/*.conf; # Actual logic - {% if WEB_WEBMAIL != '/' %} + {% if WEB_WEBMAIL != '/' and WEBROOT_REDIRECT != 'none' %} location / { {% if WEBROOT_REDIRECT %} try_files $uri {{ WEBROOT_REDIRECT }}; diff --git a/docs/configuration.rst b/docs/configuration.rst index 5ff3546a..c2c55190 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -92,14 +92,19 @@ the localpart for DMARC rua and ruf email addresses. Full-text search is enabled for IMAP is enabled by default. This feature can be disabled (e.g. for performance reasons) by setting the optional variable ``FULL_TEXT_SEARCH`` to ``off``. +.. _web_settings: + Web settings ------------ -The ``WEB_ADMIN`` contains the path to the main admin interface, while -``WEB_WEBMAIL`` contains the path to the Web email client. -The ``WEBROOT_REDIRECT`` redirects all non-found queries to the set path. -An empty ``WEBROOT_REDIRECT`` value disables redirecting and enables classic -behavior of a 404 result when not found. +- ``WEB_ADMIN`` contains the path to the main admin interface + +- ``WEB_WEBMAIL`` contains the path to the Web email client. + +- ``WEBROOT_REDIRECT`` redirects all non-found queries to the set path. + An empty ``WEBROOT_REDIRECT`` value disables redirecting and enables classic behavior of a 404 result when not found. + Alternatively, ``WEBROOT_REDIRECT`` can be set to ``none`` if you are using an Nginx override for ``location /``. + All three options need a leading slash (``/``) to work. .. note:: ``WEBROOT_REDIRECT`` has to point to a valid path on the webserver. diff --git a/docs/faq.rst b/docs/faq.rst index b292cd05..296cd59f 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -262,6 +262,8 @@ correct syntax. The following file names will be taken as override configuration - `Nginx`_ - All ``*.conf`` files in the ``nginx`` sub-directory; - `Rspamd`_ - All files in the ``rspamd`` sub-directory. +To override the root location (``/``) in Nginx ``WEBROOT_REDIRECT`` needs to be set to ``none`` in the env file (see :ref:`web settings `). + *Issue reference:* `206`_, `1368`_. I want to integrate Nextcloud 15 (and newer) with Mailu From a7d99bdedd3decc7dc02609fc070337451302eb2 Mon Sep 17 00:00:00 2001 From: Diman0 Date: Fri, 6 Aug 2021 22:35:37 +0200 Subject: [PATCH 02/44] Update CHANGELOG.md and process towncrier newsfragments. --- CHANGELOG.md | 32 +++++++++++++++++++++++++--- towncrier/newsfragments/1660.bugfix | 1 - towncrier/newsfragments/1686.bugfix | 1 - towncrier/newsfragments/1720.bugfix | 2 -- towncrier/newsfragments/1783.misc | 1 - towncrier/newsfragments/1837.bugfix | 1 - towncrier/newsfragments/1841.feature | 1 - towncrier/newsfragments/1845.feature | 1 - towncrier/newsfragments/1857.doc | 1 - towncrier/newsfragments/1861.bugfix | 1 - towncrier/newsfragments/1867.feature | 1 - towncrier/newsfragments/1874.bugfix | 1 - towncrier/newsfragments/1880.feature | 1 - towncrier/newsfragments/191.bugfix | 1 - 14 files changed, 29 insertions(+), 17 deletions(-) delete mode 100644 towncrier/newsfragments/1660.bugfix delete mode 100644 towncrier/newsfragments/1686.bugfix delete mode 100644 towncrier/newsfragments/1720.bugfix delete mode 100644 towncrier/newsfragments/1783.misc delete mode 100644 towncrier/newsfragments/1837.bugfix delete mode 100644 towncrier/newsfragments/1841.feature delete mode 100644 towncrier/newsfragments/1845.feature delete mode 100644 towncrier/newsfragments/1857.doc delete mode 100644 towncrier/newsfragments/1861.bugfix delete mode 100644 towncrier/newsfragments/1867.feature delete mode 100644 towncrier/newsfragments/1874.bugfix delete mode 100644 towncrier/newsfragments/1880.feature delete mode 100644 towncrier/newsfragments/191.bugfix diff --git a/CHANGELOG.md b/CHANGELOG.md index 579f3e82..09b9f68f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,18 +4,44 @@ Changelog Upgrade should run fine as long as you generate a new compose or stack configuration and upgrade your mailu.env. -Please note that the current 1.8 is what we call a "soft release": It’s there for everyone to see and use, but to limit possible user-impact of this very big release, it’s not yet the default in the setup-utility for new users. When upgrading, please treat it with some care, and be sure to always have backups! - There are some changes to the configuration overrides. Override files are now mounted read-only into the containers. The Dovecot and Postfix overrides are moved in their own sub-directory. If there are local override files, they will need to be moved from overrides/ to overrides/dovecot and overrides/postfix/. See https://mailu.io/1.8/faq.html#how-can-i-override-settings for all the mappings. +<<<<<<< HEAD Please note that the shipped image for PostgreSQL database is deprecated. We advise to switch to an external database server. +======= +One major change for the docker compose file is that the antispam needs a fixed hostname [#1837](https://github.com/Mailu/Mailu/issues/1837). +This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. + +Please not that the shipped image for PostgreSQL database is deprecated. +We advise to switch to an external PostgreSQL database server. +>>>>>>> afaacf5a... Update CHANGELOG.md and process towncrier newsfragments. -v1.8.0 - 2020-09-28 +1.8.0 - 2021-08-06 +-------------------- + +- Features: Update version of roundcube webmail and carddav plugin. This is a security update. ([#1841](https://github.com/Mailu/Mailu/issues/1841)) +- Features: Update version of rainloop webmail to 1.16.0. This is a security update. ([#1845](https://github.com/Mailu/Mailu/issues/1845)) +- Features: Changed default value of AUTH_RATELIMIT_SUBNET to false. Increased default value of the rate limit in setup utility (AUTH_RATELIMIT) to a higher value. ([#1867](https://github.com/Mailu/Mailu/issues/1867)) +- Features: Update jquery used in setup. Set pinned versions in requirements.txt for setup. This is a security update. ([#1880](https://github.com/Mailu/Mailu/issues/1880)) +- Bugfixes: Replace PUBLIC_HOSTNAME and PUBLIC_IP in "Received" headers to ensure that no undue spam points are attributed ([#191](https://github.com/Mailu/Mailu/issues/191)) +- Bugfixes: Don't replace nested headers (typically in attached emails) ([#1660](https://github.com/Mailu/Mailu/issues/1660)) +- Bugfixes: Fix letsencrypt access to certbot for the mail-letsencrypt flavour ([#1686](https://github.com/Mailu/Mailu/issues/1686)) +- Bugfixes: Fix CVE-2020-25275 and CVE-2020-24386 by using alpine 3.13 for + dovecot which contains a fixed dovecot version. ([#1720](https://github.com/Mailu/Mailu/issues/1720)) +- Bugfixes: Antispam service now uses a static hostname. Rspamd history is only retained when the service has a fixed hostname. ([#1837](https://github.com/Mailu/Mailu/issues/1837)) +- Bugfixes: Fix a bug preventing colons from being used in passwords when using radicale/webdav. ([#1861](https://github.com/Mailu/Mailu/issues/1861)) +- Bugfixes: Remove dot in blueprint name to prevent critical flask startup error in setup. ([#1874](https://github.com/Mailu/Mailu/issues/1874)) +- Bugfixes: fix punycode encoding of domain names ([#1891](https://github.com/Mailu/Mailu/issues/1891)) +- Improved Documentation: Update fail2ban documentation to use systemd backend instead of filepath for journald ([#1857](https://github.com/Mailu/Mailu/issues/1857)) +- Misc: ([#1783](https://github.com/Mailu/Mailu/issues/1783)) + + +v1.8.0rc - 2020-09-28 -------------------- - Features: Add support for backward-forwarding using SRS ([#328](https://github.com/Mailu/Mailu/issues/328)) diff --git a/towncrier/newsfragments/1660.bugfix b/towncrier/newsfragments/1660.bugfix deleted file mode 100644 index a90fb099..00000000 --- a/towncrier/newsfragments/1660.bugfix +++ /dev/null @@ -1 +0,0 @@ -Don't replace nested headers (typically in attached emails) diff --git a/towncrier/newsfragments/1686.bugfix b/towncrier/newsfragments/1686.bugfix deleted file mode 100644 index 932d7d7c..00000000 --- a/towncrier/newsfragments/1686.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix letsencrypt access to certbot for the mail-letsencrypt flavour diff --git a/towncrier/newsfragments/1720.bugfix b/towncrier/newsfragments/1720.bugfix deleted file mode 100644 index 0bf2b8e6..00000000 --- a/towncrier/newsfragments/1720.bugfix +++ /dev/null @@ -1,2 +0,0 @@ -Fix CVE-2020-25275 and CVE-2020-24386 by using alpine 3.13 for -dovecot which contains a fixed dovecot version. diff --git a/towncrier/newsfragments/1783.misc b/towncrier/newsfragments/1783.misc deleted file mode 100644 index 2ee4c97f..00000000 --- a/towncrier/newsfragments/1783.misc +++ /dev/null @@ -1 +0,0 @@ -Switch from client side sessions (cookies) to server-side sessions (Redis). This simplies the security model a lot and allows for an easier recovery should a cookie ever land in the hands of an attacker. diff --git a/towncrier/newsfragments/1837.bugfix b/towncrier/newsfragments/1837.bugfix deleted file mode 100644 index dcabcc6b..00000000 --- a/towncrier/newsfragments/1837.bugfix +++ /dev/null @@ -1 +0,0 @@ -Antispam service now uses a static hostname. Rspamd history is only retained when the service has a fixed hostname. diff --git a/towncrier/newsfragments/1841.feature b/towncrier/newsfragments/1841.feature deleted file mode 100644 index c91f805f..00000000 --- a/towncrier/newsfragments/1841.feature +++ /dev/null @@ -1 +0,0 @@ -Update version of roundcube webmail and carddav plugin. This is a security update. \ No newline at end of file diff --git a/towncrier/newsfragments/1845.feature b/towncrier/newsfragments/1845.feature deleted file mode 100644 index afde9313..00000000 --- a/towncrier/newsfragments/1845.feature +++ /dev/null @@ -1 +0,0 @@ -Update version of rainloop webmail to 1.16.0. This is a security update. diff --git a/towncrier/newsfragments/1857.doc b/towncrier/newsfragments/1857.doc deleted file mode 100644 index 06cb91ab..00000000 --- a/towncrier/newsfragments/1857.doc +++ /dev/null @@ -1 +0,0 @@ -Update fail2ban documentation to use systemd backend instead of filepath for journald \ No newline at end of file diff --git a/towncrier/newsfragments/1861.bugfix b/towncrier/newsfragments/1861.bugfix deleted file mode 100644 index 1e28d1b6..00000000 --- a/towncrier/newsfragments/1861.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a bug preventing colons from being used in passwords when using radicale/webdav. diff --git a/towncrier/newsfragments/1867.feature b/towncrier/newsfragments/1867.feature deleted file mode 100644 index fbd3a7d7..00000000 --- a/towncrier/newsfragments/1867.feature +++ /dev/null @@ -1 +0,0 @@ -Changed default value of AUTH_RATELIMIT_SUBNET to false. Increased default value of the rate limit in setup utility (AUTH_RATELIMIT) to a higher value. diff --git a/towncrier/newsfragments/1874.bugfix b/towncrier/newsfragments/1874.bugfix deleted file mode 100644 index a301835e..00000000 --- a/towncrier/newsfragments/1874.bugfix +++ /dev/null @@ -1 +0,0 @@ -Remove dot in blueprint name to prevent critical flask startup error in setup. diff --git a/towncrier/newsfragments/1880.feature b/towncrier/newsfragments/1880.feature deleted file mode 100644 index 212dc906..00000000 --- a/towncrier/newsfragments/1880.feature +++ /dev/null @@ -1 +0,0 @@ -Update jquery used in setup. Set pinned versions in requirements.txt for setup. This is a security update. diff --git a/towncrier/newsfragments/191.bugfix b/towncrier/newsfragments/191.bugfix deleted file mode 100644 index 185d3074..00000000 --- a/towncrier/newsfragments/191.bugfix +++ /dev/null @@ -1 +0,0 @@ -Replace PUBLIC_HOSTNAME and PUBLIC_IP in "Received" headers to ensure that no undue spam points are attributed From 4b89143362d9ef8cfb985c030047ce74642f7952 Mon Sep 17 00:00:00 2001 From: Diman0 Date: Fri, 6 Aug 2021 23:00:27 +0200 Subject: [PATCH 03/44] Update documentation config and release notes page. --- docs/conf.py | 2 +- docs/releases.rst | 65 +++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 64 insertions(+), 3 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 8f174b64..db7008b3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -36,7 +36,7 @@ html_context = { 'github_user': 'mailu', 'github_repo': 'mailu', 'github_version': version, - 'stable_version': '1.7', + 'stable_version': '1.8', 'versions': [ ('1.5', '/1.5/'), ('1.6', '/1.6/'), diff --git a/docs/releases.rst b/docs/releases.rst index 7a15d1fa..7473b033 100644 --- a/docs/releases.rst +++ b/docs/releases.rst @@ -1,8 +1,69 @@ Release notes ============= -Mailu 1.8 - 2020-10-02 ----------------------- +Mailu 1.8 - 2021-08-7 +--------------------- + +The full 1.8 release is finally ready. There have been some changes in the contributors team. Many people from the contributors team have stepped back due to changed priorities in their life. +We are very grateful for all their contributions and hope we will see them back again in the future. +This is the main reason why it took so long for 1.8 to be fully released. + +Fortunately more people have decided to join the project. Some very nice contributions have been made which will become part of the next 1.9 release. +We hope that future Mailu releases will be released more quickly now we have more active contributors again. + +For a list of all changes refer to `CHANGELOG.md` in the root folder of the Mailu github project. Please read the 'Override location changes' section further on this page. It contains important information for the people who use the overrides folder. + +New Functionality & Improvements +```````````````````````````````` + +Here’s a short summary of new features: + +- Roundcube and Rainloop have been updated. +- All dependencies have been updated to the latest security update. +- Fail2ban documentation has been improved. +- Switch from client side (cookie) sessions to server side sessions. +- Full-text-search is back after having been disabled for a while due to nasty bugs. It can still be disabled via the mailu.env file. +- Tons of documentation improvements, especially geared towards new users. +- (Experimental) support for different architectures, such as ARM. +- Improvements around webmails, such as CardDAV, GPG and a new skin for an updated roundcube, and support for MySQL for it. Updated Rainloop, too. +- Improvements around relaying, such as AUTH LOGIN and non-standard port support. +- Update to alpine:3.14 as baseimage for most containers. +- Setup warns users about compose-IPv6 deployments which have caused open relays in the past. +- Improved handling of upper-vs-lowercase aliases and user-addresses. +- Improved rate-limiting system. +- Support for SRS. +- Japanese localisation is now available. + + +Upgrading +````````` + +Upgrade should run fine as long as you generate a new compose or stack +configuration and upgrade your mailu.env. + +Please not that the shipped image for PostgreSQL database is deprecated. +The shipped image for PostgreSQL is not maintained anymore from release 1.8. +We recommend switching to an external PostgreSQL image as soon as possible. + +Override location changes +^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you have regenerated the Docker compose and environment files, there are some changes to the configuration overrides. +Override files are now mounted read-only into the containers. The Dovecot and Postfix overrides are moved in their own sub-directory. If there are local override files, they will need to be moved from ``overrides/`` to ``overrides/dovecot`` and ``overrides/postfix/``. + +Update your DNS SPF Records +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It has become known that the SPF DNS records generated by the admin interface are not completely standard compliant anymore. Please check the DNS records for your domains and compare them to what the new admin-interface instructs you to use. In most cases, this should be a simple copy-paste operation for you …. + +Fixed hostname for antispam service +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +For history to be retained in Rspamd, the antispam container requires a static hostname. When you re-generate your docker-compose.yml file (or helm-chart), this will be covered. + + +Mailu 1.8rc - 2020-10-02 +------------------------ Release 1.8 has come a long way again. Due to corona the project slowed down to a crawl. Fortunately new contributors have joined the team what enabled us to still release Mailu 1.8 this year. From 6581f8f087d62b841e9939ea3cf3814ea9e9a518 Mon Sep 17 00:00:00 2001 From: Dimitri Huisman <52963853+Diman0@users.noreply.github.com> Date: Fri, 6 Aug 2021 23:17:41 +0200 Subject: [PATCH 04/44] Resolve merge conflict --- CHANGELOG.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 09b9f68f..3ad0061b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,18 +9,14 @@ The Dovecot and Postfix overrides are moved in their own sub-directory. If there are local override files, they will need to be moved from overrides/ to overrides/dovecot and overrides/postfix/. See https://mailu.io/1.8/faq.html#how-can-i-override-settings for all the mappings. -<<<<<<< HEAD -Please note that the shipped image for PostgreSQL database is deprecated. -We advise to switch to an external database server. -======= -One major change for the docker compose file is that the antispam needs a fixed hostname [#1837](https://github.com/Mailu/Mailu/issues/1837). -This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. +One major change for the docker compose file is that the antispam container needs a fixed hostname [#1837](https://github.com/Mailu/Mailu/issues/1837). +This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. +This is also handled in the helm-chart repo. Please not that the shipped image for PostgreSQL database is deprecated. We advise to switch to an external PostgreSQL database server. ->>>>>>> afaacf5a... Update CHANGELOG.md and process towncrier newsfragments. - + 1.8.0 - 2021-08-06 -------------------- From f0997ed0fd5b3be5e8a6964bfd3197ced1aa5ef9 Mon Sep 17 00:00:00 2001 From: Diman0 Date: Sat, 7 Aug 2021 09:12:43 +0200 Subject: [PATCH 05/44] Improved changelog entry --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ad0061b..0a128163 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,7 +27,7 @@ We advise to switch to an external PostgreSQL database server. - Bugfixes: Replace PUBLIC_HOSTNAME and PUBLIC_IP in "Received" headers to ensure that no undue spam points are attributed ([#191](https://github.com/Mailu/Mailu/issues/191)) - Bugfixes: Don't replace nested headers (typically in attached emails) ([#1660](https://github.com/Mailu/Mailu/issues/1660)) - Bugfixes: Fix letsencrypt access to certbot for the mail-letsencrypt flavour ([#1686](https://github.com/Mailu/Mailu/issues/1686)) -- Bugfixes: Fix CVE-2020-25275 and CVE-2020-24386 by using alpine 3.13 for +- Bugfixes: Fix CVE-2020-25275 and CVE-2020-24386 by upgrading alpine for dovecot which contains a fixed dovecot version. ([#1720](https://github.com/Mailu/Mailu/issues/1720)) - Bugfixes: Antispam service now uses a static hostname. Rspamd history is only retained when the service has a fixed hostname. ([#1837](https://github.com/Mailu/Mailu/issues/1837)) - Bugfixes: Fix a bug preventing colons from being used in passwords when using radicale/webdav. ([#1861](https://github.com/Mailu/Mailu/issues/1861)) From 21e7a338e75437afd59ae4185c87f59d18f8f4e9 Mon Sep 17 00:00:00 2001 From: Diman0 Date: Sat, 7 Aug 2021 09:14:09 +0200 Subject: [PATCH 06/44] Fixed typing error. --- CHANGELOG.md | 2 +- docs/releases.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a128163..82f04acc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ One major change for the docker compose file is that the antispam container need This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. This is also handled in the helm-chart repo. -Please not that the shipped image for PostgreSQL database is deprecated. +Please note that the shipped image for PostgreSQL database is deprecated. We advise to switch to an external PostgreSQL database server. diff --git a/docs/releases.rst b/docs/releases.rst index 7473b033..3ae25f48 100644 --- a/docs/releases.rst +++ b/docs/releases.rst @@ -41,7 +41,7 @@ Upgrading Upgrade should run fine as long as you generate a new compose or stack configuration and upgrade your mailu.env. -Please not that the shipped image for PostgreSQL database is deprecated. +Please note that the shipped image for PostgreSQL database is deprecated. The shipped image for PostgreSQL is not maintained anymore from release 1.8. We recommend switching to an external PostgreSQL image as soon as possible. From 14a18715111e4d21fd9cac0c7e18a293771f3fd0 Mon Sep 17 00:00:00 2001 From: Diman0 Date: Sat, 7 Aug 2021 09:25:40 +0200 Subject: [PATCH 07/44] enhanced security changelog entry and added recommendation to recreate secret_key --- CHANGELOG.md | 11 ++++++++++- docs/releases.rst | 14 +++++++++++++- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 82f04acc..da945c72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,15 @@ One major change for the docker compose file is that the antispam container need This is handled when you regenerate the docker-compose file. A fixed hostname is required to retain rspamd history. This is also handled in the helm-chart repo. +Improvements have been made to protect again session-fixation attacks. +To be fully protected, it is required to change your SECRET_KEY in Mailu.env after upgrading. +A new SECRET_KEY is generated when you recreate your docker-compose.yml & mailu.env file via setup.mailu.io. + +The SECRET_KEY is an uppercase alphanumeric string of length 16. You can manually create such a string via +```cat /dev/urandom | tr -dc 'A-Z0-9' | fold -w ${1:-16} | head -n 1``` + +After changing mailu.env, it is required to recreate all containers for the changes to be propagated. + Please note that the shipped image for PostgreSQL database is deprecated. We advise to switch to an external PostgreSQL database server. @@ -34,7 +43,7 @@ We advise to switch to an external PostgreSQL database server. - Bugfixes: Remove dot in blueprint name to prevent critical flask startup error in setup. ([#1874](https://github.com/Mailu/Mailu/issues/1874)) - Bugfixes: fix punycode encoding of domain names ([#1891](https://github.com/Mailu/Mailu/issues/1891)) - Improved Documentation: Update fail2ban documentation to use systemd backend instead of filepath for journald ([#1857](https://github.com/Mailu/Mailu/issues/1857)) -- Misc: ([#1783](https://github.com/Mailu/Mailu/issues/1783)) +- Misc: Switch from client side (cookie) sessions to server side sessions and protect against session-fixation attacks. We recommend that you change your SECRET_KEY after upgrading. ([#1783](https://github.com/Mailu/Mailu/issues/1783)) v1.8.0rc - 2020-09-28 diff --git a/docs/releases.rst b/docs/releases.rst index 3ae25f48..6c672538 100644 --- a/docs/releases.rst +++ b/docs/releases.rst @@ -21,7 +21,7 @@ Here’s a short summary of new features: - Roundcube and Rainloop have been updated. - All dependencies have been updated to the latest security update. - Fail2ban documentation has been improved. -- Switch from client side (cookie) sessions to server side sessions. +- Switch from client side (cookie) sessions to server side sessions and protect against session-fixation attacks. We recommend that you change your SECRET_KEY after upgrading. - Full-text-search is back after having been disabled for a while due to nasty bugs. It can still be disabled via the mailu.env file. - Tons of documentation improvements, especially geared towards new users. - (Experimental) support for different architectures, such as ARM. @@ -51,6 +51,18 @@ Override location changes If you have regenerated the Docker compose and environment files, there are some changes to the configuration overrides. Override files are now mounted read-only into the containers. The Dovecot and Postfix overrides are moved in their own sub-directory. If there are local override files, they will need to be moved from ``overrides/`` to ``overrides/dovecot`` and ``overrides/postfix/``. +Recreate SECRET_KEY after upgrading +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Improvements have been made to protect again session-fixation attacks. +To be fully protected, it is required to change your SECRET_KEY in Mailu.env after upgrading. +A new SECRET_KEY is generated when you recreate your docker-compose.yml & mailu.env file via setup.mailu.io. + +The SECRET_KEY is an uppercase alphanumeric string of length 16. You can manually create such a string via +```cat /dev/urandom | tr -dc 'A-Z0-9' | fold -w ${1:-16} | head -n 1``` + +After changing mailu.env, it is required to recreate all containers for the changes to be propagated. + Update your DNS SPF Records ^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 3157fc3623424f7af991f2de86981c2aeced4cca Mon Sep 17 00:00:00 2001 From: Diman0 Date: Sat, 7 Aug 2021 09:27:47 +0200 Subject: [PATCH 08/44] Give docker containers in each test one more minute for starting. --- .github/workflows/CI.yml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 19a445b4..e2a535dd 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -121,7 +121,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test core suite - run: python tests/compose/test.py core 1 + run: python tests/compose/test.py core 2 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} @@ -168,7 +168,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test fetch - run: python tests/compose/test.py fetchmail 1 + run: python tests/compose/test.py fetchmail 2 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} @@ -215,7 +215,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test clamvav - run: python tests/compose/test.py filters 2 + run: python tests/compose/test.py filters 3 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} @@ -262,7 +262,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test rainloop - run: python tests/compose/test.py rainloop 1 + run: python tests/compose/test.py rainloop 2 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} @@ -309,7 +309,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test roundcube - run: python tests/compose/test.py roundcube 1 + run: python tests/compose/test.py roundcube 2 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} @@ -356,7 +356,7 @@ jobs: - name: Copy all certs run: sudo -- sh -c 'mkdir -p /mailu && cp -r tests/certs /mailu && chmod 600 /mailu/certs/*' - name: Test webdav - run: python tests/compose/test.py webdav 1 + run: python tests/compose/test.py webdav 2 env: MAILU_VERSION: ${{ env.MAILU_VERSION }} TRAVIS_BRANCH: ${{ env.BRANCH }} From 1438253a069da3b10831ef89dc119177f16f5216 Mon Sep 17 00:00:00 2001 From: Florent Daigniere Date: Sun, 8 Aug 2021 09:21:14 +0200 Subject: [PATCH 09/44] Ratelimit outgoing emails per user --- core/admin/mailu/configuration.py | 1 + core/admin/mailu/internal/views/postfix.py | 10 ++++++++-- core/admin/mailu/models.py | 8 +++++++- core/admin/mailu/ui/templates/user/list.html | 5 ++++- core/postfix/conf/main.cf | 1 + core/postfix/start.py | 3 ++- setup/flavors/compose/mailu.env | 5 +++++ setup/templates/steps/config.html | 7 +++++++ towncrier/newsfragments/1031.feature | 1 + 9 files changed, 36 insertions(+), 5 deletions(-) create mode 100644 towncrier/newsfragments/1031.feature diff --git a/core/admin/mailu/configuration.py b/core/admin/mailu/configuration.py index d2d34d88..50733d52 100644 --- a/core/admin/mailu/configuration.py +++ b/core/admin/mailu/configuration.py @@ -46,6 +46,7 @@ DEFAULT_CONFIG = { 'DKIM_SELECTOR': 'dkim', 'DKIM_PATH': '/dkim/{domain}.{selector}.key', 'DEFAULT_QUOTA': 1000000000, + 'MESSAGE_RATELIMIT': '100/hour', # Web settings 'SITENAME': 'Mailu', 'WEBSITE': 'https://mailu.io', diff --git a/core/admin/mailu/internal/views/postfix.py b/core/admin/mailu/internal/views/postfix.py index c358c37f..06918c61 100644 --- a/core/admin/mailu/internal/views/postfix.py +++ b/core/admin/mailu/internal/views/postfix.py @@ -1,5 +1,6 @@ -from mailu import models +from mailu import models, utils from mailu.internal import internal +from flask import current_app as app import flask import idna @@ -31,7 +32,6 @@ def postfix_alias_map(alias): destination = models.Email.resolve_destination(localpart, domain_name) return flask.jsonify(",".join(destination)) if destination else flask.abort(404) - @internal.route("/postfix/transport/") def postfix_transport(email): if email == '*' or re.match("(^|.*@)\[.*\]$", email): @@ -139,6 +139,12 @@ def postfix_sender_login(sender): destination = models.Email.resolve_destination(localpart, domain_name, True) return flask.jsonify(",".join(destination)) if destination else flask.abort(404) +@internal.route("/postfix/sender/rate/") +def postfix_sender_rate(sender): + """ Rate limit outbound emails per sender login + """ + user = models.User.get(sender) or flask.abort(404) + return flask.abort(404) if user.sender_limiter.hit() else flask.jsonify("REJECT") @internal.route("/postfix/sender/access/") def postfix_sender_access(sender): diff --git a/core/admin/mailu/models.py b/core/admin/mailu/models.py index 3a299786..5760c27f 100644 --- a/core/admin/mailu/models.py +++ b/core/admin/mailu/models.py @@ -27,7 +27,7 @@ from sqlalchemy.ext.hybrid import hybrid_property from sqlalchemy.inspection import inspect from werkzeug.utils import cached_property -from mailu import dkim +from mailu import dkim, utils db = flask_sqlalchemy.SQLAlchemy() @@ -501,6 +501,12 @@ class User(Base, Email): self.reply_enddate > now ) + @property + def sender_limiter(self): + return utils.limiter.get_limiter( + app.config["MESSAGE_RATELIMIT"], "sender", self.email + ) + @classmethod def get_password_context(cls): """ create password context for hashing and verification diff --git a/core/admin/mailu/ui/templates/user/list.html b/core/admin/mailu/ui/templates/user/list.html index 2aff662f..746afd45 100644 --- a/core/admin/mailu/ui/templates/user/list.html +++ b/core/admin/mailu/ui/templates/user/list.html @@ -19,7 +19,8 @@ {% trans %}User settings{% endtrans %} {% trans %}Email{% endtrans %} {% trans %}Features{% endtrans %} - {% trans %}Quota{% endtrans %} + {% trans %}Storage Quota{% endtrans %} + {% trans %}Sending Quota{% endtrans %} {% trans %}Comment{% endtrans %} {% trans %}Created{% endtrans %} {% trans %}Last edit{% endtrans %} @@ -41,6 +42,8 @@ {% if user.enable_pop %}pop3{% endif %} {{ user.quota_bytes_used | filesizeformat }} / {{ (user.quota_bytes | filesizeformat) if user.quota_bytes else '∞' }} + {% set limiter = user.sender_limiter %} + {{ limiter.get_window_stats()[1] }} / {{ limiter.limit }} {{ user.comment or '-' }} {{ user.created_at }} {{ user.updated_at or '' }} diff --git a/core/postfix/conf/main.cf b/core/postfix/conf/main.cf index 8f35f609..6f5a20b8 100644 --- a/core/postfix/conf/main.cf +++ b/core/postfix/conf/main.cf @@ -100,6 +100,7 @@ smtpd_sender_login_maps = ${podop}senderlogin smtpd_helo_required = yes smtpd_client_restrictions = + check_sasl_access ${podop}senderrate, permit_mynetworks, check_sender_access ${podop}senderaccess, reject_non_fqdn_sender, diff --git a/core/postfix/start.py b/core/postfix/start.py index e0c781b7..139616b2 100755 --- a/core/postfix/start.py +++ b/core/postfix/start.py @@ -25,7 +25,8 @@ def start_podop(): ("recipientmap", "url", url + "recipient/map/§"), ("sendermap", "url", url + "sender/map/§"), ("senderaccess", "url", url + "sender/access/§"), - ("senderlogin", "url", url + "sender/login/§") + ("senderlogin", "url", url + "sender/login/§"), + ("senderrate", "url", url + "sender/rate/§") ]) def is_valid_postconf_line(line): diff --git a/setup/flavors/compose/mailu.env b/setup/flavors/compose/mailu.env index d45f5517..52f4ee04 100644 --- a/setup/flavors/compose/mailu.env +++ b/setup/flavors/compose/mailu.env @@ -62,6 +62,11 @@ ANTIVIRUS={{ antivirus_enabled or 'none' }} # Max attachment size will be 33% smaller MESSAGE_SIZE_LIMIT={{ message_size_limit or '50000000' }} +# Message rate limit (per user) +{% if message_ratelimit_pd > '0' %} +MESSAGE_RATELIMIT={{ message_ratelimit_pd }}/day +{% endif %} + # Networks granted relay permissions # Use this with care, all hosts in this networks will be able to send mail without authentication! RELAYNETS= diff --git a/setup/templates/steps/config.html b/setup/templates/steps/config.html index 72b83915..87410fca 100644 --- a/setup/templates/steps/config.html +++ b/setup/templates/steps/config.html @@ -55,6 +55,13 @@ Or in plain english: if receivers start to classify your mail as spam, this post

+
+ + +

/ day +

+
+