1916: Ratelimit outgoing emails per user r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
A conflict-free version of #1360 implementing per-user sender limits
### Related issue(s)
- close#1360
- close#1031
- close#1774
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
1871: Replace PUBLIC_HOSTNAME/IP with real data in Received headers r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
This will ensure that we don't get spam points for not respecting the RFC (gmail is notorious for not liking it)
### Related issue(s)
- close#1448
- #466
- #191
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1858: Docs: Replace hardcoded journald logpath with systemd backend r=mergify[bot] a=networkException
The file at /var/log/messages is not universal for every
distribution. Fail2ban can access journald logs directly
by using the systemd backend.
## What type of PR?
documentation
## What does this PR do?
The path /var/log/messages does not apply for Ubuntu 20.04 for example, because of that I have looked
at alternative ways to access journald in fail2ban. The proper way seems to be to use the systemd
backend, this patch updates the documentation accordingly.
### Related issue(s)
*none*
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: networkException <git@nwex.de>
1845: Update rainloop to 1.16.0 r=mergify[bot] a=nextgens
## What type of PR?
Security-update for rainloop.
## What does this PR do?
Upgrade to rainloop v1.16
### Related issue(s)
- #1829
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1754: centralize Webmail authentication behind the admin panel (SSO) r=mergify[bot] a=nextgens
## What type of PR?
Enhancement: it centralizes the authentication of webmails to the admin interface.
## What does this PR do?
It implements the glue required for webmails to do SSO using the admin interface.
One of the main advantages of centralizing things this way is that it reduces significantly the attack surface available to an unauthenticated attacker (no webmail access until there is a valid Flask session).
Others include the ability to implement 2FA down the line and rate-limit things as required.
### Related issue(s)
- #783
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1758: Implement a simpler credential cache (alternative to #1755) r=mergify[bot] a=nextgens
## What type of PR?
Feature: it implements a credential cache to speedup authentication requests.
## What does this PR do?
Credentials are stored in cold-storage using a slow, salted/iterated hash function to prevent offline bruteforce attacks. This creates a performance bottleneck for no valid reason (see the
rationale/long version on https://github.com/Mailu/Mailu/issues/1194#issuecomment-762115549).
The new credential cache makes things fast again.
This is the simpler version of #1755 (with no new dependencies)
### Related issue(s)
- close#1411
- close#1194
- close#1755
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1776: optimize generation of transport nexthop r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix and enhancement.
## What does this PR do?
Possibly there should be more input validation when editing a relay, but for now this tries to make the best out of the existing "smtp" attribute while maintaining backwards compatibility. When relay is empty, the transport's nexthop is the MX of the relayed domain to fix#1588
```
RELAY NEXTHOP TRANSPORT
empty use MX of relay domain smtp:domain
:port use MX of relay domain and use port smtp:domain:port
target resolve A/AAAA of target smtp:[target]
target:port resolve A/AAAA of target and use port smtp:[target]:port
mx:target resolve MX of target smtp:target
mx:target:port resolve MX of target and use port smtp:target:port
lmtp:target resolve A/AAAA of target lmtp:target
lmtp:target:port resolve A/AAAA of target and use port lmtp:target:port
target can also be an IPv4 or IPv6 address (an IPv6 address must be enclosed in []: [2001:DB8::]).
```
When there is proper input validation and existing database entries are migrated this function can be made much shorter again.
### Related issue(s)
- closes#1588
- closes#1815
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1831: Fix roundcube database env configuration r=mergify[bot] a=parisni
## What type of PR?
bug-fix
## What does this PR do?
Both roundcube and mailu admin website can be backed by postgres/mysql. Before this PR, the `DB_FLAVOR` is shared by both services. However, the other roundcube dedicated DB params are prefixed with `ROUNDCUBE_`. (eg: `ROUNDCUBE_DB_NAME`)
There is no reason to share the DB_FLAVOR for both: This PR makes them be considered independently to make things clear and avoid bugs.
Also, the roundcube_db_flavor and db_flavor are made separated in this PR. However for simplicity, the template generator bind them : roundcube_db_flavor = db_flavor. This makes the template generator UI more simple. I considered most of the time people want to have both roundcube and mailu share the same RDBMS.
Also, AFAIK the internal postgresql service is deprecated and will be removed in 1.9. This is why this PR does not integrate roundcube in postgres when the internal DB is choosen: in case of internal postgres, the roundcube is backed with sqlite.
Both documentation and setup website have been updated accordingly.
### Related issue(s)
- Auto close an issue like: closes#1648#1471
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: parisni <nicolas.paris@riseup.net>
Co-authored-by: Nicolas Paris <nicolas.paris@riseup.net>
1836: Test ci parallel r=Diman0 a=Diman0
## What type of PR?
enhancement
## What does this PR do?
Changes CI workflow to run all tests in parellel. After performing some tests (see #1830 ), I determined that using actions/cache to only cache a tar.gz. file with all build images and use this for all parallel tests is the fasted solution.
With Travis builds took ~30 minutes. Now each build runs for a maximum of 20 minutes (bors test and merge on master).
Bors r+ runs take about ~16/17 minutes.
### Related issue(s)
- Auto close an issue like: closes#1830
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
1746: DNS records for client autoconfiguration (RFC6186) r=Diman0 a=nextgens
## What type of PR?
Feature
## What does this PR do?
Add instructions on how to configure rfc6186 DNS records for client autoconfiguration
### Related issue(s)
- #224
- #498
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1694: update compression algorithms for current dovecot r=nextgens a=lub
## What type of PR?
enhancement
## What does this PR do?
This adds additional compression algorithms in accordance with
https://doc.dovecot.org/configuration_manual/zlib_plugin/
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1649: Update docs/reverse.rst with Traefik v2+ info r=mergify[bot] a=patryk-tech
## What type of PR?
Documentation
## What does this PR do?
Adds information about using Traefik v2+ as a reverse proxy.
### Related issue(s)
Closes#1503
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1673: Remove rspamd unused env var from start script r=mergify[bot] a=cbachert
## What type of PR?
Cleanup
## What does this PR do?
Remove unused environment variable FRONT_ADDRESS in rspamd. FRONT_ADDRESS references were removed with commit 8172f3e in PR #727 like mentioned in chat https://matrix.to/#/!MINuyJjJSrfowljYCK:tedomum.net/$160401946364NGNmI:imninja.net?via=huisman.xyz&via=matrix.org&via=imninja.net
```
Mailu$ grep -r "FRONT_ADDRESS" core/rspamd/
core/rspamd/start.py:os.environ["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
```
### Related issue(s)
N/A
## Prerequistes
- [x] Documentation updated accordingly: No documentation to update
- [x] Add to changelog: Minor change
Co-authored-by: Patryk Tech <git@patryk.tech>
Co-authored-by: cbachert <cbachert@users.noreply.github.com>
1760: Security updates to postgresql r=mergify[bot] a=WebSpider
## What type of PR?
Security update
## What does this PR do?
It fixes vulnerabilities in the sudo package in the postgresql optional container documented in
CVE-2021-23240, CVE-2021-3156 and CVE-2021-23239
### Related issue(s)
None
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Nils Vogels <n.vogels@aves-it.nl>
1785: Fix bug #1660 (don't replace nested headers) r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Don't replace nested headers (typically in forwarded/attached emails). This will ensure we don't break cryptographic signatures.
### Related issue(s)
- close#1660
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1783: Switch to server-side sessions r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It simplifies session management.
- it ensures that sessions will eventually expire (*)
- it implements some mitigation against session-fixation attacks
- it switches from client-side to server-side sessions (in Redis)
It doesn't prevent us from (re)-implementing a "remember_me" type of feature if that's considered useful by some.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1610: add option to enforce inbound starttls r=mergify[bot] a=lub
## What type of PR?
Feature
## What does this PR do?
It implements a check in the auth_http handler to check for Auth-SSL == on and otherwise returns a 530 starttls error.
If INBOUND_TLS_ENFORCE is not set the behaviour is still the same as before, so existing installations should be unaffected.
Although there is a small difference to e.g. smtpd_tls_security_level of Postfix.
Postfix already throws a 530 after mail from, but this solution only throws it after rcpt to. auth_http is only the request after rcpt to, so it's not possible to do it earlier.
### Related issue(s)
#1328 is kinda related, although this PR doesn't solve the issue that the headers will still display ESMTP instead of ESMTPS
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1638: Remove the username from the milter_headers r=mergify[bot] a=githtz
Rspamd adds the name of the authenticated user by default. Setting add_smtp_user to false prevents the login to be leaked.
## What type of PR?
Enhancement
## What does this PR do?
This PR prevents the user login to be leaked in sent emails (for example using an alias)
### Related issue(s)
Closes https://github.com/Mailu/Mailu/issues/1465
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: anrc <15327800+githtz@users.noreply.github.com>
1711: fix typo in faq.rst r=Diman0 a=tomwojcik
1712: Add details for postfix-overrides r=mergify[bot] a=sholl
## What type of PR?
Documentation clarification
## What does this PR do?
### Related issue(s)
this clarifies the FAQ about overrides and fixes#1628
Co-authored-by: Tomasz Wójcik <tomwojcik@users.noreply.github.com>
Co-authored-by: Stephan Holl <stephan@holl-land.de>
Co-authored-by: Stephan Holl <1610827+sholl@users.noreply.github.com>
1618: add OCSP stapling to nginx.conf r=mergify[bot] a=lub
It's not added in tls.conf, because apparently the mail ssl module
doesnt' support OCSP stapling.
https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling
^ exists
https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_stapling
^ missing
When the configured certificate doesn't have OCSP information, it'll
just log a warning during startup.
## What type of PR?
enhancement
## What does this PR do?
It enables OCSP stapling for the http server. OCSP stapling reduces roundtrips for the client and reduces load on OCSP responders.
### Related issue(s)
- fixes #1616
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1607: _FILE variables for Docker swarm secrets r=mergify[bot] a=lub
## What type of PR?
enhancement
## What does this PR do?
This PR enables usage of DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY to load these values from files instead of supplying them directly. That way it's possible to use Docker secrets.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1669: Fix extract_host_port port separation r=mergify[bot] a=cbachert
Regex quantifier should be lazy to make port separation work.
## What type of PR?
bug-fix
## What does this PR do?
The "extract_host_port" function in admin/mailu/internal/nginx.py and optional/fetchmail/fetchmail.py is not actually separating host and port due to the `(.*)` part of the regex being too generous. Lazy quantifier `(.*?)` allows the other capturing groups to match.
### Related issue(s)
- No issue raised for this
## Prerequistes
- [x] Documentation updated accordingly: N/A, bug-fix
- [x] Add [changelog] entry file: Added towncrier newsfragment with second commit
1672: mark radio buttons in setup utility as required r=mergify[bot] a=lub
## What type of PR?
bug-fix
## What does this PR do?
mark radio buttons in setup utility as required
Otherwise it's possible to submit the form without selecting e.g. any
flavor, which would need additional handling on the server side.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: cbachert <cbachert@users.noreply.github.com>
Co-authored-by: lub <git@lubiland.de>
1592: Add documentation for the web administration gui. r=mergify[bot] a=Diman0
## What type of PR?
Documentation
## What does this PR do?
This PR adds the section Web Administration Interface to the documentation site which completely documents all available settings in the web administration interface.
### Related issue(s)
- Closes issue #1590 (please close this issue for me)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ Done ] In case of feature or enhancement: documentation updated accordingly
- [ Done ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
1611: Adds own server on port 80 for letsencrypt and redirect r=mergify[bot] a=elektro-wolle
## What type of PR?
Bugfix
## What does this PR do?
Handle letsencrypt route to `.well-known` by own server configuration within nginx.
### Related issue(s)
closes#1564
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Wolfgang Jung <w.jung@polyas.de>
1526: Use Radicale 3.x for webdav service r=mergify[bot] a=ofthesun9
- remove -f flag in Dockerfile CMD
- remove deprecated daemon and dns_lookup settings from radicale.conf
- move realm setting from [server] to [auth] in radicale.conf
- add newsfragment
## What type of PR?
Miscellaneous
## What does this PR do?
Modifications in Dockerfile and radicale.conf to get Radicale 3.0 service building properly.
Functional tests would be needed before merge.
### Related issue(s)
- closes#1512
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1538: Introduce environment variable to control dovecot full-text-search r=mergify[bot] a=tremlin
## What type of PR?
Enhancement
## What does this PR do?
In #1320 a full-text-search feature was enabled in Dovecot by default. Since this can have a big impact on performance, I think it's preferable to offer an option to disable the feature if it is not needed. This PR doesn't change the default behavior (FTS on).
### Related issue(s)
- #1320
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordinagly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
1478: Allow to enforce TLS for outbound r=mergify[bot] a=micw
using OUTBOUND_TLS_LEVEL=encrypt (default is 'may')
## What type of PR?
enhancement
## What does this PR do?
Add an option to postfix to enforce outbound traffic to be TLS encrypted.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1501: In setup/flavor, change DMARC RUA and RUF email default settings r=mergify[bot] a=ofthesun9
## What type of PR?
bug-fix
## What does this PR do?
This PR changes the default value used to set DMARC_RUA and DMARC_RUF:
DMARC_RUA and DMARC_RUF defaults will reuse the value defined for POSTMASTER,
instead of 'admin' as previously.
Please note that the setup tool doesn't allow (yet?) to define dmarc_rua nor dmarc_ruf, so the default value is indeed used for the time being.
### Related issue(s)
closes#1463
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1532: Replace SMPT with SMTP r=mergify[bot] a=dhoppe
1543: Disable Health checks on swarm mode r=mergify[bot] a=ofthesun9
ref: https://github.com/moby/moby/issues/35451
## What type of PR?
bug-fix
## What does this PR do?
Modify the docker-compose.yml template used by setup (swarm flavor) to disable Health checks on swarm mode for each service
### Related issue(s)
closes#1289
## Prerequistes
- [x] add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
Co-authored-by: Dennis Hoppe <github@debian-solutions.de>
- remove ==2.1.12 in Dockerfile pip3 install radicale
- remove -f flag in Dockerfile CMD
- remove deprecated daemon and dns_lookup settings from radicale.conf
- move realm setting from [server] to [auth] in radicale.conf
- add newsfragment
1298: Added carddav-plugin for roundcube webmail r=ofthesun9 a=sholl
## Feature
This PR enables the carddav contacts plugin for integration remote contact-repositories based on CardDAV.
## What does this PR do?
This PR enables the carddav contacts plugin for integration remote contact-repositories based on CardDAV.
### Related issue(s)
- Related #1230, at least for CardDAV.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly (not needed since the plugins of webmail is not mentioned in the docs.)
- [x] Changelog-entry added
Co-authored-by: Stephan Holl <stephan@holl-land.de>
Florent Daigniere
lub