1916: Ratelimit outgoing emails per user r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
A conflict-free version of #1360 implementing per-user sender limits
### Related issue(s)
- close#1360
- close#1031
- close#1774
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.
This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.
This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
I have found a list of the top100 email destinations online and ran them
through a script to ensure that all of their MX servers had valid
configuration... this is the result
1656: Add ability to set no WEBROOT_REDIRECT to Nginx r=mergify[bot] a=DavidFair
## What type of PR?
Enhancement / Documentation
## What does this PR do?
From commit:
---
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.
This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.
This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
---
I've also added bullet points to break up a long flowing sentence in `configuration.rst` - it should be a bit easier to read now
### Related issue(s)
No Related Issue - I just jumped to a PR
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
@ Maintainers - Is this worthy of the changelog, it's useful to know about but I imagine the number of people it affects is equally minimal?
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: David Fairbrother <DavidFair@users.noreply.github.com>
1901: treat localpart case insensitive again r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
fixes error introduced by #1604 where the localpart of an email address was handled case sensitive.
this screwed things up at various other places.
### Related issue(s)
closes#1895closes#1900
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1856: update asset builder dependencies r=mergify[bot] a=ghostwheel42
## What type of PR?
update asset builder dependencies
## What does this PR do?
only include needed dependencies to build mailu assets with nodejs v8
### Related issue(s)
update dependencies as discussed in #1829
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1896: save dkim key after creation r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
saves generated dkim key after creation vi web ui.
after the model change the domain object needs to be added and flushed via sqlalchemy.
### Related issue(s)
closes#1892
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1888: Use threads in gunicorn rather than workers/processes r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available
"smarter default"
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1885: fix 1884: always lookup a FQDN r=mergify[bot] a=nextgens
## What type of PR?
bugfix
## What does this PR do?
Fix bug #1884. Ensure that we avoid the musl resolver bug by always looking up a FQDN
### Related issue(s)
- closes#1884
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1890: fix Email class in model.py r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
fixes class Email - keep email, localpart and domain in sync.
### Related issue(s)
closes#1878
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available
"smarter default"
1857: disable startdate when autoreply is disabled r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
disable the reply startdate field when autoreply is disabled
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1791: Enhanced session handling r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
- replaces flask_kvsession and simplekv with a mailu-specific session store
- call cleanup_sessions before first request and not on startup.
this allows to run cmdline actions without redis (and makes it faster)
- allow running without redis for debugging purposes by setting MEMORY_SESSIONS to True
- don't sign session id, as it has plenty of entropy (as suggested by nextgens)
- adds method to prune a user's sessions
### Related issue(s)
- enhances and close#1787
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1754: centralize Webmail authentication behind the admin panel (SSO) r=mergify[bot] a=nextgens
## What type of PR?
Enhancement: it centralizes the authentication of webmails to the admin interface.
## What does this PR do?
It implements the glue required for webmails to do SSO using the admin interface.
One of the main advantages of centralizing things this way is that it reduces significantly the attack surface available to an unauthenticated attacker (no webmail access until there is a valid Flask session).
Others include the ability to implement 2FA down the line and rate-limit things as required.
### Related issue(s)
- #783
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1758: Implement a simpler credential cache (alternative to #1755) r=mergify[bot] a=nextgens
## What type of PR?
Feature: it implements a credential cache to speedup authentication requests.
## What does this PR do?
Credentials are stored in cold-storage using a slow, salted/iterated hash function to prevent offline bruteforce attacks. This creates a performance bottleneck for no valid reason (see the
rationale/long version on https://github.com/Mailu/Mailu/issues/1194#issuecomment-762115549).
The new credential cache makes things fast again.
This is the simpler version of #1755 (with no new dependencies)
### Related issue(s)
- close#1411
- close#1194
- close#1755
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1776: optimize generation of transport nexthop r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix and enhancement.
## What does this PR do?
Possibly there should be more input validation when editing a relay, but for now this tries to make the best out of the existing "smtp" attribute while maintaining backwards compatibility. When relay is empty, the transport's nexthop is the MX of the relayed domain to fix#1588
```
RELAY NEXTHOP TRANSPORT
empty use MX of relay domain smtp:domain
:port use MX of relay domain and use port smtp:domain:port
target resolve A/AAAA of target smtp:[target]
target:port resolve A/AAAA of target and use port smtp:[target]:port
mx:target resolve MX of target smtp:target
mx:target:port resolve MX of target and use port smtp:target:port
lmtp:target resolve A/AAAA of target lmtp:target
lmtp:target:port resolve A/AAAA of target and use port lmtp:target:port
target can also be an IPv4 or IPv6 address (an IPv6 address must be enclosed in []: [2001:DB8::]).
```
When there is proper input validation and existing database entries are migrated this function can be made much shorter again.
### Related issue(s)
- closes#1588
- closes#1815
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1746: DNS records for client autoconfiguration (RFC6186) r=Diman0 a=nextgens
## What type of PR?
Feature
## What does this PR do?
Add instructions on how to configure rfc6186 DNS records for client autoconfiguration
### Related issue(s)
- #224
- #498
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1694: update compression algorithms for current dovecot r=nextgens a=lub
## What type of PR?
enhancement
## What does this PR do?
This adds additional compression algorithms in accordance with
https://doc.dovecot.org/configuration_manual/zlib_plugin/
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1649: Update docs/reverse.rst with Traefik v2+ info r=mergify[bot] a=patryk-tech
## What type of PR?
Documentation
## What does this PR do?
Adds information about using Traefik v2+ as a reverse proxy.
### Related issue(s)
Closes#1503
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1673: Remove rspamd unused env var from start script r=mergify[bot] a=cbachert
## What type of PR?
Cleanup
## What does this PR do?
Remove unused environment variable FRONT_ADDRESS in rspamd. FRONT_ADDRESS references were removed with commit 8172f3e in PR #727 like mentioned in chat https://matrix.to/#/!MINuyJjJSrfowljYCK:tedomum.net/$160401946364NGNmI:imninja.net?via=huisman.xyz&via=matrix.org&via=imninja.net
```
Mailu$ grep -r "FRONT_ADDRESS" core/rspamd/
core/rspamd/start.py:os.environ["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
```
### Related issue(s)
N/A
## Prerequistes
- [x] Documentation updated accordingly: No documentation to update
- [x] Add to changelog: Minor change
Co-authored-by: Patryk Tech <git@patryk.tech>
Co-authored-by: cbachert <cbachert@users.noreply.github.com>
- the session key is now generated using
- a hash of the uid seeded by the apps secret_key (size: SESSION_KEY_BITS)
- a random token (size: 128 bits)
- the session's creation time (size: 32 bits)
- redis server side sessions are now refreshed after 1/2 the session lifetime
even if not modified
- the cookie is also updated if necessary
This can be used to delete all sessions belonging to a user/login.
For no it just iterates over all sessions.
This could be enhanced by using a prefix for and deleting by prefix.
- call cleanup_sessions on first kvstore access
this allows to run cmdline actions without redis (and makes it faster)
- Allow development using DictStore by setting REDIS_ADDRESS to the empty string in env
- don't sign 64bit random session id as suggested by nextgens
1785: Fix bug #1660 (don't replace nested headers) r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Don't replace nested headers (typically in forwarded/attached emails). This will ensure we don't break cryptographic signatures.
### Related issue(s)
- close#1660
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1783: Switch to server-side sessions r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It simplifies session management.
- it ensures that sessions will eventually expire (*)
- it implements some mitigation against session-fixation attacks
- it switches from client-side to server-side sessions (in Redis)
It doesn't prevent us from (re)-implementing a "remember_me" type of feature if that's considered useful by some.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1610: add option to enforce inbound starttls r=mergify[bot] a=lub
## What type of PR?
Feature
## What does this PR do?
It implements a check in the auth_http handler to check for Auth-SSL == on and otherwise returns a 530 starttls error.
If INBOUND_TLS_ENFORCE is not set the behaviour is still the same as before, so existing installations should be unaffected.
Although there is a small difference to e.g. smtpd_tls_security_level of Postfix.
Postfix already throws a 530 after mail from, but this solution only throws it after rcpt to. auth_http is only the request after rcpt to, so it's not possible to do it earlier.
### Related issue(s)
#1328 is kinda related, although this PR doesn't solve the issue that the headers will still display ESMTP instead of ESMTPS
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1638: Remove the username from the milter_headers r=mergify[bot] a=githtz
Rspamd adds the name of the authenticated user by default. Setting add_smtp_user to false prevents the login to be leaked.
## What type of PR?
Enhancement
## What does this PR do?
This PR prevents the user login to be leaked in sent emails (for example using an alias)
### Related issue(s)
Closes https://github.com/Mailu/Mailu/issues/1465
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: anrc <15327800+githtz@users.noreply.github.com>
- remove PASSWORD_SCHEME altogether
- introduce CREDENTIAL_ROUNDS
- migrate all old hashes to the current format
- auto-detect/enable all hash types that passlib supports
- upgrade passlib to 1.7.4 (see #1706: ldap_salted_sha512 support)
1765: Set sensible cookie flags on the admin app r=mergify[bot] a=nextgens
## What type of PR?
Bugfix
## What does this PR do?
It sets the right flags on the session cookie issued by the admin app.
This should probably be backported as the lack of secure flag on TLS-enabled setup is a high risk vulnerability.
SameSite is hardening / helps against CSRF on modern browsers
HTTPOnly is hardening / helps reduce the impact of XSS
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1763: show flash messages again r=mergify[bot] a=lub
## What type of PR?
bug-fix
## What does this PR do?
This basically restores the behaviour, that got removed in
ecdf0c25b3 during refactoring.
### Related issue(s)
- noticed it while reviewing #1756
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [-] In case of feature or enhancement: documentation updated accordingly
- [-] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1618: add OCSP stapling to nginx.conf r=mergify[bot] a=lub
It's not added in tls.conf, because apparently the mail ssl module
doesnt' support OCSP stapling.
https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling
^ exists
https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_stapling
^ missing
When the configured certificate doesn't have OCSP information, it'll
just log a warning during startup.
## What type of PR?
enhancement
## What does this PR do?
It enables OCSP stapling for the http server. OCSP stapling reduces roundtrips for the client and reduces load on OCSP responders.
### Related issue(s)
- fixes #1616
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1607: _FILE variables for Docker swarm secrets r=mergify[bot] a=lub
## What type of PR?
enhancement
## What does this PR do?
This PR enables usage of DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY to load these values from files instead of supplying them directly. That way it's possible to use Docker secrets.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
With `CHUNKING`set as a capability, nginx advertises this capability to
clients at a stage where the SMTP dialog does not seem to be forwarded
to the proxy-target (postfix) yet. Nginx' SMTP parser itself does not
support the `BDAT` command issued as part of a chunke-d dialog. This makes
Nginx respond with a `250 2.0.0 OK` and close the connection, after the
mail-data got sent by the client — without forwarding this to the
proxy-target.
With this, users mail can be lost.
Furthermore, when a user uses a sieve filter to forward mail, dovecot
sometimes chunks the forwarded mail when sending it through `front`.
These forwards then fail.
Removing `CHUNKING` from the capabilities fixes this behavior.
- yaml-import is now logged via schema.Logger
- iremoved relative imports - not used in other mailu modules
- removed develepment comments
- added Mailconfig.check method to check for duplicate domain names
- converted .format() to .format_map() where possible
- switched to yaml multiline dump for dkim_key
- converted dkim_key import from regex to string functions
- automatically unhide/unexclude explicitly specified attributes on dump
- use field order when loading to stabilize import
- fail when using 'hash_password' without 'password'
- fixed logging of dkim_key
- fixed pruning and deleting of lists
- modified error messages
- added debug flag and two verbosity levels
- idna.encode does not encode upper-case letters,
so .lower() has to be called on value not on result
- split email-address on '@' only once
- converted '*'.format(*) to f-strings
- added docstrings
- removed from_dict method
- code cleanup/style (list concat, exceptions, return&else, line-length)
- added TODO comments on possible future changes
- revived original config-update function for backwards compability
- renamed config-dump to config-export to be in line with config-import
- converted '*'.format(*) to f-strings
- converted string-concatenation to f-strings
renamed single letter variables (m => match)
renamed classmethod arguments to cls (model)
removed shadowing of variables (hash, context)
shortened unneeded lambda functions (id)
converted type ... is to isinstance(...)
removed unneded imports (flask)
Updated ConfigManager to only modify app.config and not replace it.
Swagger does not play well, when app.config is not a real dict and
it is not necessary to keep ConfigManager around after init.
Also added "API" flag to config (default: disabled).
1654: Ensure that the rendered file ends with newline in order to make `pos… r=mergify[bot] a=tremlin
## What type of PR?
Bugfix
## What does this PR do?
This fixes#1580
### Related issue(s)
- closes#1580
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.
This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.
This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
1611: Adds own server on port 80 for letsencrypt and redirect r=mergify[bot] a=elektro-wolle
## What type of PR?
Bugfix
## What does this PR do?
Handle letsencrypt route to `.well-known` by own server configuration within nginx.
### Related issue(s)
closes#1564
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Wolfgang Jung <w.jung@polyas.de>
1635: Add support for AUTH LOGIN authentication mechanism for relaying emai… r=mergify[bot] a=Diman0
…l via smart hosts.
## What type of PR?
Feature
## What does this PR do?
This PR adds support to postfix for AUTH LOGIN authentication mechanism. This enables using smart hosts which only offer AUTH LOGIN.
### Related issue(s)
- Auto close an issue like: closes#1633
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Because basically every MTA out there uses opportunistic TLS _in
the best case_, it's actually counter productive to use such strict
settings.
The alternative to a handshake error is often an unencrypted submission,
which is basically the opposite of what strict ssl_protocols and
ssl_ciphers tries to achieve.
Even big and established providers like Amazon SES are incompatible with the current
settings.
This reverts commit 2ddf46ad2b.
this enables usage of e.g. docker swarm secrets instead of exposing the
passwords directly via environment variables
just use DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY
- stabilized CommaSeparatedList by sorting values
- CommaSeparatedList can now handle list and set input
- from_dict now handles mapped keys
- from_dict now handles null values
- class Domain: handle dkim-key None correctly
- class User: delete obsolete keys after converting
- class Alias: now uses Email._dict_input
1557: Explicitly define ProxyFix options r=mergify[bot] a=brian-maloney
## What type of PR?
bug-fix
## What does this PR do?
This PR explicitly defines the options for the ProxyFix module, which fixes a regression in admin behind a reverse proxy.
### Related issue(s)
- #1309
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
This is a bugfix, so not doc changes, and it's an extremely minor change.
Co-authored-by: Brian Maloney <3286425+brian-maloney@users.noreply.github.com>
1538: Introduce environment variable to control dovecot full-text-search r=mergify[bot] a=tremlin
## What type of PR?
Enhancement
## What does this PR do?
In #1320 a full-text-search feature was enabled in Dovecot by default. Since this can have a big impact on performance, I think it's preferable to offer an option to disable the feature if it is not needed. This PR doesn't change the default behavior (FTS on).
### Related issue(s)
- #1320
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordinagly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
1478: Allow to enforce TLS for outbound r=mergify[bot] a=micw
using OUTBOUND_TLS_LEVEL=encrypt (default is 'may')
## What type of PR?
enhancement
## What does this PR do?
Add an option to postfix to enforce outbound traffic to be TLS encrypted.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1501: In setup/flavor, change DMARC RUA and RUF email default settings r=mergify[bot] a=ofthesun9
## What type of PR?
bug-fix
## What does this PR do?
This PR changes the default value used to set DMARC_RUA and DMARC_RUF:
DMARC_RUA and DMARC_RUF defaults will reuse the value defined for POSTMASTER,
instead of 'admin' as previously.
Please note that the setup tool doesn't allow (yet?) to define dmarc_rua nor dmarc_ruf, so the default value is indeed used for the time being.
### Related issue(s)
closes#1463
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1532: Replace SMPT with SMTP r=mergify[bot] a=dhoppe
1543: Disable Health checks on swarm mode r=mergify[bot] a=ofthesun9
ref: https://github.com/moby/moby/issues/35451
## What type of PR?
bug-fix
## What does this PR do?
Modify the docker-compose.yml template used by setup (swarm flavor) to disable Health checks on swarm mode for each service
### Related issue(s)
closes#1289
## Prerequistes
- [x] add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
Co-authored-by: Dennis Hoppe <github@debian-solutions.de>
1293: Remove `reject_unverified_recipient` from `smtpd_client_restrictions` r=mergify[bot] a=SunMar
## What type of PR?
Bug-fix
## What does this PR do?
It removes recipient verification, as it broke my catch-all address.
Fix for #1292, though I'm not sure if this is the right way to fix the issue. It was added in 175349a224.
### Related issue(s)
Fix for #1292 and a revert of 175349a224.
Co-authored-by: SunMar <SunMar@users.noreply.github.com>
1399: Remove SPF type SPF record #1394 r=mergify[bot] a=bladeswords
As mentioned in #1394 - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
## What type of PR?
Documentation
## What does this PR do?
Removes the recommendation to add a SPF RR for SPF records, as this is no longer RFC complaint and often causes issues to maintain two records.
### Related issue(s)
- closes#1394
## Prerequistes
None
1417: docker-compose exec needs a -T flag if no TTY is allocated r=mergify[bot] a=ofthesun9
This flag is missing in 00_create_users.sh and is failing the tests on travis arm architecture
## What type of PR?
This PR is an enhancement/bugfix needed to allow usage of travis to test and deploy on arm platform
Before the PR, tests are failing with the msg: "the input device is not a TTY"
## What does this PR do?
This PR add -T flag for the docker-compose exec occurences found in 00_create_users.sh
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1389: Prefer specific alias over wildcard, regardless of case r=mergify[bot] a=Nebukadneza
## What type of PR?
bug-fix
## What does this PR do?
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
## Notes
I realize that the if-hell down there isn’t nice. What it is, however, is quite clear and easy to read. I’m hoping that if anyone ever gets confused in the future, this will make the current behavior transparent. For me, that was more important than a minimal amount of statements/branches …
### Related issue(s)
closes#1387
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1392: Use environment variables for cert paths/names in nginx certwatcher r=mergify[bot] a=Nebukadneza
## What type of PR?
bug-fix
## What does this PR do?
Previously, nginx certwatcher would only react to the hardcoded paths. It should have
honored the enviroment variables that are used by config.py too for this.
### Related issue(s)
closes#903
## Prerequistes
- [x] no feature or enhancement
- [x] minor/internal change
Co-authored-by: Dario Ernst <github@kanojo.de>
1398: Update crypto to be modern and inline with tls.conf r=mergify[bot] a=bladeswords
Updated to match tls.conf and be aligned to more modern cryptographic standards and only use currently secure protocols and ciphers.
## What type of PR?
bugfix
## What does this PR do?
Update to use more modern cryptographic techniques
### Related issue(s)
- Addresses comment raised in 4f973f6
## Prerequistes
None
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
1393: Ignore newlines and comment-lines in postfix overrides r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
### Related issue(s)
closes#1098
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1296: fetchmail: print unhandled exceptions, but don't crash r=Nebukadneza a=Al2Klimov
fixes#1295
1322: Bump validators from 0.12.5 to 0.12.6 in /core/admin r=Nebukadneza a=dependabot[bot]
Bumps [validators](https://github.com/kvesteri/validators) from 0.12.5 to 0.12.6.
<details>
<summary>Changelog</summary>
*Sourced from [validators's changelog](https://github.com/kvesteri/validators/blob/master/CHANGES.rst).*
> 0.12.6 (2019-05-08)
> ^^^^^^^^^^^^^^^^^^^
>
> - Fixed domain validator for single character domains ([#118](https://github-redirect.dependabot.com/kvesteri/validators/issues/118), pull request courtesy kingbuzzman)
</details>
<details>
<summary>Commits</summary>
- See full diff in [compare view](https://github.com/kvesteri/validators/commits)
</details>
<br />
[](https://help.github.com/articles/configuring-automated-security-fixes)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).
</details>
1337: Add IPv6 to allow_nets r=Nebukadneza a=PhilRW
Roundcube was not connecting to sieve with IPv6 enabled.
Fixes#1336
1358: Add port to relay if it contains a colon r=Nebukadneza a=PhilRW
## What type of PR?
enhancement
## What does this PR do?
Allows relaying domains to non-standard SMTP ports by appending `:port` to the destination host/IP. E.g., `mx1.internal:2525`
### Related issue(s)
Closes#1357
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Philip Rosenberg-Watt <p.rosenberg-watt@cablelabs.com>
1395: Remove duplicate ports line r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
### Related issue(s)
closes#1079
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1396: Use pyyaml safe_load instead of load r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
Since load in unsafe (ref: https://msg.pyyaml.org/load),
switch the only occurrance of `yaml.load` that i could
find to safe_load.
### Related issue(s)
closes#1085
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1320: Add xapian full-text-search plugin to dovecot r=mergify[bot] a=Nebukadneza
## What type of PR?
Enhancement
## What does this PR do?
Currently we are not able to offer our users a FTS experience after the
demise of lucene due to unfixed coredumps with musl/alpine.
We now add lucene, the only remaining maintained small/lean FTS plugin
for dovecot. It is quite simple to add to our stack: A two-stage docker
build is used to compile the fts plugin in the first stage, and copy
over only the resulting plugin-artifact to the second stage, which is
our usual dovecot container. Configuration is also minimal.
There was a upstream issue where bodies were not able to be searched for subwords, but fortunately it was fixed quite quickly. We currently need to wait for a new release to use a stable tag in our `Dockerfile`.
### Related issue(s)
- https://github.com/Mailu/Mailu/pull/1176
- https://github.com/Mailu/Mailu/pull/1297
- https://github.com/Mailu/Mailu/issues/751
- **Upstream-issues which is the cause for the `TODO` in the `Dockerfile`**: https://github.com/grosjo/fts-xapian/issues/32
## Prerequistes
- [ ] Wait for upstream to prepare new release after https://github.com/grosjo/fts-xapian/issues/32 — so that we can use a stable tag in our `Dockerfile`
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
As mentioned in #1394 - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
closes#1098
Previously, nginx certwatcher would only react to the hardcoded paths. It should have
honored the enviroment variables that are used by config.py too for this.
closes#903
1359: Refactor the rate limiting code r=mergify[bot] a=kaiyou
## What type of PR?
Enhancement
## What does this PR do?
Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.
Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.
### Related issue(s)
No specific issue.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
1321: Upgrading nginx TLS configuration r=mergify[bot] a=radtkedev
## What type of PR?
Enhancement
## What does this PR do?
Upgrades the TLS protocols and ciphers to the recommended "Intermediate Configuration" and sets the "Old Configuration" for port 25 (SMTP) based on the [Mozilla SSL Configuration Generator](https://ssl-config.mozilla.org/) and adjusted for the nginx mail proxy.
Co-authored-by: Tom Radtke <tom@radtke.dev>
1349: Add support for SRS, related to #328 r=mergify[bot] a=kaiyou
## What type of PR?
Feature
## What does this PR do?
It implements SRS using a Python SRS library.
### Related issue(s)
- closes#328
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
closes#1387
Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.
Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.
1278: Limiter implementation r=kaiyou a=micw
## What type of PR?
(Feature, enhancement, bug-fix, documentation)
## What does this PR do?
Adds a custom limter based on the "limits" lirary that counts up on failed auths only
### Related issue(s)
- closes#1195
- closes#634
## Prerequistes
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: micw <michael@wyraz.de>
1316: Fix the encoding of incoming user email and password r=mergify[bot] a=kaiyou
## What type of PR?
Bug fix
## What does this PR do?
As described in the changes, RFC2616 states that header should be considered ISO8859-1 in HTTP, which obviously nginx does not really care about when forwarding the password from SMTP authentication to the backend. Hence, we need to encode-then-decode passwords to get the proper value in case a special char is in there.
### Related issue(s)
- This fixes#1139
- This is also related to #1281
- This is also related to #1139
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
Currently we are not able to offer our users a FTS experience after the
demise of lucene due to unfixed coredumps with musl/alpine.
We now add lucene, the only remaining maintained small/lean FTS plugin
for dovecot. It is quite simple to add to our stack: A two-stage docker
build is used to compile the fts plugin in the first stage, and copy
over only the resulting plugin-artifact to the second stage, which is
our usual dovecot container. Configuration is also minimal.
1299: Don't remove the address extension in postfix r=mergify[bot] a=RobertMe
## What type of PR?
Bugfix
## What does this PR do?
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.
This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.
### Related issue(s)
Fixes#982
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Robert Meijers <robert.meijers@gmail.com>
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.
This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.
Fixes#982
985: Permit raspberry pi (and other architectures) builds r=mergify[bot] a=abondis
## What type of PR?
Enhancement
## What does this PR do?
Add an option to select base images and permit building for different CPU architectures.
### Related issue(s)
N/A
## Prerequistes
- [X] documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Aurélien Bondis <aurelien.bondis@gmail.com>
Co-authored-by: Aurelien <aurelien.bondis@gmail.com>
1213: Bump werkzeug from 0.15.2 to 0.15.3 in /core/admin r=mergify[bot] a=dependabot[bot]
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 0.15.2 to 0.15.3.
<details>
<summary>Release notes</summary>
*Sourced from [werkzeug's releases](https://github.com/pallets/werkzeug/releases).*
> ## 0.15.3
> * Blog: https://palletsprojects.com/blog/werkzeug-0-15-3-released/
> * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-3
>
</details>
<details>
<summary>Changelog</summary>
*Sourced from [werkzeug's changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst).*
> Version 0.15.3
> --------------
>
> Released 2019-05-14
>
> - Properly handle multi-line header folding in development server in
> Python 2.7. (:issue:`1080`)
> - Restore the ``response`` argument to :exc:`~exceptions.Unauthorized`.
> (:pr:`1527`)
> - :exc:`~exceptions.Unauthorized` doesn't add the ``WWW-Authenticate``
> header if ``www_authenticate`` is not given. (:issue:`1516`)
> - The default URL converter correctly encodes bytes to string rather
> than representing them with ``b''``. (:issue:`1502`)
> - Fix the filename format string in
> :class:`~middleware.profiler.ProfilerMiddleware` to correctly handle
> float values. (:issue:`1511`)
> - Update :class:`~middleware.lint.LintMiddleware` to work on Python 3.
> (:issue:`1510`)
> - The debugger detects cycles in chained exceptions and does not time
> out in that case. (:issue:`1536`)
> - When running the development server in Docker, the debugger security
> pin is now unique per container.
</details>
<details>
<summary>Commits</summary>
- [`9b1123a`](9b1123a779) release version 0.15.3
- [`00bc43b`](00bc43b167) unique debugger pin in Docker containers
- [`2cbdf2b`](2cbdf2b022) Merge pull request [#1542](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1542) from asottile/exceptions_arent_always_hashable
- [`0e669f6`](0e669f6be5) Fix unhashable exception types
- [`bdc17e4`](bdc17e4cd1) Merge pull request [#1540](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1540) from pallets/break-tb-cycle
- [`44e38c2`](44e38c2985) break cycle in chained exceptions
- [`777500b`](777500b646) Merge pull request [#1518](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1518) from NiklasMM/fix/1510_lint-middleware-python3-compa...
- [`e00c7c2`](e00c7c2ced) Make LintMiddleware Python 3 compatible and add tests
- [`d590cc7`](d590cc7cf2) Merge pull request [#1539](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1539) from pallets/profiler-format
- [`0388fc9`](0388fc95e6) update filename_format for ProfilerMiddleware.
- Additional commits viewable in [compare view](https://github.com/pallets/werkzeug/compare/0.15.2...0.15.3)
</details>
<br />
[](https://help.github.com/articles/configuring-automated-security-fixes)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).
</details>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1211: Split HOST_ANTISPAM in HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
Fixes#1190 by separating HOST_ANTISPAM into HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI
### Related issue(s)
- closes#1190
- closes#1150
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1198: Enable access log of admin service only for log levels of INFO and finer r=muhlemmer a=micw
## What type of PR?
bug fix
## What does this PR do?
### Related issue(s)
- closes#1197
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1204: Add initial admin account to kubernetes example r=muhlemmer a=micw
## What type of PR?
documentation
## What does this PR do?
Add INITIAL_ADMIN_* example to kubernetes configmap.yaml
### Related issue(s)
- closes#1201
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- docs example only
1207: Add Japanese translation r=muhlemmer a=IchikawaYukko
## What type of PR?
Translation
## What does this PR do?
Provide completed Japanese translation.
### Related issue(s)
None
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [*] In case of feature or enhancement: documentation updated accordingly
- [*] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1208: Persist mailqueue r=muhlemmer a=micw
## What type of PR?
bug-fix
## What does this PR do?
Makes postfix mailqueue presistent (for docker, swarm and kubernetes)
### Related issue(s)
- closes#1161
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: Michael Wyraz <michael.wyraz@evermind.de>
Co-authored-by: U-COREI3-3120M\市川ゆり子 <ichikawayuriko@yahoo.co.jp>
Co-authored-by: micw <michael@wyraz.de>
1181: Update to address issue #1178 (HTTP headers) r=muhlemmer a=bladeswords
This change should remove the duplicate `x-xss-protection` header and also the `x-powered-by` header. Hopefully a pull request to main is appropriate, but may be worth back porting to 1.7.
Tested config by modifying live 1.7 nginx config and reloading. Has had the desired outcome of removing the headers.
```/etc/nginx # nginx -t -c /etc/nginx/nginx.conf
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
/etc/nginx # nginx -s reload
```
These steps were based on:
- https://serverfault.com/questions/928912/how-do-i-remove-a-server-added-header-from-proxied-location
- https://serverfault.com/questions/929571/overwrite-http-headers-comming-back-from-a-web-application-server-proxied-in-ngi
- http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header
## What type of PR?
Enhancement
## What does this PR do?
Removes duplicate and unneeded headers. See issue #1178
### Related issue(s)
- issue: #1178
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ X ] In case of feature or enhancement: documentation updated accordingly
- [ X ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
1183: Fix rspamd-learn when moving mail from/to junk folder r=mergify[bot] a=Nebukadneza
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
## What type of PR?
bug-fix
## What does this PR do?
Instead of relying on dovecot passing our environment cleanly to sieve-called scripts, this explicitly injects the antispam ip/port into the spam/ham scripts used when moving files from/to the spam-folder. This required some management of the files, such as setting proper permissions after the jinja-run.
### Related issue(s)
fixes#1177
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
1196: Change default password scheme to PBKDF2 (#1194) r=mergify[bot] a=hoellen
## What type of PR?
enhancement
## What does this PR do?
This PR change the default password scheme to PBKDF2. It is already changed in some places (e.g. [docs/compose/.env](https://github.com/Mailu/Mailu/blob/master/docs/compose/.env#L142)).
### Related issue(s)
closes#1194
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1180: Add a title to Mailu-Admin pages r=mergify[bot] a=Niduroki
## What type of PR?
Enhancement
## What does this PR do?
This simply adds a title to be displayed in the browser tab for Mailu-Admin pages.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Considering this is only a one-line change I don't think these apply.
-----
I made the title `Mailu-Admin - SITENAME` – if this seems unfitting feel free to change it to something different.
Co-authored-by: Niduroki <chris@niduroki.net>
1158: Use nginx for kubernetes ingress r=kaiyou a=micw
## What type of PR?
enhancement
## What does this PR do?
Currently, kubernetes uses a complex ingress setting which is not portable across different ingress controllers. This PR simplifies the ingress and delegates everythins special to Mailu to the front container,
### Related issue(s)
- closes#1121
- closes#1117
- closes#1021
- closes#1045
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog]
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
1114: Resolve HOST to ADDRESS only if ADDRESS is not already set r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
~Makes the rsolving from hosts to ips at startup configurable~
I rewrote the pull request after #940 was merged. Now it resolves HOSTs to ADDRESSes only of ADDRESSes are not already set. So on kubernetes we can jsut set the address and have working service discovery.
### Related issue(s)
- closes#1113
## Prerequistes
~Minor change, backward compatible~
Changelog will be added
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1130: Remove unnecessary host variable assignments r=mergify[bot] a=kaiyou
## What type of PR?
Bug fix
## What does this PR do?
### Related issue(s)
- Closes#1126
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1132: Release Mailu 1.7 r=mergify[bot] a=kaiyou
## What type of PR?
Release
## What does this PR do?
It provides the changelog and migration notes for release 1.7.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
940: Fix host variables r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
- Fix#884
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
883: Admin create user enhancement r=mergify[bot] a=cr1st1p
## What type of PR?
Enhancement
## What does this PR do?
It allows the admin docker image to also create the admin user.
The idea is that in my kubernetes setup, I do not want to manually do anything, as such, I need a way for the admin user to also be created automatically without me getting inside the pod.
So I had to change the manage.py function that creates the user to allow different 'modes' (me, I'll be using 'ifmissing') and also start.py to call that functionality if appropriate environment variables are present.
So now, in my Deployment, I add 3 more environment variables and I get the admin user created, IF not already present.
### Related issue(s)
none?
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: cristi <cristi.posoiu@gmail.com>
Co-authored-by: cr1st1p <cristi.posoiu@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
1088: Support domain literals (fix#1087) r=mergify[bot] a=hoellen
## What type of PR?
bug-fix
## What does this PR do?
This PR adds error handling for idna enocding. With telnet you now get a "Bad sender address syntax" message.
```
> telnet mail.example.com 25
Connected to example.com.
Escape character is '^]'.
220 mail.example.com ESMTP ready
EHLO dummy.example.com
250-mail.example.com
250 STARTTLS
MAIL FROM: does-not-exist@[116.203.165.200]
250 2.0.0 OK
RCPT TO: some-user@example.com
501 5.1.7 Bad sender address syntax
Connection closed by foreign host.
```
### Related issue(s)
fix#1087
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1052: Upgrade alpine to 3.10 r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement / bug-fix
## What does this PR do?
Upgrade the alpine base image to 3.10 and clean up ensuing problems. Also directly uses postfix foreground-running with stdout logging.
### Related issue(s)
closes#1049closes#1051
Note: This is a duplicate effort of #1050#1039 …
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
With these images, I have tested manually:
- Email receive to user
- on main domain
- on additional domain
- via an alias on main-domain
- via an alias on additional domain
- via catchall
- Email reject
- of eicar testfiles
- Email sending
- Fetchmail from legacy POP
- Front LE certificates
- Simple overriding for postfix (only postfix.cf), nginx, dovecot, rspamd
- Creating a CalDAV calendar and CardDAV entry using davx5
I have not (yet) tested:
- certdumper
- databases other than sqlite
Todo:
- [x] Get rid of podop at `git+https://github.com/Nebukadneza/Podop.git@fix_py37` once https://github.com/Mailu/Podop/pull/6 is merged
- [x] Bend Mailustart back to original repository
- [x] Test more (? also help wanted ?)
Co-authored-by: Thomas Sänger <thomas@gecko.space>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Reto Glauser <git@blinkeye.ch>
1048: Refactor admin using webpack r=mergify[bot] a=kaiyou
## What type of PR?
Refactoring
## What does this PR do?
This PR brings some refactoring to the admin container :
- remove the dependency to mailustart and replace it with socrate
- remove static assets from the repo and use Webpack for building them
Co-authored-by: hoellen <dev@hoellen.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
931: Resolve webmail in admin r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
Implement mailustart to resolve webmail in admin
### Related issue(s)
Fix#716
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
962: Open some admin sidebar links in a new browser tab r=kaiyou a=mariooos
## What type of PR?
Enhancement
## What does this PR do?
Add target=_blank attribute to html <a href ...> tag in following admin sidebar links:
- Antispam
- Webmail
- Website
- Help
Co-authored-by: mariooos <48351788+mariooos@users.noreply.github.com>
993: Make aliases case-insensitive (too) r=mergify[bot] a=Nebukadneza
Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.
For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …
closes#867
## What type of PR?
bug-fix
## What does this PR do?
Make aliases optionally case-insensitive: After attempting to resolve an alias in its preserved case, also attempt to match it case-insensitively.
This followed after some more thought from #868 …
### Related issue(s)
closes#867
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
945: Add extra newline to main.cf r=mergify[bot] a=flopes89
This should prevent jinja from stripping the newline, which causes overrides to be appended after the comment section
## What type of PR?
Bugfix
## What does this PR do?
Adds a new empty newline a the end of `conf/main.cf` so prevent jinja from stripping it, by which overrides done with `postconf -e` are not appended correctly.
### Related issue(s)
see #941
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
Co-authored-by: Florian Peschka <flopes89@users.noreply.github.com>
1005: distinguish disabled users in user list view by row color r=kaiyou a=zhuangyy
## What type of PR?
enhancement
## What does this PR do?
distinguish disabled users in user list view by row color
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Zhuang Yuyao <jssyan@qq.com>
Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.
For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …
closes#867
934: Pass optional arguments to domain model r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
Passing optional args to domain model when using cli.
### Related issue(s)
Fixes#933
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
919: Install bash in alpine based images. r=mergify[bot] a=firvida
This fixes#918
Bash shell is used by default in Kubernetes' dashboard console, which is very
useful for admins.
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
- closes#918
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Abel Alfonso Fírvida Donéstevez <abel@merchise.org>
886: Ipv6 support r=mergify[bot] a=muhlemmer
## What type of PR?
(Feature, enhancement, bug-fix, documentation) -> A bit of everything
## What does this PR do?
Document how to use ipv6nat. This, however triggers some kind of flaky behavior with the Docker DNS resolver, resulting in lookup failures between containers. So all resolving needs to be done during container startup/configuration.
In order not to pollute every single start.py file, we've created a small library called [Mailu/MailuStart](https://github.com/Mailu/MailuStart). As an addition, this library also defines the template generation function, including its logging facility.
Note: `docker-compose.yml` downgrade is necessary, as IPv6 settings are not supported by the Docker Compose file format 3 😞
### Related issue(s)
Supersedes PR #844
- Fixes#827
- Hopefully helps with #829 and #834
## No backport yet
This PR directly imports MailuStart from git. This makes it a bit more simple to implement on the short term an do some testing and probably some future improvements. When everything is proved stable, we will create a proper PyPi package with versioning and consider back porting.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
At some places, the string that DOMAIN_REGISTRATION is got used like a boolean
(an easy misassumption to make while in python and dealing with the config
dict), making `DOMAIN_REGISTRATION=False` act as a truthy value. To stop such
future problems from happening, coerce environment config strings to real
bools.
closes#830
Since it’s common for wildcard~ish systems to prefer concrete objects over
wildcards, and aliases can be broad-wildcards (think catchall, %@xxx.tld), it
may be more intuitive for users that user-names rank higher than aliases. This
makes it impossible for user-names to be unreachable, since they can be
completely overridden by a catchall otherwise.
This changes default behavior, and is not configurable.
closes#815
As discussed with hoellen on matrix, since postfix indeed supports including
the recipient delimiter character in a verbatim alias, we should support so too
— and handle its precedence correctly. The clearer and simpler formulation of
the precedence-clauses are credit to @hoellen. Thanks!
This fixes delivery to an alias minus recipient delimiter in cases where a
wildcard alias would also match. For example,
* foo@xxx.tld
* %@xxx.tld
Sending to foo+spam@xxx.tld would get eaten by the catchall before this fix.
Now, the order of alias resolution is made clearer.
closes#813
In the process we found that the previous way of tenacity syntax caused it not to honor any args.
In this commit we've refactored to use the @decorator syntax, in which tenacity seems to behave better.
CVE-2017-18342
Vulnerable versions: < 4.2b1
Patched version: 4.2b1
In PyYAML before 4.1, the yaml.load() API could execute arbitrary code. In other words, yaml.safe_load is not used.
Since postfix now asks us for the complete email over podop, which
includes the recipient-delimiter-and-what-follows not stripped, we need
to attempt to find both the verbatim localpart, as well as the localpart
stripped of the delimited part ….
Fixes#755
The init script was pushing an application context, which maked
flask.g global and persisted across requests. This was evaluated
to have a minimal security impact.
This explains/fixes #738: flask_wtf caches the csrf token in the
application context to have a single token per request, and only
sets the session attribute after the first generation.
Supporting multiple backends requires that specific sqlite
collations are not used, thus lowercase is applied to all non
case-sensitive columns. However, lowercasing the database requires
temporary disabling foreign key constraints, which is not possible
on SQLite and requires we specify the constraint names.
This migration specific to sqlite and postgresql drops every
constraint, whether it is named or not, and recreates all of them
with known names so we can later disable them.
In case of TLS_FLAVOR=[mail,cert], the user supplies their own certificates.
However, since nginx is not aware of changes to these files, it cannot
reload itself e.g. when the certs get renewed.
To solve this, let’s add a small daemon in the place of
`letsencrypt.py`, which uses a flexible file-watching framework and
reloads nginx in the case the certificates change ….
This will make network configuration and host based authentication
more robust, across different deployment platforms.
The options `RELAYNETS` and`POD_ADDRESS_RANGE` are kept for compatibility.
However, their usage have become optional.
To facilitate this, the default redirect at / can be disabled, even if
the default remains at redirecting to the webmailer.
The extensions are within the host scope and are read from
$ROOT/overrides/nginx/*.conf.
At present postfix checks this view for matches in the domain table and is used to accept/deny messages sent into it however it never checks for matches in the alternative table.
Fixes#718
In Docker Swarm mode the services listed below can get stuck in their start script, while they
are waiting for other services become available. Now, with HEALTHCHECK enabled, docker does not resolve
names of services that not pass HEALTHCHECK yet. Meaning that if one of the depenend services is not yet
available, it will create a chain of failing services.
The services below retry to resolve 100 time, with an average of 3.5 seconds. Hence, the --start-time
flag is now set at 350 seconds.
- dovecot (imap)
- postfix (smtp)
- rspamd (antispam)
- Specified seperated /health path in order to allow for healthcheck even if webmail and admin are not seletectd. This also allows healthchecking fom external services like DNS load balancers;
- Make curl not to fail on TLS because localhost is not included in the certificates.
Florent Daigniere
lub