- remove PASSWORD_SCHEME altogether
- introduce CREDENTIAL_ROUNDS
- migrate all old hashes to the current format
- auto-detect/enable all hash types that passlib supports
- upgrade passlib to 1.7.4 (see #1706: ldap_salted_sha512 support)
1765: Set sensible cookie flags on the admin app r=mergify[bot] a=nextgens
## What type of PR?
Bugfix
## What does this PR do?
It sets the right flags on the session cookie issued by the admin app.
This should probably be backported as the lack of secure flag on TLS-enabled setup is a high risk vulnerability.
SameSite is hardening / helps against CSRF on modern browsers
HTTPOnly is hardening / helps reduce the impact of XSS
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1763: show flash messages again r=mergify[bot] a=lub
## What type of PR?
bug-fix
## What does this PR do?
This basically restores the behaviour, that got removed in
ecdf0c25b3 during refactoring.
### Related issue(s)
- noticed it while reviewing #1756
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [-] In case of feature or enhancement: documentation updated accordingly
- [-] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1618: add OCSP stapling to nginx.conf r=mergify[bot] a=lub
It's not added in tls.conf, because apparently the mail ssl module
doesnt' support OCSP stapling.
https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling
^ exists
https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_stapling
^ missing
When the configured certificate doesn't have OCSP information, it'll
just log a warning during startup.
## What type of PR?
enhancement
## What does this PR do?
It enables OCSP stapling for the http server. OCSP stapling reduces roundtrips for the client and reduces load on OCSP responders.
### Related issue(s)
- fixes #1616
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1607: _FILE variables for Docker swarm secrets r=mergify[bot] a=lub
## What type of PR?
enhancement
## What does this PR do?
This PR enables usage of DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY to load these values from files instead of supplying them directly. That way it's possible to use Docker secrets.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
With `CHUNKING`set as a capability, nginx advertises this capability to
clients at a stage where the SMTP dialog does not seem to be forwarded
to the proxy-target (postfix) yet. Nginx' SMTP parser itself does not
support the `BDAT` command issued as part of a chunke-d dialog. This makes
Nginx respond with a `250 2.0.0 OK` and close the connection, after the
mail-data got sent by the client — without forwarding this to the
proxy-target.
With this, users mail can be lost.
Furthermore, when a user uses a sieve filter to forward mail, dovecot
sometimes chunks the forwarded mail when sending it through `front`.
These forwards then fail.
Removing `CHUNKING` from the capabilities fixes this behavior.
- yaml-import is now logged via schema.Logger
- iremoved relative imports - not used in other mailu modules
- removed develepment comments
- added Mailconfig.check method to check for duplicate domain names
- converted .format() to .format_map() where possible
- switched to yaml multiline dump for dkim_key
- converted dkim_key import from regex to string functions
- automatically unhide/unexclude explicitly specified attributes on dump
- use field order when loading to stabilize import
- fail when using 'hash_password' without 'password'
- fixed logging of dkim_key
- fixed pruning and deleting of lists
- modified error messages
- added debug flag and two verbosity levels
- idna.encode does not encode upper-case letters,
so .lower() has to be called on value not on result
- split email-address on '@' only once
- converted '*'.format(*) to f-strings
- added docstrings
- removed from_dict method
- code cleanup/style (list concat, exceptions, return&else, line-length)
- added TODO comments on possible future changes
- revived original config-update function for backwards compability
- renamed config-dump to config-export to be in line with config-import
- converted '*'.format(*) to f-strings
- converted string-concatenation to f-strings
renamed single letter variables (m => match)
renamed classmethod arguments to cls (model)
removed shadowing of variables (hash, context)
shortened unneeded lambda functions (id)
converted type ... is to isinstance(...)
removed unneded imports (flask)
Updated ConfigManager to only modify app.config and not replace it.
Swagger does not play well, when app.config is not a real dict and
it is not necessary to keep ConfigManager around after init.
Also added "API" flag to config (default: disabled).
1654: Ensure that the rendered file ends with newline in order to make `pos… r=mergify[bot] a=tremlin
## What type of PR?
Bugfix
## What does this PR do?
This fixes#1580
### Related issue(s)
- closes#1580
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.
This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.
This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
1611: Adds own server on port 80 for letsencrypt and redirect r=mergify[bot] a=elektro-wolle
## What type of PR?
Bugfix
## What does this PR do?
Handle letsencrypt route to `.well-known` by own server configuration within nginx.
### Related issue(s)
closes#1564
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Wolfgang Jung <w.jung@polyas.de>
1635: Add support for AUTH LOGIN authentication mechanism for relaying emai… r=mergify[bot] a=Diman0
…l via smart hosts.
## What type of PR?
Feature
## What does this PR do?
This PR adds support to postfix for AUTH LOGIN authentication mechanism. This enables using smart hosts which only offer AUTH LOGIN.
### Related issue(s)
- Auto close an issue like: closes#1633
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Because basically every MTA out there uses opportunistic TLS _in
the best case_, it's actually counter productive to use such strict
settings.
The alternative to a handshake error is often an unencrypted submission,
which is basically the opposite of what strict ssl_protocols and
ssl_ciphers tries to achieve.
Even big and established providers like Amazon SES are incompatible with the current
settings.
This reverts commit 2ddf46ad2b.
this enables usage of e.g. docker swarm secrets instead of exposing the
passwords directly via environment variables
just use DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY
- stabilized CommaSeparatedList by sorting values
- CommaSeparatedList can now handle list and set input
- from_dict now handles mapped keys
- from_dict now handles null values
- class Domain: handle dkim-key None correctly
- class User: delete obsolete keys after converting
- class Alias: now uses Email._dict_input
1557: Explicitly define ProxyFix options r=mergify[bot] a=brian-maloney
## What type of PR?
bug-fix
## What does this PR do?
This PR explicitly defines the options for the ProxyFix module, which fixes a regression in admin behind a reverse proxy.
### Related issue(s)
- #1309
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
This is a bugfix, so not doc changes, and it's an extremely minor change.
Co-authored-by: Brian Maloney <3286425+brian-maloney@users.noreply.github.com>
1538: Introduce environment variable to control dovecot full-text-search r=mergify[bot] a=tremlin
## What type of PR?
Enhancement
## What does this PR do?
In #1320 a full-text-search feature was enabled in Dovecot by default. Since this can have a big impact on performance, I think it's preferable to offer an option to disable the feature if it is not needed. This PR doesn't change the default behavior (FTS on).
### Related issue(s)
- #1320
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordinagly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
1478: Allow to enforce TLS for outbound r=mergify[bot] a=micw
using OUTBOUND_TLS_LEVEL=encrypt (default is 'may')
## What type of PR?
enhancement
## What does this PR do?
Add an option to postfix to enforce outbound traffic to be TLS encrypted.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1501: In setup/flavor, change DMARC RUA and RUF email default settings r=mergify[bot] a=ofthesun9
## What type of PR?
bug-fix
## What does this PR do?
This PR changes the default value used to set DMARC_RUA and DMARC_RUF:
DMARC_RUA and DMARC_RUF defaults will reuse the value defined for POSTMASTER,
instead of 'admin' as previously.
Please note that the setup tool doesn't allow (yet?) to define dmarc_rua nor dmarc_ruf, so the default value is indeed used for the time being.
### Related issue(s)
closes#1463
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1532: Replace SMPT with SMTP r=mergify[bot] a=dhoppe
1543: Disable Health checks on swarm mode r=mergify[bot] a=ofthesun9
ref: https://github.com/moby/moby/issues/35451
## What type of PR?
bug-fix
## What does this PR do?
Modify the docker-compose.yml template used by setup (swarm flavor) to disable Health checks on swarm mode for each service
### Related issue(s)
closes#1289
## Prerequistes
- [x] add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
Co-authored-by: Dennis Hoppe <github@debian-solutions.de>
1293: Remove `reject_unverified_recipient` from `smtpd_client_restrictions` r=mergify[bot] a=SunMar
## What type of PR?
Bug-fix
## What does this PR do?
It removes recipient verification, as it broke my catch-all address.
Fix for #1292, though I'm not sure if this is the right way to fix the issue. It was added in 175349a224.
### Related issue(s)
Fix for #1292 and a revert of 175349a224.
Co-authored-by: SunMar <SunMar@users.noreply.github.com>
1399: Remove SPF type SPF record #1394 r=mergify[bot] a=bladeswords
As mentioned in #1394 - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
## What type of PR?
Documentation
## What does this PR do?
Removes the recommendation to add a SPF RR for SPF records, as this is no longer RFC complaint and often causes issues to maintain two records.
### Related issue(s)
- closes#1394
## Prerequistes
None
1417: docker-compose exec needs a -T flag if no TTY is allocated r=mergify[bot] a=ofthesun9
This flag is missing in 00_create_users.sh and is failing the tests on travis arm architecture
## What type of PR?
This PR is an enhancement/bugfix needed to allow usage of travis to test and deploy on arm platform
Before the PR, tests are failing with the msg: "the input device is not a TTY"
## What does this PR do?
This PR add -T flag for the docker-compose exec occurences found in 00_create_users.sh
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1389: Prefer specific alias over wildcard, regardless of case r=mergify[bot] a=Nebukadneza
## What type of PR?
bug-fix
## What does this PR do?
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
## Notes
I realize that the if-hell down there isn’t nice. What it is, however, is quite clear and easy to read. I’m hoping that if anyone ever gets confused in the future, this will make the current behavior transparent. For me, that was more important than a minimal amount of statements/branches …
### Related issue(s)
closes#1387
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1392: Use environment variables for cert paths/names in nginx certwatcher r=mergify[bot] a=Nebukadneza
## What type of PR?
bug-fix
## What does this PR do?
Previously, nginx certwatcher would only react to the hardcoded paths. It should have
honored the enviroment variables that are used by config.py too for this.
### Related issue(s)
closes#903
## Prerequistes
- [x] no feature or enhancement
- [x] minor/internal change
Co-authored-by: Dario Ernst <github@kanojo.de>
1398: Update crypto to be modern and inline with tls.conf r=mergify[bot] a=bladeswords
Updated to match tls.conf and be aligned to more modern cryptographic standards and only use currently secure protocols and ciphers.
## What type of PR?
bugfix
## What does this PR do?
Update to use more modern cryptographic techniques
### Related issue(s)
- Addresses comment raised in 4f973f6
## Prerequistes
None
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
1393: Ignore newlines and comment-lines in postfix overrides r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
### Related issue(s)
closes#1098
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1296: fetchmail: print unhandled exceptions, but don't crash r=Nebukadneza a=Al2Klimov
fixes#1295
1322: Bump validators from 0.12.5 to 0.12.6 in /core/admin r=Nebukadneza a=dependabot[bot]
Bumps [validators](https://github.com/kvesteri/validators) from 0.12.5 to 0.12.6.
<details>
<summary>Changelog</summary>
*Sourced from [validators's changelog](https://github.com/kvesteri/validators/blob/master/CHANGES.rst).*
> 0.12.6 (2019-05-08)
> ^^^^^^^^^^^^^^^^^^^
>
> - Fixed domain validator for single character domains ([#118](https://github-redirect.dependabot.com/kvesteri/validators/issues/118), pull request courtesy kingbuzzman)
</details>
<details>
<summary>Commits</summary>
- See full diff in [compare view](https://github.com/kvesteri/validators/commits)
</details>
<br />
[](https://help.github.com/articles/configuring-automated-security-fixes)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).
</details>
1337: Add IPv6 to allow_nets r=Nebukadneza a=PhilRW
Roundcube was not connecting to sieve with IPv6 enabled.
Fixes#1336
1358: Add port to relay if it contains a colon r=Nebukadneza a=PhilRW
## What type of PR?
enhancement
## What does this PR do?
Allows relaying domains to non-standard SMTP ports by appending `:port` to the destination host/IP. E.g., `mx1.internal:2525`
### Related issue(s)
Closes#1357
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Philip Rosenberg-Watt <p.rosenberg-watt@cablelabs.com>
1395: Remove duplicate ports line r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
### Related issue(s)
closes#1079
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1396: Use pyyaml safe_load instead of load r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
Since load in unsafe (ref: https://msg.pyyaml.org/load),
switch the only occurrance of `yaml.load` that i could
find to safe_load.
### Related issue(s)
closes#1085
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1320: Add xapian full-text-search plugin to dovecot r=mergify[bot] a=Nebukadneza
## What type of PR?
Enhancement
## What does this PR do?
Currently we are not able to offer our users a FTS experience after the
demise of lucene due to unfixed coredumps with musl/alpine.
We now add lucene, the only remaining maintained small/lean FTS plugin
for dovecot. It is quite simple to add to our stack: A two-stage docker
build is used to compile the fts plugin in the first stage, and copy
over only the resulting plugin-artifact to the second stage, which is
our usual dovecot container. Configuration is also minimal.
There was a upstream issue where bodies were not able to be searched for subwords, but fortunately it was fixed quite quickly. We currently need to wait for a new release to use a stable tag in our `Dockerfile`.
### Related issue(s)
- https://github.com/Mailu/Mailu/pull/1176
- https://github.com/Mailu/Mailu/pull/1297
- https://github.com/Mailu/Mailu/issues/751
- **Upstream-issues which is the cause for the `TODO` in the `Dockerfile`**: https://github.com/grosjo/fts-xapian/issues/32
## Prerequistes
- [ ] Wait for upstream to prepare new release after https://github.com/grosjo/fts-xapian/issues/32 — so that we can use a stable tag in our `Dockerfile`
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
As mentioned in #1394 - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
closes#1098
Previously, nginx certwatcher would only react to the hardcoded paths. It should have
honored the enviroment variables that are used by config.py too for this.
closes#903
1359: Refactor the rate limiting code r=mergify[bot] a=kaiyou
## What type of PR?
Enhancement
## What does this PR do?
Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.
Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.
### Related issue(s)
No specific issue.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
1321: Upgrading nginx TLS configuration r=mergify[bot] a=radtkedev
## What type of PR?
Enhancement
## What does this PR do?
Upgrades the TLS protocols and ciphers to the recommended "Intermediate Configuration" and sets the "Old Configuration" for port 25 (SMTP) based on the [Mozilla SSL Configuration Generator](https://ssl-config.mozilla.org/) and adjusted for the nginx mail proxy.
Co-authored-by: Tom Radtke <tom@radtke.dev>
1349: Add support for SRS, related to #328 r=mergify[bot] a=kaiyou
## What type of PR?
Feature
## What does this PR do?
It implements SRS using a Python SRS library.
### Related issue(s)
- closes#328
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
closes#1387
Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.
Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.
1278: Limiter implementation r=kaiyou a=micw
## What type of PR?
(Feature, enhancement, bug-fix, documentation)
## What does this PR do?
Adds a custom limter based on the "limits" lirary that counts up on failed auths only
### Related issue(s)
- closes#1195
- closes#634
## Prerequistes
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: micw <michael@wyraz.de>
1316: Fix the encoding of incoming user email and password r=mergify[bot] a=kaiyou
## What type of PR?
Bug fix
## What does this PR do?
As described in the changes, RFC2616 states that header should be considered ISO8859-1 in HTTP, which obviously nginx does not really care about when forwarding the password from SMTP authentication to the backend. Hence, we need to encode-then-decode passwords to get the proper value in case a special char is in there.
### Related issue(s)
- This fixes#1139
- This is also related to #1281
- This is also related to #1139
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
Currently we are not able to offer our users a FTS experience after the
demise of lucene due to unfixed coredumps with musl/alpine.
We now add lucene, the only remaining maintained small/lean FTS plugin
for dovecot. It is quite simple to add to our stack: A two-stage docker
build is used to compile the fts plugin in the first stage, and copy
over only the resulting plugin-artifact to the second stage, which is
our usual dovecot container. Configuration is also minimal.
1299: Don't remove the address extension in postfix r=mergify[bot] a=RobertMe
## What type of PR?
Bugfix
## What does this PR do?
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.
This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.
### Related issue(s)
Fixes#982
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Robert Meijers <robert.meijers@gmail.com>
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.
This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.
Fixes#982
985: Permit raspberry pi (and other architectures) builds r=mergify[bot] a=abondis
## What type of PR?
Enhancement
## What does this PR do?
Add an option to select base images and permit building for different CPU architectures.
### Related issue(s)
N/A
## Prerequistes
- [X] documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Aurélien Bondis <aurelien.bondis@gmail.com>
Co-authored-by: Aurelien <aurelien.bondis@gmail.com>
1213: Bump werkzeug from 0.15.2 to 0.15.3 in /core/admin r=mergify[bot] a=dependabot[bot]
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 0.15.2 to 0.15.3.
<details>
<summary>Release notes</summary>
*Sourced from [werkzeug's releases](https://github.com/pallets/werkzeug/releases).*
> ## 0.15.3
> * Blog: https://palletsprojects.com/blog/werkzeug-0-15-3-released/
> * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-3
>
</details>
<details>
<summary>Changelog</summary>
*Sourced from [werkzeug's changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst).*
> Version 0.15.3
> --------------
>
> Released 2019-05-14
>
> - Properly handle multi-line header folding in development server in
> Python 2.7. (:issue:`1080`)
> - Restore the ``response`` argument to :exc:`~exceptions.Unauthorized`.
> (:pr:`1527`)
> - :exc:`~exceptions.Unauthorized` doesn't add the ``WWW-Authenticate``
> header if ``www_authenticate`` is not given. (:issue:`1516`)
> - The default URL converter correctly encodes bytes to string rather
> than representing them with ``b''``. (:issue:`1502`)
> - Fix the filename format string in
> :class:`~middleware.profiler.ProfilerMiddleware` to correctly handle
> float values. (:issue:`1511`)
> - Update :class:`~middleware.lint.LintMiddleware` to work on Python 3.
> (:issue:`1510`)
> - The debugger detects cycles in chained exceptions and does not time
> out in that case. (:issue:`1536`)
> - When running the development server in Docker, the debugger security
> pin is now unique per container.
</details>
<details>
<summary>Commits</summary>
- [`9b1123a`](9b1123a779) release version 0.15.3
- [`00bc43b`](00bc43b167) unique debugger pin in Docker containers
- [`2cbdf2b`](2cbdf2b022) Merge pull request [#1542](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1542) from asottile/exceptions_arent_always_hashable
- [`0e669f6`](0e669f6be5) Fix unhashable exception types
- [`bdc17e4`](bdc17e4cd1) Merge pull request [#1540](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1540) from pallets/break-tb-cycle
- [`44e38c2`](44e38c2985) break cycle in chained exceptions
- [`777500b`](777500b646) Merge pull request [#1518](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1518) from NiklasMM/fix/1510_lint-middleware-python3-compa...
- [`e00c7c2`](e00c7c2ced) Make LintMiddleware Python 3 compatible and add tests
- [`d590cc7`](d590cc7cf2) Merge pull request [#1539](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1539) from pallets/profiler-format
- [`0388fc9`](0388fc95e6) update filename_format for ProfilerMiddleware.
- Additional commits viewable in [compare view](https://github.com/pallets/werkzeug/compare/0.15.2...0.15.3)
</details>
<br />
[](https://help.github.com/articles/configuring-automated-security-fixes)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).
</details>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1211: Split HOST_ANTISPAM in HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
Fixes#1190 by separating HOST_ANTISPAM into HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI
### Related issue(s)
- closes#1190
- closes#1150
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1198: Enable access log of admin service only for log levels of INFO and finer r=muhlemmer a=micw
## What type of PR?
bug fix
## What does this PR do?
### Related issue(s)
- closes#1197
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1204: Add initial admin account to kubernetes example r=muhlemmer a=micw
## What type of PR?
documentation
## What does this PR do?
Add INITIAL_ADMIN_* example to kubernetes configmap.yaml
### Related issue(s)
- closes#1201
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- docs example only
1207: Add Japanese translation r=muhlemmer a=IchikawaYukko
## What type of PR?
Translation
## What does this PR do?
Provide completed Japanese translation.
### Related issue(s)
None
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [*] In case of feature or enhancement: documentation updated accordingly
- [*] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1208: Persist mailqueue r=muhlemmer a=micw
## What type of PR?
bug-fix
## What does this PR do?
Makes postfix mailqueue presistent (for docker, swarm and kubernetes)
### Related issue(s)
- closes#1161
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: Michael Wyraz <michael.wyraz@evermind.de>
Co-authored-by: U-COREI3-3120M\市川ゆり子 <ichikawayuriko@yahoo.co.jp>
Co-authored-by: micw <michael@wyraz.de>
1181: Update to address issue #1178 (HTTP headers) r=muhlemmer a=bladeswords
This change should remove the duplicate `x-xss-protection` header and also the `x-powered-by` header. Hopefully a pull request to main is appropriate, but may be worth back porting to 1.7.
Tested config by modifying live 1.7 nginx config and reloading. Has had the desired outcome of removing the headers.
```/etc/nginx # nginx -t -c /etc/nginx/nginx.conf
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
/etc/nginx # nginx -s reload
```
These steps were based on:
- https://serverfault.com/questions/928912/how-do-i-remove-a-server-added-header-from-proxied-location
- https://serverfault.com/questions/929571/overwrite-http-headers-comming-back-from-a-web-application-server-proxied-in-ngi
- http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_hide_header
## What type of PR?
Enhancement
## What does this PR do?
Removes duplicate and unneeded headers. See issue #1178
### Related issue(s)
- issue: #1178
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ X ] In case of feature or enhancement: documentation updated accordingly
- [ X ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
1183: Fix rspamd-learn when moving mail from/to junk folder r=mergify[bot] a=Nebukadneza
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
## What type of PR?
bug-fix
## What does this PR do?
Instead of relying on dovecot passing our environment cleanly to sieve-called scripts, this explicitly injects the antispam ip/port into the spam/ham scripts used when moving files from/to the spam-folder. This required some management of the files, such as setting proper permissions after the jinja-run.
### Related issue(s)
fixes#1177
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
1196: Change default password scheme to PBKDF2 (#1194) r=mergify[bot] a=hoellen
## What type of PR?
enhancement
## What does this PR do?
This PR change the default password scheme to PBKDF2. It is already changed in some places (e.g. [docs/compose/.env](https://github.com/Mailu/Mailu/blob/master/docs/compose/.env#L142)).
### Related issue(s)
closes#1194
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1180: Add a title to Mailu-Admin pages r=mergify[bot] a=Niduroki
## What type of PR?
Enhancement
## What does this PR do?
This simply adds a title to be displayed in the browser tab for Mailu-Admin pages.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Considering this is only a one-line change I don't think these apply.
-----
I made the title `Mailu-Admin - SITENAME` – if this seems unfitting feel free to change it to something different.
Co-authored-by: Niduroki <chris@niduroki.net>
1158: Use nginx for kubernetes ingress r=kaiyou a=micw
## What type of PR?
enhancement
## What does this PR do?
Currently, kubernetes uses a complex ingress setting which is not portable across different ingress controllers. This PR simplifies the ingress and delegates everythins special to Mailu to the front container,
### Related issue(s)
- closes#1121
- closes#1117
- closes#1021
- closes#1045
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog]
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
1114: Resolve HOST to ADDRESS only if ADDRESS is not already set r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
~Makes the rsolving from hosts to ips at startup configurable~
I rewrote the pull request after #940 was merged. Now it resolves HOSTs to ADDRESSes only of ADDRESSes are not already set. So on kubernetes we can jsut set the address and have working service discovery.
### Related issue(s)
- closes#1113
## Prerequistes
~Minor change, backward compatible~
Changelog will be added
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1130: Remove unnecessary host variable assignments r=mergify[bot] a=kaiyou
## What type of PR?
Bug fix
## What does this PR do?
### Related issue(s)
- Closes#1126
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1132: Release Mailu 1.7 r=mergify[bot] a=kaiyou
## What type of PR?
Release
## What does this PR do?
It provides the changelog and migration notes for release 1.7.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
940: Fix host variables r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
- Fix#884
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
883: Admin create user enhancement r=mergify[bot] a=cr1st1p
## What type of PR?
Enhancement
## What does this PR do?
It allows the admin docker image to also create the admin user.
The idea is that in my kubernetes setup, I do not want to manually do anything, as such, I need a way for the admin user to also be created automatically without me getting inside the pod.
So I had to change the manage.py function that creates the user to allow different 'modes' (me, I'll be using 'ifmissing') and also start.py to call that functionality if appropriate environment variables are present.
So now, in my Deployment, I add 3 more environment variables and I get the admin user created, IF not already present.
### Related issue(s)
none?
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: cristi <cristi.posoiu@gmail.com>
Co-authored-by: cr1st1p <cristi.posoiu@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
1088: Support domain literals (fix#1087) r=mergify[bot] a=hoellen
## What type of PR?
bug-fix
## What does this PR do?
This PR adds error handling for idna enocding. With telnet you now get a "Bad sender address syntax" message.
```
> telnet mail.example.com 25
Connected to example.com.
Escape character is '^]'.
220 mail.example.com ESMTP ready
EHLO dummy.example.com
250-mail.example.com
250 STARTTLS
MAIL FROM: does-not-exist@[116.203.165.200]
250 2.0.0 OK
RCPT TO: some-user@example.com
501 5.1.7 Bad sender address syntax
Connection closed by foreign host.
```
### Related issue(s)
fix#1087
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1052: Upgrade alpine to 3.10 r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement / bug-fix
## What does this PR do?
Upgrade the alpine base image to 3.10 and clean up ensuing problems. Also directly uses postfix foreground-running with stdout logging.
### Related issue(s)
closes#1049closes#1051
Note: This is a duplicate effort of #1050#1039 …
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
With these images, I have tested manually:
- Email receive to user
- on main domain
- on additional domain
- via an alias on main-domain
- via an alias on additional domain
- via catchall
- Email reject
- of eicar testfiles
- Email sending
- Fetchmail from legacy POP
- Front LE certificates
- Simple overriding for postfix (only postfix.cf), nginx, dovecot, rspamd
- Creating a CalDAV calendar and CardDAV entry using davx5
I have not (yet) tested:
- certdumper
- databases other than sqlite
Todo:
- [x] Get rid of podop at `git+https://github.com/Nebukadneza/Podop.git@fix_py37` once https://github.com/Mailu/Podop/pull/6 is merged
- [x] Bend Mailustart back to original repository
- [x] Test more (? also help wanted ?)
Co-authored-by: Thomas Sänger <thomas@gecko.space>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Reto Glauser <git@blinkeye.ch>
Florent Daigniere
lub