1754: centralize Webmail authentication behind the admin panel (SSO) r=mergify[bot] a=nextgens
## What type of PR?
Enhancement: it centralizes the authentication of webmails to the admin interface.
## What does this PR do?
It implements the glue required for webmails to do SSO using the admin interface.
One of the main advantages of centralizing things this way is that it reduces significantly the attack surface available to an unauthenticated attacker (no webmail access until there is a valid Flask session).
Others include the ability to implement 2FA down the line and rate-limit things as required.
### Related issue(s)
- #783
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1758: Implement a simpler credential cache (alternative to #1755) r=mergify[bot] a=nextgens
## What type of PR?
Feature: it implements a credential cache to speedup authentication requests.
## What does this PR do?
Credentials are stored in cold-storage using a slow, salted/iterated hash function to prevent offline bruteforce attacks. This creates a performance bottleneck for no valid reason (see the
rationale/long version on https://github.com/Mailu/Mailu/issues/1194#issuecomment-762115549).
The new credential cache makes things fast again.
This is the simpler version of #1755 (with no new dependencies)
### Related issue(s)
- close#1411
- close#1194
- close#1755
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1776: optimize generation of transport nexthop r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix and enhancement.
## What does this PR do?
Possibly there should be more input validation when editing a relay, but for now this tries to make the best out of the existing "smtp" attribute while maintaining backwards compatibility. When relay is empty, the transport's nexthop is the MX of the relayed domain to fix#1588
```
RELAY NEXTHOP TRANSPORT
empty use MX of relay domain smtp:domain
:port use MX of relay domain and use port smtp:domain:port
target resolve A/AAAA of target smtp:[target]
target:port resolve A/AAAA of target and use port smtp:[target]:port
mx:target resolve MX of target smtp:target
mx:target:port resolve MX of target and use port smtp:target:port
lmtp:target resolve A/AAAA of target lmtp:target
lmtp:target:port resolve A/AAAA of target and use port lmtp:target:port
target can also be an IPv4 or IPv6 address (an IPv6 address must be enclosed in []: [2001:DB8::]).
```
When there is proper input validation and existing database entries are migrated this function can be made much shorter again.
### Related issue(s)
- closes#1588
- closes#1815
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1831: Fix roundcube database env configuration r=mergify[bot] a=parisni
## What type of PR?
bug-fix
## What does this PR do?
Both roundcube and mailu admin website can be backed by postgres/mysql. Before this PR, the `DB_FLAVOR` is shared by both services. However, the other roundcube dedicated DB params are prefixed with `ROUNDCUBE_`. (eg: `ROUNDCUBE_DB_NAME`)
There is no reason to share the DB_FLAVOR for both: This PR makes them be considered independently to make things clear and avoid bugs.
Also, the roundcube_db_flavor and db_flavor are made separated in this PR. However for simplicity, the template generator bind them : roundcube_db_flavor = db_flavor. This makes the template generator UI more simple. I considered most of the time people want to have both roundcube and mailu share the same RDBMS.
Also, AFAIK the internal postgresql service is deprecated and will be removed in 1.9. This is why this PR does not integrate roundcube in postgres when the internal DB is choosen: in case of internal postgres, the roundcube is backed with sqlite.
Both documentation and setup website have been updated accordingly.
### Related issue(s)
- Auto close an issue like: closes#1648#1471
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: parisni <nicolas.paris@riseup.net>
Co-authored-by: Nicolas Paris <nicolas.paris@riseup.net>
1836: Test ci parallel r=Diman0 a=Diman0
## What type of PR?
enhancement
## What does this PR do?
Changes CI workflow to run all tests in parellel. After performing some tests (see #1830 ), I determined that using actions/cache to only cache a tar.gz. file with all build images and use this for all parallel tests is the fasted solution.
With Travis builds took ~30 minutes. Now each build runs for a maximum of 20 minutes (bors test and merge on master).
Bors r+ runs take about ~16/17 minutes.
### Related issue(s)
- Auto close an issue like: closes#1830
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
1746: DNS records for client autoconfiguration (RFC6186) r=Diman0 a=nextgens
## What type of PR?
Feature
## What does this PR do?
Add instructions on how to configure rfc6186 DNS records for client autoconfiguration
### Related issue(s)
- #224
- #498
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1694: update compression algorithms for current dovecot r=nextgens a=lub
## What type of PR?
enhancement
## What does this PR do?
This adds additional compression algorithms in accordance with
https://doc.dovecot.org/configuration_manual/zlib_plugin/
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1649: Update docs/reverse.rst with Traefik v2+ info r=mergify[bot] a=patryk-tech
## What type of PR?
Documentation
## What does this PR do?
Adds information about using Traefik v2+ as a reverse proxy.
### Related issue(s)
Closes#1503
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1673: Remove rspamd unused env var from start script r=mergify[bot] a=cbachert
## What type of PR?
Cleanup
## What does this PR do?
Remove unused environment variable FRONT_ADDRESS in rspamd. FRONT_ADDRESS references were removed with commit 8172f3e in PR #727 like mentioned in chat https://matrix.to/#/!MINuyJjJSrfowljYCK:tedomum.net/$160401946364NGNmI:imninja.net?via=huisman.xyz&via=matrix.org&via=imninja.net
```
Mailu$ grep -r "FRONT_ADDRESS" core/rspamd/
core/rspamd/start.py:os.environ["FRONT_ADDRESS"] = system.get_host_address_from_environment("FRONT", "front")
```
### Related issue(s)
N/A
## Prerequistes
- [x] Documentation updated accordingly: No documentation to update
- [x] Add to changelog: Minor change
Co-authored-by: Patryk Tech <git@patryk.tech>
Co-authored-by: cbachert <cbachert@users.noreply.github.com>
1760: Security updates to postgresql r=mergify[bot] a=WebSpider
## What type of PR?
Security update
## What does this PR do?
It fixes vulnerabilities in the sudo package in the postgresql optional container documented in
CVE-2021-23240, CVE-2021-3156 and CVE-2021-23239
### Related issue(s)
None
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Nils Vogels <n.vogels@aves-it.nl>
1785: Fix bug #1660 (don't replace nested headers) r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Don't replace nested headers (typically in forwarded/attached emails). This will ensure we don't break cryptographic signatures.
### Related issue(s)
- close#1660
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1783: Switch to server-side sessions r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It simplifies session management.
- it ensures that sessions will eventually expire (*)
- it implements some mitigation against session-fixation attacks
- it switches from client-side to server-side sessions (in Redis)
It doesn't prevent us from (re)-implementing a "remember_me" type of feature if that's considered useful by some.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1610: add option to enforce inbound starttls r=mergify[bot] a=lub
## What type of PR?
Feature
## What does this PR do?
It implements a check in the auth_http handler to check for Auth-SSL == on and otherwise returns a 530 starttls error.
If INBOUND_TLS_ENFORCE is not set the behaviour is still the same as before, so existing installations should be unaffected.
Although there is a small difference to e.g. smtpd_tls_security_level of Postfix.
Postfix already throws a 530 after mail from, but this solution only throws it after rcpt to. auth_http is only the request after rcpt to, so it's not possible to do it earlier.
### Related issue(s)
#1328 is kinda related, although this PR doesn't solve the issue that the headers will still display ESMTP instead of ESMTPS
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1638: Remove the username from the milter_headers r=mergify[bot] a=githtz
Rspamd adds the name of the authenticated user by default. Setting add_smtp_user to false prevents the login to be leaked.
## What type of PR?
Enhancement
## What does this PR do?
This PR prevents the user login to be leaked in sent emails (for example using an alias)
### Related issue(s)
Closes https://github.com/Mailu/Mailu/issues/1465
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: anrc <15327800+githtz@users.noreply.github.com>
1711: fix typo in faq.rst r=Diman0 a=tomwojcik
1712: Add details for postfix-overrides r=mergify[bot] a=sholl
## What type of PR?
Documentation clarification
## What does this PR do?
### Related issue(s)
this clarifies the FAQ about overrides and fixes#1628
Co-authored-by: Tomasz Wójcik <tomwojcik@users.noreply.github.com>
Co-authored-by: Stephan Holl <stephan@holl-land.de>
Co-authored-by: Stephan Holl <1610827+sholl@users.noreply.github.com>
1618: add OCSP stapling to nginx.conf r=mergify[bot] a=lub
It's not added in tls.conf, because apparently the mail ssl module
doesnt' support OCSP stapling.
https://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_stapling
^ exists
https://nginx.org/en/docs/mail/ngx_mail_ssl_module.html#ssl_stapling
^ missing
When the configured certificate doesn't have OCSP information, it'll
just log a warning during startup.
## What type of PR?
enhancement
## What does this PR do?
It enables OCSP stapling for the http server. OCSP stapling reduces roundtrips for the client and reduces load on OCSP responders.
### Related issue(s)
- fixes #1616
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1607: _FILE variables for Docker swarm secrets r=mergify[bot] a=lub
## What type of PR?
enhancement
## What does this PR do?
This PR enables usage of DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY to load these values from files instead of supplying them directly. That way it's possible to use Docker secrets.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: lub <git@lubiland.de>
1669: Fix extract_host_port port separation r=mergify[bot] a=cbachert
Regex quantifier should be lazy to make port separation work.
## What type of PR?
bug-fix
## What does this PR do?
The "extract_host_port" function in admin/mailu/internal/nginx.py and optional/fetchmail/fetchmail.py is not actually separating host and port due to the `(.*)` part of the regex being too generous. Lazy quantifier `(.*?)` allows the other capturing groups to match.
### Related issue(s)
- No issue raised for this
## Prerequistes
- [x] Documentation updated accordingly: N/A, bug-fix
- [x] Add [changelog] entry file: Added towncrier newsfragment with second commit
1672: mark radio buttons in setup utility as required r=mergify[bot] a=lub
## What type of PR?
bug-fix
## What does this PR do?
mark radio buttons in setup utility as required
Otherwise it's possible to submit the form without selecting e.g. any
flavor, which would need additional handling on the server side.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: cbachert <cbachert@users.noreply.github.com>
Co-authored-by: lub <git@lubiland.de>
1592: Add documentation for the web administration gui. r=mergify[bot] a=Diman0
## What type of PR?
Documentation
## What does this PR do?
This PR adds the section Web Administration Interface to the documentation site which completely documents all available settings in the web administration interface.
### Related issue(s)
- Closes issue #1590 (please close this issue for me)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ Done ] In case of feature or enhancement: documentation updated accordingly
- [ Done ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
1611: Adds own server on port 80 for letsencrypt and redirect r=mergify[bot] a=elektro-wolle
## What type of PR?
Bugfix
## What does this PR do?
Handle letsencrypt route to `.well-known` by own server configuration within nginx.
### Related issue(s)
closes#1564
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Wolfgang Jung <w.jung@polyas.de>
1526: Use Radicale 3.x for webdav service r=mergify[bot] a=ofthesun9
- remove -f flag in Dockerfile CMD
- remove deprecated daemon and dns_lookup settings from radicale.conf
- move realm setting from [server] to [auth] in radicale.conf
- add newsfragment
## What type of PR?
Miscellaneous
## What does this PR do?
Modifications in Dockerfile and radicale.conf to get Radicale 3.0 service building properly.
Functional tests would be needed before merge.
### Related issue(s)
- closes#1512
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1538: Introduce environment variable to control dovecot full-text-search r=mergify[bot] a=tremlin
## What type of PR?
Enhancement
## What does this PR do?
In #1320 a full-text-search feature was enabled in Dovecot by default. Since this can have a big impact on performance, I think it's preferable to offer an option to disable the feature if it is not needed. This PR doesn't change the default behavior (FTS on).
### Related issue(s)
- #1320
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordinagly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Thomas Rehn <thomas.rehn@initos.com>
1478: Allow to enforce TLS for outbound r=mergify[bot] a=micw
using OUTBOUND_TLS_LEVEL=encrypt (default is 'may')
## What type of PR?
enhancement
## What does this PR do?
Add an option to postfix to enforce outbound traffic to be TLS encrypted.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1501: In setup/flavor, change DMARC RUA and RUF email default settings r=mergify[bot] a=ofthesun9
## What type of PR?
bug-fix
## What does this PR do?
This PR changes the default value used to set DMARC_RUA and DMARC_RUF:
DMARC_RUA and DMARC_RUF defaults will reuse the value defined for POSTMASTER,
instead of 'admin' as previously.
Please note that the setup tool doesn't allow (yet?) to define dmarc_rua nor dmarc_ruf, so the default value is indeed used for the time being.
### Related issue(s)
closes#1463
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1532: Replace SMPT with SMTP r=mergify[bot] a=dhoppe
1543: Disable Health checks on swarm mode r=mergify[bot] a=ofthesun9
ref: https://github.com/moby/moby/issues/35451
## What type of PR?
bug-fix
## What does this PR do?
Modify the docker-compose.yml template used by setup (swarm flavor) to disable Health checks on swarm mode for each service
### Related issue(s)
closes#1289
## Prerequistes
- [x] add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
Co-authored-by: Dennis Hoppe <github@debian-solutions.de>
- remove ==2.1.12 in Dockerfile pip3 install radicale
- remove -f flag in Dockerfile CMD
- remove deprecated daemon and dns_lookup settings from radicale.conf
- move realm setting from [server] to [auth] in radicale.conf
- add newsfragment
1298: Added carddav-plugin for roundcube webmail r=ofthesun9 a=sholl
## Feature
This PR enables the carddav contacts plugin for integration remote contact-repositories based on CardDAV.
## What does this PR do?
This PR enables the carddav contacts plugin for integration remote contact-repositories based on CardDAV.
### Related issue(s)
- Related #1230, at least for CardDAV.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly (not needed since the plugins of webmail is not mentioned in the docs.)
- [x] Changelog-entry added
Co-authored-by: Stephan Holl <stephan@holl-land.de>
1487: Fix postfix queue permissions r=mergify[bot] a=ofthesun9
## What type of PR?
bug-fix (by anticipation of a potential change in base image)
## What does this PR do?
We need to check /queue permissions before starting postfix.
In case of postfix/postdrop uid/gid change, postfix would fail to start
### Related issue(s)
closes#1486
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1380: [Roundcube] DKIM sign message delivery reports r=mergify[bot] a=TheLegend875
This PR enables the From header for message delivery reports in Roundcube.
This ensures that the message delivery report is DKIM signed and therefore not not blocked or considered spam by receiving mailservers.
Co-authored-by: TheLegend875 <40040530+TheLegend875@users.noreply.github.com>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1399: Remove SPF type SPF record #1394 r=mergify[bot] a=bladeswords
As mentioned in #1394 - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
## What type of PR?
Documentation
## What does this PR do?
Removes the recommendation to add a SPF RR for SPF records, as this is no longer RFC complaint and often causes issues to maintain two records.
### Related issue(s)
- closes#1394
## Prerequistes
None
1417: docker-compose exec needs a -T flag if no TTY is allocated r=mergify[bot] a=ofthesun9
This flag is missing in 00_create_users.sh and is failing the tests on travis arm architecture
## What type of PR?
This PR is an enhancement/bugfix needed to allow usage of travis to test and deploy on arm platform
Before the PR, tests are failing with the msg: "the input device is not a TTY"
## What does this PR do?
This PR add -T flag for the docker-compose exec occurences found in 00_create_users.sh
Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
1389: Prefer specific alias over wildcard, regardless of case r=mergify[bot] a=Nebukadneza
## What type of PR?
bug-fix
## What does this PR do?
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
## Notes
I realize that the if-hell down there isn’t nice. What it is, however, is quite clear and easy to read. I’m hoping that if anyone ever gets confused in the future, this will make the current behavior transparent. For me, that was more important than a minimal amount of statements/branches …
### Related issue(s)
closes#1387
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1393: Ignore newlines and comment-lines in postfix overrides r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement
## What does this PR do?
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
### Related issue(s)
closes#1098
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <github@kanojo.de>
1296: fetchmail: print unhandled exceptions, but don't crash r=Nebukadneza a=Al2Klimov
fixes#1295
1322: Bump validators from 0.12.5 to 0.12.6 in /core/admin r=Nebukadneza a=dependabot[bot]
Bumps [validators](https://github.com/kvesteri/validators) from 0.12.5 to 0.12.6.
<details>
<summary>Changelog</summary>
*Sourced from [validators's changelog](https://github.com/kvesteri/validators/blob/master/CHANGES.rst).*
> 0.12.6 (2019-05-08)
> ^^^^^^^^^^^^^^^^^^^
>
> - Fixed domain validator for single character domains ([#118](https://github-redirect.dependabot.com/kvesteri/validators/issues/118), pull request courtesy kingbuzzman)
</details>
<details>
<summary>Commits</summary>
- See full diff in [compare view](https://github.com/kvesteri/validators/commits)
</details>
<br />
[](https://help.github.com/articles/configuring-automated-security-fixes)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).
</details>
1337: Add IPv6 to allow_nets r=Nebukadneza a=PhilRW
Roundcube was not connecting to sieve with IPv6 enabled.
Fixes#1336
1358: Add port to relay if it contains a colon r=Nebukadneza a=PhilRW
## What type of PR?
enhancement
## What does this PR do?
Allows relaying domains to non-standard SMTP ports by appending `:port` to the destination host/IP. E.g., `mx1.internal:2525`
### Related issue(s)
Closes#1357
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Philip Rosenberg-Watt <p.rosenberg-watt@cablelabs.com>
1268: Roundcube db r=Nebukadneza a=micw
## What type of PR?
feature
## What does this PR do?
- makes roundcube work with mysql
- runs db init/upgrade scripts on startup
- redirects roundcube logs to stdout
### Related issue(s)
- preparations to solve #1226
- closes#1157 (side effect ;-) )
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: micw <michael@wyraz.de>
1320: Add xapian full-text-search plugin to dovecot r=mergify[bot] a=Nebukadneza
## What type of PR?
Enhancement
## What does this PR do?
Currently we are not able to offer our users a FTS experience after the
demise of lucene due to unfixed coredumps with musl/alpine.
We now add lucene, the only remaining maintained small/lean FTS plugin
for dovecot. It is quite simple to add to our stack: A two-stage docker
build is used to compile the fts plugin in the first stage, and copy
over only the resulting plugin-artifact to the second stage, which is
our usual dovecot container. Configuration is also minimal.
There was a upstream issue where bodies were not able to be searched for subwords, but fortunately it was fixed quite quickly. We currently need to wait for a new release to use a stable tag in our `Dockerfile`.
### Related issue(s)
- https://github.com/Mailu/Mailu/pull/1176
- https://github.com/Mailu/Mailu/pull/1297
- https://github.com/Mailu/Mailu/issues/751
- **Upstream-issues which is the cause for the `TODO` in the `Dockerfile`**: https://github.com/grosjo/fts-xapian/issues/32
## Prerequistes
- [ ] Wait for upstream to prepare new release after https://github.com/grosjo/fts-xapian/issues/32 — so that we can use a stable tag in our `Dockerfile`
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
To make postfix override files understandable and readable, users may
want to insert empty newlines and #-commented lines in their postfix
override files too. This will now ignore such bogus-lines and not send
them to `postconf`, which produced ugly errors in the past.
closes#1098
1349: Add support for SRS, related to #328 r=mergify[bot] a=kaiyou
## What type of PR?
Feature
## What does this PR do?
It implements SRS using a Python SRS library.
### Related issue(s)
- closes#328
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.
closes#1387
1278: Limiter implementation r=kaiyou a=micw
## What type of PR?
(Feature, enhancement, bug-fix, documentation)
## What does this PR do?
Adds a custom limter based on the "limits" lirary that counts up on failed auths only
### Related issue(s)
- closes#1195
- closes#634
## Prerequistes
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: micw <michael@wyraz.de>
1308: Use redis 5 on k8s & add selector r=mergify[bot] a=der-eismann
## What type of PR?
Enhancement
## What does this PR do?
This PR is updating Redis to version 5 in the kubernetes manifests. It is already used in the compose and swarm files, so I don't expect any incompatibilities. There is no necessary migration, you just can't go back.
In addition I added a selector to the manifest and applied a consistent formatting.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Philipp Trulson <philipp@trulson.de>
1241: Change extensions/v1beta to apps/v1 to be compliant with Kubernetes 1.16 r=mergify[bot] a=WebSpider
## What type of PR?
Enhancement
## What does this PR do?
Changes to Kubernetes YAML files to be able to use this in Kubernetes 1.16
### Related issue(s)
- Fixes#1237
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- N/A In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: WebSpider <webspider@bitlair.nl>
Co-authored-by: micw <michael@wyraz.de>
1215: Allow specifying the traefik version for cert dumping r=mergify[bot] a=timoschwarzer
## What type of PR?
Enhancement
## What does this PR do?
### Related issue(s)
- #1011
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Timo Schwarzer <me@timoschwarzer.com>
985: Permit raspberry pi (and other architectures) builds r=mergify[bot] a=abondis
## What type of PR?
Enhancement
## What does this PR do?
Add an option to select base images and permit building for different CPU architectures.
### Related issue(s)
N/A
## Prerequistes
- [X] documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Aurélien Bondis <aurelien.bondis@gmail.com>
Co-authored-by: Aurelien <aurelien.bondis@gmail.com>
1211: Split HOST_ANTISPAM in HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
Fixes#1190 by separating HOST_ANTISPAM into HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI
### Related issue(s)
- closes#1190
- closes#1150
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1198: Enable access log of admin service only for log levels of INFO and finer r=muhlemmer a=micw
## What type of PR?
bug fix
## What does this PR do?
### Related issue(s)
- closes#1197
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1204: Add initial admin account to kubernetes example r=muhlemmer a=micw
## What type of PR?
documentation
## What does this PR do?
Add INITIAL_ADMIN_* example to kubernetes configmap.yaml
### Related issue(s)
- closes#1201
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- docs example only
1207: Add Japanese translation r=muhlemmer a=IchikawaYukko
## What type of PR?
Translation
## What does this PR do?
Provide completed Japanese translation.
### Related issue(s)
None
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [*] In case of feature or enhancement: documentation updated accordingly
- [*] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1208: Persist mailqueue r=muhlemmer a=micw
## What type of PR?
bug-fix
## What does this PR do?
Makes postfix mailqueue presistent (for docker, swarm and kubernetes)
### Related issue(s)
- closes#1161
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: Michael Wyraz <michael.wyraz@evermind.de>
Co-authored-by: U-COREI3-3120M\市川ゆり子 <ichikawayuriko@yahoo.co.jp>
Co-authored-by: micw <michael@wyraz.de>
1183: Fix rspamd-learn when moving mail from/to junk folder r=mergify[bot] a=Nebukadneza
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
## What type of PR?
bug-fix
## What does this PR do?
Instead of relying on dovecot passing our environment cleanly to sieve-called scripts, this explicitly injects the antispam ip/port into the spam/ham scripts used when moving files from/to the spam-folder. This required some management of the files, such as setting proper permissions after the jinja-run.
### Related issue(s)
fixes#1177
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Dario Ernst <dario.ernst@rommelag.com>
1158: Use nginx for kubernetes ingress r=kaiyou a=micw
## What type of PR?
enhancement
## What does this PR do?
Currently, kubernetes uses a complex ingress setting which is not portable across different ingress controllers. This PR simplifies the ingress and delegates everythins special to Mailu to the front container,
### Related issue(s)
- closes#1121
- closes#1117
- closes#1021
- closes#1045
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog]
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
1114: Resolve HOST to ADDRESS only if ADDRESS is not already set r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
~Makes the rsolving from hosts to ips at startup configurable~
I rewrote the pull request after #940 was merged. Now it resolves HOSTs to ADDRESSes only of ADDRESSes are not already set. So on kubernetes we can jsut set the address and have working service discovery.
### Related issue(s)
- closes#1113
## Prerequistes
~Minor change, backward compatible~
Changelog will be added
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1088: Support domain literals (fix#1087) r=mergify[bot] a=hoellen
## What type of PR?
bug-fix
## What does this PR do?
This PR adds error handling for idna enocding. With telnet you now get a "Bad sender address syntax" message.
```
> telnet mail.example.com 25
Connected to example.com.
Escape character is '^]'.
220 mail.example.com ESMTP ready
EHLO dummy.example.com
250-mail.example.com
250 STARTTLS
MAIL FROM: does-not-exist@[116.203.165.200]
250 2.0.0 OK
RCPT TO: some-user@example.com
501 5.1.7 Bad sender address syntax
Connection closed by foreign host.
```
### Related issue(s)
fix#1087
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1084: Allow subnet with host bit set in setup r=mergify[bot] a=hoellen
## What type of PR?
bug-fix
## What does this PR do?
This allows to set the host bit in the subnet while using the [setup](https://setup.mailu.io/1.6/). The host bits are now masked out to determine the appropriate network address. This means that e.g. `172.17.0.1/12` is now considered as `172.17.0.0/12`.
So new users are not confused by error messages while setup a new Mailu instance (as you can see the multiple requests in the Matrix chat).
ref: https://docs.python.org/3/library/ipaddress.html#ipaddress.IPv4Network
### Related issue(s)
closes#1083
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1082: Use socrate instead of Mailustart r=mergify[bot] a=hoellen
## What type of PR?
enhancement
## What does this PR do?
This PR updates the `Dockerfile`, `setup.py` and `config.py` of each image to support the new [Mailu/socrate](https://github.com/Mailu/socrate) python package. So [MailuStart](https://github.com/Mailu/MailuStart) is not used anymore for resolving DNS and configuration processing.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1052: Upgrade alpine to 3.10 r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement / bug-fix
## What does this PR do?
Upgrade the alpine base image to 3.10 and clean up ensuing problems. Also directly uses postfix foreground-running with stdout logging.
### Related issue(s)
closes#1049closes#1051
Note: This is a duplicate effort of #1050#1039 …
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
With these images, I have tested manually:
- Email receive to user
- on main domain
- on additional domain
- via an alias on main-domain
- via an alias on additional domain
- via catchall
- Email reject
- of eicar testfiles
- Email sending
- Fetchmail from legacy POP
- Front LE certificates
- Simple overriding for postfix (only postfix.cf), nginx, dovecot, rspamd
- Creating a CalDAV calendar and CardDAV entry using davx5
I have not (yet) tested:
- certdumper
- databases other than sqlite
Todo:
- [x] Get rid of podop at `git+https://github.com/Nebukadneza/Podop.git@fix_py37` once https://github.com/Mailu/Podop/pull/6 is merged
- [x] Bend Mailustart back to original repository
- [x] Test more (? also help wanted ?)
Co-authored-by: Thomas Sänger <thomas@gecko.space>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Reto Glauser <git@blinkeye.ch>
1048: Refactor admin using webpack r=mergify[bot] a=kaiyou
## What type of PR?
Refactoring
## What does this PR do?
This PR brings some refactoring to the admin container :
- remove the dependency to mailustart and replace it with socrate
- remove static assets from the repo and use Webpack for building them
Co-authored-by: hoellen <dev@hoellen.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
1070: use HTTP/1.1 for proxyied connections r=mergify[bot] a=HorayNarea
## What type of PR?
Feature/Enhancement
## What does this PR do?
[by default nginx uses HTTP/1.0 when proxying connections](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version), this PR changes that to HTTP/1.1 so keep-alive and other efficiency-improvements are available
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Thomas Sänger <thomas@gecko.space>
1067: Update password in commandline r=hoellen a=hoellen
## What type of PR?
Feature
## What does this PR do?
Adds the ability to set/reset a user's password from the Mailu command line. Similar to the user command:
```
docker-compose exec admin flask mailu password myuser example.net 'password123'
```
With this we can reset the demo server password every few minutes (#987)
### Related issue(s)
closes#1066
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
Alexander Graf
lub