2603: Enable HAPROXY protocol on SUBNET r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
- Enable HAPROXY in between front and imap: With this we avoid running into the limitations of ``mail_max_userip_connections`` and the logfiles reflect the real IP.
- Enable HAPROXY in between front and smtp: with this postfix and rspamd are aware of whether TLS was used or not on the last hop. In practice this won't work as nginx doesn't send PROTO yet.
- Discard redundant log messages from postfix
With all of this, not only are the logs easier to understand but ``doveadm who`` also works as one would expect.
### Related issue(s)
- closes#894
- #1328
- closes#1364
- #1705
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2528: Implement #2510: oletools integration r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
OLETools now flags documents with macros and rejects suspicious ones. We also block executable file extensions by default (but don't perform inspection in archives: you can tell users to zip-up whatever needs sending).
### Related issue(s)
- closes#2510
- closes#2511
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2562: Dynamic address resolution everywhere r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Use dynamic address resolution everywhere.
Derive a new key for admin/SECRET_KEY
Cleanup the environment
This should allow restarting containers.
### Related issue(s)
- closes#1341
- closes#1013
- closes#1430
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2577: Autofocus the login form on /sso/login r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Autofocus the login form on /sso/login
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2542: Implement header authentication via external proxy r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
Implement header authentication via external proxy
### Related issue(s)
- closes#1972
- closes#2183
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2559: Turns out that php81-ctype is required by roundcube r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It solves:
```
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "NOTICE: PHP message: PHP Fatal error: Uncaught Error: Call to undefined function Masterminds\HTML5\Parser\ctype_alpha() in /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php:140"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "Stack trace:"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#0 /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php(82): Masterminds\HTML5\Parser\Tokenizer->consumeData()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#1 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(161): Masterminds\HTML5\Parser\Tokenizer->parse()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#2 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(89): Masterminds\HTML5->parse('<html>\n <hea...', Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#3 /var/www/roundcube/program/lib/Roundcube/rcube_washtml.php(700): Masterminds\HTML5->loadHTML('<html>\n <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#4 /var/www/roundcube/program/actions/mail/index.php(975): rcube_washtml->wash('<html>\n <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#5 /var/www/roundcube/program/actions/mail/index.php(1019): rcmail_action_mail_index::wash_html('<!doctype html>...', Array, Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#6 /var/www/roundcube/program/actions/mail/show.php(720): rcmail_action_mail_index::pr..."
```
see https://github.com/roundcube/roundcubemail/issues/7049
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2546: Implement a GUI for WILDCARD_SENDERS r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
- Implement a GUI for WILDCARD_SENDERS
### Related issue(s)
- closes#2372
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2544: Fix#2242: Make quotas adjustable in 50MiB increments r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Make quotas adjustable in 50MiB increments
### Related issue(s)
- closes#2242
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2550: Webmail hardening r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Add [Snuffleupagus](https://github.com/jvoisin/snuffleupagus/) (a modern Suhosin replacement) to protect webmails.
It may be possible to harden further, by encrypting some of the cookies and auditing the usage of gpg more closely.
This seems to work for me.
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2543: Fix#2231: make public announcements work r=nextgens a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure public announcements bypass filters.
They can still time-out... but this is already a big improvement that we should be able to backport.
### Related issue(s)
- closes#2231
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2526: Upgrade Snappymail to 2.21 and merge the webmail containers r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Upgrade Snappymail to 2.21 and merge the webmail containers. This will make the CI faster and should simplify things going forward (hardening but also allow running more than one webmail at the time, ...).
- enable APCu
- add new test to ensure we redirect to SSO and have disabled the admin panel
- add all the packaged dictionaries for spell checking
- harden the configuration of the webmails a bit (more to come in a separate PR)
- turn off deprecation warnings (php8.1 is too new)
- turn off error reporting (log them instead)
- return HTTP302 when we should
- gpg-verify the signature of the webmails we ship
- upgrade to snappymail 2.21, switch to the new json config format
- use socrates as it's meant to so that helm users can do their thing
- run the HTTPd and PHP as different users
- redirect the PHP errors to stderr
## Related issue(s)
- closes#2466
- closes#948
- closes#2250
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2529: Improve fetchmail r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Improve fetchmail:
- allow delivery via LMTP (faster, bypassing the filters)
- allow several folders to be retrieved
- run fetchmail as non-root
- tweak the compose file to ensure we have all the dependencies
### Related issue(s)
- closes#1231
- closes#2246
- closes#711
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2483: Introduce FETCHMAIL_ENABLED r=mergify[bot] a=DjVinnii
## What type of PR?
Enhancement
## What does this PR do?
Add `FETCHMAIL_ENABLED` to enable/disable the Fetchmail functionality in the Admin UI.
### Related issue(s)
- closes#2127
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2535: fix the linux/arm/v7 build r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
The arm builder is running aarch64 ... and there is no package for arm/v7
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2479: Rework the anti-spoofing rule r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
We shouldn't assume that Mailu is the only MTA allowed to send emails on behalf of the domains it hosts.
We should also ensure that it's non-trivial for email-spoofing of hosted domains to happen
Previously we were preventing any spoofing of the envelope from; Now we are preventing spoofing of both the envelope from and the header from unless some form of authentication passes (is a RELAYHOST, SPF, DKIM, ARC)
### Related issue(s)
- close#2475
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2498: Implement ITERATE in podop r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
This makes ``doveadm -A`` work.
The easiest way to try it out is:
```
doveadm dict iter proxy:/tmp/podop.socket:auth shared/userdb
or
doveadm user '*'
```
The protocol is described at https://doc.dovecot.org/developer_manual/design/dict_protocol/
The current version of dovecot is not using flags... so there's little gain in implementing them.
### Related issue(s)
- close#2499
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2497: Upgrade to alpine 3.16.2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
This may fix the build issues on arm (troubles building cryptography)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2328: Feature: Configurable default spam threshold used for new users r=mergify[bot] a=enginefeeder101
## What type of PR?
Feature
## What does this PR do?
This PR adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` is
used for this purpose. When not set, it defaults back to 80%, as the
default value was before.
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in. Should that be done?
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: enginefeeder101 <enginefeeder101@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2458: Fix: Don't update updated_at on quota_bytes_used change r=mergify[bot] a=DjVinnii
## What type of PR?
bug-fix
## What does this PR do?
This PR makes sure that the `updated_at` field is not updated when `quota_bytes_used` is updated. All other updates to the `User` model still updates the `updated_at` field.
This is done by explicitly using an method in the `Base` class triggering [`flag_modified`][url-flag-modified].
### Related issue(s)
- closes#1363
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
<!-- LINKS-->
[url-flag-modified]: https://docs.sqlalchemy.org/en/14/orm/session_api.html#sqlalchemy.orm.attributes.flag_modified
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Florent Daigniere