2525: Switch to GrapheneOS's hardened_malloc r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
Switch to GrapheneOS's hardened_malloc
This was suggested during the dev meeting of the 18/09/22.
It may break things and it may make things unbearably slow... but it should also make the exploitation of memory corruption bugs a lot harder.
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2534: Close#2533: document SQLALCHEMY_DATABASE_URI r=mergify[bot] a=nextgens
## What type of PR?
documentation
## What does this PR do?
document SQLALCHEMY_DATABASE_URI
### Related issue(s)
- closes#2533
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2530: disable SESSION_COOKIE_SECURE when TLS_FLAVOR=notls r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
People are unlikely to proxy everything
### Related issue(s)
- closes#2527
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2514: Update deps r=mergify[bot] a=ghostwheel42
## What type of PR?
update python dependencies
## What does this PR do?
Update python deps in base image
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2523: fix JS error r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It fixes a bug whereby one may have to click twice on the submit button depending on timing.
e.trigger() will error out on most browsers.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2479: Rework the anti-spoofing rule r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
We shouldn't assume that Mailu is the only MTA allowed to send emails on behalf of the domains it hosts.
We should also ensure that it's non-trivial for email-spoofing of hosted domains to happen
Previously we were preventing any spoofing of the envelope from; Now we are preventing spoofing of both the envelope from and the header from unless some form of authentication passes (is a RELAYHOST, SPF, DKIM, ARC)
### Related issue(s)
- close#2475
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2498: Implement ITERATE in podop r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
This makes ``doveadm -A`` work.
The easiest way to try it out is:
```
doveadm dict iter proxy:/tmp/podop.socket:auth shared/userdb
or
doveadm user '*'
```
The protocol is described at https://doc.dovecot.org/developer_manual/design/dict_protocol/
The current version of dovecot is not using flags... so there's little gain in implementing them.
### Related issue(s)
- close#2499
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2502: Resolve using socrate function r=mergify[bot] a=ghostwheel42
## What type of PR?
enhancement
## What does this PR do?
nginx.py had a copy of the socrate function resolve_hostname.
This removes the duplicated code and uses the socrate function.
The socrate functions does the same but prefers ipv4 addresses when resolving.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2497: Upgrade to alpine 3.16.2 r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
This may fix the build issues on arm (troubles building cryptography)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2328: Feature: Configurable default spam threshold used for new users r=mergify[bot] a=enginefeeder101
## What type of PR?
Feature
## What does this PR do?
This PR adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` is
used for this purpose. When not set, it defaults back to 80%, as the
default value was before.
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in. Should that be done?
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: enginefeeder101 <enginefeeder101@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2458: Fix: Don't update updated_at on quota_bytes_used change r=mergify[bot] a=DjVinnii
## What type of PR?
bug-fix
## What does this PR do?
This PR makes sure that the `updated_at` field is not updated when `quota_bytes_used` is updated. All other updates to the `User` model still updates the `updated_at` field.
This is done by explicitly using an method in the `Base` class triggering [`flag_modified`][url-flag-modified].
### Related issue(s)
- closes#1363
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
<!-- LINKS-->
[url-flag-modified]: https://docs.sqlalchemy.org/en/14/orm/session_api.html#sqlalchemy.orm.attributes.flag_modified
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
2460: Switch to a base image containing base tools and the podop and socrate libs r=mergify[bot] a=ghostwheel42
## What type of PR?
enhancement of build process
## What does this PR do?
Changes build.hcl to build core images using a base image.
Also adds a "assets" base image for the admin container.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Pierre Jaury <pierre@jaury.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
2468: Ensure that Mailu keeps working even if it can't obtain a certificate from LE r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Ensure that Mailu keeps working even if it can't obtain a certificate from letsencrypt for one of the HOSTNAMES
Without this TLS configuration would fail and Mailu would operate without TLS completely.
I haven't tested it but thought this used to work previously... maybe certbot has changed something
### Related issue(s)
- closes#2467
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2455: Fix/missing tanslations r=mergify[bot] a=DjVinnii
## What type of PR?
Fix/Enhancement
## What does this PR do?
Add missing Dutch translation, as well as the German translation for `Start of vacation`
### Related issue(s)
- closes#2217
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2448: Give a chance to rspamd's bayes classifier r=mergify[bot] a=nextgens
## What type of PR?
bug-fix + documentation
## What does this PR do?
As pointed out in #2442, the bayesian filter of rspamd doesn't get any chance to run as ``min_learns`` is set to 200 and we never teach it any HAM.
This PR enables rspamd's autolearn feature, that will "reinforce" good/bad by learning from the scoring of other modules. It ensures both that we will eventually reach the 200 mark but also that the data stays fresh.
I've also taken this opportunity to update the documentation & FAQ accordingly, to ensure that users teach their HAM & SPAM to both the fuzzy and bayes classifiers.
Thank you to [woj-tek](https://github.com/woj-tek) for doing the ground work on this.
### Related issue(s)
- closes#2442
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
This new advanced setting to harden cipher configuration on port 25. Changing the default is strongly discouraged, please read the documentation before doing so.
2440: The ARM wheels don't work r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
Remove piwheels to ensure we always rebuild on ARM
### Related issue(s)
- closes#2439
- #1200
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Antispam.rst contained a syntax error.
Move config description to common section which is more fitting.
Fixed wrong assignment of default value for DEFAULT_SPAM_THRESHOLD in models.py.
2403: Feature: switch CI/CD from build to buildx r=mergify[bot] a=Diman0
## What type of PR?
Feature and enhancement
## What does this PR do?
Switch from docker build to buildx for CI/CD.
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in much faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it. The arm images are built on merge on master and release branch (x.y). They do not influence the normal CI/CD workflow used for bors (for PR) and real releases (merge on master and branch x.y for x86_64).
- Arm images (and normal x86_64 images) can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images. This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
The main workflow makes use of github actions cache to store the cache layer. This layer is used to quickly rebuilt the images in the testing step and deploy step.
Unfortunately the building the arm images fails sometimes due to timeouts. Sometimes the connection to github actions cache is very slow. Restarting the workflow from the last failed step resolves this. I have not observed this with the normal build.
Just as previous time, you can use a forked project for testing the changes (https://github.com/Diman0/Mailu_Fork). You should still have owner access. I have created branch 1.11 for testing. You can see I already push 4 times to branch 1.11 (current version is 1.11.3).
### Related issue(s)
- Mention an issue like: #001
- closes#2383
- closes#1830
- closes#1200
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2221: Add support for custom NGINX config r=mergify[bot] a=easybe
## What type of PR?
enhancement
## What does this PR do?
Add support for custom NGINX config. Including *.conf files in /etc/nginx/conf.d same as the default NGINX configuration gives the user more flexibility.
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Ezra Buehler <ezra@easyb.ch>
2357: Switch to ffdhe3072 to enable RFC 7919 r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
The idea being:
- it's a "nothing up my sleeves" group
- it may help shave off some bytes of the SSL handshake; That being said, I doubt that clients that are modern enough to support this RFC won't offer an EC kex
https://raw.githubusercontent.com/internetstandards/dhe_groups/master/ffdhe3072.pem
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2404: Forwarding emails option in user settings did not support 1 letter do… r=mergify[bot] a=Diman0
…mains.
## What type of PR?
Bug-fix
## What does this PR do?
Forwarding emails option in user setting did not support 1 letter domains. The regex for checking the validity of multiple email addresses string has been modified to allow 1 letter domains and to allow 1 letter local part.
### Related issue(s)
- closes#2402
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2397: Fix resolving alias addresses for postfix when target is a punycode domain r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
- fix splitting of localpart in resolve_destination
- idna-enode domain-part of email addresses before returning to postfix
### Related issue(s)
- closes#2393
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it.
- Arm images can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images.
This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
Remove unneeded IF statement in /admin block in nginx.conf of front.
Fix contributions made to Dockerfile, add missing trailing \ and add back curl
Change healthcheck to monitoring page of fpm. Now we check nginx and fpm.
This is required for passing rDNS/ptr information to postfix.
The mail proxy uses the resolver info for passing XCLIENT info.
See http://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient
Without this info rspamd will flag all messages with DHFILTER_HOSTNAME_UNKNOWN due to the missing rDNS/ptr info.
Per requested changes added the ``DEFAULT_SPAM_THRESHOLD`` to the main
application configuration dictionary in ``configuration.py`` and updated
``models.py`` accordingly.
No error handling is added, as that was not required.
This commit adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` can
be used for this purpose. When not set, it defaults back to 80%, as the
default value was before
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in.
2325: postfix: wrap IPv6 CIDRs in square brackets for RELAYNETS r=mergify[bot] a=pommi
## What type of PR?
bug-fix
## What does this PR do?
This PR wraps IPv6 CIDRs in the `RELAYNETS` environment variable in square brackets for the postfix configuration.
The `RELAYNETS` environment variable is used for configuring both postfix `mynetworks` and rspamd `local_networks`. Postfix requires IPv6 addresses to be wrapped in square brackets (eg. `[2001:db8::]/64`).
When an IPv6 address is not wrapped in square brackets in the postfix configuration for `mynetworks` it results in this error while processing an incoming email from an IPv6 sender:
```
postfix/smtpd[340]: warning: 2001:db8::/64 is unavailable. unsupported dictionary type: 2001
postfix/smtpd[340]: warning: smtpd_client_event_limit_exceptions: 2001:db8::/64: table lookup problem
```
The sender sees an error and the incoming email is refused:
```
451 4.3.0 <unknown[2001:xxx:xxx:xxx:xxx:xxx:xxx:xxx]>: Temporary lookup failure
```
I tried to work around this issue by wrapping the IPv6 CIDR in square brackets in the `RELAYNETS` environment variable, but it segfaults rspamd, because it can't deal with this non-standard IPv6 notation used by postfix:
```
kernel: [4305632.603704] rspamd[1954299]: segfault at 0 ip 00007fb848983871 sp 00007ffe02cc6d1
8 error 4 in ld-musl-x86_64.so.1[7fb848948000+48000]
```
### Related issue(s)
- #2293
- #2272
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
**No changelog or documentation necessary for this minor change.**
Co-authored-by: Pim van den Berg <pim@nethuis.nl>
The RELAYNETS environment variable is used for configuring both postfix
`mynetworks` and rspamd `local_networks`. Postfix requires IPv6
addresses to be wrapped in square brackets (eg. [2001:db8::]/64).
2310: Update deprecated rspamd config option r=mergify[bot] a=henniaufmrenni
## What type of PR?
Configuration update
## What does this PR do?
This is just a small config update to get rid of the following warning message:
`lua; antivirus.lua:109: CLAM_VIRUS [clamav]: Using attachments_only is deprecated. Please use scan_mime_parts = true instead`
As per the rspamd documentation https://rspamd.com/doc/modules/antivirus.html
> attachments_only = true; # Before 1.8.1
> scan_mime_parts = true; # After 1.8.1
The currently used version of rspamd is 3.1.
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: henniaufmrenni <henniaufmrenni@keinvergessen.org>
bors[bot]