Commit Graph

1508 Commits (699be6f9fa62ef538fdac0f09003b2c54838eda4)

Author SHA1 Message Date
kaiyou 080e76f972
Merge pull request from rakshith-ravi/patch-1
Fixed a small typo
kaiyou 23e5aa2e05
Escape strings properly in the Dovecot dict dialect
kaiyou 814bb1f36d
Properly miss when the web api returns 404
kaiyou d2b98ae323
Update to 0.2.2
kaiyou 81d171f978
Add some debug logging to the table class
Pierre Jaury d640da8787
Include package data in the package
Pierre Jaury c5fa0280a0
Add support for dovecot dict_set operations
Pierre Jaury eb6b1866f1
Specify dependencies in the setup script
Pierre Jaury b1b0aeb69d
Initial commit
Alexander Graf 9fe452e3d1
Use base image when building core images
Alexander Graf 5e552bae69
Add base image
Alexander Graf 295d7ea675
Move assets to own Dockerfile
bors[bot] e600f20762
Merge
2468: Ensure that Mailu keeps working even if it can't obtain a certificate from LE r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Ensure that Mailu keeps working even if it can't obtain a certificate from letsencrypt for one of the HOSTNAMES

Without this TLS configuration would fail and Mailu would operate without TLS completely.

I haven't tested it but thought this used to work previously... maybe certbot has changed something

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 1630a18dd8 Ensure that Mailu keeps working even if it can't obtain a certificate from letsencrypt for one of the HOSTNAMES
Vincent Kling bda404182f Replace before update listener with method in the Base class
bors[bot] 1cdc4e76b4
Merge
2455: Fix/missing tanslations r=mergify[bot] a=DjVinnii

## What type of PR?

Fix/Enhancement

## What does this PR do?
Add missing Dutch translation, as well as the German translation for `Start of vacation`

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] b5e7cad2d3
Merge
2448: Give a chance to rspamd's bayes classifier r=mergify[bot] a=nextgens

## What type of PR?

bug-fix + documentation

## What does this PR do?

As pointed out in , the bayesian filter of rspamd doesn't get any chance to run as ``min_learns`` is set to 200 and we never teach it any HAM.

This PR enables rspamd's autolearn feature, that will "reinforce" good/bad by learning from the scoring of other modules. It ensures both that we will eventually reach the 200 mark but also that the data stays fresh.

I've also taken this opportunity to update the documentation & FAQ accordingly, to ensure that users teach their HAM & SPAM to both the fuzzy and bayes classifiers.

Thank you to [woj-tek](https://github.com/woj-tek) for doing the ground work on this.

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Dimitri Huisman 5b21fae968 Add missing Dutch translation
Alexander Graf 7682b4fa7f
Fix typo and translate Website with Webseite
Alexander Graf ffa7d6c565
Updated german translation
Vincent Kling 102d96bc7d Implement event lister to keep updated_at unchanged on quota_bytes_used updates
Vincent Kling a02a2c26a7 Fix typo
Vincent Kling 486dd06ca8 Add missing German translation for Start of vacation
Vincent Kling 84f60116ea Add missing Dutch translations
Florent Daigniere 85a2aafcdf ghostwheel42's suggestions
Florent Daigniere 6a0e881522 Introduce TLS_PERMISSIVE for port 25
This new advanced setting to harden cipher configuration on port 25. Changing the default is strongly discouraged, please read the documentation before doing so.
Florent Daigniere 5d09390147 enable rspamd's autolearn feature
Vincent Kling bab3f0f5a4 Remove POD_ADDRESS_RANGE
bors[bot] 7ed1da5bf1
Merge
2440: The ARM wheels don't work r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Remove piwheels to ensure we always rebuild on ARM

### Related issue(s)
- closes 
- 


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 35a794cfd6 The ARM wheels don't work
Florent Daigniere 355589a23c Apparently SQLAlchemy needs to be told explictely
Florent Daigniere 30b3a3771e Prevent signups with accounts where an alias exists
Florent Daigniere 6fb554bc2e Do the same for the postfix container
Dimitri Huisman 81c9e01d24 finishing touches for PR# 2328
Antispam.rst contained a syntax error.
Move config description to common section which is more fitting.
Fixed wrong assignment of default value for DEFAULT_SPAM_THRESHOLD in models.py.
Florent Daigniere 1500936232 Some people will need this at runtime
Florent Daigniere bd5fd9536d doh
Florent Daigniere e0643cf45c Disable the cache; don't upgrade pip if not req
Florent Daigniere f760024812 These are required for the healthcheck
Florent Daigniere 19eda03a49 Build wheels only if we have to.
Alexander Graf 822abc9136
Put ipv6 resolver address in square brackets
bors[bot] 53de7b7d60
Merge
2403: Feature: switch CI/CD from build to buildx r=mergify[bot] a=Diman0

## What type of PR?

Feature and enhancement

## What does this PR do?

Switch from docker build to buildx for CI/CD.
    - The main workflow file has been optimised and simplified.
    - Images are built in parallel when building locally resulting in much faster build times.
    - The github action workflow is about 50% faster.
    - Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it. The arm images are built on merge on master and release branch (x.y). They do not influence the normal CI/CD workflow used for bors (for PR) and real releases (merge on master and branch x.y for x86_64). 
    - Arm images (and normal x86_64 images) can also be built locally.
    - Reusable workflow is introduced for building, testing and deploying the images. This allows the workflow to be reused for other purposes in the future.
    - Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.

The main workflow makes use of github actions cache to store the cache layer. This layer is used to quickly rebuilt the images in the testing step and deploy step.

Unfortunately the building the arm images fails sometimes due to timeouts. Sometimes the connection to github actions cache is very slow. Restarting the workflow from the last failed step resolves this. I have not observed this with the normal build.

Just as previous time, you can use a forked project for testing the changes (https://github.com/Diman0/Mailu_Fork). You should still have owner access. I have created branch 1.11 for testing. You can see I already push 4 times to branch 1.11 (current version is 1.11.3).

### Related issue(s)
- Mention an issue like: 
- closes  
- closes 
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] 3327500f96
Merge
2221: Add support for custom NGINX config r=mergify[bot] a=easybe

## What type of PR?

enhancement

## What does this PR do?

Add support for custom NGINX config. Including *.conf files in /etc/nginx/conf.d same as the default NGINX configuration gives the user more flexibility.

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Ezra Buehler <ezra@easyb.ch>
bors[bot] 1069c02bc8
Merge
2357: Switch to ffdhe3072 to enable RFC 7919 r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

The idea being:
- it's a "nothing up my sleeves" group
- it may help shave off some bytes of the SSL handshake; That being said, I doubt that clients that are modern enough to support this RFC won't offer an EC kex

https://raw.githubusercontent.com/internetstandards/dhe_groups/master/ffdhe3072.pem

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
bors[bot] bae15c0af3
Merge
2404: Forwarding emails option in user settings did not support 1 letter do… r=mergify[bot] a=Diman0

…mains.

## What type of PR?

Bug-fix

## What does this PR do?

Forwarding emails option in user setting did not support 1 letter domains. The regex for checking the validity of  multiple email addresses string has been modified to allow 1 letter domains and to allow 1 letter local part.

### Related issue(s)
- closes  

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Dimitri Huisman 57865495d4 Forwarding emails option in user settings did not support 1 letter domains.
bors[bot] 51945aa316
Merge
2397: Fix resolving alias addresses for postfix when target is a punycode domain r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

- fix splitting of localpart in resolve_destination
- idna-enode domain-part of email addresses before returning to postfix

### Related issue(s)
- closes 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Dimitri Huisman 3aafecafe7 Merge branch 'master' into feat-switch-buildx
Dimitri Huisman f6de2b2938 Switch from docker build to buildx for CI/CD.
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it.
- Arm images can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images.
  This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
Alexander Graf c478e26d68
Encode domain part of email addresses before returning.
Alexander Graf 5179cf0618
Fix localpart splitting and make code more readable.
Dimitri Huisman ee78a34da4 Process code review feedback
Remove unneeded IF statement in /admin block in nginx.conf of front.
Fix contributions made to Dockerfile, add missing trailing \ and add back curl
Change healthcheck to monitoring page of fpm. Now we check nginx and fpm.
Dimitri Huisman d19208d3d1 Merge branch 'master' of github.com:Mailu/Mailu into feature-switch-snappymail
Dimitri Huisman 4b491d9de5 Re-enable the built-in nginx resolver for traffic going through the mail plugin.
This is required for passing rDNS/ptr information to postfix.
The mail proxy uses the resolver info for passing XCLIENT info.
See http://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient
Without this info rspamd will flag all messages with DHFILTER_HOSTNAME_UNKNOWN due to the missing rDNS/ptr info.
enginefeeder101 82860d0f80
Moved parsing environment variable to global application config dictionary
Per requested changes added the ``DEFAULT_SPAM_THRESHOLD`` to the main
application configuration dictionary in ``configuration.py`` and updated
``models.py`` accordingly.
No error handling is added, as that was not required.
enginefeeder101 6c83d25312
Configurable default spam threshold used for new users
This commit adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` can
be used for this purpose. When not set, it defaults back to 80%, as the
default value was before
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in.
bors[bot] c2d85ecc32
Merge
2325: postfix: wrap IPv6 CIDRs in square brackets for RELAYNETS r=mergify[bot] a=pommi

## What type of PR?

bug-fix

## What does this PR do?

This PR wraps IPv6 CIDRs in the `RELAYNETS` environment variable in square brackets for the postfix configuration.

The `RELAYNETS` environment variable is used for configuring both postfix `mynetworks` and rspamd `local_networks`. Postfix requires IPv6 addresses to be wrapped in square brackets (eg. `[2001:db8::]/64`).

When an IPv6 address is not wrapped in square brackets in the postfix configuration for `mynetworks` it results in this error while processing an incoming email from an IPv6 sender:
```
postfix/smtpd[340]: warning: 2001:db8::/64 is unavailable. unsupported dictionary type: 2001
postfix/smtpd[340]: warning: smtpd_client_event_limit_exceptions: 2001:db8::/64: table lookup problem
```

The sender sees an error and the incoming email is refused:
```
451 4.3.0 <unknown[2001:xxx:xxx:xxx:xxx:xxx:xxx:xxx]>: Temporary lookup failure
```

I tried to work around this issue by wrapping the IPv6 CIDR in square brackets in the `RELAYNETS` environment variable, but it segfaults rspamd, because it can't deal with this non-standard IPv6 notation used by postfix:
```
kernel: [4305632.603704] rspamd[1954299]: segfault at 0 ip 00007fb848983871 sp 00007ffe02cc6d1
8 error 4 in ld-musl-x86_64.so.1[7fb848948000+48000]
```

### Related issue(s)
- 
- 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

**No changelog or documentation necessary for this minor change.**

Co-authored-by: Pim van den Berg <pim@nethuis.nl>
Pim van den Berg d495052b52 postfix: wrap IPv6 CIDRs in square brackets for RELAYNETS
The RELAYNETS environment variable is used for configuring both postfix
`mynetworks` and rspamd `local_networks`. Postfix requires IPv6
addresses to be wrapped in square brackets (eg. [2001:db8::]/64).
Alexander Graf e75201bb34
Add default to column spam_mark_as_read
Florent Daigniere 74c5e92628 Switch to ffdhe3072 to enable RFC 7919
The idea being:
- it's a "nothing up my sleeves" group
- it may help shave off some bytes of the SSL handshake; That being
said, I doubt that clients that are modern enough to support this RFC
won't offer an EC kex

https://raw.githubusercontent.com/internetstandards/dhe_groups/master/ffdhe3072.pem
Florent Daigniere 04b7ddfffd Merge remote-tracking branch 'upstream/master' into Riscue-master
Florent Daigniere d2aa647a9f l10n
bors[bot] e519ec9ae6
Merge
2310: Update deprecated rspamd config option r=mergify[bot] a=henniaufmrenni

## What type of PR?

Configuration update

## What does this PR do?

This is just a small config update to get rid of the following warning message:
`lua; antivirus.lua:109: CLAM_VIRUS [clamav]: Using attachments_only is deprecated. Please use scan_mime_parts = true instead`

As per the rspamd documentation https://rspamd.com/doc/modules/antivirus.html
> attachments_only = true; # Before 1.8.1
> scan_mime_parts = true; # After 1.8.1

The currently used version of rspamd is 3.1.

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: henniaufmrenni <henniaufmrenni@keinvergessen.org>
bors[bot] e92c67b118
Merge
2338: Update X-XSS-Protection to current recommendation r=mergify[bot] a=AvverbioPronome

See:

- https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection and
- https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html#x-xss-protection

## What type of PR?

Slight enhancement

## What does this PR do?

This PR turns off the XSS auditor in the few browsers that still have one.

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ?] In case of feature or enhancement: documentation updated accordingly
- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Giuseppe C <1191978+AvverbioPronome@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Florent Daigniere cb656fc9fd Silence some errors in nginx
"could not be resolved (3: Host not found) while in resolving client
address, client:"
Your Name f7a3ecee2c remove X-XSS-Protection header from nginx.conf
Giuseppe C 389438d18b
Update X-XSS-Protection to current recommendation
See:

- https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection and
- https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html#x-xss-protection
bors[bot] c09253ede3
Merge
2323: Fix Postfix FileExistsError on startup r=mergify[bot] a=Pumba98

## What type of PR?

bug-fix

## What does this PR do?

I'm running mailu with the mailu helm-chart on kubernetes. Sometimes when a Pod restarts I get the following error during startup:
```
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/start.py", line 18, in start_podop
os.mkdir('/dev/shm/postfix',mode=0o700)
FileExistsError: [Errno 17] File exists: '/dev/shm/postfix'
INFO:MAIN:MTA-STS daemon starting...
```

But that does not prevent the container startup. When mails arrive it will fail with something like:

```
postfix/trivial-rewrite[94979]: warning: connect to /tmp/podop.socket: No such file or directory
postfix/trivial-rewrite[94979]: warning: table socketmap:unix:/tmp/podop.socket:transport lookup error: No such file or directory
postfix/trivial-rewrite[94979]: warning: socketmap:unix:/tmp/podop.socket:transport lookup error for "*"
```

I'm running this quick fix now since almost two months without problems. Maybe you got a better approach how to solve this, but this works fine for me.

### Related issue(s)
- none

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

<!--
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
--->

**No changelog or documentation necessary for this minor change.**


Co-authored-by: Pumba98 <mail@pumba98.de>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Florent Daigniere 193d835abe
Use os.makedirs instead
henniaufmrenni 8eb8cb1f48 Update deprecated rspamd config option
This gets rid of the following error message:
lua; antivirus.lua:109: CLAM_VIRUS [clamav]: Using attachments_only is deprecated. Please use scan_mime_parts = true instead

As per the rspamd documentation https://rspamd.com/doc/modules/antivirus.html
attachments_only = true; # Before 1.8.1
scan_mime_parts = true; # After 1.8.1

The currently used version is rspamd 3.1.
bors[bot] 6f89209f9f
Merge
2302: Update alpine-linux to 3.14.5 - Zlib security FIX r=mergify[bot] a=willofr

## What type of PR?
Security fix

## What does this PR do?
Update alpine docker image to alpine-3.14.5

- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Will <will@packer-output-c8fcfb40-3d93-4475-8f87-e14a9dd683b6>
Will a54a784168 Update alpine-linux to 3.14.5 - Zlib security FIX
hitech95 fc8926493c admin: graceful fail on user fetch in basic auth
Signed-off-by: hitech95 <nicveronese@gmail.com>
Dimitri Huisman f2f859280c Merge remote-tracking branch 'origin/master' into feature-switch-snappymail
Dimitri Huisman 9519d07ba2 Switch from RainLoop to SnappyMail
bors[bot] c15e4e6015
Merge
2276: Autoconfig of email clients r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

It provides auto-configuration templates for email clients and encourages them to use implicit TLS (see https://nostarttls.secvuln.info/)

There are numerous caveats:
- it will only work if suitable DNS records are created and certificates obtained (autoconfig, autodiscover, ...)
- the mobileconfig file isn't signed
- the credentials will be prompted... we could/should provision a token on each request instead
- it currently doesn't advertise caldav
- it's IMAP only

### Related issue(s)
- close  

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 9b952da6c2 Allow nginx to lookup IPv6 addresses
It creates issues with RSPAMD/HFILTER_HOSTNAME_UNKNOWN on v6 enabled
setups see
https://github.com/Mailu/Mailu/issues/2260#issuecomment-1066797661
bors[bot] 8cc91bad75
Merge
2281: Update alpine-linux to 3.14.4 - OpenSSL security FIX r=mergify[bot] a=willofr

## What type of PR?
Security fix

## What does this PR do?
Update Dockerfiles to use alpine-linux 3.14.4 which contains a security fix for openssl
https://alpinelinux.org/posts/Alpine-3.12.10-3.13.8-3.14.4-released.html

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2285: Update names of language json files r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

language json files of datatables i18n have been renamed
this updates the mappings to the current names


2286: Fix typo in Traefik reverse proxy docs r=mergify[bot] a=ghostwheel42

Slight typo in the Traefik reverse proxy docs. Found through running into the issue on my own instance.

## What type of PR?

documentation

## What does this PR do?

Adds   to master


2287: Fix typo in docs: cert not certs r=mergify[bot] a=ghostwheel42

## What type of PR?

documentation

## What does this PR do?

just a typo

Co-authored-by: Will <will@packer-output-c8fcfb40-3d93-4475-8f87-e14a9dd683b6>
Co-authored-by: willofr <willofr@users.noreply.github.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: DAHPr0gram3r <cbillwork02@gmail.com>
bors[bot] a7149b83d4
Merge
2284: Fixing AUTH_RATELIMIT_IP not working on imap/pop3/smtp r=mergify[bot] a=fischerscode



## What type of PR?

bug-fix

## What does this PR do?
This fixes AUTH_RATELIMIT_IP not working on imap/pop3/smtp.

### Related issue(s)
closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

Co-authored-by: Maximilian Fischer <github@maaeps.de>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 68d3d67b8c
Merge
2255: Create a polite and turtle delivery queue to accommodate destinations that expect emails to be sent slowly r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

Create a polite and turtle delivery queue to accommodate destinations that expect emails to be sent slowly

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf d70596d431
Update names of language json files
Alexander Graf 64ad6931e9
Move 'is_valid_user = user is not None' into else
Alexander Graf 630a4e9b5e
Update auth.py
Add spaces
Maximilian Fischer 8775dc5b15 Fixing AUTH_RATELIMIT_IP not working on imap/pop3/smtp
Will d02296c3bc Update alpine-linux to 3.14.4 - OpenSSL security FIX
Florent Daigniere ce9dc3a335 ghostwheel42's suggestion
Florent Daigniere 83140322e0 ghostwheel42's suggestion
Florent Daigniere 3aa735cc2d ghostwheel42's suggestion
Florent Daigniere 6d80eea649 ghostwheel42's suggestion
Florent Daigniere 9bc963f19b don't think the escaping is required but it was there
Florent Daigniere 3e6f3a95a4 Reflect the data from the POST
Florent Daigniere 14931c4acd doh
Florent Daigniere c6c444cfa7 simplify
Florent Daigniere 373e6d2161 doh
Florent Daigniere 184c9bc566 Add json redirect
Florent Daigniere 9a2d8d63a3 Search and replace wasn't a good idea
Florent Daigniere c50750054b Allow POST
Florent Daigniere 71897f4ff0 Doh
Florent Daigniere d677c465a7 Handle spaces too
Florent Daigniere 6fc1273b58 Add a link to autoconfigure apple devices
Florent Daigniere 0bccb5045c STARTTLS is a bad idea
Florent Daigniere 3a56525e21 As discussed on #mailu-dev
Don't attempt to guess what the user wants
Florent Daigniere 81b592f3cb try to get LE certs for the new names
Florent Daigniere a3f9e2beee Use priorities instead
Florent Daigniere 2b62a6327a Do explicit TLS where possible
Florent Daigniere c817eaf608 Add the SRV record for autodiscover
Florent Daigniere cdc92aa65b Mobileconfig apple style
Florent Daigniere ccd2cad4f1 Autodiscovery microsoft style
Florent Daigniere 523cee1680 Autoconfig mozilla-style
bors[bot] 0b25854de0
Merge
2210: Add input validation for domain creation r=mergify[bot] a=0pc0deFR

## What type of PR?

bug-fix

## What does this PR do?

This patch add the input validation for domain creation.

### Related issue(s)
- Mention an issue like: 
- Auto close an issue like: closes 


Co-authored-by: Kevin Falcoz <0pc0defr@gmail.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
İbrahim Akyel f65e2fc469 Feature: Marking "Read" spam mails
Florent Daigniere a7f9a35fa1
Merge branch 'master' into fix2274
Florent Daigniere a4ed464170 doh
Florent Daigniere 0bfbb3bcd4
doh
Florent Daigniere cd3eee4c51 ghostwheel42's suggestion
Florent Daigniere d723326b8e style
Florent Daigniere f01d8cd9b9 improve
Florent Daigniere 7b9c4e01f7 improve
Florent Daigniere 91de20c49c Fix exception in logs
This was occuring when you had square brackets in the domain part
Florent Daigniere 8cf76afbab Catch the ValueError instead
Florent Daigniere 08aa32a5df Revert "Don't bother running the query without an address"
This reverts commit dc81979550.
Florent Daigniere 7ce7f2096b belt, braces and suspenders
Florent Daigniere dc81979550 Don't bother running the query without an address
This should solve the following in admin logs:
"WARNING in nginx: Invalid user 'xxxx': (builtins.ValueError)
invalid email address (no "@")"
Pumba98 f1952d0e97
Update start.py
bors[bot] 2e9b14d536
Merge
2254: Send ISRG_X1 on port 25, make DANE pin that r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Ensure we send ISRG_X1 in the handshake on port 25 (non-interactive, size doesn't really matter).

Update the DANE pin to reflect the change.

I am not sure whether we will need to add --preferred-chain= in the future; This may be the case when letsencrypt decides to use X2/the ECDSA chain

This needs to be tested on a letsencrypt account that isn't mine (I'm opted in for the alternate cert chains)

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

There's already a towncrier news for it

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere f9869b1d79 ghostwheel42's suggestions
Dimitri Huisman c40a0f4b80 Change link in warning to master. Master is always available. 1.9 will be unavaiable in the future.
Florent Daigniere ab35492589 the first time the loop runs we don't have the second cert
Florent Daigniere 0816cb9497 simplify as per ghostwheel42's suggestion
Florent Daigniere 7166e7d2b2 Implement : slow transports
Florent Daigniere e4a32b55f5 Send ISRG_X1 on port 25, make DANE pin that
Florent Daigniere d3e7ea5389 spell it out
Florent Daigniere a8dc20962a workaround a bug in coredns
Dimitri Huisman 55a601de5a Add missing import for validators, improve behaviour when an error occurs.
Dimitri Huisman 7d801c560c Improve if statement
Florent Daigniere 9466ad4131 fix
Ezra Buehler 5d6b295013 Add support for custom NGINX config
Including *.conf files in /etc/nginx/conf.d same as the default NGINX
configuration gives the user more flexibility.
bors[bot] 855f3b065b
Merge
2211: Ensure we use IMAP IDLE like it's supposed to work r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Increase IMAP IDLE time from 2min to 29mins: this should massively help reduce network traffic & increase battery life of clients

See https://peterkieser.com/2011/03/25/androids-k-9-mail-battery-life-and-dovecots-push-imap/

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Florent Daigniere 224880822f
remove space
Florent Daigniere 3d7b9fe194 Ensure we use IMAP IDLE like it's supposed to work
imap_idle_notify_interval = 2 mins -> 29 mins

See https://peterkieser.com/2011/03/25/androids-k-9-mail-battery-life-and-dovecots-push-imap/
Kevin Falcoz 278d74ce6f
Add title attribute on user-panel div
Kevin Falcoz 3fe1dbe881
Add input validation for domain creation
Kevin Falcoz c69f886a73 Update code with ghostwheel42 comments
Kevin Falcoz 3e394faf92
Patch function "Display Name" into admin page
Florent Daigniere f6ebf9fda2
Update tls.conf
Florent Daigniere 68ff6c8337
Use ISRG_ROOT_X1 as DST_ROOT is not available
Sebastian Klemke a6b4b9ae52 Removed ssl_trusted_certificate configuration setting from nginx.
Resolves an nginx startup issue when letsencrypt or
mail-letsencrypt is enabled.

Fixes 
Sebastian Klemke 89a86e9dda disabled rsyslogd pidfile
Florent Daigniere b9e614145f there too
Florent Daigniere b7fb8c661a switch to new API
Billy Chan 90394d7d8c 🎨 use resolver.resolve
shing6326 32446f03e7
Update start.py
fix missing leading . for the resolver test
bors[bot] 1e53530164
Merge
2144: Enable unbound by default, warn if the DNS resolver doesn't work r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Enable unbound by default, warn if the DNS resolver doesn't work

### Related issue(s)
- close 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Florent Daigniere a9da0c084a
syntax error
Florent Daigniere a2f6243382
remove the error variable
Florent Daigniere b12616b93f
Make the recommendation clearer
Alexander Graf f809be39bf
supply missing fields argument
bors[bot] e3e3700187
Merge
2150: fix 2145: exceptions may be thrown when login is invalid or rate-limits exceeded r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Exceptions may be thrown when login is invalid or rate-limits exceeded for those running very recent builds of 1.9

For some reason I haven't caught it while testing #2130... that's when it was introduced.

### Related issue(s)
- close 
- close 
- 



Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 7bd1fd3489 fix 2145
Florent Daigniere 6425f440d3 fix 2147
Florent Daigniere 379fe18f7a test dns resolvers at startup
Florent Daigniere 98973223fd
reduce TTL to 1d
Florent Daigniere 792893caae change TTL to 1y
Florent Daigniere 671f3e382a Fix 2138: Pin DANE with the full cert
Florent Daigniere 7f89a29790 Fix 2125
Make the caller responsible to know whether the rate-limit code should
be called or not
bors[bot] 65d905fe62
Merge
2099: update Dockerfile to alpine 3.14.3 r=mergify[bot] a=willofr

## What type of PR?
Security fix

## What does this PR do?
Updated the Dockerfile to use the latest alpine version 3.14.3 where several CVEs have been fixed: https://alpinelinux.org/posts/Alpine-3.14.3-released.html
New images successfully built on my test env.

### Related issue(s)
None

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Will <will@packer-output-c8fcfb40-3d93-4475-8f87-e14a9dd683b6>
Co-authored-by: willofr <willofr@users.noreply.github.com>
bors[bot] 3eca813182
Merge
2116: fix 2114: redirect old path r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Old paths may still be cached in browsers, it's easy enough to redirect them

### Related issue(s)
- close 


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere c4675e6e75 fix 2114: redirect old path
Dimitri Huisman b4d3d4b3c9 Preparations for 1.9 release.
Erriez 4b0694705c Fix build dependencies pycares
Dimitri Huisman 51d94b8d14 Fix issue 2102
Will b2abbc8856 update Dockerfile to alpine 3.14.3
Florent Daigniere bee6e980e3 doh
Florent Daigniere 58d0faff7f ensure we clear the token on delete()
Florent Daigniere 2b29cfb3f0 fix cleanup_sessions()
Florent Daigniere f0247a2faf Use self where appropriate
Florent Daigniere c161a2c987 syntax
bors[bot] 18865bf03b
Merge
2094: Sessions tweaks r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

- Make all sessions permanent, introduce SESSION_TIMEOUT and PERMANENT_SESSION_LIFETIME.
- Prevent the creation of a session before there is a login attempt
- Ensure that webmail tokens are in sync with sessions

### Related issue(s)
- close  

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Dimitri Huisman d40be05117 Fix missing edit buttons in alias, relay and fetchmail lists in admin.
Florent Daigniere a28c7f903e do it once
Dimitri Huisman f88daa1e77 Add missing cast to int
Florent Daigniere 5f313310d4 regenerate() shouldn't extend lifetime
Florent Daigniere fe18cf9743 Fix 2080
Ensure that webmail tokens are in sync with sessions
Florent Daigniere 02c93c44f2 Tweak sessions
simplify:
- make all sessions permanent by default
- update the TTL of sessions on access (save always)
- fix session-expiry, modulo 8byte precision
Florent Daigniere ea96a68eb4 don't create a session if we don't have to
bors[bot] 7c03878347
Merge
1441: Rsyslog logging for postfix r=mergify[bot] a=micw


## What type of PR?

enhancement

## What does this PR do?
Changes postfix logging from stdout to rsyslog:
* stdout logging still enabled
* internal test request log messages are filtered out by rsyslog
* optional logging to file via POSTFIX_LOG_FILE env variable

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


2090: fix 2086 r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Fix a bug I've introduced in ae8db08bd

### Related issue(s)
- close 

Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 346ace5fb3 Make webmail the default action
bors[bot] 634318adba
Merge
2072: use dovecot-fts-xapian from alpine package r=mergify[bot] a=willofr

## What type of PR?

enhancement

## What does this PR do?
use dovecot-fts-xapian from alpine packages repository (newer) instead of compiling an older version from source
see https://pkgs.alpinelinux.org/package/edge/community/x86/dovecot-fts-xapian

### Related issue(s)
No

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: willofr <willofr@users.noreply.github.com>
Florent Daigniere 09926702d6 fix 2086
bors[bot] e7f77875e2
Merge
2084: Fix  (login to webmail did not work when WEB_WEBMAIL=/ was set) r=mergify[bot] a=Diman0

## What type of PR?

bug-fix

## What does this PR do?
It fixes . Login from SSO page to webmail did not work if WEB_WEBMAIL=/ was set in mailu.env.

I tested that it works with
- WEB_WEBMAIL=/webmail
- WEB_WEBMAIL=/

### Related issue(s)
- closes  

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] n/a In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Florent Daigniere d7a8235b89
Simplify
bors[bot] 08be233607
Merge
2058: Implement versioning for CI/CD workflow. r=mergify[bot] a=Diman0

## What type of PR?

Feature!

## What does this PR do?
This PR introduces 3 things
- Add versioning (tagging) for branch x.y (1.8). E.g. 1.8.0, 1.8.1 etc.
  - docker repo will contain x.y (latest) and x.y.z (pinned version) images.
  - The X.Y.Z tag is incremented automatically. E.g. if 1.8.0 already exists, then the next merge on 1.8 will result in the new tag 1.8.1 being used.
- Make the version available in the image.
  -  For X.Y and X.Y.Z write the version (X.Y.Z) into /version on the image and add a label with version=X.Y.Z
	  -  This means that the latest X.Y image shows the pinned version (X.Y.Z e.g. 1.8.1) it was based on. Via the tag X.Y.Z you can see the commit hash that triggered the built.
  -  For master write the commit hash into /version on the image and add a label with version={commit hash}
-  Automatic releases. For x.y triggered builts (e.g. merge on 1.9) do a new github release for the pinned x.y.z (e.g. 1.9.2). 
  -  Release shows a static message (see RELEASE_TEMPLATE.md) that explains how to reach the newsfragments folder and change the branch to the tag (x.y.z) mentioned in the release. Now you can get the changelog by reading all newsfragment files in this folder.

This PR does not change anything to our workflow (what we (human persons) do). Our processes are still exactly the same. The above introduced logic is automatic. When we backport to X.Y all the magic for creating the pinned version X.Y.Z is handled by the CI/CD workflow.

### Related issue(s)
- closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

## Testing
Suggested testing steps. This should cover all situations including BORS. It does require that you use your own docker repo or temporarily create a new one.
Suggested testing steps.
1. Create new github repo.
2. Add the required docker secrets to the project (see beginning of CI.yml for the secret names), DOCKER_UN, DOCKER_PW, DOCKER_ORG, DOCKER_ORG_TESTS.
3. Clone the project.
4. Copy the contents of the PR to the cloned project.
5. Push to your new github repo.
6. Now master images are built. Check that images with tag master are pushed to your docker repo
7. Check with docker inspect nginx:master that it has the label version={commit hash}.
8. Run an image, run `docker-compose exec <name> cat /version`. Note that /version also contains the pinned version. For master the pinned version is the commit hash.
9. Create branch 1.8. 
10. Push branch 1.8 to repo.
11. Note that tags 1.8 and 1.8.0 are built and pushed to docker repo
12. Inspect label and /version. Note that 1.8 and 1.8.0 both show version 1.8.0.
13. Push another commit to branch 1.8.
14. Note that tags 1.8 and 1.8.1 are built and pushed to docker repo
15. Inspect label and /version. Note that 1.8 and 1.8.1 both show version 1.8.1.
16. Let's check BORS stuff.
17. Create branch testing.
18. Push the commit with the exact commit text (IMPORTANT!!): `Try #1234:`'.
19. Note that images are built and pushed for tag `pr-1234`.
20. Inspect label and /version. Note that the version is `pr-1234`.
20. Create branch staging.
21. Push the commit with commit text: `Merge #1234`.
22. Note that this image is not pushed to docker (as expected).

but you could also check the GH repo and docker repo I used:
https://github.com/Diman0/Mailu_Fork
https://hub.docker.com/r/diman/rainloop/tags

Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] d2a2a3a8bf
Merge
2076: fix the default for DEFER_ON_TLS_ERROR r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

The default wasn't set anywhere

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Dimitri Huisman fdb10cfb85 Start crond when POSTFIX_LOG_FILE is set
Dimitri Huisman 5bedcc1cb1 Fix
Dimitri Huisman d76773b1df Also check the SMTP port for webmail/token
Dimitri Huisman f26fa8da84 Fix Webmail token check. Fix Auth-Port for Webmail.
Florent Daigniere 593e3ac5a4 fix DEFER_ON_TLS_ERROR
willofr 841b29e794
revert back to alpine 3.14.2 as requested
willofr 73f5291cdb
Merge branch 'Mailu:master' into patch-1
Dimitri Huisman 53975684b8 Using Syslog is the new standard. It is not optional anymore.