1685 Commits (d017b3f22a5dc00261c3227eb5149a5d62790257)

Author SHA1 Message Date
Florent Daigniere a566cb07d6 fix 2 years ago
Florent Daigniere 08b3a2814b Merge branch 'master' of https://github.com/Mailu/Mailu into notls 2 years ago
Florent Daigniere 6474108056 Use a join() instead 2 years ago
Florent Daigniere c0c91691fd Fix the issue on /admin/fetch/edit 2 years ago
Alexander Graf b0b64a8e63
Use FLASK_DEBUG, fix assets, show startup errors. 2 years ago
Florent Daigniere 505bb79a78 Don't set the secure Cookie flag if TLS_FLAVOR=notls 2 years ago
Florent Daigniere 08a9ab9a56 Improve fetchmail 2 years ago
Florent Daigniere 455180043d doh 2 years ago
bors[bot] 8a90f83bd0
Merge #2514
2514: Update deps r=mergify[bot] a=ghostwheel42

## What type of PR?

update python dependencies

## What does this PR do?

Update python deps in base image


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
Florent Daigniere f11c451403 Restrict it to arch where there is a package 2 years ago
Florent Daigniere 97df65e9ef Switch to GrapheneOS's hardened_malloc
This was suggested during the dev meeting of the 18/09/22.

It may break things and it may make things unbearably slow
2 years ago
bors[bot] 745c211c4a
Merge #2523
2523: fix JS error r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It fixes a bug whereby one may have to click twice on the submit button depending on timing.

e.trigger() will error out on most browsers.

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
bors[bot] 0839490beb
Merge #2479
2479: Rework the anti-spoofing rule r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

We shouldn't assume that Mailu is the only MTA allowed to send emails on behalf of the domains it hosts.
We should also ensure that it's non-trivial for email-spoofing of hosted domains to happen

Previously we were preventing any spoofing of the envelope from; Now we are preventing spoofing of both the envelope from and the header from unless some form of authentication passes (is a RELAYHOST, SPF, DKIM, ARC)

### Related issue(s)
- close #2475

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere c91c9df134 fix error 2 years ago
Alexander Graf e0d2432c6b
Rename data-ordered to data-sort 2 years ago
Alexander Graf 2a4402cdc2
Fix datatable for list fo sign-up domains 2 years ago
Alexander Graf af6cf5fd1d
Fix language selector without session 2 years ago
Alexander Graf 2778641e78
Fix screen reader title of language selector 2 years ago
Alexander Graf 4776094ea7
Configure datatables on missing tables, add sign in button to sso page. 2 years ago
Alexander Graf 6218b36372
configure datatables via html5 data attributes 2 years ago
Alexander Graf 1ae9156756
Add bcyrpt as direct dependency for better crypto. Also some updates 2 years ago
Alexander Graf a74396a9ef
Fix wtforms usage 2 years ago
Alexander Graf 047413185e
Mask Flask-SQLAlchemy >= 3.0.0 for now as it breaks mailu 2 years ago
Alexander Graf 7e36694b64
Update python deps 2 years ago
Alexander Graf 4b179d9008
Merge branch 'master' into hibp 2 years ago
Alexander Graf 36019a8ce9
Don't show Dockerfile before building 2 years ago
Alexander Graf 91e12d510d
Use default password used everywhere else 2 years ago
Alexander Graf defd533319
Don't duplicate hidden fields 2 years ago
Alexander Graf db87a0f3a1
Move temporary db into container and show docker run command 2 years ago
Alexander Graf f7caaddbec
Speed up asset building when developing 2 years ago
Alexander Graf 71263f1a8c
Add more env variables and restyle code 2 years ago
Alexander Graf fd8570ec34
Remove unused QUOTA_STORAGE_URL 2 years ago
Alexander Graf bbeb211d72
Listen to localhost by default 2 years ago
Alexander Graf 1d90dc3ea3
Allow running without redis 2 years ago
Alexander Graf c507b765be
Improve dev runner 2 years ago
Alexander Graf 8732b70b30
Add shell script to run admin dev environment 2 years ago
Alexander Graf ea636a1835
Fix hibp test 2 years ago
Alexander Graf 311f41c331
Add missing hidden fields 2 years ago
Alexander Graf 27a5f9db65
Reformatting 2 years ago
Vincent Kling 83fdc07a6f Default FETCHMAIL_ENABLED to True 2 years ago
Florent Daigniere 54e9858633 this 2 years ago
Florent Daigniere 14f802fb4a untested but that should work 2 years ago
bors[bot] e0ff135a00
Merge #2498
2498: Implement ITERATE in podop r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

This makes ``doveadm -A`` work.

The easiest way to try it out is:
```
doveadm dict iter proxy:/tmp/podop.socket:auth shared/userdb

or 

doveadm user '*'
```

The protocol is described at https://doc.dovecot.org/developer_manual/design/dict_protocol/
The current version of dovecot is not using flags... so there's little gain in implementing them.

### Related issue(s)
- close #2499

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
Alexander Graf c57706ad27
Duh 2 years ago
Alexander Graf 46773f639b
Return 404 is user-id cannot be parsed 2 years ago
Alexander Graf 595b32cf97
Fix quota return value 2 years ago
Alexander Graf bec0b1c3b2
Fix variable name 2 years ago
Florent Daigniere 001acd60ac doh2 2 years ago
Alexander Graf dec5309ef9
Fix typo 2 years ago
Florent Daigniere 6b7026ef69 Here too 2 years ago
Florent Daigniere 24b2c7c04a doh 2 years ago
Florent Daigniere 66250e396c refactor 2 years ago
wkr d920b3d037 fix(auto-reply): include start and end dates in the auto-reply period; issue #2512 2 years ago
Florent Daigniere ff9f152a52 This may be helpful too 2 years ago
Florent Daigniere 5137b235e9 whitelist what we know works
If other people use other arch and want their builds to go faster we can
whitelist them too after they have confirmed it works.
2 years ago
Alexander Graf a2d43be6de
Fix building wheels when deps need to compile 2 years ago
bors[bot] 659cf8894c
Merge #2502
2502: Resolve using socrate function r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement

## What does this PR do?

nginx.py had a copy of the socrate function resolve_hostname.
This removes the duplicated code and uses the socrate function.
The socrate functions does the same but prefers ipv4 addresses when resolving.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
Alexander Graf 91f86a4c2a
Resolve using socrate function 2 years ago
Alexander Graf bba98b320e
Fix armv7 build by manually downloading crates.io index 2 years ago
Florent Daigniere 9cb8df57c6 enforce at least 8 chars 2 years ago
Florent Daigniere afbaabd8cd v1 2 years ago
Florent Daigniere 6def1b555b doh 2 years ago
Florent Daigniere c1f571a4c3 Speed things up.
If we want to go further than this we should change podop's incr(), pass
the flags and make admin process the results.
2 years ago
Florent Daigniere 96d9289630 No need to send an extra \n 2 years ago
Florent Daigniere cdc9b63a46 Guard the message logging too 2 years ago
Florent Daigniere 2a417dbfc2 doesn't belong here 2 years ago
Florent Daigniere 1ce889b91b Do it the pythonic way 2 years ago
Florent Daigniere e10527a4bf This is not used anymore 2 years ago
Florent Daigniere 1ae4c37cb9 Don't do fancy, just re-raise it 2 years ago
Florent Daigniere 5ec4277e1e Make it async. I'm not sure it's a good idea 2 years ago
Florent Daigniere cf34be967c Implement ITERATE 2 years ago
bors[bot] 62c919da09
Merge #2497
2497: Upgrade to alpine 3.16.2 r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

This may fix the build issues on arm (troubles building cryptography)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 340e359096 doh 2 years ago
Florent Daigniere 076d67b513 follow the protocol 2 years ago
Florent Daigniere 2e467092a2 The newer dovecot sends data podop should ignore 2 years ago
bors[bot] 12480ccbff
Merge #2328
2328: Feature: Configurable default spam threshold used for new users r=mergify[bot] a=enginefeeder101

## What type of PR?

Feature

## What does this PR do?

This PR adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` is
used for this purpose. When not set, it defaults back to 80%, as the
default value was before.

If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in. Should that be done?

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: enginefeeder101 <enginefeeder101@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
Florent Daigniere 8942448561 Upgrade to alpine 3.16.2
This may fix the build issues on arm
2 years ago
Dimitri Huisman 06b784da57
Shorten default function by using lambda 2 years ago
bors[bot] 9975a793fe
Merge #2458
2458: Fix: Don't update updated_at on quota_bytes_used change r=mergify[bot] a=DjVinnii

## What type of PR?

bug-fix

## What does this PR do?

This PR makes sure that the `updated_at` field is not updated when `quota_bytes_used` is updated. All other updates to the `User` model still updates the `updated_at` field. 

This is done by explicitly using an method in the `Base` class triggering [`flag_modified`][url-flag-modified].

### Related issue(s)
- closes #1363

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

<!-- LINKS-->
[url-flag-modified]: https://docs.sqlalchemy.org/en/14/orm/session_api.html#sqlalchemy.orm.attributes.flag_modified


Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
2 years ago
Florent Daigniere ec4224123b Use the logger 2 years ago
bors[bot] 5703e97c73
Merge #2460
2460: Switch to a base image containing base tools and the podop and socrate libs r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement of build process

## What does this PR do?

Changes build.hcl to build core images using a base image.
Also adds a "assets" base image for the admin container.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Pierre Jaury <pierre@jaury.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
2 years ago
Alexander Graf 024b0573b3
Update build reqs and fix armv7 build 2 years ago
Vincent Kling 6363acf30a Add dont_change_updated_at to fetch_done 2 years ago
Vincent Kling 6b785abb01 Rename flag_updated_at_as_modified to dont_change_updated_at 2 years ago
Florent Daigniere 89f7d983b4 Don't start rspamd until admin is up and working 2 years ago
Florent Daigniere d8cf0c3848 Revert "Admin may not have started up when this loads"
This reverts commit 0f17299b4e.
2 years ago
Florent Daigniere 0f17299b4e Admin may not have started up when this loads 2 years ago
Florent Daigniere 95a3a3d342 doh 2 years ago
Florent Daigniere 84a722eabc Optimize the query 2 years ago
Florent Daigniere bd1b73032c Poke a hole for mailing lists 2 years ago
Blaž Zupan 56617bbe12 Quote SMTP SIZE to avoid splitting keyword and parameter in EHLO response 2 years ago
Florent Daigniere c4fcaed7d4 doh 2 years ago
Vincent Kling 8a60b658b4 Implement FETCHMAIL_ENABLED 2 years ago
Florent Daigniere 8929f54de5 clarify
Also cover the case where the DKIM sig is for another domain and there
is no explicit DMARC policy
2 years ago
Florent Daigniere 8da6117bb9 clarify 2 years ago
Florent Daigniere af87456faf this works for me 2 years ago
Vincent Kling 23d06a5761 Fix a bunch of typos 2 years ago
Florent Daigniere be4dd6d84a Spell it out 2 years ago
Florent Daigniere f7b3aad831 Ensure we REJECT when we don't have a DMARC policy
This restores the old behaviour
2 years ago
Florent Daigniere 0204c9e59d doh 2 years ago
Florent Daigniere 8775a2bf04 untested code that may just work 2 years ago
Florent Daigniere 5ebcecf4dd Don't need that anymore either 2 years ago
Florent Daigniere 3e51d15b03 Remove the strict anti-spoofing rule. 2 years ago
Alexander Graf d9bf6875e1
Optimize build order for better caching 2 years ago
Alexander Graf 7441a420c4
Fix and speed-up arm build. Allow chosing of prod/dev env. 2 years ago
Alexander Graf 146921f619
Move curl to base image 2 years ago
Alexander Graf 4c1071a497
Move all requirements*.txt to base image 2 years ago
Alexander Graf a29f066858
Move even more python deps to base image 2 years ago
Alexander Graf 52dd09d452
Fix assets build process #2 2 years ago
Alexander Graf 768c0cc1ce
Fix assets build process 2 years ago
Alexander Graf 8668b269cd
Add requirements.txt for base image 2 years ago
Alexander Graf 9f511faf64
Merge pull request #8 from NeverBehave/master
fix: resolve IPv6 container hostname
2 years ago
Dimitri Huisman b711f930ef
Merge pull request #9 from vavanade/patch-1
fix docstring
2 years ago
Dimitri Huisman c0066abd01
Merge pull request #6 from micw/log-failed-dns
Add logging for failed DNS lookups
2 years ago
kaiyou f63837b8e1
Update to 0.2.0 2 years ago
kaiyou 68d44201ab
Merge pull request #4 from micw/resolve-host-if-address-not-set
Resolve host if address not set
2 years ago
kaiyou b198fde756
Merge pull request #3 from micw/fix-random-failures
Change test hostnames for stable test results
2 years ago
kaiyou 7f6d51904b
Remove wrong dependency to importlib 2 years ago
kaiyou ef344c62f6
Add automated tests 2 years ago
kaiyou 74a3e87de3
Fix a couple syntax typos 2 years ago
kaiyou 351b05b92d
Allow jinja to load from file path or handle 2 years ago
kaiyou 0370b26f3e
Initial commit 2 years ago
Alexander Graf ce9d886195
Merge pull request #10 from ghostwheel42/add_gitignore
Add .gitignore file
2 years ago
kaiyou dbec5f0a6c
Switch to setuptools and bump the version 2 years ago
kaiyou 3d0d831c76
Version 0.2.4 2 years ago
kaiyou e2979f9103
Merge pull request #6 from Nebukadneza/fix_py37
Don’t use deprecated now-keyword "async"
2 years ago
kaiyou 6fadd39aea
Merge pull request #3 from Nebukadneza/add_key_url_quoting
URL-Quote the key in HTTP requests
2 years ago
kaiyou 080e76f972
Merge pull request #1 from rakshith-ravi/patch-1
Fixed a small typo
2 years ago
kaiyou 23e5aa2e05
Escape strings properly in the Dovecot dict dialect 2 years ago
kaiyou 814bb1f36d
Properly miss when the web api returns 404 2 years ago
kaiyou d2b98ae323
Update to 0.2.2 2 years ago
kaiyou 81d171f978
Add some debug logging to the table class 2 years ago
Pierre Jaury d640da8787
Include package data in the package 2 years ago
Pierre Jaury c5fa0280a0
Add support for dovecot dict_set operations 2 years ago
Pierre Jaury eb6b1866f1
Specify dependencies in the setup script 2 years ago
Pierre Jaury b1b0aeb69d
Initial commit 2 years ago
Alexander Graf 9fe452e3d1
Use base image when building core images 2 years ago
Alexander Graf 5e552bae69
Add base image 2 years ago
Alexander Graf 295d7ea675
Move assets to own Dockerfile 2 years ago
bors[bot] e600f20762
Merge #2468
2468: Ensure that Mailu keeps working even if it can't obtain a certificate from LE r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Ensure that Mailu keeps working even if it can't obtain a certificate from letsencrypt for one of the HOSTNAMES

Without this TLS configuration would fail and Mailu would operate without TLS completely.

I haven't tested it but thought this used to work previously... maybe certbot has changed something

### Related issue(s)
- closes #2467

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 1630a18dd8 Ensure that Mailu keeps working even if it can't obtain a certificate from letsencrypt for one of the HOSTNAMES 2 years ago
Vincent Kling bda404182f Replace before update listener with method in the Base class 2 years ago
bors[bot] 1cdc4e76b4
Merge #2455
2455: Fix/missing tanslations r=mergify[bot] a=DjVinnii

## What type of PR?

Fix/Enhancement

## What does this PR do?
Add missing Dutch translation, as well as the German translation for `Start of vacation`

### Related issue(s)
- closes #2217

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
bors[bot] b5e7cad2d3
Merge #2448
2448: Give a chance to rspamd's bayes classifier r=mergify[bot] a=nextgens

## What type of PR?

bug-fix + documentation

## What does this PR do?

As pointed out in #2442, the bayesian filter of rspamd doesn't get any chance to run as ``min_learns`` is set to 200 and we never teach it any HAM.

This PR enables rspamd's autolearn feature, that will "reinforce" good/bad by learning from the scoring of other modules. It ensures both that we will eventually reach the 200 mark but also that the data stays fresh.

I've also taken this opportunity to update the documentation & FAQ accordingly, to ensure that users teach their HAM & SPAM to both the fuzzy and bayes classifiers.

Thank you to [woj-tek](https://github.com/woj-tek) for doing the ground work on this.

### Related issue(s)
- closes #2442

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Dimitri Huisman 5b21fae968 Add missing Dutch translation 2 years ago
Alexander Graf 7682b4fa7f
Fix typo and translate Website with Webseite 2 years ago
Alexander Graf ffa7d6c565
Updated german translation 2 years ago
Vincent Kling 102d96bc7d Implement event lister to keep updated_at unchanged on quota_bytes_used updates 2 years ago
Vincent Kling a02a2c26a7 Fix typo 2 years ago
Vincent Kling 486dd06ca8 Add missing German translation for Start of vacation 2 years ago
Vincent Kling 84f60116ea Add missing Dutch translations 2 years ago
Florent Daigniere 85a2aafcdf ghostwheel42's suggestions 2 years ago
Florent Daigniere 6a0e881522 Introduce TLS_PERMISSIVE for port 25
This new advanced setting to harden cipher configuration on port 25. Changing the default is strongly discouraged, please read the documentation before doing so.
2 years ago
Florent Daigniere 5d09390147 enable rspamd's autolearn feature 2 years ago
Vincent Kling bab3f0f5a4 Remove POD_ADDRESS_RANGE 2 years ago
bors[bot] 7ed1da5bf1
Merge #2440
2440: The ARM wheels don't work r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Remove piwheels to ensure we always rebuild on ARM

### Related issue(s)
- closes #2439
- #1200


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 35a794cfd6 The ARM wheels don't work 2 years ago
Florent Daigniere 355589a23c Apparently SQLAlchemy needs to be told explictely 2 years ago
Florent Daigniere 30b3a3771e Prevent signups with accounts where an alias exists 2 years ago
Florent Daigniere 6fb554bc2e Do the same for the postfix container 2 years ago
Dimitri Huisman 81c9e01d24 finishing touches for PR# 2328
Antispam.rst contained a syntax error.
Move config description to common section which is more fitting.
Fixed wrong assignment of default value for DEFAULT_SPAM_THRESHOLD in models.py.
2 years ago
Florent Daigniere 1500936232 Some people will need this at runtime 2 years ago
Florent Daigniere bd5fd9536d doh 2 years ago
Florent Daigniere e0643cf45c Disable the cache; don't upgrade pip if not req 2 years ago
Florent Daigniere f760024812 These are required for the healthcheck 2 years ago
Florent Daigniere 19eda03a49 Build wheels only if we have to. 2 years ago
Alexander Graf 822abc9136
Put ipv6 resolver address in square brackets 2 years ago
bors[bot] 53de7b7d60
Merge #2403
2403: Feature: switch CI/CD from build to buildx r=mergify[bot] a=Diman0

## What type of PR?

Feature and enhancement

## What does this PR do?

Switch from docker build to buildx for CI/CD.
    - The main workflow file has been optimised and simplified.
    - Images are built in parallel when building locally resulting in much faster build times.
    - The github action workflow is about 50% faster.
    - Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it. The arm images are built on merge on master and release branch (x.y). They do not influence the normal CI/CD workflow used for bors (for PR) and real releases (merge on master and branch x.y for x86_64). 
    - Arm images (and normal x86_64 images) can also be built locally.
    - Reusable workflow is introduced for building, testing and deploying the images. This allows the workflow to be reused for other purposes in the future.
    - Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.

The main workflow makes use of github actions cache to store the cache layer. This layer is used to quickly rebuilt the images in the testing step and deploy step.

Unfortunately the building the arm images fails sometimes due to timeouts. Sometimes the connection to github actions cache is very slow. Restarting the workflow from the last failed step resolves this. I have not observed this with the normal build.

Just as previous time, you can use a forked project for testing the changes (https://github.com/Diman0/Mailu_Fork). You should still have owner access. I have created branch 1.11 for testing. You can see I already push 4 times to branch 1.11 (current version is 1.11.3).

### Related issue(s)
- Mention an issue like: #001
- closes #2383 
- closes #1830
- closes #1200

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
bors[bot] 3327500f96
Merge #2221
2221: Add support for custom NGINX config r=mergify[bot] a=easybe

## What type of PR?

enhancement

## What does this PR do?

Add support for custom NGINX config. Including *.conf files in /etc/nginx/conf.d same as the default NGINX configuration gives the user more flexibility.

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Ezra Buehler <ezra@easyb.ch>
2 years ago
bors[bot] 1069c02bc8
Merge #2357
2357: Switch to ffdhe3072 to enable RFC 7919 r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

The idea being:
- it's a "nothing up my sleeves" group
- it may help shave off some bytes of the SSL handshake; That being said, I doubt that clients that are modern enough to support this RFC won't offer an EC kex

https://raw.githubusercontent.com/internetstandards/dhe_groups/master/ffdhe3072.pem

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
bors[bot] bae15c0af3
Merge #2404
2404: Forwarding emails option in user settings did not support 1 letter do… r=mergify[bot] a=Diman0

…mains.

## What type of PR?

Bug-fix

## What does this PR do?

Forwarding emails option in user setting did not support 1 letter domains. The regex for checking the validity of  multiple email addresses string has been modified to allow 1 letter domains and to allow 1 letter local part.

### Related issue(s)
- closes #2402 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [n/a] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
Dimitri Huisman 57865495d4 Forwarding emails option in user settings did not support 1 letter domains. 2 years ago
bors[bot] 51945aa316
Merge #2397
2397: Fix resolving alias addresses for postfix when target is a punycode domain r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

- fix splitting of localpart in resolve_destination
- idna-enode domain-part of email addresses before returning to postfix

### Related issue(s)
- closes #2393


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
Dimitri Huisman 3aafecafe7 Merge branch 'master' into feat-switch-buildx 2 years ago
Dimitri Huisman f6de2b2938 Switch from docker build to buildx for CI/CD.
- The main workflow file has been optimised and simplified.
- Images are built in parallel when building locally resulting in faster build times.
- The github action workflow is about 50% faster.
- Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it.
- Arm images can also be built locally.
- Reusable workflow is introduced for building, testing and deploying the images.
  This allows the workflow to be reused for other purposes in the future.
- Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.
2 years ago
Alexander Graf c478e26d68
Encode domain part of email addresses before returning. 2 years ago
Alexander Graf 5179cf0618
Fix localpart splitting and make code more readable. 2 years ago
Dimitri Huisman ee78a34da4 Process code review feedback
Remove unneeded IF statement in /admin block in nginx.conf of front.
Fix contributions made to Dockerfile, add missing trailing \ and add back curl
Change healthcheck to monitoring page of fpm. Now we check nginx and fpm.
2 years ago
Dimitri Huisman d19208d3d1 Merge branch 'master' of github.com:Mailu/Mailu into feature-switch-snappymail 2 years ago
Dimitri Huisman 4b491d9de5 Re-enable the built-in nginx resolver for traffic going through the mail plugin.
This is required for passing rDNS/ptr information to postfix.
The mail proxy uses the resolver info for passing XCLIENT info.
See http://nginx.org/en/docs/mail/ngx_mail_proxy_module.html#xclient
Without this info rspamd will flag all messages with DHFILTER_HOSTNAME_UNKNOWN due to the missing rDNS/ptr info.
2 years ago
enginefeeder101 82860d0f80
Moved parsing environment variable to global application config dictionary
Per requested changes added the ``DEFAULT_SPAM_THRESHOLD`` to the main
application configuration dictionary in ``configuration.py`` and updated
``models.py`` accordingly.
No error handling is added, as that was not required.
3 years ago
enginefeeder101 6c83d25312
Configurable default spam threshold used for new users
This commit adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` can
be used for this purpose. When not set, it defaults back to 80%, as the
default value was before
If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in.
3 years ago
bors[bot] c2d85ecc32
Merge #2325
2325: postfix: wrap IPv6 CIDRs in square brackets for RELAYNETS r=mergify[bot] a=pommi

## What type of PR?

bug-fix

## What does this PR do?

This PR wraps IPv6 CIDRs in the `RELAYNETS` environment variable in square brackets for the postfix configuration.

The `RELAYNETS` environment variable is used for configuring both postfix `mynetworks` and rspamd `local_networks`. Postfix requires IPv6 addresses to be wrapped in square brackets (eg. `[2001:db8::]/64`).

When an IPv6 address is not wrapped in square brackets in the postfix configuration for `mynetworks` it results in this error while processing an incoming email from an IPv6 sender:
```
postfix/smtpd[340]: warning: 2001:db8::/64 is unavailable. unsupported dictionary type: 2001
postfix/smtpd[340]: warning: smtpd_client_event_limit_exceptions: 2001:db8::/64: table lookup problem
```

The sender sees an error and the incoming email is refused:
```
451 4.3.0 <unknown[2001:xxx:xxx:xxx:xxx:xxx:xxx:xxx]>: Temporary lookup failure
```

I tried to work around this issue by wrapping the IPv6 CIDR in square brackets in the `RELAYNETS` environment variable, but it segfaults rspamd, because it can't deal with this non-standard IPv6 notation used by postfix:
```
kernel: [4305632.603704] rspamd[1954299]: segfault at 0 ip 00007fb848983871 sp 00007ffe02cc6d1
8 error 4 in ld-musl-x86_64.so.1[7fb848948000+48000]
```

### Related issue(s)
- #2293
- #2272

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

**No changelog or documentation necessary for this minor change.**

Co-authored-by: Pim van den Berg <pim@nethuis.nl>
3 years ago
Pim van den Berg d495052b52 postfix: wrap IPv6 CIDRs in square brackets for RELAYNETS
The RELAYNETS environment variable is used for configuring both postfix
`mynetworks` and rspamd `local_networks`. Postfix requires IPv6
addresses to be wrapped in square brackets (eg. [2001:db8::]/64).
3 years ago
Alexander Graf e75201bb34
Add default to column spam_mark_as_read 3 years ago
Florent Daigniere 74c5e92628 Switch to ffdhe3072 to enable RFC 7919
The idea being:
- it's a "nothing up my sleeves" group
- it may help shave off some bytes of the SSL handshake; That being
said, I doubt that clients that are modern enough to support this RFC
won't offer an EC kex

https://raw.githubusercontent.com/internetstandards/dhe_groups/master/ffdhe3072.pem
3 years ago
Florent Daigniere 04b7ddfffd Merge remote-tracking branch 'upstream/master' into Riscue-master 3 years ago
Florent Daigniere d2aa647a9f l10n 3 years ago
bors[bot] e519ec9ae6
Merge #2310
2310: Update deprecated rspamd config option r=mergify[bot] a=henniaufmrenni

## What type of PR?

Configuration update

## What does this PR do?

This is just a small config update to get rid of the following warning message:
`lua; antivirus.lua:109: CLAM_VIRUS [clamav]: Using attachments_only is deprecated. Please use scan_mime_parts = true instead`

As per the rspamd documentation https://rspamd.com/doc/modules/antivirus.html
> attachments_only = true; # Before 1.8.1
> scan_mime_parts = true; # After 1.8.1

The currently used version of rspamd is 3.1.

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: henniaufmrenni <henniaufmrenni@keinvergessen.org>
3 years ago
bors[bot] e92c67b118
Merge #2338
2338: Update X-XSS-Protection to current recommendation r=mergify[bot] a=AvverbioPronome

See:

- https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection and
- https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html#x-xss-protection

## What type of PR?

Slight enhancement

## What does this PR do?

This PR turns off the XSS auditor in the few browsers that still have one.

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ?] In case of feature or enhancement: documentation updated accordingly
- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Giuseppe C <1191978+AvverbioPronome@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
3 years ago
Florent Daigniere cb656fc9fd Silence some errors in nginx
"could not be resolved (3: Host not found) while in resolving client
address, client:"
3 years ago
Your Name f7a3ecee2c remove X-XSS-Protection header from nginx.conf 3 years ago
Giuseppe C 389438d18b
Update X-XSS-Protection to current recommendation
See:

- https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection and
- https://cheatsheetseries.owasp.org/cheatsheets/HTTP_Headers_Cheat_Sheet.html#x-xss-protection
3 years ago
bors[bot] c09253ede3
Merge #2323
2323: Fix Postfix FileExistsError on startup r=mergify[bot] a=Pumba98

## What type of PR?

bug-fix

## What does this PR do?

I'm running mailu with the mailu helm-chart on kubernetes. Sometimes when a Pod restarts I get the following error during startup:
```
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/start.py", line 18, in start_podop
os.mkdir('/dev/shm/postfix',mode=0o700)
FileExistsError: [Errno 17] File exists: '/dev/shm/postfix'
INFO:MAIN:MTA-STS daemon starting...
```

But that does not prevent the container startup. When mails arrive it will fail with something like:

```
postfix/trivial-rewrite[94979]: warning: connect to /tmp/podop.socket: No such file or directory
postfix/trivial-rewrite[94979]: warning: table socketmap:unix:/tmp/podop.socket:transport lookup error: No such file or directory
postfix/trivial-rewrite[94979]: warning: socketmap:unix:/tmp/podop.socket:transport lookup error for "*"
```

I'm running this quick fix now since almost two months without problems. Maybe you got a better approach how to solve this, but this works fine for me.

### Related issue(s)
- none

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

<!--
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
--->

**No changelog or documentation necessary for this minor change.**


Co-authored-by: Pumba98 <mail@pumba98.de>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
3 years ago
Florent Daigniere 193d835abe
Use os.makedirs instead 3 years ago
henniaufmrenni 8eb8cb1f48 Update deprecated rspamd config option
This gets rid of the following error message:
lua; antivirus.lua:109: CLAM_VIRUS [clamav]: Using attachments_only is deprecated. Please use scan_mime_parts = true instead

As per the rspamd documentation https://rspamd.com/doc/modules/antivirus.html
attachments_only = true; # Before 1.8.1
scan_mime_parts = true; # After 1.8.1

The currently used version is rspamd 3.1.
3 years ago
bors[bot] 6f89209f9f
Merge #2302
2302: Update alpine-linux to 3.14.5 - Zlib security FIX r=mergify[bot] a=willofr

## What type of PR?
Security fix

## What does this PR do?
Update alpine docker image to alpine-3.14.5

- closes #2291

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Will <will@packer-output-c8fcfb40-3d93-4475-8f87-e14a9dd683b6>
3 years ago
Will a54a784168 Update alpine-linux to 3.14.5 - Zlib security FIX 3 years ago
hitech95 fc8926493c admin: graceful fail on user fetch in basic auth
Signed-off-by: hitech95 <nicveronese@gmail.com>
3 years ago
Dimitri Huisman f2f859280c Merge remote-tracking branch 'origin/master' into feature-switch-snappymail 3 years ago
Dimitri Huisman 9519d07ba2 Switch from RainLoop to SnappyMail 3 years ago
bors[bot] c15e4e6015
Merge #2276
2276: Autoconfig of email clients r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

It provides auto-configuration templates for email clients and encourages them to use implicit TLS (see https://nostarttls.secvuln.info/)

There are numerous caveats:
- it will only work if suitable DNS records are created and certificates obtained (autoconfig, autodiscover, ...)
- the mobileconfig file isn't signed
- the credentials will be prompted... we could/should provision a token on each request instead
- it currently doesn't advertise caldav
- it's IMAP only

### Related issue(s)
- close #224 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3 years ago
Florent Daigniere 9b952da6c2 Allow nginx to lookup IPv6 addresses
It creates issues with RSPAMD/HFILTER_HOSTNAME_UNKNOWN on v6 enabled
setups see
https://github.com/Mailu/Mailu/issues/2260#issuecomment-1066797661
3 years ago
bors[bot] 8cc91bad75
Merge #2281 #2285 #2286 #2287
2281: Update alpine-linux to 3.14.4 - OpenSSL security FIX r=mergify[bot] a=willofr

## What type of PR?
Security fix

## What does this PR do?
Update Dockerfiles to use alpine-linux 3.14.4 which contains a security fix for openssl
https://alpinelinux.org/posts/Alpine-3.12.10-3.13.8-3.14.4-released.html

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2285: Update names of language json files r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

language json files of datatables i18n have been renamed
this updates the mappings to the current names


2286: Fix typo in Traefik reverse proxy docs r=mergify[bot] a=ghostwheel42

Slight typo in the Traefik reverse proxy docs. Found through running into the issue on my own instance.

## What type of PR?

documentation

## What does this PR do?

Adds  #2282 to master


2287: Fix typo in docs: cert not certs r=mergify[bot] a=ghostwheel42

## What type of PR?

documentation

## What does this PR do?

just a typo

Co-authored-by: Will <will@packer-output-c8fcfb40-3d93-4475-8f87-e14a9dd683b6>
Co-authored-by: willofr <willofr@users.noreply.github.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: DAHPr0gram3r <cbillwork02@gmail.com>
3 years ago
bors[bot] a7149b83d4
Merge #2284
2284: Fixing AUTH_RATELIMIT_IP not working on imap/pop3/smtp r=mergify[bot] a=fischerscode

#2283

## What type of PR?

bug-fix

## What does this PR do?
This fixes AUTH_RATELIMIT_IP not working on imap/pop3/smtp.

### Related issue(s)
closes #2283

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

Co-authored-by: Maximilian Fischer <github@maaeps.de>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
3 years ago
bors[bot] 68d3d67b8c
Merge #2255
2255: Create a polite and turtle delivery queue to accommodate destinations that expect emails to be sent slowly r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

Create a polite and turtle delivery queue to accommodate destinations that expect emails to be sent slowly

### Related issue(s)
- closes #2213

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
3 years ago
Alexander Graf d70596d431
Update names of language json files 3 years ago
Alexander Graf 64ad6931e9
Move 'is_valid_user = user is not None' into else 3 years ago
Alexander Graf 630a4e9b5e
Update auth.py
Add spaces
3 years ago
Maximilian Fischer 8775dc5b15 Fixing AUTH_RATELIMIT_IP not working on imap/pop3/smtp
#2283
3 years ago
Will d02296c3bc Update alpine-linux to 3.14.4 - OpenSSL security FIX 3 years ago
Florent Daigniere ce9dc3a335 ghostwheel42's suggestion 3 years ago
Florent Daigniere 83140322e0 ghostwheel42's suggestion 3 years ago
Florent Daigniere 3aa735cc2d ghostwheel42's suggestion 3 years ago
Florent Daigniere 6d80eea649 ghostwheel42's suggestion 3 years ago
Florent Daigniere 9bc963f19b don't think the escaping is required but it was there 3 years ago
Florent Daigniere 3e6f3a95a4 Reflect the data from the POST 3 years ago
Florent Daigniere 14931c4acd doh 3 years ago
Florent Daigniere c6c444cfa7 simplify 3 years ago
Florent Daigniere 373e6d2161 doh 3 years ago
Florent Daigniere 184c9bc566 Add json redirect 3 years ago
Florent Daigniere 9a2d8d63a3 Search and replace wasn't a good idea 3 years ago
Florent Daigniere c50750054b Allow POST 3 years ago
Florent Daigniere 71897f4ff0 Doh 3 years ago
Florent Daigniere d677c465a7 Handle spaces too 3 years ago
Florent Daigniere 6fc1273b58 Add a link to autoconfigure apple devices 3 years ago
Florent Daigniere 0bccb5045c STARTTLS is a bad idea 3 years ago
Florent Daigniere 3a56525e21 As discussed on #mailu-dev
Don't attempt to guess what the user wants
3 years ago
Florent Daigniere 81b592f3cb try to get LE certs for the new names 3 years ago
Florent Daigniere a3f9e2beee Use priorities instead 3 years ago
Florent Daigniere 2b62a6327a Do explicit TLS where possible 3 years ago
Florent Daigniere c817eaf608 Add the SRV record for autodiscover 3 years ago
Florent Daigniere cdc92aa65b Mobileconfig apple style 3 years ago
Florent Daigniere ccd2cad4f1 Autodiscovery microsoft style 3 years ago
Florent Daigniere 523cee1680 Autoconfig mozilla-style 3 years ago
bors[bot] 0b25854de0
Merge #2210
2210: Add input validation for domain creation r=mergify[bot] a=0pc0deFR

## What type of PR?

bug-fix

## What does this PR do?

This patch add the input validation for domain creation.

### Related issue(s)
- Mention an issue like: #1817
- Auto close an issue like: closes #1817


Co-authored-by: Kevin Falcoz <0pc0defr@gmail.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
3 years ago
İbrahim Akyel f65e2fc469 Feature: Marking "Read" spam mails 3 years ago
Florent Daigniere a7f9a35fa1
Merge branch 'master' into fix2274 3 years ago
Florent Daigniere a4ed464170 doh 3 years ago
Florent Daigniere 0bfbb3bcd4
doh 3 years ago
Florent Daigniere cd3eee4c51 ghostwheel42's suggestion 3 years ago
Florent Daigniere d723326b8e style 3 years ago
Florent Daigniere f01d8cd9b9 improve 3 years ago
Florent Daigniere 7b9c4e01f7 improve 3 years ago
Florent Daigniere 91de20c49c Fix exception in logs
This was occuring when you had square brackets in the domain part
3 years ago
Florent Daigniere 8cf76afbab Catch the ValueError instead 3 years ago
Florent Daigniere 08aa32a5df Revert "Don't bother running the query without an address"
This reverts commit dc81979550.
3 years ago
Florent Daigniere 7ce7f2096b belt, braces and suspenders 3 years ago
Florent Daigniere dc81979550 Don't bother running the query without an address
This should solve the following in admin logs:
"WARNING in nginx: Invalid user 'xxxx': (builtins.ValueError)
invalid email address (no "@")"
3 years ago
Pumba98 f1952d0e97
Update start.py 3 years ago