- Moved run to bottom of Dockerfile to allow using unmodified / cached states.
- Simplified bash code in deploy.sh.
- Improved the large bash one-liner in CI.yml. It could not handle >9 for 1.x.
2044: Vault/rspamd: don't return any key for relayed domains r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR
Don't return any key for relayed domains. We may want to revisit this (ARC signing)... but in the meantime it saves from a scary message in rspamd.
```signing failure: cannot request data from the vault url: /internal/rspamd/vault/v1/dkim/ ...```
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2042: Add MESSAGE_RATELIMIT_EXEMPTION r=mergify[bot] a=nextgens
## What type of PR?
Enhancement
## What does this PR do?
Add a new knob called ```MESSAGE_RATELIMIT_EXEMPTION```.
### Related issue(s)
- #1774
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2034: Add timezone to containers r=mergify[bot] a=DjVinnii
## What type of PR?
Enhancement
## What does this PR do?
This PR adds the tzdata package so that the environment variable `TZ` can be used to set the timezone of containers.
### Related issue(s)
- closes#1154
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: DjVinnii <vincentkling@msn.com>
We may want to revisit this (ARC signing)... but in the meantime
it saves from a scary message in rspamd
signing failure: cannot request data from the vault url: /internal/rspamd/vault/v1/dkim/ ...
this is working fine, but introduces a sqlalchemy warning
when using config-import:
/app/mailu/schemas.py:822:
SAWarning: Identity map already had an identity for (...),
replacing it with newly flushed object.
Are there load operations occurring inside of an event handler
within the flush?
ConfigManager should not replace app.config - this is causing trouble
with some other flask modules (swagger).
Updated ConfigManager to only modify app.config and not replace it.
2036: round display of range inputs to 2 decimals r=mergify[bot] a=ghostwheel42
## What type of PR?
small fix
## What does this PR do?
rounds display of range inputs to 2 decimals
### Related issue(s)
- small fix to #1966
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2027: Make logs more quiet r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
It silences various useless log messages in front, specifically:
```
Oct 30 03:11:04 instance-20210109-1612 docker-front[1963]: 127.0.0.1 - - [30/Oct/2021:03:11:04 +0000] "GET /health HTTP/1.1" 301 162 "-" "curl/7.78.0"
Oct 30 03:11:04 instance-20210109-1612 docker-front[1963]: 127.0.0.1 - - [30/Oct/2021:03:11:04 +0000] "GET /health HTTP/2.0" 204 0 "-" "curl/7.78.0"
Oct 30 03:11:04 instance-20210109-1612 docker-front[1963]: 2021/10/30 03:11:04 [info] 476302#476302: *2622679 client 127.0.0.1 closed keepalive connection
Oct 30 03:13:02 instance-20210109-1612 docker-front[1963]: 127.0.0.1 - - [30/Oct/2021:03:13:02 +0000] "GET /auth/email HTTP/1.0" 200 0 "-" "-"
```
`@micw` has requested it for k8s
2030: Fix RELAYNETS r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
RELAYNETS should be comma separated like everything else; rspamd should also be aware of what is considered "trusted".
I am not sure whether ```local_networks``` is the right configuration option for it though
- close#360
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1987: Enhancement to the rate limits r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Turn the rate-limiters into something useful (that won't fire for no reason).
- fix rate-limiting on /webdav/
- it changes the rate-limiting behaviour from limiting a single IP address to a subnet of a reasonable size (/24 on v4 and /56 on v6 both are now configurable) : AUTH_RATELIMIT_IP / AUTH_RATELIMIT_IP_V4_MASK / AUTH_RATELIMIT_IP_V6_MASK
- It ensures we only use IP-based rate-limits for attempts on accounts that do not exist
- it creates a new rate limit preventing attackers from targetting a specific user account (separate from what's above) : AUTH_RATELIMIT_USER
- it introduces a rate limiting exemption mechanism whereby, upon authentication, users will see their source-ip address being exempt for a specific amount of time AUTH_RATELIMIT_EXEMPTION_LENGTH. A similar mechanism is available for web-based sessions (see below)
- It introduces in AUTH_RATELIMIT_EXEMPTION a comma separated list of network CIDRs that will be exempt from both types of rate limiting
- it implements device-tokens, as described on https://owasp.org/www-community/Slow_Down_Online_Guessing_Attacks_with_Device_Cookies to ensure that genuine users aren't locked-out by a malicious attacker abusing the rate-limit feature.
Things that could be improved include:
- the IP-based rate limiter flags attempts against "non-existing" accounts: it could go further and flag the number of unique non-existing accounts attempted (to prevent the case of a user making a typo in his MUA configuration)
- the IP address exemption mechanism doesn't pin the exemption to a specific username: any real user can trivially bypass the rate limits (and attempt to brute-force someone else's account)
### Related issue(s)
- close#1926
- close#1745
- close#1915
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2018: show dmarc record for report domain in domain details r=mergify[bot] a=ghostwheel42
## What type of PR?
documentation
## What does this PR do?
show dmarc record for report domain in domain details
### Related issue(s)
closes#1382
## Prerequisites
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1986: Document how to setup client autoconfig r=mergify[bot] a=nextgens
## What type of PR?
enhancement
## What does this PR do?
Document how to setup autoconfig. This works with most open-source MUAs (thunderbird, evolution, ...)
We could go further than that by providing dynamic configuration (issue an auth token for each MUA request)... but it won't work unless a new DNS entry (and matching certificate) is created.
### Related issue(s)
- #224
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
2014: Update Chinese translation r=mergify[bot] a=qy117121
## What type of PR?
translation
## What does this PR do?
Update Chinese translation. Use `zh` instead of `zh_CN`.
### Related issue(s)
none
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: qy117121 <mixuan121@gmail.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2017: rspamd: get dkim keys via REST API instead of filesystem r=mergify[bot] a=ghostwheel42
## What type of PR?
enhancement
## What does this PR do?
rspamd now uses hashicorp's vault api v1 to get dkim keys and selectors for a domain.
this allows future enhancement (multiple keys) without reconfiguring and restarting rspamd.
it also makes mounting the /dkim volume into the rspamd container unnecessary.
### Related issue(s)
- improves and closes#2012
- allows to implement key rotation using multiple selectors (see #1700)
- allows to implement dkim for alternate domains (see #1519)
- fixes and closes#1345 (selector transmitted by admin container is used)
- closes#1179 (no keys on disk)
- allows to implement key rotation from the outside (ie. via a helper script talking to some dns provider's api) (see #547)
## Prerequisites
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1997: Prevent traceback when using non-email in login r=mergify[bot] a=ghostwheel42
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:
=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
param.append(processors[key](compiled_params[key]))
File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
return process_param(value, dialect)
File "/app/mailu/models.py", line 60, in process_bind_param
localpart, domain_name = value.lower().rsplit('`@',` 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```
=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "`@")`
```
## What type of PR?
enhancement
## What does this PR do?
replace traceback (ERROR) with error message (WARNING)
### Related issue(s)
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2007: allow sending emails as user+detail@domain.tld r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix or enhancement
## What does this PR do?
Allows sending emails with an added "+detail" in the local part.
### Related issue(s)
closes#1948
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: root <ghostwheel42@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:
=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
context = constructor(dialect, self, conn, *args)
File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
param.append(processors[key](compiled_params[key]))
File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
return process_param(value, dialect)
File "/app/mailu/models.py", line 60, in process_bind_param
localpart, domain_name = value.lower().rsplit('@', 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```
=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "@")
```
1966: AdminLTE3 optimizations & compression and caching r=mergify[bot] a=ghostwheel42
## What type of PR?
enhancement, bugfix
## What does this PR do?
Optimization and cleanup of styles and javascript code for AdminLTE 3
Adds caching headers, gzip and robots.txt to nginx.
### Related issue(s)
Makes #1800 even better. Thanks to `@DjVinnii` and `@Diman0` for the good work.
Closes#1905
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
1971: Updated Polish translation. r=mergify[bot] a=ghostwheel42
## What type of PR?
translation
## What does this PR do?
Update polish translation. Used `pl/LC_MESSAGES/messages.po` from PR #1751 created by `@martys71`
Part of Discussion of 1.9 roadmap #1930
### Related issue(s)
closes#1751
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1975: Replace traceback with error message when creating initial admin user r=mergify[bot] a=ghostwheel42
## What type of PR?
small enhancement
## What does this PR do?
when creating the admin user via cli a traceback is shown when this user is already present in the database.
This is confusing users. I've replaced the traceback with an error message.
### Related issue(s)
#1921
## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1968: optimize handle_authentication r=mergify[bot] a=ghostwheel42
## What type of PR?
bug-fix
## What does this PR do?
catch utf-8 decoding errors and log a warning in handle_authentication instead of writing a traceback into the log.
### Related issue(s)
closes#1361
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
1967: fix 1789: ensure that nginx resolves ipv4 addresses r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
This fixes ipv6 enabled setup by disabling it. If you were using SUBNET6 in your configuration, odds are it's broken since gunicorn isn't bound on an on an ipv6 enabled socket.
Should we backport this?
### Related issue(s)
- close#1789
- close#1802
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
- fixed copy of qemu-arm-static for alpine
- added 'set -eu' safeguard
- silenced npm update notification
- added color to webpack call
- changed Admin-LTE default blue
(core/admin/Dockerfile)
- AdminLTE 3 style tweaks
(core/admin/assets/app.css)
(core/admin/mailu/ui/templates/base.html)
(core/admin/mailu/ui/templates/sidebar.html)
- localized datatables
(core/admin/Dockerfile)
(core/admin/assets/app.js)
(core/admin/package.json)
- moved external javascript code to vendor.js
(core/admin/assets/app.js)
(core/admin/assets/vendor.js)
(core/admin/webpack.config.js)
- added mailu logo
(core/admin/assets/app.js)
(core/admin/assets/app.css)
(core/admin/assets/mailu.png)
- moved all inline javascript to app.js
(core/admin/assets/app.js)
(core/admin/mailu/ui/templates/domain/create.html)
(core/admin/mailu/ui/templates/user/create.html)
- added iframe display of rspamd page
(core/admin/assets/app.js)
(core/admin/mailu/ui/views/base.py)
(core/admin/mailu/ui/templates/sidebar.html)
(core/admin/mailu/ui/templates/antispam.html)
- updated language-selector to display full language names and use post
(core/admin/assets/app.js)
(core/admin/mailu/__init__.py)
(core/admin/mailu/utils.py)
(core/admin/mailu/ui/views/languages.py)
- added fieldset to group and en/disable input fields
(core/admin/assets/app.js)
(core/admin/mailu/ui/templates/macros.html)
(core/admin/mailu/ui/templates/user/settings.html)
(core/admin/mailu/ui/templates/user/reply.html)
- added clipboard copy buttons
(core/admin/assets/app.js)
(core/admin/assets/vendor.js)
(core/admin/mailu/ui/templates/macros.html)
(core/admin/mailu/ui/templates/domain/details.html)
- cleaned external javascript imports
(core/admin/assets/vendor.js)
- pre-split first hostname for further use
(core/admin/mailu/__init__.py)
(core/admin/mailu/models.py)
(core/admin/mailu/ui/templates/client.html)
(core/admin/mailu/ui/templates/domain/signup.html)
- cache dns_* properties of domain object (immutable during runtime)
(core/admin/mailu/models.py)
(core/admin/mailu/ui/templates/domain/details.html)
- fixed and splitted dns_dkim property of domain object (space missing)
- added autoconfig and tlsa properties to domain object
(core/admin/mailu/models.py)
- suppressed extra vertical spacing in jinja2 templates
- improved accessibility for screen reader
(core/admin/mailu/ui/templates/**.html)
- deleted unused/broken /user/forward route
(core/admin/mailu/ui/templates/user/forward.html)
(core/admin/mailu/ui/views/users.py)
- updated gunicorn to 20.1.0 to get rid of buffering error at startup
(core/admin/requirements-prod.txt)
- switched webpack to production mode
(core/admin/webpack.config.js)
- added css and javascript minimization
- added pre-compression of assets (gzip)
(core/admin/webpack.config.js)
(core/admin/package.json)
- removed obsolte dependencies
- switched from node-sass to dart-sass
(core/admin/package.json)
- changed startup cleaning message from error to info
(core/admin/mailu/utils.py)
- move client config to "my account" section when logged in
(core/admin/mailu/ui/templates/sidebar.html)
1800: AdminLTE 3 r=mergify[bot] a=DjVinnii
## What type of PR?
Enhancement
## What does this PR do?
This PR implements AdminLTE 3 for the admin interface. It also includes the implementation of DataTables and a language selector.
### Related issue(s)
- closes: #1567
- closes: #1764
## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Vincent Kling <vincentkling@msn.com>
Co-authored-by: DjVinnii <vincentkling@msn.com>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
This will default to True and defer emails that fail even "loose"
validation of DANE or MTA-STS
It should work most of the time but if it doesn't and you would rather
see your emails delivered, you can turn it off.
1873: Completed Hebrew translation r=mergify[bot] a=yarons
The Hebrew translation is incomplete so I've completed it.
Co-authored-by: Yaron Shahrabani <sh.yaron@gmail.com>
1939: Ensure that we don't do multiple DNS lookups in the sieve script r=mergify[bot] a=nextgens
## What type of PR?
bug-fix
## What does this PR do?
It ensures that DNS lookups don't introduce inconsistent state. We may want to go further and actually check the return codes of rspamc too.
I haven't tested it but it should work.
### Related issue(s)
- #1938
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
1916: Ratelimit outgoing emails per user r=mergify[bot] a=nextgens
## What type of PR?
Feature
## What does this PR do?
A conflict-free version of #1360 implementing per-user sender limits
### Related issue(s)
- close#1360
- close#1031
- close#1774
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.
This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.
This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
I have found a list of the top100 email destinations online and ran them
through a script to ensure that all of their MX servers had valid
configuration... this is the result
Dimitri Huisman