Commit Graph

913 Commits (22fe65e4d8424a3ee23752b9639230f74d10d58b)

Author SHA1 Message Date
bors[bot] 6ea4e3217a
Merge
1901: treat localpart case insensitive again r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

fixes error introduced by  where the localpart of an email address was handled case sensitive.
this screwed things up at various other places.
 
### Related issue(s)

closes 
closes 

Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 6856c2c80f treat localpart case insensitive again
by lowercasing it where necessary
bors[bot] 656cf22126
Merge
1856: update asset builder dependencies r=mergify[bot] a=ghostwheel42

## What type of PR?

update asset builder dependencies

## What does this PR do?

only include needed dependencies to build mailu assets with nodejs v8

### Related issue(s)

update dependencies as discussed in 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9289fa6420
Merge
1896: save dkim key after creation r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

saves generated dkim key after creation vi web ui.
after the model change the domain object needs to be added and flushed via sqlalchemy.

### Related issue(s)

closes 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9a4c6385e5
Merge
1888: Use threads in gunicorn rather than workers/processes r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available

"smarter default"

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 54b46a13c6 save dkim key after creation
Alexander Graf c2c3030a2f rephrased comments
Alexander Graf ad1b036f20 fix Email class
Florent Daigniere 8d9f3214cc Use threads in gunicorn rather than processes
This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available

"smarter default"
Yaron Shahrabani e0bf75ae17
Completed Hebrew translation
bors[bot] c5ff72d657
Merge
1857: disable startdate when autoreply is disabled r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

disable the reply startdate field when autoreply is disabled


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Florent Daigniere a0dcd46483 fix : Handle colons in passwords
Alexander Graf 180026bd77 also disable startdate
Alexander Graf 56cfcf8b64 converted tabs to spaces
Alexander Graf 6377ccb2cb re-add jquery and select2 used in app.js
Alexander Graf 3c8a8aa8f0 use less v3 to make less-loader happy
Alexander Graf 1bb059f4c1 switched to newest possible versions for nodejs v8
Alexander Graf 858312a5cb remove explicit jQuery dependency
Alexander Graf 3f91dcb7af compile scheme list using a generator
Alexander Graf 3bb0d68ead add cargo to build cryptography
Alexander Graf 9790dcdabe updated dependencies
Florent Daigniere 420afa53f8 Upgrade to alpine 3.14
bors[bot] 4a5f6b1f92
Merge
1791: Enhanced session handling r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

- replaces flask_kvsession and simplekv with a mailu-specific session store
- call cleanup_sessions before first request and not on startup.
  this allows to run cmdline actions without redis (and makes it faster)
- allow running without redis for debugging purposes by setting MEMORY_SESSIONS to True
- don't sign session id, as it has plenty of entropy (as suggested by nextgens)
- adds method to prune a user's sessions

### Related issue(s)
- enhances and close 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 8b71a92219 use fixed msg for key derivation
Alexander Graf 92896ae646 fix bugs in model and schema introduced by
Alexander Graf 6740c77e43 small bugfix for exception
Alexander Graf fab3168c23 Merge remote-tracking branch 'upstream/master' into kvsession
Alexander Graf fbd945390d cleaned imports and fixed datetime and passlib use
Dimitri Huisman 6dc1a19390
Merge branch 'master' into import-export
bors[bot] fc1a663da2
Merge
1754: centralize Webmail authentication behind the admin panel (SSO) r=mergify[bot] a=nextgens

## What type of PR?

Enhancement: it centralizes the authentication of webmails to the admin interface.

## What does this PR do?

It implements the glue required for webmails to do SSO using the admin interface.
One of the main advantages of centralizing things this way is that it reduces significantly the attack surface available to an unauthenticated attacker (no webmail access until there is a valid Flask session).

Others include the ability to implement 2FA down the line and rate-limit things as required.

### Related issue(s)
- 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
bors[bot] 4ff90683ca
Merge
1758: Implement a simpler credential cache (alternative to ) r=mergify[bot] a=nextgens

## What type of PR?

Feature: it implements a credential cache to speedup authentication requests.

## What does this PR do?

Credentials are stored in cold-storage using a slow, salted/iterated hash function to prevent offline bruteforce attacks. This creates a performance bottleneck for no valid reason (see the
rationale/long version on https://github.com/Mailu/Mailu/issues/1194#issuecomment-762115549).

The new credential cache makes things fast again.

This is the simpler version of  (with no new dependencies)

### Related issue(s)
- close 
- close  
- close 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


1776: optimize generation of transport nexthop r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix and enhancement.

## What does this PR do?

Possibly there should be more input validation when editing a relay, but for now this tries to make the best out of the existing "smtp" attribute while maintaining backwards compatibility. When relay is empty, the transport's nexthop is the MX of the relayed domain to fix  

```
RELAY			NEXTHOP						TRANSPORT
empty			use MX of relay domain				smtp:domain
:port			use MX of relay domain and use port	smtp:domain:port
target			resolve A/AAAA of target			smtp:[target]
target:port		resolve A/AAAA of target and use port	smtp:[target]:port
mx:target		resolve MX of target				smtp:target
mx:target:port	resolve MX of target and use port	smtp:target:port
lmtp:target		resolve A/AAAA of target			lmtp:target
lmtp:target:port	resolve A/AAAA of target and use port	lmtp:target:port

target can also be an IPv4 or IPv6 address (an IPv6 address must be enclosed in []: [2001:DB8::]).
```

When there is proper input validation and existing database entries are migrated this function can be made much shorter again.

### Related issue(s)
- closes  
- closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] d9da8e4bb2
Merge
1746: DNS records for client autoconfiguration (RFC6186) r=Diman0 a=nextgens

## What type of PR?

Feature

## What does this PR do?

Add instructions on how to configure rfc6186 DNS records for client autoconfiguration

### Related issue(s)
- 
- 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 3f23e199f6 modified generation of session key and added refresh
- the session key is now generated using
  - a hash of the uid seeded by the apps secret_key (size: SESSION_KEY_BITS)
  - a random token (size: 128 bits)
  - the session's creation time (size: 32 bits)

- redis server side sessions are now refreshed after 1/2 the session lifetime
  even if not modified
- the cookie is also updated if necessary
Alexander Graf 9ef8aaf698 removed double confiog and fixed shaker
Alexander Graf a1fd44fced added lmtp: prefix and documentation
Florent Daigniere 875308d405 Revert "In fact it could be global"
This reverts commit f52984e4c3.
Florent Daigniere f52984e4c3 In fact it could be global
Florent Daigniere ae9206e968 Implement a simple credential cache
DjVinnii 419fed5e6e Add language selector
Alexander Graf 731ce8ede9 fix permanent sessions. hash uid using SECRET_KEY
clean session in redis only once when starting
Alexander Graf 4b8bbf760b default to 128 bits
Alexander Graf 4b71bd56c4 replace flask_kvsession with mailu's own storage
DjVinnii 7dafa22762 Add /language/<language> route for changing the locale using a session variable
DjVinnii f30cca1263 Do imports based on AdminLTE plugins
DjVinnii a4bb42faeb Remove extra space between 'AdminLTE' and 'on' in footer
DjVinnii b2498e8c8f Refactor box macro to card
DjVinnii 5ddea07c9a Fix form input append class
DjVinnii 1db0a870f3 Fix log in icon in sidebar
DjVinnii 51346c4860 Fix pre- and append styling
DjVinnii e963e7495d Create datatable based on dataTable class instead of table class
DjVinnii 0984173504 Change label to badge
DjVinnii 8246497d16 Add card header to tables
DjVinnii 49d68fa6d1 Fix horizontal scrollbar in sidebar
DjVinnii 7d3c9d412d Change tables to datatables
DjVinnii cdfa94c243 Make main action float right
DjVinnii 0c5fda3fca Change macros.box to macros.card
DjVinnii deca6e0c4a update user/settings
DjVinnii 6b3170cb4c Update side menu
DjVinnii c97728289b Update node version for building the image (AdminLTE requires node 10 or higher)
DjVinnii e46d9e1fc9 Update admin-lte version in package.json
Vincent Kling c6d0ef229f
Update messages.po
Alexander Graf f0f79b23a3 Allow cleanup of sessions by key&value in data
This can be used to delete all sessions belonging to a user/login.
For no it just iterates over all sessions.
This could be enhanced by using a prefix for and deleting by prefix.
Alexander Graf 83b1fbb9d6 Lazy loading of KVSessionExtension
- call cleanup_sessions on first kvstore access
  this allows to run cmdline actions without redis (and makes it faster)
- Allow development using DictStore by setting REDIS_ADDRESS to the empty string in env
- don't sign 64bit random session id as suggested by nextgens
Alexander Graf 8bc4445572 Sync update of localpart, domain_name and email
Alexander Graf 0c38128c4e Add pygments to requirements
Alexander Graf 9cb6962335 Moved MyYamlLexer into logger
now cmdline runs without pygments
Alexander Graf ce9a9ec572 always init Logger first
Alexander Graf c17bfae240 correct rfc3339 datetime serialization
now using correct timezone
Alexander Graf dc5464f254 Merge remote-tracking branch 'upstream/master' into import-export
Alexander Graf e90d5548a6 use RFC3339 for last_check
fixed to UTC for now
Florent Daigniere dd3d03f06d Merge remote-tracking branch 'upstream/master' into webmail-sso
bors[bot] 25e8910b89
Merge
1783: Switch to server-side sessions r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It simplifies session management.
- it ensures that sessions will eventually expire (*)
- it implements some mitigation against session-fixation attacks
- it switches from client-side to server-side sessions (in Redis)

It doesn't prevent us from (re)-implementing a "remember_me" type of feature if that's considered useful by some.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
lub f3f0a4d86d
Merge branch 'master' into enforce-tls-admin
Florent Daigniere 64d757582d Disable anti-csrf on the login form
The rationale is that the attacker doesn't have the password...
and that doing it this way we avoid creating useless sessions
Florent Daigniere 481cb67392 cleanup old sessions on startup
Florent Daigniere b9becd8649 make sessions expire
Florent Daigniere a1d32568d6 Regenerate session-ids to prevent session fixation
Florent Daigniere d459c37432 make session IDs 128bits
Florent Daigniere 22af5b8432 Switch to server-side sessions in redis
Alexander Graf dd2e218375 Merge remote-tracking branch 'upstream/master' into import-export
Florent Daigniere 96ae54d04d CryptContext should be a singleton
Florent Daigniere 5f05fee8b3 Don't need regexps anymore
Florent Daigniere 1c5b58cba4 Remove scheme_dict
Florent Daigniere df230cb482 Refactor auth under nginx.check_credentials()
Florent Daigniere f9ed517b39 Be specific token length
Florent Daigniere d0b34f8e24 Move CREDENTIAL_ROUNDS to advanced settings
Florent Daigniere fda758e2b4 remove merge artifact
Florent Daigniere 57a6abaf50 Remove {scheme} from the DB if mailu has set it
Florent Daigniere 7137ba6ff1 Misc improvements to PASSWORD_SCHEME
- remove PASSWORD_SCHEME altogether
- introduce CREDENTIAL_ROUNDS
- migrate all old hashes to the current format
- auto-detect/enable all hash types that passlib supports
- upgrade passlib to 1.7.4 (see : ldap_salted_sha512 support)
Florent Daigniere 00b001f76b Improve the token storage format
shortcomings of the previous format included:
- 1000x slower than it should be (no point in adding rounds since there
 is enough entropy: they are not bruteforceable)
- vulnerable to DoS as explained in
https://passlib.readthedocs.io/en/stable/lib/passlib.hash.sha256_crypt.html#security-issues
Florent Daigniere eb7895bd1c Don't do more work than necessary (/webdav)
This is also fixing tokens on /webdav/
Florent Daigniere 58b2cdc428 Don't do more work than necessary
bors[bot] 464e46b02b
Merge
1765: Set sensible cookie flags on the admin app r=mergify[bot] a=nextgens

## What type of PR?

Bugfix

## What does this PR do?

It sets the right flags on the session cookie issued by the admin app.
This should probably be backported as the lack of secure flag on TLS-enabled setup is a high risk vulnerability.

SameSite is hardening / helps against CSRF on modern browsers
HTTPOnly is hardening / helps reduce the impact of XSS

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
bors[bot] 47d6c697d0
Merge
1763: show flash messages again r=mergify[bot] a=lub

## What type of PR?

bug-fix

## What does this PR do?
This basically restores the behaviour, that got removed in
ecdf0c25b3 during refactoring.

### Related issue(s)
- noticed it while reviewing 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [-] In case of feature or enhancement: documentation updated accordingly
- [-] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: lub <git@lubiland.de>
bors[bot] cca4b50915
Merge
1607: _FILE variables for Docker swarm secrets r=mergify[bot] a=lub

## What type of PR?

enhancement

## What does this PR do?

This PR enables usage of DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY to load these values from files instead of supplying them directly. That way it's possible to use Docker secrets.

### Related issue(s)


## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: lub <git@lubiland.de>
Florent Daigniere 0dcc059cd6 Add a new knob as discussed on matrix with lub
Jaume Barber 5bb67dfcbb Translated using Weblate (Basque)
Currently translated at 100.0% (151 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/eu/
Jaume Barber a49b9d7974 Translated using Weblate (Catalan)
Currently translated at 99.3% (150 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ca/
Jaume Barber cd9992f79c Translated using Weblate (Swedish)
Currently translated at 74.2% (121 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/sv/
Jaume Barber afae5d1c24 Translated using Weblate (Russian)
Currently translated at 88.3% (144 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ru/
Jaume Barber 7a01a63389 Translated using Weblate (Portuguese)
Currently translated at 88.3% (144 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/pt/
Jaume Barber 480ec29d3d Translated using Weblate (Italian)
Currently translated at 91.4% (149 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/it/
Jaume Barber 5e96a4bfcf Translated using Weblate (Spanish)
Currently translated at 91.4% (149 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/es/
Jaume Barber 6143d66eb8 Translated using Weblate (English)
Currently translated at 39.2% (64 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Anonymous 6da5978870 Translated using Weblate (German)
Currently translated at 88.3% (144 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/de/
Anonymous 58c22fd2c6 Translated using Weblate (English)
Currently translated at 38.6% (63 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Jaume Barber 0dc8817f32 Translated using Weblate (English)
Currently translated at 38.6% (63 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Anonymous 3d17000ceb Translated using Weblate (English)
Currently translated at 29.4% (48 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Jaume Barber a2933d00f3 Translated using Weblate (English)
Currently translated at 29.4% (48 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Jaume Barber 7c0158c5f8 Translated using Weblate (English)
Currently translated at 17.7% (29 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Anonymous 7de94275a0 Translated using Weblate (English)
Currently translated at 17.7% (29 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Jaume Barber 43133d8515 Added translation using Weblate (Basque)
Jaume Barber 5e0aa65c8d Translated using Weblate (Italian)
Currently translated at 96.3% (157 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/it/
Jaume Barber 725cdc270c Translated using Weblate (Spanish)
Currently translated at 100.0% (163 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/es/
Weblate a571704a9d Merge branch 'origin/master' into Weblate.
Jaume Barber b9c2dc1a79 Translated using Weblate (Catalan)
Currently translated at 98.6% (149 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ca/
Anonymous 3a9a133226 Translated using Weblate (English)
Currently translated at 11.0% (18 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Jaume Barber af251216b0 Translated using Weblate (English)
Currently translated at 11.0% (18 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
Alexander Graf b55b53b781 optimize generation of transport nexthop
Alexander Graf 0a9f732faa added docstring to Logger. use generators.
Alexander Graf bde7a2b6c4 moved import logging to schema
- yaml-import is now logged via schema.Logger
- iremoved relative imports - not used in other mailu modules
- removed develepment comments
- added Mailconfig.check method to check for duplicate domain names
- converted .format() to .format_map() where possible
- switched to yaml multiline dump for dkim_key
- converted dkim_key import from regex to string functions
- automatically unhide/unexclude explicitly specified attributes on dump
- use field order when loading to stabilize import
- fail when using 'hash_password' without 'password'
- fixed logging of dkim_key
- fixed pruning and deleting of lists
- modified error messages
- added debug flag and two verbosity levels
Florent Daigniere aa8cb98906 Set sensible cookie options
Alexander Graf e4c83e162d fixed colorize auto detection
Alexander Graf e46d4737b0 merged changes from api without api
Alexander Graf 4b9886b139 Merge remote-tracking branch 'upstream/master' into import-export
Alexander Graf 10435114ec updated remarks and docs
Alexander Graf 1e2b5f26ab don't handle nested lists
Alexander Graf 70a1c79f81 handle prune and delete for lists and backrefs
Alexander Graf 8929912dea remove OrderedDict - not necessary in python>=3.7
Alexander Graf 3937986e76 Convert OrderedDict to dict for output
Alexander Graf 68caf50154 new import/export using marshmallow
lub 88f992de16 show flash messages again
This basically restores the behaviour, that got removed in
ecdf0c25b3 during refactoring.
Florent Daigniere 2e749abe61 DNS records for client autoconfiguration (RFC6186)
Florent Daigniere b49554bec1 merge artifact
Florent Daigniere ef637f51b7 derive the SSO keys from a KDF
Florent Daigniere 906a051925 Make rainloop use internal auth
Alexander Graf 1c9abf6e48 updated requirements for import/export
api reqs (flask-restx, ...) are still missing
Alexander Graf 902b398127 next step for import/export yaml & json
Mordi Sacks f56af3053a
Removed email address
Alexander Graf 65b1ad46d9 order yaml data and allow callback on import
- in yaml the primary key is now always first
- calling a function on import allows import to be more verbose
- skip "fetches" when empty
Alexander Graf 8213d044b2 added docstrings, use f-strings, cleanup
- idna.encode does not encode upper-case letters,
  so .lower() has to be called on value not on result
- split email-address on '@' only once
- converted '*'.format(*) to f-strings
- added docstrings
- removed from_dict method
- code cleanup/style (list concat, exceptions, return&else, line-length)
- added TODO comments on possible future changes
Alexander Graf 31a903f959 revived & renamed config-fns. cosmetics.
- revived original config-update function for backwards compability
- renamed config-dump to config-export to be in line with config-import
- converted '*'.format(*) to f-strings
- converted string-concatenation to f-strings
Alexander Graf c24bff1c1b added config_import using marshmallow
Alexander Graf 7413f9b7b4 config_dump now using marshmallow
Alexander Graf dc42d375e2 added filtering of keys and default value
Alexander Graf 82cf0d843f fix sqlalchemy column definitions
Alexander Graf b3f8dacdad add docstrings and make linter happy
Alexander Graf 6629aa3ff8 first try at api using flask-restx & marshmallow
Alexander Graf 4c258f5a6b cosmetic changes & make linter happy
renamed single letter variables (m => match)
renamed classmethod arguments to cls (model)
removed shadowing of variables (hash, context)
shortened unneeded lambda functions (id)
converted type ... is to isinstance(...)
removed unneded imports (flask)
Alexander Graf 7229c89de1 ConfigManager should not replace app.config
Updated ConfigManager to only modify app.config and not replace it.
Swagger does not play well, when app.config is not a real dict and
it is not necessary to keep ConfigManager around after init.

Also added "API" flag to config (default: disabled).
Alexander Graf 3b35180b41 cosmetic changes
Alexander Graf 815f47667b update dkim-key on commit only
Alexander Graf 0a594aaa2c cosmetic changes
Alexander Graf 3064a1dcff removed call to (undefined) cli
Alexander Graf 0051b93077 removed unused variable
Alexander Graf 2cd3acdc1a Merge remote-tracking branch 'upstream/master' into import-export
Alexander Graf 63176f4878 Merge remote-tracking branch 'upstream/master' into import-export
dependabot[bot] 54ccfdf975
Bump cryptography from 2.6.1 to 3.2 in /core/admin
Bumps [cryptography](https://github.com/pyca/cryptography) from 2.6.1 to 3.2.
- [Release notes](https://github.com/pyca/cryptography/releases)
- [Changelog](https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/2.6.1...3.2)

Signed-off-by: dependabot[bot] <support@github.com>
Alexander Graf adc9c70c3e added dump option to dump dns data of domains
Alexander Graf 2a5c46c890 Allow to dump only selected sections
Alexander Graf 500967b2f5 ignore dkim_publickey when updating config
Alexander Graf c46f9328f7 also dump dkim_publickey. allow key generation.
Alexander Graf acc728109b validate dkim keys and allow removal
cbachert 72a9ec5b7c Fix extract_host_port port separation
Regex quantifier should be lazy to make port separation work.
Alexander Graf dfc34b2165 Merge remote-tracking branch 'upstream/master' into import-export
Dimitri Huisman 78890a97ff Preparations for 1.8 release.
Alexander Graf 8e14aa80ee documented options and added help text
Alexander Graf 9d2327b0f1 add space for more human readable indentation
add a newline before main sections
add some spaces to indent
lub 05e2af1802
fix small typo in Auth-SSL
lub f0f873ffe7 add option to enforce inbound starttls
lub 02cfe326d3 support using files for SECRET_KEY and DB_PW
this enables usage of e.g. docker swarm secrets instead of exposing the
passwords directly via environment variables

just use DB_PW_FILE and SECRET_KEY_FILE instead of DB_PW and SECRET_KEY
Alexander Graf 69ccf791d2 fixed data import via from_dict
- stabilized CommaSeparatedList by sorting values
- CommaSeparatedList can now handle list and set input

- from_dict now handles mapped keys
- from_dict now handles null values

- class Domain: handle dkim-key None correctly
- class User: delete obsolete keys after converting
- class Alias: now uses Email._dict_input
Alexander Graf 190e7a709b renamed config-dump option --verbose to --full
Alexander Graf 5c0efe82cf implemented config_update and config_dump
enhanced data model with to_dict and from_dict methods
added config_dump function to manage command
config_update now uses new data model methods
Alexander Graf c26ddd3c68 fixed user's destination property
self.forward_destination is a list (and not string)
Alexander Graf 5dfccdafe9 fixed some minor typos, removed unused variable
ofthesun9 539114a3d6
Merge branch 'master' into test-alpine-3.12
Brian Maloney 6bd14506c0
Explicitly define ProxyFix options
Even though these seem to be the defaults, since 1.7 x_proto was not being honored (see ), this fixes this issue for me.
ofthesun9 1d35b1283d Adjust python required packages for alpine:3.12
ofthesun9 cff2e76269 Switching to alpine:3.12
bors[bot] 15a0d7303c
Merge
1399: Remove SPF type SPF record  r=mergify[bot] a=bladeswords

As mentioned in  - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...

## What type of PR?

Documentation

## What does this PR do?
Removes the recommendation to add a SPF RR for SPF records, as this is no longer RFC complaint and often causes issues to maintain two records.

### Related issue(s)
- closes 

## Prerequistes
None


1417: docker-compose exec needs a -T flag if no TTY is allocated r=mergify[bot] a=ofthesun9

This flag is missing in 00_create_users.sh and is failing the tests on travis arm architecture

## What type of PR?
This PR is an enhancement/bugfix needed to allow usage of travis to test and deploy on arm platform
Before the PR, tests are failing with the msg: "the input device is not a TTY"

## What does this PR do?
This PR add -T flag for the docker-compose exec occurences found in 00_create_users.sh


Co-authored-by: bladeswords <bladeswords@users.noreply.github.com>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: ofthesun9 <olivier@ofthesun.net>
Weblate 066f2bac07 Merge branch 'origin/master' into Weblate.
Jaume Barber 6c25d20c83 Translated using Weblate (Catalan)
Currently translated at 100.0% (151 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ca/
bors[bot] 60b9a3e2f0
Merge
1389: Prefer specific alias over wildcard, regardless of case r=mergify[bot] a=Nebukadneza

## What type of PR?
bug-fix

## What does this PR do?
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.

## Notes
I realize that the if-hell down there isn’t nice. What it is, however, is quite clear and easy to read. I’m hoping that if anyone ever gets confused in the future, this will make the current behavior transparent. For me, that was more important than a minimal amount of statements/branches …

### Related issue(s)
closes 

## Prerequistes
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Dario Ernst <github@kanojo.de>
bors[bot] 575f6b1691
Merge
1296: fetchmail: print unhandled exceptions, but don't crash r=Nebukadneza a=Al2Klimov

fixes 

1322: Bump validators from 0.12.5 to 0.12.6 in /core/admin r=Nebukadneza a=dependabot[bot]

Bumps [validators](https://github.com/kvesteri/validators) from 0.12.5 to 0.12.6.
<details>
<summary>Changelog</summary>

*Sourced from [validators's changelog](https://github.com/kvesteri/validators/blob/master/CHANGES.rst).*

> 0.12.6 (2019-05-08)
> ^^^^^^^^^^^^^^^^^^^
> 
> - Fixed domain validator for single character domains ([](https://github-redirect.dependabot.com/kvesteri/validators/issues/118), pull request courtesy kingbuzzman)
</details>
<details>
<summary>Commits</summary>

- See full diff in [compare view](https://github.com/kvesteri/validators/commits)
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=validators&package-manager=pip&previous-version=0.12.5&new-version=0.12.6)](https://help.github.com/articles/configuring-automated-security-fixes)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).

</details>

1337: Add IPv6 to allow_nets r=Nebukadneza a=PhilRW

Roundcube was not connecting to sieve with IPv6 enabled.

Fixes 

1358: Add port to relay if it contains a colon r=Nebukadneza a=PhilRW

## What type of PR?

enhancement

## What does this PR do?

Allows relaying domains to non-standard SMTP ports by appending `:port` to the destination host/IP. E.g., `mx1.internal:2525`

### Related issue(s)

Closes  


## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Philip Rosenberg-Watt <p.rosenberg-watt@cablelabs.com>
Weblate e9ddb2ddcc Merge branch 'origin/master' into Weblate.
Jaume Barber a2fa52170c Translated using Weblate (Catalan)
Currently translated at 98.6% (149 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ca/
Jaume Barber aafcbadb23 Translated using Weblate (Italian)
Currently translated at 98.7% (161 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/it/
Jaume Barber ecb8e07da2 Translated using Weblate (Spanish)
Currently translated at 98.7% (161 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/es/
Jae Beojkkoch ca82380bcf Translated using Weblate (English)
Currently translated at 7.9% (13 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/en/
bladeswords 8010595dd2
Remove SPF type SPF record
As mentioned in  - In accordance with RFC 7208, offer only TXT RRs for SPF.
Agree with @Nebukadneza - but not sure how to go about telling people to remove the old record...
Dario Ernst 23f21f8b9c Use pyyaml safe_load instead of load
Since load in unsafe (ref: https://msg.pyyaml.org/load),
switch the only occurrance of `yaml.load` that i could
find to safe_load.

closes 
bors[bot] b8b1699f9e
Merge
1359: Refactor the rate limiting code r=mergify[bot] a=kaiyou

## What type of PR?

Enhancement

## What does this PR do?

Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.

Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.

### Related issue(s)

No specific issue.

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: kaiyou <pierre@jaury.eu>
dependabot[bot] 94cfc31e04
Bump validators from 0.12.5 to 0.12.6 in /core/admin
Bumps [validators](https://github.com/kvesteri/validators) from 0.12.5 to 0.12.6.
- [Release notes](https://github.com/kvesteri/validators/releases)
- [Changelog](https://github.com/kvesteri/validators/blob/master/CHANGES.rst)
- [Commits](https://github.com/kvesteri/validators/commits)

Signed-off-by: dependabot[bot] <support@github.com>
bors[bot] 1ca4d6769c
Merge
1349: Add support for SRS, related to  r=mergify[bot] a=kaiyou

## What type of PR?

Feature

## What does this PR do?

It implements SRS using a Python SRS library.

### Related issue(s)
- closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: kaiyou <pierre@jaury.eu>
Dario Ernst da2dda49d4 Prefer specific alias over wildcard, regardless of case
Since direct addresses (not aliases) are case-insensitive since a while,
it makes sense for aliases to behave the same. Up until now, a wildcard
alias could trump a alias not-matching-the-case of the incoming address.
This clarifies this behavior.

closes 
NeroPcStation 365f21007d Translated using Weblate (Polish)
Currently translated at 90.2% (147 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/pl/
kaiyou 8e88f1b8c3 Refactor the rate limiting code
Rate limiting was already redesigned to use Python limits. This
introduced some unexpected behavior, including the fact that only
one criteria is supported per limiter. Docs and setup utility are
updated with this in mind.

Also, the code was made more generic, so limiters can be delivered
for something else than authentication. Authentication-specific
code was moved directly to the authentication routine.
Philip Rosenberg-Watt ff1dfec39a Add port to relay if it contains a colon
This closes 
Philip Rosenberg-Watt 27e37577c6 Add IPv6 to allow_nets
Roundcube was not connecting to sieve with IPv6 enabled.

Fixes 
Weblate b248f6a800 Merge branch 'origin/master' into Weblate
Andrási István 395a0d14dc Translated using Weblate (Hungarian)
Currently translated at 100.0% (163 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/hu/
bors[bot] 96f832835a
Merge
1278: Limiter implementation r=kaiyou a=micw

## What type of PR?

(Feature, enhancement, bug-fix, documentation)

## What does this PR do?

Adds a custom limter based on the "limits" lirary that counts up on failed auths only

### Related issue(s)
- closes 
- closes 

## Prerequistes

- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: micw <michael@wyraz.de>
bors[bot] 761fade9a9
Merge
1316: Fix the encoding of incoming user email and password r=mergify[bot] a=kaiyou

## What type of PR?

Bug fix

## What does this PR do?

As described in the changes, RFC2616 states that header should be considered ISO8859-1 in HTTP, which obviously nginx does not really care about when forwarding the password from SMTP authentication to the backend. Hence, we need to encode-then-decode passwords to get the proper value in case a special char is in there.

### Related issue(s)
- This fixes  
- This is also related to  
- This is also related to 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: kaiyou <pierre@jaury.eu>
Tom Radtke 9d213b213a
Upgrading to a 2048-bit DKIM key
Torben Jensen 6f910c5738 Translated using Weblate (Danish)
Currently translated at 80.8% (122 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/da/
kaiyou bd69b7a491 Add support for SRS, related to
kaiyou 9b7a027d6f Fix the encoding of incoming user email and password
Weblate 869f230e0d Merge branch 'origin/master' into Weblate
Torben Jensen 619a87a821 Added translation using Weblate (Danish)
bors[bot] 812439332a
Merge
1299: Don't remove the address extension in postfix r=mergify[bot] a=RobertMe

## What type of PR?
Bugfix

## What does this PR do?
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.

This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.

### Related issue(s)
Fixes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Robert Meijers <robert.meijers@gmail.com>
Angedestenebres c09f046ba7 Translated using Weblate (French)
Currently translated at 100.0% (163 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/fr/
micw 7688caa784
Add missing self.
Robert Meijers 989e4d5db5 Don't remove the address extension in postfix
Currently when the mail address is looked up by Postfix (using the admin
part) the address extension is removed. This is due to the address
extension being removed to look up the user, and afterwards returning
the users mail address. But by not returning the mail address including
the address extension it also isn't part anymore in the LMTP
communication to Dovecot. So Dovecot doesn't know about the extension,
and in turn the address extension can't be used in Sieve mail filtering.

This change fixes that by returning the original address by just
concatinating the "localpart" and domain again when the user is found.

Fixes 
Marc Riera ba7364d5e9 Added translation using Weblate (Catalan)
Michael Wyraz 70f797dbd9 Don't raise rate limit exception on hit(), only on check()
Michael Wyraz a7f787f914 Make rate limit for subnet (webmail) configurable
Michael Wyraz bee80b5c64 Remove rate limit reset
Michael Wyraz 889386b4a6 Limiter implementation
Mordi Sacks ebc39b5308 Translated using Weblate (Hebrew)
Currently translated at 6.0% (9 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/he/
Mordi Sacks 5b23e30b39 Added translation using Weblate (Hebrew)
Weblate e736abbe6f Merge branch 'origin/master' into Weblate
Simen Kildahl Eriksen 87c6984b99 Translated using Weblate (Norwegian Bokmål)
Currently translated at 100.0% (151 of 151 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/nb_NO/
bors[bot] 0417c791ff
Merge
985: Permit raspberry pi (and other architectures) builds r=mergify[bot] a=abondis

## What type of PR?

Enhancement

## What does this PR do?

Add an option to select base images and permit building for different CPU architectures.

### Related issue(s)
N/A

## Prerequistes

- [X] documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Aurélien Bondis <aurelien.bondis@gmail.com>
Co-authored-by: Aurelien <aurelien.bondis@gmail.com>
Weblate 60e5f551f9 Merge branch 'origin/master' into Weblate
Simen Kildahl Eriksen 4c3af0b905 Added translation using Weblate (Norwegian Bokmål)
bors[bot] 9bae9b3078
Merge
1213: Bump werkzeug from 0.15.2 to 0.15.3 in /core/admin r=mergify[bot] a=dependabot[bot]

Bumps [werkzeug](https://github.com/pallets/werkzeug) from 0.15.2 to 0.15.3.
<details>
<summary>Release notes</summary>

*Sourced from [werkzeug's releases](https://github.com/pallets/werkzeug/releases).*

> ## 0.15.3
> * Blog: https://palletsprojects.com/blog/werkzeug-0-15-3-released/
> * Changes: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-3
> 
</details>
<details>
<summary>Changelog</summary>

*Sourced from [werkzeug's changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst).*

> Version 0.15.3
> --------------
> 
> Released 2019-05-14
> 
> -   Properly handle multi-line header folding in development server in
>     Python 2.7. (:issue:`1080`)
> -   Restore the ``response`` argument to :exc:`~exceptions.Unauthorized`.
>     (:pr:`1527`)
> -   :exc:`~exceptions.Unauthorized` doesn't add the ``WWW-Authenticate``
>     header if ``www_authenticate`` is not given. (:issue:`1516`)
> -   The default URL converter correctly encodes bytes to string rather
>     than representing them with ``b''``. (:issue:`1502`)
> -   Fix the filename format string in
>     :class:`~middleware.profiler.ProfilerMiddleware` to correctly handle
>     float values. (:issue:`1511`)
> -   Update :class:`~middleware.lint.LintMiddleware` to work on Python 3.
>     (:issue:`1510`)
> -   The debugger detects cycles in chained exceptions and does not time
>     out in that case. (:issue:`1536`)
> -   When running the development server in Docker, the debugger security
>     pin is now unique per container.
</details>
<details>
<summary>Commits</summary>

- [`9b1123a`](9b1123a779) release version 0.15.3
- [`00bc43b`](00bc43b167) unique debugger pin in Docker containers
- [`2cbdf2b`](2cbdf2b022) Merge pull request [](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1542) from asottile/exceptions_arent_always_hashable
- [`0e669f6`](0e669f6be5) Fix unhashable exception types
- [`bdc17e4`](bdc17e4cd1) Merge pull request [](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1540) from pallets/break-tb-cycle
- [`44e38c2`](44e38c2985) break cycle in chained exceptions
- [`777500b`](777500b646) Merge pull request [](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1518) from NiklasMM/fix/1510_lint-middleware-python3-compa...
- [`e00c7c2`](e00c7c2ced) Make LintMiddleware Python 3 compatible and add tests
- [`d590cc7`](d590cc7cf2) Merge pull request [](https://github-redirect.dependabot.com/pallets/werkzeug/issues/1539) from pallets/profiler-format
- [`0388fc9`](0388fc95e6) update filename_format for ProfilerMiddleware.
- Additional commits viewable in [compare view](https://github.com/pallets/werkzeug/compare/0.15.2...0.15.3)
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=werkzeug&package-manager=pip&previous-version=0.15.2&new-version=0.15.3)](https://help.github.com/articles/configuring-automated-security-fixes)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot ignore this [patch|minor|major] version` will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/Mailu/Mailu/network/alerts).

</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
bors[bot] dcda412b99
Merge
1211: Split HOST_ANTISPAM in HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI r=mergify[bot] a=micw

## What type of PR?

bug-fix

## What does this PR do?

Fixes  by separating HOST_ANTISPAM into HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI

### Related issue(s)
- closes 
- closes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Michael Wyraz <michael@wyraz.de>
bors[bot] 35160b770d
Merge
1198: Enable access log of admin service only for log levels of INFO and finer r=muhlemmer a=micw

## What type of PR?

bug fix

## What does this PR do?

### Related issue(s)
- closes 

## Prerequistes

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


1204: Add initial admin account to kubernetes example r=muhlemmer a=micw

## What type of PR?

documentation

## What does this PR do?

Add INITIAL_ADMIN_* example to kubernetes configmap.yaml

### Related issue(s)

- closes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- docs example only

1207: Add Japanese translation r=muhlemmer a=IchikawaYukko

## What type of PR?

Translation

## What does this PR do?

Provide completed Japanese translation.

### Related issue(s)

None

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [*] In case of feature or enhancement: documentation updated accordingly
- [*] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


1208: Persist mailqueue r=muhlemmer a=micw

## What type of PR?

bug-fix

## What does this PR do?

Makes postfix mailqueue presistent (for docker, swarm and kubernetes)

### Related issue(s)
- closes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Michael Wyraz <michael@wyraz.de>
Co-authored-by: Michael Wyraz <michael.wyraz@evermind.de>
Co-authored-by: U-COREI3-3120M\市川ゆり子 <ichikawayuriko@yahoo.co.jp>
Co-authored-by: micw <michael@wyraz.de>
Michael Wyraz a907fe4cac Split HOST_ANTISPAM in HOST_ANTISPAM_MILTER and HOST_ANTISPAM_WEBUI
dependabot[bot] ed204766b2
Bump werkzeug from 0.15.2 to 0.15.3 in /core/admin
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 0.15.2 to 0.15.3.
- [Release notes](https://github.com/pallets/werkzeug/releases)
- [Changelog](https://github.com/pallets/werkzeug/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/werkzeug/compare/0.15.2...0.15.3)

Signed-off-by: dependabot[bot] <support@github.com>
U-COREI3-3120M\市川ゆり子 26955df7ae remove POT-Creation-Date, PO-Revision-Date
U-COREI3-3120M\市川ゆり子 efd628c338 add Japanese translation
bors[bot] 950a88bf0e
Merge
1196: Change default password scheme to PBKDF2 () r=mergify[bot] a=hoellen

## What type of PR?
enhancement

## What does this PR do?
This PR change the default password scheme to PBKDF2. It is already changed in some places (e.g. [docs/compose/.env](https://github.com/Mailu/Mailu/blob/master/docs/compose/.env#L142)).

### Related issue(s)
closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: hoellen <dev@hoellen.eu>
Michael Wyraz d689a8eeb3 Enable access log of admin service only for log levels of INFO and finer
hoellen d3dd4802f4 Change default password scheme to PBKDF2 ()
Niduroki b58f06c03e Add a title to Mailu-Admin pages
Michael Wyraz de2f166bd1 Resolve HOST_* to *_ADDRESS only if *_ADDRESS is not already set
Aurélien Bondis 34079244a6 fix ARG positions
Aurélien Bondis e1a4bf32a6 only insert qemu binary if building for arm
kaiyou d50504fa2b Only set the redis address once, fixes
bors[bot] 3e626b8500
Merge
940: Fix host variables r=mergify[bot] a=ionutfilip

## What type of PR?
bug-fix

## What does this PR do?

### Related issue(s)
- Fix 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
bors[bot] 2785bca1f4
Merge
883: Admin create user enhancement r=mergify[bot] a=cr1st1p

## What type of PR?
Enhancement

## What does this PR do?
It allows the admin docker image to also create the admin user.
The idea is that in my kubernetes setup, I do not want to manually do anything, as such, I need a way for the admin user to also be created automatically without me getting inside the pod.
So I had to change the manage.py function that creates the user to allow different 'modes' (me, I'll be using 'ifmissing') and also start.py to call that functionality if appropriate environment variables are present.

So now, in my Deployment, I add 3 more environment variables and I get the admin user created, IF not already present.

### Related issue(s)
none?

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: cristi <cristi.posoiu@gmail.com>
Co-authored-by: cr1st1p <cristi.posoiu@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
Tim Möhlmann 348ea1a572
Remove obsolete log call
Ionut Filip 075417bf90 Merged master and fixed conflicts
Aurélien Bondis 124b1d4c71 rebase and update for 3.10, avoid adding qemu file to x86 images
Tim Möhlmann 7d964874e8
Cleanup spaces around '='
Tim Möhlmann 52971ed42e
Use os.environment.get()
Mildred Ki'Lya 95dce5575b Parameterize redis address
hoellen b720bedf72 Support domain literals for receiving emails
hoellen bed2c6ea09 Revert "Error handling for idna encoding"
This reverts commit 10034526a1.
hoellen 10034526a1 Error handling for idna encoding
bors[bot] 2788909a13 Merge
1052: Upgrade alpine to 3.10 r=mergify[bot] a=Nebukadneza

## What type of PR?
enhancement / bug-fix

## What does this PR do?
Upgrade the alpine base image to 3.10 and clean up ensuing problems. Also directly uses postfix foreground-running with stdout logging.

### Related issue(s)
closes 
closes 

Note: This is a duplicate effort of   …

## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.

With these images, I have tested manually:
- Email receive to user
  - on main domain
  - on additional domain
  - via an alias on main-domain
  - via an alias on additional domain
  - via catchall
- Email reject
  - of eicar testfiles
- Email sending
- Fetchmail from legacy POP
- Front LE certificates
- Simple overriding for postfix (only postfix.cf), nginx, dovecot, rspamd
- Creating a CalDAV calendar and CardDAV entry using davx5

I have not (yet) tested:
- certdumper
- databases other than sqlite

Todo:
- [x] Get rid of podop at `git+https://github.com/Nebukadneza/Podop.git@fix_py37` once https://github.com/Mailu/Podop/pull/6 is merged
- [x] Bend Mailustart back to original repository
- [x] Test more (? also help wanted ?)

Co-authored-by: Thomas Sänger <thomas@gecko.space>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Reto Glauser <git@blinkeye.ch>
kaiyou 23871d9180 Translated using Weblate (Russian)
Currently translated at 100.0% (163 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/ru/
kaiyou 0281b5a595 Translated using Weblate (Italian)
Currently translated at 71.8% (117 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/it/
kaiyou 45789d3f22 Translated using Weblate (Hungarian)
Currently translated at 100.0% (163 of 163 strings)

Translation: Mailu/admin
Translate-URL: https://translate.tedomum.net/projects/mailu/admin/hu/
Bambie07 e70d1e1455
fix dns dkim entry
Dario Ernst 8c2e22f666 Merge branch 'master' into HorayNarea-feat-upgrade-alpine
bors[bot] f77e1bdd0e Merge
1048: Refactor admin using webpack r=mergify[bot] a=kaiyou

## What type of PR?

Refactoring

## What does this PR do?

This PR brings some refactoring to the admin container :
- remove the dependency to mailustart and replace it with socrate
- remove static assets from the repo and use Webpack for building them


Co-authored-by: hoellen <dev@hoellen.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
Dario Ernst 0306be1eed Re-add missing MailuStar in admin
It turns out we were all blind and admin *does* use MailuStart
Dario Ernst ce0c24e076 Merge branch 'master' into HorayNarea-feat-upgrade-alpine
Dario Ernst 53f754f5ac Remove MailuStart from admin and correct layer-sharing comments
hoellen 31464c5c44 Change image back to alpine
hoellen b351841418 Expose jQuery for use outside Webpack build
hoellen 90f678de52 add app.css and fix align of menu button
hoellen 616d40d225 Update password in commandline
Dario Ernst bb2edb6eb6 Revert "Move alpine version definition out to variable"
This reverts commit c787e4bdbd.
hoellen 81a8acf9ec fix resolve issue
Dario Ernst c787e4bdbd Move alpine version definition out to variable
Dario Ernst a253ca47fe Use official Mailu/MailuStart
Dario Ernst d1f80cca99 Update Dockerfiles to most recent alpine 3.10
Thomas Sänger ef3c6c407a upgrade alpine base-image
kaiyou baa5a8a4e0 Fix hostname resolution
kaiyou c20a502695 Do not forcefully resolve optional hostnames
kaiyou a3abf8c353 Add the assets source
kaiyou 4b620ba5d1 Merge branch 'hoellen-fix-sidebar-toggle' into refactor-admin-webpack
kaiyou c147a371d7 Merge branch 'fix-sidebar-toggle' of https://github.com/hoellen/Mailu into hoellen-fix-sidebar-toggle
kaiyou 2d3560b396 Fix select2 using webpack-built assets
kaiyou c2a04ab1d4 Update the build configuration to use webpack and compiled assets
kaiyou ecdf0c25b3 Use webpack for building static assets
kaiyou d7747639e9 Remove the dependency to mailustart, introducing socrate
cristi f2a009795b Merge branch 'master' into admin_create_user_enhancement
bors[bot] f1f5aef7d3 Merge
931: Resolve webmail in admin r=mergify[bot] a=ionutfilip

## What type of PR?
bug-fix

## What does this PR do?
Implement mailustart to resolve webmail in admin

### Related issue(s)
Fix  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
bors[bot] 2c90ac3334 Merge
962: Open some admin sidebar links in a new browser tab r=kaiyou a=mariooos

## What type of PR?

Enhancement

## What does this PR do?

Add target=_blank attribute to html <a href ...> tag in following admin sidebar links:
- Antispam
- Webmail
- Website
- Help


Co-authored-by: mariooos <48351788+mariooos@users.noreply.github.com>
bors[bot] 7021278bfb Merge
993: Make aliases case-insensitive (too) r=mergify[bot] a=Nebukadneza

Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.

For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …

closes 

## What type of PR?
bug-fix

## What does this PR do?
Make aliases optionally case-insensitive: After attempting to resolve an alias in its preserved case, also attempt to match it case-insensitively.

This followed after some more thought from   …

### Related issue(s)
closes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Dario Ernst <dario@kanojo.de>
bors[bot] d76ba5ee39 Merge
1005: distinguish disabled users in user list view by row color r=kaiyou a=zhuangyy

## What type of PR?

enhancement

## What does this PR do?

distinguish disabled users in user list view by row color

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Zhuang Yuyao <jssyan@qq.com>
kaiyou 645c423d09 Update messages.po (POEditor.com)
kaiyou 08bdc715bb Update messages.po (POEditor.com)
kaiyou a150315c1d Added translation using Weblate (Japanese)
Weblate abece125f4 Merge branch 'origin/master' into Weblate
kaiyou d99ad5fe3e Added translation using Weblate (Hungarian)
kaiyou b91c7f64ee Update messages.po (POEditor.com)
kaiyou d929413784 Update messages.po (POEditor.com)
kaiyou 91c8ca1279 Update messages.po (POEditor.com)
kaiyou c0263a2ae9 Update messages.po (POEditor.com)
kaiyou 80c57b597b Update messages.po (POEditor.com)
kaiyou 0566b00664 Update messages.po (POEditor.com)
kaiyou c939b30e6b Update messages.po (POEditor.com)
kaiyou 308521ff9a Update messages.po (POEditor.com)
kaiyou 0ca18035e1 Update messages.po (POEditor.com)
kaiyou 8bcb4deb7e Update messages.po (POEditor.com)
kaiyou 65a61ad5c8 Added translation using Weblate (Icelandic)
kaiyou 57385df15c Deleted translation using Weblate (Icelandic)
kaiyou cfc224352a Added translation using Weblate (Icelandic)
Zhuang Yuyao 67d5c8fcb3 distinguish disabled user in user list view by row color
Dario Ernst e22324adcd Make aliases case-insensitive (too)
Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.

For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …

closes 
hoellen 167e5a87e1 add sidebar toggle button
hoellen 92ef73fa74 update AdminLTE to version 2.4
mariooos 253ae9fad8 Open some admin sidebar links (Antispam, Webmail, Website, Help) in a new browser tab
Ionut Filip ac0fc41421
Merge branch 'master' into replace-gethostbyname
Ionut Filip dd7710951e
Replaced double quotes with single ones
Ionut Filip 0ce0b5ec02 Fix addresses in front
Ionut Filip 4c25c83419 HOST_* and *_ADDRESS variables cleanup
Ionut Filip f9e3cd3c5d Use corret host_* variables
Ionut Filip ef49357eb3 Update redis urls
Ionut Filip 43abbf4d63 Resolve redis and add logging
bors[bot] 211136f9b1 Merge
934: Pass optional arguments to domain model r=mergify[bot] a=ionutfilip

## What type of PR?
bug-fix

## What does this PR do?
Passing optional args to domain model when using cli.

### Related issue(s)
Fixes 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
bors[bot] d129733fac Merge
919: Install bash in alpine based images. r=mergify[bot] a=firvida

This fixes 

Bash shell is used by default in Kubernetes' dashboard console, which is very
useful for admins.

## What type of PR?

bug-fix

## What does this PR do?

### Related issue(s)
- closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: Abel Alfonso Fírvida Donéstevez <abel@merchise.org>
hoellen 62511b723b
Merge branch 'master' into fix-doc-commands-1
Ionut Filip 4d2844c1f5 Fixed arguments passed to domain
Roman Hoellen 1dfef1609a add domain cli to docs
Ionut Filip cebc64a280 Resolve HOST_WEBMAIL in admin
Roman Hoellen bc617cf8ec replace _ with - in cli options
Tim Möhlmann 111dae351a
Fix typo in migration file
Fixes 
Abel Alfonso Fírvida Donéstevez 39444c794e Install bash in alpine based images.
This fix https://github.com/Mailu/Mailu/issues/918

Bash shell is used by default in Kubernetes' dashboard console, which is very
useful for admins.
bors[bot] 86b4242f82 Merge
886: Ipv6 support r=mergify[bot] a=muhlemmer

## What type of PR?

(Feature, enhancement, bug-fix, documentation) -> A bit of everything

## What does this PR do?

Document how to use ipv6nat. This, however triggers some kind of flaky behavior with the Docker DNS resolver, resulting in lookup failures between containers.  So all resolving needs to be done during container startup/configuration.

In order not to pollute every single start.py file, we've created a small library called [Mailu/MailuStart](https://github.com/Mailu/MailuStart). As an addition, this library also defines the template generation function, including its logging facility.

Note: `docker-compose.yml` downgrade is necessary, as IPv6 settings are not supported by the Docker Compose file format 3 😞  

### Related issue(s)
Supersedes  PR 
- Fixes  
- Hopefully helps with  and 

## No backport yet

This PR directly imports MailuStart from git. This makes it a bit more simple to implement on the short term an do some testing and probably some future improvements. When everything is proved stable, we will create a proper PyPi package with versioning and consider back porting.

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.


Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
Tim Möhlmann 06457e9617
Merge branch 'master' into fix_fetches_create
cr1st1p 9b65a6c862
Merge branch 'master' into admin_create_user_enhancement
Ionut Filip f8dffe5a19
Resolve hosts in admin
Dario Ernst 34b31727c4 Fix password validator for creating fetched accounts
cristi a071181c61 Allow to automatically create admin account during startup.
hoellen fba246e9a1 add wildcard option to cli (alias)
hoellen 808092c65e
Merge pull request from hoellen/token-ipv6
allow ipv6 addresses for tokens
Tim Möhlmann e341e0141f
Merge remote-tracking branch 'upstream/master' into token-ipv6
Manuel Vázquez Acosta dfe5ddb18e
Check if sender address is not null before trying to check the domain.
Tim Möhlmann fd236e4ea5
Fix syntax error
nathan-sain 23bad0e042
Set value for SQLALCHEMY_DATABASE_URI in admin DEFAULT_CONFIG to SQLite URI
nathan-sain db8977e17a
Update SQLite DB file in DEFAULT_CONFIG and DB_TEMPLATES
hoellen 6dea8b422a
Merge pull request from hoellen/fix-create-onupdate-default
fix default value for created_at and updated_at
hoellen 79c4edcaf1
Merge pull request from hoellen/schema-maxUserQuota-1
Change quota columns type to BigInteger
hoellen 54169db7e3
Merge branch 'master' into fix-forward-validation
hoellen 9721df0bc5 fix default value for created_at and updated_at
Use date instead of datetime for created_at and updated_at.
hoellen cfa7ca3838 Merge branch 'master' into schema-maxUserQuota-1
hoellen f1e1c96c3b create migration file for changing quota to big integer
Ionut Filip 2d34f0ee52 Fixed auto-forward email validation
hoellen c8758a6526 allow ipv6 addresses for tokens
hoellen 2af540a1c9 change quota columns to bigint
Dario Ernst 66df7a31b0 Unify and coerce booleans from env used in admin
At some places, the string that DOMAIN_REGISTRATION is got used like a boolean
(an easy misassumption to make while in python and dealing with the config
dict), making `DOMAIN_REGISTRATION=False` act as a truthy value. To stop such
future problems from happening, coerce environment config strings to real
bools.

closes 
mergify[bot] 9175b15d49
Merge pull request from Nebukadneza/reverse_user_alias_pref
Reverse the resolution order of user and alias
mergify[bot] 278bcfb13a
Merge pull request from Nebukadneza/fix_delimiter_alias
Deliver mails to alias-stripped-of-delimeter, even if catchall exists