Commit Graph

990 Commits (071ad15a97df5135b8edddec05d26a69ac6dc90b)

Author SHA1 Message Date
Florent Daigniere 6bf1a178b9 Go with ghostwheel42's suggestion
Florent Daigniere b68033eb43 only parse it once
Alexander Graf 82e14f1292
Merge branch 'master' into update_deps
bors[bot] f0188d9623
Merge
2034: Add timezone to containers r=mergify[bot] a=DjVinnii

## What type of PR?

Enhancement

## What does this PR do?
This PR adds the tzdata package so that the environment variable `TZ` can be used to set the timezone of containers.

### Related issue(s)
- closes  

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: DjVinnii <vincentkling@msn.com>
Florent Daigniere dc6e970a7f handle HTTP too
Florent Daigniere bbef4bee27 Don't return any key for relayed domains
We may want to revisit this (ARC signing)... but in the meantime
it saves from a scary message in rspamd

signing failure: cannot request data from the vault url: /internal/rspamd/vault/v1/dkim/ ...
Florent Daigniere 6c6b0b161c Set the right flags on the rate_limit cookie
Florent Daigniere f9373eacab Merge remote-tracking branch 'upstream/master' into misc
Florent Daigniere 5714b4f4b0 introduce MESSAGE_RATELIMIT_EXEMPTION
DjVinnii 30d7e72765 Move TZ to Advanced settings
DjVinnii 225160610b Set default TZ in Dockerfiles
DjVinnii 81e33d3679 Add default TZ to config manager
Alexander Graf 97e79a973f fix sso login button spacing again
Alexander Graf 73ab4327c2 updated database libraries (sqlalchemy etc.)
this is working fine, but introduces a sqlalchemy warning
when using config-import:

  /app/mailu/schemas.py:822:
    SAWarning: Identity map already had an identity for (...),
    replacing it with newly flushed object.
    Are there load operations occurring inside of an event handler
    within the flush?
Alexander Graf 4669374b9e use python wheels
Alexander Graf 85d86d4156 some more libs updated
Alexander Graf ffd99c3fa8 updated flask
ConfigManager should not replace app.config - this is causing trouble
with some other flask modules (swagger).
Updated ConfigManager to only modify app.config and not replace it.
Alexander Graf 87884213c4 update misc helper libs
Alexander Graf 56f65d724d update babel
Alexander Graf 5238b00f0b update alembic
Alexander Graf f613205fe1 update tenacity
Alexander Graf 833ccb5544 reload page using GET when selecting language
Alexander Graf 8b15820b01 fix sso login button spacing
Alexander Graf 26fb108a3f updated Flask-Login
Alexander Graf abc4112242 updated Werkzeug, Click and Flask-Migrate
Alexander Graf f1d7bedd1b fix display of range inputs (again)
Alexander Graf 13e6793c9f Merge remote-tracking branch 'upstream/master' into update_deps
Alexander Graf aca1e13648 update socrate - will be removed later
Alexander Graf 866741bcbe updated WTForms-Components deps
Alexander Graf ef19869cde updated redis
Alexander Graf d8efd3057c updated idna
Alexander Graf 8ad8cde0e2 removed some obsolete requirements
Alexander Graf 3ac1b3d86c update pyyaml and pygments
Alexander Graf 40cdff4911 updated dnspython
Alexander Graf dcbe55f062 updated crypto
Alexander Graf 771b2d1112 duh
Alexander Graf 23d0cd0466 update tabluate. fix audit.py and include in container
Alexander Graf 8d90a74624 update werkzeug to 1.x
bors[bot] 5e212ea46d
Merge
2036: round display of range inputs to 2 decimals r=mergify[bot] a=ghostwheel42

## What type of PR?

small fix

## What does this PR do?

rounds display of range inputs to 2 decimals 

### Related issue(s)

- small fix to 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 80be3506da upgrade pip. completed reqs via pip freeze
Alexander Graf 598b2df5a0 update wtforms
Alexander Graf e8b5f1a185 round display of range inputs to 2 decimals
DjVinnii 1d6809193b Add tzdata to core
Florent Daigniere f3c93212c6 The Rate-limiter should run after the deny
Alexander Graf 9bc685c30b removed some more whitespace
Alexander Graf 8c31699baf fixed locale selector for no_NB
Alexander Graf 882a27f87c simplified if's and added external link icon
Alexander Graf 3141ffe791 removed some whitespace
Dimitri Huisman 6b16756d92 Fix acessing antispam via sidebar.
Dimitri Huisman 3449b67c86 Process code review remarks PR2023
Dimitri Huisman 8784971b7f Merge rate limiting and failed login logging
Dimitri Huisman 503044ef6e Reintroduce ProxyFix. Use two buttons for logging in.
Dimitri Huisman c42ad8e71e Forgot to include changes for url_for of base.html
Dimitri Huisman fb0f005343 Get rid of complicated prefix logic. Further simplify /static handling and nginx config.
Dimitri Huisman da788ddee3 Merge branch 'fix-sso-1929' of github.com:Diman0/Mailu into fix-sso-1929
Dimitri Huisman bdcc183165 Redirect to configured ENV VAR for Admin/Webmail, further simplify nginx config.
Florent Daigniere fee13e6c4b Save a redirect
Florent Daigniere d3f07a0882 Simplify the handling of /static
Dimitri Huisman a47afec4ee Make logic more readable.
Dimitri Huisman 48764f0400 Ensure all requests from the page sso go through the page sso.
Dimitri Huisman aab258d284 Move handling of logging out in admin, to sso logout page.
Dimitri Huisman 615743b331 Improve indendation of conditions.
Dimitri Huisman 5d81846c5d Introduce the shared stub /static for providing all static files
Dimitri Huisman 44d2448412 Updated SSO logic for webmails. Fixed small bug rate limiting.
Dimitri Huisman f9eee0cbaf Adapt HEALTHCHECK to new URL
Dimitri Huisman ed7adf52a6 Merge branch 'master' of github.com:Diman0/Mailu into fix-sso-1929
Dimitri Huisman 913a6304a7 Finishing touches. Introduce /static stub for handling all static files.
bors[bot] a1192d8039
Merge
1987: Enhancement to the rate limits r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Turn the rate-limiters into something useful (that won't fire for no reason).

- fix rate-limiting on /webdav/
- it changes the rate-limiting behaviour from limiting a single IP address to a subnet of a reasonable size (/24 on v4 and /56 on v6 both are now configurable) : AUTH_RATELIMIT_IP / AUTH_RATELIMIT_IP_V4_MASK / AUTH_RATELIMIT_IP_V6_MASK
- It ensures we only use IP-based rate-limits for attempts on accounts that do not exist
- it creates a new rate limit preventing attackers from targetting a specific user account (separate from what's above) : AUTH_RATELIMIT_USER
- it introduces a rate limiting exemption mechanism whereby, upon authentication, users will see their source-ip address being exempt for a specific amount of time AUTH_RATELIMIT_EXEMPTION_LENGTH. A similar mechanism is available for web-based sessions (see below)
- It introduces in AUTH_RATELIMIT_EXEMPTION a comma separated list of network CIDRs that will be exempt from both types of rate limiting
- it implements device-tokens, as described on https://owasp.org/www-community/Slow_Down_Online_Guessing_Attacks_with_Device_Cookies to ensure that genuine users aren't locked-out by a malicious attacker abusing the rate-limit feature.

Things that could be improved include:
- the IP-based rate limiter flags attempts against "non-existing" accounts: it could go further and flag the number of unique non-existing accounts attempted (to prevent the case of a user making a typo in his MUA configuration)
- the IP address exemption mechanism doesn't pin the exemption to a specific username: any real user can trivially bypass the rate limits (and attempt to brute-force someone else's account)

### Related issue(s)
- close 
- close  
- close 


## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Florent Daigniere 693b578bbb The second strip isn't necessary
Florent Daigniere 1c6165213c better that way
Florent Daigniere 34497cff20 doh
Florent Daigniere e8871dd77f doh
Florent Daigniere 5b72c32251 Doh
Florent Daigniere 19b784b198 Parse the network configuration only once
thanks @ghostwheel42
Florent Daigniere 98742268e6 Make it more readable
Florent Daigniere 94bbed9746 Ensure we have the right IP
Florent Daigniere c5bd82650f doh
Florent Daigniere 99c81c20a7 Introduce AUTH_RATELIMIT_EXEMPTION
This disables rate limiting on specific CIDRs
Florent Daigniere c674f1567a Merge branch 'ratelimits' of https://github.com/nextgens/Mailu into ratelimits
Florent Daigniere 8414dd5cf0 Merge remote-tracking branch 'upstream/master' into ratelimits
Florent Daigniere e14d2e7c03 Error out explictely if Auth-Port isn't set
Florent Daigniere abaa2e8cc3 simplify client_ip
Florent Daigniere de276a6822 Simplify extract_network_from_ip
Florent Daigniere 3bda8368e4 simplify the Auth-Status check
Florent Daigniere 2dd9ea1506 simplify
Florent Daigniere 068170c0ff Use app instead of flask.current_app where possible
Florent Daigniere 57b0dd490c Initialize user_email in all cases
qy117121 b1425015ef
Update messages.po
Fix wrong text
bors[bot] afffe4063e
Merge
2018: show dmarc record for report domain in domain details r=mergify[bot] a=ghostwheel42

## What type of PR?

documentation

## What does this PR do?

show dmarc record for report domain in domain details

### Related issue(s)

closes 

## Prerequisites

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9f2aa0aadc
Merge
1986: Document how to setup client autoconfig r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Document how to setup autoconfig. This works with most open-source MUAs (thunderbird, evolution, ...)

We could go further than that by providing dynamic configuration (issue an auth token for each MUA request)... but it won't work unless a new DNS entry (and matching certificate) is created.

### Related issue(s)
- 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2014: Update Chinese translation r=mergify[bot] a=qy117121

## What type of PR?

translation

## What does this PR do?

Update Chinese translation. Use `zh` instead of `zh_CN`.

### Related issue(s)

none

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: qy117121 <mixuan121@gmail.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 7fe15ea9cf added dmarc record for report domain
bors[bot] a5b1d36171
Merge
2017: rspamd: get dkim keys via REST API instead of filesystem r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement

## What does this PR do?

rspamd now uses hashicorp's vault api v1 to get dkim keys and selectors for a domain.
this allows future enhancement (multiple keys) without reconfiguring and restarting rspamd.
it also makes mounting the /dkim volume into the rspamd container unnecessary.

### Related issue(s)

- improves and closes  
- allows to implement key rotation using multiple selectors (see )
- allows to implement dkim for alternate domains (see )
- fixes and closes  (selector transmitted by admin container is used)
- closes  (no keys on disk)
- allows to implement key rotation from the outside (ie. via a helper script talking to some dns provider's api) (see )

## Prerequisites

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 7b0c5935a8 only support GET method in vault
Alexander Graf 303fae00fb cleanup modules. use dkim selector from config
Alexander Graf dc9f970a91 removed zh_CN and updated locale-map for datatables
Alexander Graf 893705169e PoC rspamd use dkimkeys from admin using vault api
Florent Daigniere 632ce663ee Prevent logins with no password
qy117121 866f784d06
Create messages.po
Update the translation
qy117121 251eea5553
Update messages.po
Updated translation
Florent Daigniere 7277e0b4e4
Merge branch 'master' into ratelimits
bors[bot] 8c8c1b2015
Merge
1997: Prevent traceback when using non-email in login r=mergify[bot] a=ghostwheel42

There's a traceback when the username used to log via SMTPAUTH
in is not an email address:

=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 60, in process_bind_param
    localpart, domain_name = value.lower().rsplit('`@',` 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```

=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "`@")`
```

## What type of PR?

enhancement

## What does this PR do?

replace traceback (ERROR) with error message (WARNING)

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9b01e663b2
Merge
2007: allow sending emails as user+detail@domain.tld r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix or enhancement

## What does this PR do?

Allows sending emails with an added "+detail" in the local part.
 
### Related issue(s)

closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: root <ghostwheel42@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 14360f8926 RECIPIENT_DELIMITER can have several characters
root 8c59f35697 use RECIPIENT_DELIMITER for splitting
Alexander Graf 1d571dedfc split localpart into user and tag
Florent Daigniere d131d863ba The if needs to be inside the block
Alexander Graf aaf3ddd002 moved javascript to app.js
Florent Daigniere b48779ea70 SESSION_COOKIE_SECURE and HTTP won't work
Florent Daigniere 10d78a888b Derive a new subkey for SRS
Alexander Graf 65133a960a Prevent traceback when using non-email in login
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:

=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 60, in process_bind_param
    localpart, domain_name = value.lower().rsplit('@', 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```

=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "@")
```
Diman0 f4cde61148 Make header translatable. More finishing touches.
Florent Daigniere 7d56ed3b70 Merge branch 'master' of https://github.com/Mailu/Mailu into ratelimits
Diman0 fbe0a446b9 Merge branch 'master' of github.com:Mailu/Mailu into fix-sso-1929
Florent Daigniere 1e07b85fa1 doh
Diman0 9894b49cbd Merge/Update with changes from master
Florent Daigniere 24aadf2f52 ensure we log when the rate limiter hits
Florent Daigniere 64bc7972cc Make AUTH_RATELIMIT_IP 60/hour as discussed
Florent Daigniere cab0ce2017 doh
Florent Daigniere a9340e61f5 Log auth attempts on /admin
Florent Daigniere 89ea51d570 Implement rate-limits
Diman0 bf0aad9820 Merge branch 'master' of github.com:Mailu/Mailu into fix-sso-1929
bors[bot] 4c5c6c3b5f
Merge
1966: AdminLTE3 optimizations & compression and caching r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement, bugfix

## What does this PR do?

Optimization and cleanup of styles and javascript code for AdminLTE 3
Adds caching headers, gzip and robots.txt to nginx.

### Related issue(s)

Makes  even better. Thanks to `@DjVinnii` and `@Diman0` for the good work.
Closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] b329971b87
Merge
1971: Updated Polish translation. r=mergify[bot] a=ghostwheel42

## What type of PR?

translation

## What does this PR do?

Update polish translation. Used `pl/LC_MESSAGES/messages.po` from PR  created by `@martys71`
Part of Discussion of 1.9 roadmap 

### Related issue(s)

closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 25cf8b5358 better help formatting
Alexander Graf b63081cb48 display error (not exception) when creating admin
repleace misleading python exception (mailu broken)
with error message stating that the admin user is
already present
Alexander Graf 7bec8029a4 strip not necessary anymore
Alexander Graf 1e8b41f731 Merge remote-tracking branch 'upstream/master' into adminlte3_fixes
Alexander Graf b883e3c4a6 duh.
Alexander Graf bb40ccc4b0 normalize HOSTNAMES
should be moved to python lib and normalized in start.py
Alexander Graf 45a2be3766 Updated Polish translation.
Used pl/LC_MESSAGES/messages.po from PR#1751 created by martys71
bors[bot] d464187477
Merge
1964: Alpine3.14.2 r=mergify[bot] a=nextgens

Upgrade to alpine 3.14.2, retry upgrading unbound & switch back to libressl

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 0094268410 allow to change logo. default color for flash msg
- two new environment variables allow to change logo background color
  and graphic
- flash messages are now green (not cyan)
Alexander Graf d8b4a016af use blue color from https://mailu.io/
bors[bot] 6fe265b548
Merge
1968: optimize handle_authentication r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

catch utf-8 decoding errors and log a warning in handle_authentication instead of writing a traceback into the log.

### Related issue(s)

closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 90c96bdddc optimize handle_authentication
- catch decoding of nginx headers (utf-8 exception)
- re-ordered function
Florent Daigniere 0ee52ba65b Doh
Florent Daigniere 0f0459e9b2 suggestions from @ghostwheel42
Alexander Graf 7bede55fce more verbose cleaning message
Florent Daigniere a9a1b3e55e Reduce the EDNS0 size to 1232
@see
https://github.com/dns-violations/dnsflagday/issues/125
Alexander Graf 7fd605cc21 fixed brand link target for normal users
Diman0 b148e41d9b Fix nginx config
Florent Daigniere d8c22db547 Merge remote-tracking branch 'upstream/master' into policyd-mta-sts
Alexander Graf 8cdd7e911d duh. removed debug
Alexander Graf 2ba0d552e0 Merge remote-tracking branch 'upstream/master' into passlib
Alexander Graf 34df8b3168 AdminLTE3 optimizations & compression and caching
- fixed copy of qemu-arm-static for alpine
- added 'set -eu' safeguard
- silenced npm update notification
- added color to webpack call
- changed Admin-LTE default blue
  (core/admin/Dockerfile)

- AdminLTE 3 style tweaks
  (core/admin/assets/app.css)
  (core/admin/mailu/ui/templates/base.html)
  (core/admin/mailu/ui/templates/sidebar.html)

- localized datatables
  (core/admin/Dockerfile)
  (core/admin/assets/app.js)
  (core/admin/package.json)

- moved external javascript code to vendor.js
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/webpack.config.js)

- added mailu logo
  (core/admin/assets/app.js)
  (core/admin/assets/app.css)
  (core/admin/assets/mailu.png)

- moved all inline javascript to app.js
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/domain/create.html)
  (core/admin/mailu/ui/templates/user/create.html)

- added iframe display of rspamd page
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/views/base.py)
  (core/admin/mailu/ui/templates/sidebar.html)
  (core/admin/mailu/ui/templates/antispam.html)

- updated language-selector to display full language names and use post
  (core/admin/assets/app.js)
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/utils.py)
  (core/admin/mailu/ui/views/languages.py)

- added fieldset to group and en/disable input fields
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/user/settings.html)
  (core/admin/mailu/ui/templates/user/reply.html)

- added clipboard copy buttons
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/domain/details.html)

- cleaned external javascript imports
  (core/admin/assets/vendor.js)

- pre-split first hostname for further use
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/client.html)
  (core/admin/mailu/ui/templates/domain/signup.html)

- cache dns_* properties of domain object (immutable during runtime)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/domain/details.html)

- fixed and splitted dns_dkim property of domain object (space missing)
- added autoconfig and tlsa properties to domain object
  (core/admin/mailu/models.py)

- suppressed extra vertical spacing in jinja2 templates
- improved accessibility for screen reader
  (core/admin/mailu/ui/templates/**.html)

- deleted unused/broken /user/forward route
  (core/admin/mailu/ui/templates/user/forward.html)
  (core/admin/mailu/ui/views/users.py)

- updated gunicorn to 20.1.0 to get rid of buffering error at startup
  (core/admin/requirements-prod.txt)

- switched webpack to production mode
  (core/admin/webpack.config.js)

- added css and javascript minimization
- added pre-compression of assets (gzip)
  (core/admin/webpack.config.js)
  (core/admin/package.json)

- removed obsolte dependencies
- switched from node-sass to dart-sass
  (core/admin/package.json)

- changed startup cleaning message from error to info
  (core/admin/mailu/utils.py)

- move client config to "my account" section when logged in
  (core/admin/mailu/ui/templates/sidebar.html)
Diman0 960033525d configure sso in nginx
Diman0 8868aec0dc Merge master. Make sso login working for admin.
Diman0 1cfc9ee1c4 Merge branch 'master' of github.com:Diman0/Mailu into fix-sso-1929
Diman0 9fac3d7ad3 Initial implementation for standalone sso page
Florent Daigniere d7c2b510c7 Give alpine 3.14.2 a shot
Florent Daigniere fe186afb6f Revert "Switch to openssl to workaround alpine #12763"
This reverts commit f8362d04e4.
Florent Daigniere c1d94bb725 Ensure that postfix will be able to use the TLSA records
see https://www.huque.com/dane/testsite/ for the testcases
Florent Daigniere 9f66e2672b Use DEFER_ON_TLS_ERROR here too
We just don't know whether the lookup failed because we are under attack
or whether it's a glitch; the safe behaviour is to defer
Florent Daigniere a1da4daa4c Implement the DANE-only lookup policyd
https://github.com/Snawoot/postfix-mta-sts-resolver/issues/67 for
context
Dimitri Huisman 5f18860669 Remove workaround. Remove deprecated url-loader.
Dimitri Huisman 60be06e298 Temporary workaround to get FontAwesome icons working.
Dimitri Huisman 5da7a06675 Resolve webpack.config.js error
Dimitri Huisman 00276d8b70
Merge branch 'master' into AdminLTE-3
bors[bot] 6e32092abd
Merge
1873: Completed Hebrew translation r=mergify[bot] a=yarons

The Hebrew translation is incomplete so I've completed it.

Co-authored-by: Yaron Shahrabani <sh.yaron@gmail.com>
Dimitri Huisman 169a540692 Use punycode for HTTP header for radicale and create changelog
Dimitri Huisman 4f5cb0974e Make sure HTTP header only contains ASCII
Florent Daigniere b4102ba464 doh
Florent Daigniere 9ec7590171 Merge branch 'master' of https://github.com/Mailu/Mailu into wildcard_senders
Florent Daigniere 7252a73e11 WILDCARD_SENDERS can have spaces
bors[bot] b57df78dac
Merge
1916: Ratelimit outgoing emails per user r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

A conflict-free version of  implementing per-user sender limits

### Related issue(s)
- close  
- close 
- close  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Dimitri Huisman e5972bd9ec Set default message rate limit to 200/day
Florent Daigniere facc4b6427 Allow specific users to send email from any address
Diman0 5afbf37292 Resolve build issues
Dimitri Huisman df64601b28
Merge branch 'master' into AdminLTE-3
Florent Daigniere dccd8afd51 Thanks @Diman0!
ENEEDSLEEP
Florent Daigniere 5e7d5adf17 AUTH shouldn't happen on port 25
Florent Daigniere 6d244222da better error message
Florent Daigniere 1438253a06 Ratelimit outgoing emails per user
Diman0 588904078e Set default of AUTH_RATELIMIT_SUBNET to False. Increase default AUTH_RATELIMIT value.
Florent Daigniere defea3258d update arm builds too
Florent Daigniere d44608ed04 Merge remote-tracking branch 'upstream/master' into upgrade-alpine
Florent Daigniere f8362d04e4 Switch to openssl to workaround alpine
bors[bot] 6ea4e3217a
Merge
1901: treat localpart case insensitive again r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

fixes error introduced by  where the localpart of an email address was handled case sensitive.
this screwed things up at various other places.
 
### Related issue(s)

closes 
closes 

Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 6856c2c80f treat localpart case insensitive again
by lowercasing it where necessary
bors[bot] 656cf22126
Merge
1856: update asset builder dependencies r=mergify[bot] a=ghostwheel42

## What type of PR?

update asset builder dependencies

## What does this PR do?

only include needed dependencies to build mailu assets with nodejs v8

### Related issue(s)

update dependencies as discussed in 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9289fa6420
Merge
1896: save dkim key after creation r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

saves generated dkim key after creation vi web ui.
after the model change the domain object needs to be added and flushed via sqlalchemy.

### Related issue(s)

closes 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9a4c6385e5
Merge
1888: Use threads in gunicorn rather than workers/processes r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available

"smarter default"

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 54b46a13c6 save dkim key after creation
Alexander Graf c2c3030a2f rephrased comments
Alexander Graf ad1b036f20 fix Email class
Florent Daigniere 8d9f3214cc Use threads in gunicorn rather than processes
This ensures that we share the auth-cache... will enable memory savings
and may improve performances when a higher number of cores is available

"smarter default"
Yaron Shahrabani e0bf75ae17
Completed Hebrew translation
bors[bot] c5ff72d657
Merge
1857: disable startdate when autoreply is disabled r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

disable the reply startdate field when autoreply is disabled


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Florent Daigniere a0dcd46483 fix : Handle colons in passwords
Alexander Graf 180026bd77 also disable startdate
Alexander Graf 56cfcf8b64 converted tabs to spaces
Alexander Graf 6377ccb2cb re-add jquery and select2 used in app.js
Alexander Graf 3c8a8aa8f0 use less v3 to make less-loader happy
Alexander Graf 1bb059f4c1 switched to newest possible versions for nodejs v8
Alexander Graf 858312a5cb remove explicit jQuery dependency
Alexander Graf 3f91dcb7af compile scheme list using a generator
Alexander Graf 3bb0d68ead add cargo to build cryptography
Alexander Graf 9790dcdabe updated dependencies
Florent Daigniere 420afa53f8 Upgrade to alpine 3.14
bors[bot] 4a5f6b1f92
Merge
1791: Enhanced session handling r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

- replaces flask_kvsession and simplekv with a mailu-specific session store
- call cleanup_sessions before first request and not on startup.
  this allows to run cmdline actions without redis (and makes it faster)
- allow running without redis for debugging purposes by setting MEMORY_SESSIONS to True
- don't sign session id, as it has plenty of entropy (as suggested by nextgens)
- adds method to prune a user's sessions

### Related issue(s)
- enhances and close 


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 8b71a92219 use fixed msg for key derivation
Alexander Graf 92896ae646 fix bugs in model and schema introduced by
Alexander Graf 6740c77e43 small bugfix for exception
Alexander Graf fab3168c23 Merge remote-tracking branch 'upstream/master' into kvsession
Alexander Graf fbd945390d cleaned imports and fixed datetime and passlib use
Dimitri Huisman 6dc1a19390
Merge branch 'master' into import-export
bors[bot] fc1a663da2
Merge
1754: centralize Webmail authentication behind the admin panel (SSO) r=mergify[bot] a=nextgens

## What type of PR?

Enhancement: it centralizes the authentication of webmails to the admin interface.

## What does this PR do?

It implements the glue required for webmails to do SSO using the admin interface.
One of the main advantages of centralizing things this way is that it reduces significantly the attack surface available to an unauthenticated attacker (no webmail access until there is a valid Flask session).

Others include the ability to implement 2FA down the line and rate-limit things as required.

### Related issue(s)
- 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
bors[bot] 4ff90683ca
Merge
1758: Implement a simpler credential cache (alternative to ) r=mergify[bot] a=nextgens

## What type of PR?

Feature: it implements a credential cache to speedup authentication requests.

## What does this PR do?

Credentials are stored in cold-storage using a slow, salted/iterated hash function to prevent offline bruteforce attacks. This creates a performance bottleneck for no valid reason (see the
rationale/long version on https://github.com/Mailu/Mailu/issues/1194#issuecomment-762115549).

The new credential cache makes things fast again.

This is the simpler version of  (with no new dependencies)

### Related issue(s)
- close 
- close  
- close 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


1776: optimize generation of transport nexthop r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix and enhancement.

## What does this PR do?

Possibly there should be more input validation when editing a relay, but for now this tries to make the best out of the existing "smtp" attribute while maintaining backwards compatibility. When relay is empty, the transport's nexthop is the MX of the relayed domain to fix  

```
RELAY			NEXTHOP						TRANSPORT
empty			use MX of relay domain				smtp:domain
:port			use MX of relay domain and use port	smtp:domain:port
target			resolve A/AAAA of target			smtp:[target]
target:port		resolve A/AAAA of target and use port	smtp:[target]:port
mx:target		resolve MX of target				smtp:target
mx:target:port	resolve MX of target and use port	smtp:target:port
lmtp:target		resolve A/AAAA of target			lmtp:target
lmtp:target:port	resolve A/AAAA of target and use port	lmtp:target:port

target can also be an IPv4 or IPv6 address (an IPv6 address must be enclosed in []: [2001:DB8::]).
```

When there is proper input validation and existing database entries are migrated this function can be made much shorter again.

### Related issue(s)
- closes  
- closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] d9da8e4bb2
Merge
1746: DNS records for client autoconfiguration (RFC6186) r=Diman0 a=nextgens

## What type of PR?

Feature

## What does this PR do?

Add instructions on how to configure rfc6186 DNS records for client autoconfiguration

### Related issue(s)
- 
- 

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 3f23e199f6 modified generation of session key and added refresh
- the session key is now generated using
  - a hash of the uid seeded by the apps secret_key (size: SESSION_KEY_BITS)
  - a random token (size: 128 bits)
  - the session's creation time (size: 32 bits)

- redis server side sessions are now refreshed after 1/2 the session lifetime
  even if not modified
- the cookie is also updated if necessary
Alexander Graf 9ef8aaf698 removed double confiog and fixed shaker
Alexander Graf a1fd44fced added lmtp: prefix and documentation
Florent Daigniere 875308d405 Revert "In fact it could be global"
This reverts commit f52984e4c3.
Florent Daigniere f52984e4c3 In fact it could be global
Florent Daigniere ae9206e968 Implement a simple credential cache
DjVinnii 419fed5e6e Add language selector
Alexander Graf 731ce8ede9 fix permanent sessions. hash uid using SECRET_KEY
clean session in redis only once when starting
Alexander Graf 4b8bbf760b default to 128 bits
Alexander Graf 4b71bd56c4 replace flask_kvsession with mailu's own storage
DjVinnii 7dafa22762 Add /language/<language> route for changing the locale using a session variable
DjVinnii f30cca1263 Do imports based on AdminLTE plugins
DjVinnii a4bb42faeb Remove extra space between 'AdminLTE' and 'on' in footer
DjVinnii b2498e8c8f Refactor box macro to card
DjVinnii 5ddea07c9a Fix form input append class
DjVinnii 1db0a870f3 Fix log in icon in sidebar
DjVinnii 51346c4860 Fix pre- and append styling
DjVinnii e963e7495d Create datatable based on dataTable class instead of table class
DjVinnii 0984173504 Change label to badge
DjVinnii 8246497d16 Add card header to tables
DjVinnii 49d68fa6d1 Fix horizontal scrollbar in sidebar
DjVinnii 7d3c9d412d Change tables to datatables
DjVinnii cdfa94c243 Make main action float right
DjVinnii 0c5fda3fca Change macros.box to macros.card
DjVinnii deca6e0c4a update user/settings
DjVinnii 6b3170cb4c Update side menu
DjVinnii c97728289b Update node version for building the image (AdminLTE requires node 10 or higher)
DjVinnii e46d9e1fc9 Update admin-lte version in package.json
Vincent Kling c6d0ef229f
Update messages.po
Alexander Graf f0f79b23a3 Allow cleanup of sessions by key&value in data
This can be used to delete all sessions belonging to a user/login.
For no it just iterates over all sessions.
This could be enhanced by using a prefix for and deleting by prefix.
Alexander Graf 83b1fbb9d6 Lazy loading of KVSessionExtension
- call cleanup_sessions on first kvstore access
  this allows to run cmdline actions without redis (and makes it faster)
- Allow development using DictStore by setting REDIS_ADDRESS to the empty string in env
- don't sign 64bit random session id as suggested by nextgens
Alexander Graf 8bc4445572 Sync update of localpart, domain_name and email
Alexander Graf 0c38128c4e Add pygments to requirements
Alexander Graf 9cb6962335 Moved MyYamlLexer into logger
now cmdline runs without pygments
Alexander Graf ce9a9ec572 always init Logger first
Alexander Graf c17bfae240 correct rfc3339 datetime serialization
now using correct timezone
Alexander Graf dc5464f254 Merge remote-tracking branch 'upstream/master' into import-export
Alexander Graf e90d5548a6 use RFC3339 for last_check
fixed to UTC for now
Florent Daigniere dd3d03f06d Merge remote-tracking branch 'upstream/master' into webmail-sso
bors[bot] 25e8910b89
Merge
1783: Switch to server-side sessions r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It simplifies session management.
- it ensures that sessions will eventually expire (*)
- it implements some mitigation against session-fixation attacks
- it switches from client-side to server-side sessions (in Redis)

It doesn't prevent us from (re)-implementing a "remember_me" type of feature if that's considered useful by some.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
lub f3f0a4d86d
Merge branch 'master' into enforce-tls-admin