Commit Graph

1092 Commits (11bbceb9cc4527516f66c80c56c96f0a9e394534)

Author SHA1 Message Date
Florent Daigniere abaa2e8cc3 simplify client_ip
Florent Daigniere de276a6822 Simplify extract_network_from_ip
Florent Daigniere 3bda8368e4 simplify the Auth-Status check
Florent Daigniere 2dd9ea1506 simplify
Florent Daigniere 068170c0ff Use app instead of flask.current_app where possible
Florent Daigniere 57b0dd490c Initialize user_email in all cases
qy117121 b1425015ef
Update messages.po
Fix wrong text
bors[bot] afffe4063e
Merge
2018: show dmarc record for report domain in domain details r=mergify[bot] a=ghostwheel42

## What type of PR?

documentation

## What does this PR do?

show dmarc record for report domain in domain details

### Related issue(s)

closes 

## Prerequisites

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9f2aa0aadc
Merge
1986: Document how to setup client autoconfig r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Document how to setup autoconfig. This works with most open-source MUAs (thunderbird, evolution, ...)

We could go further than that by providing dynamic configuration (issue an auth token for each MUA request)... but it won't work unless a new DNS entry (and matching certificate) is created.

### Related issue(s)
- 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2014: Update Chinese translation r=mergify[bot] a=qy117121

## What type of PR?

translation

## What does this PR do?

Update Chinese translation. Use `zh` instead of `zh_CN`.

### Related issue(s)

none

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: qy117121 <mixuan121@gmail.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 7fe15ea9cf added dmarc record for report domain
bors[bot] a5b1d36171
Merge
2017: rspamd: get dkim keys via REST API instead of filesystem r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement

## What does this PR do?

rspamd now uses hashicorp's vault api v1 to get dkim keys and selectors for a domain.
this allows future enhancement (multiple keys) without reconfiguring and restarting rspamd.
it also makes mounting the /dkim volume into the rspamd container unnecessary.

### Related issue(s)

- improves and closes  
- allows to implement key rotation using multiple selectors (see )
- allows to implement dkim for alternate domains (see )
- fixes and closes  (selector transmitted by admin container is used)
- closes  (no keys on disk)
- allows to implement key rotation from the outside (ie. via a helper script talking to some dns provider's api) (see )

## Prerequisites

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 7b0c5935a8 only support GET method in vault
Alexander Graf 303fae00fb cleanup modules. use dkim selector from config
Alexander Graf dc9f970a91 removed zh_CN and updated locale-map for datatables
Alexander Graf 893705169e PoC rspamd use dkimkeys from admin using vault api
Florent Daigniere 632ce663ee Prevent logins with no password
qy117121 866f784d06
Create messages.po
Update the translation
qy117121 251eea5553
Update messages.po
Updated translation
Florent Daigniere 7277e0b4e4
Merge branch 'master' into ratelimits
bors[bot] 8c8c1b2015
Merge
1997: Prevent traceback when using non-email in login r=mergify[bot] a=ghostwheel42

There's a traceback when the username used to log via SMTPAUTH
in is not an email address:

=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 60, in process_bind_param
    localpart, domain_name = value.lower().rsplit('`@',` 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```

=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "`@")`
```

## What type of PR?

enhancement

## What does this PR do?

replace traceback (ERROR) with error message (WARNING)

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 9b01e663b2
Merge
2007: allow sending emails as user+detail@domain.tld r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix or enhancement

## What does this PR do?

Allows sending emails with an added "+detail" in the local part.
 
### Related issue(s)

closes 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: root <ghostwheel42@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 14360f8926 RECIPIENT_DELIMITER can have several characters
root 8c59f35697 use RECIPIENT_DELIMITER for splitting
Alexander Graf 1d571dedfc split localpart into user and tag
Florent Daigniere d131d863ba The if needs to be inside the block
Alexander Graf aaf3ddd002 moved javascript to app.js
Florent Daigniere b48779ea70 SESSION_COOKIE_SECURE and HTTP won't work
Florent Daigniere 502affbe66 Use the regexp engine since we have one
Florent Daigniere a349190e52 simplify
Florent Daigniere 10d78a888b Derive a new subkey for SRS
Florent Daigniere 995ce8d437 Remove OUTCLEAN_ADDRESS
I believe that this isn't relevant anymore as we don't use OpenDKIM
anymore

Background on:
https://bofhskull.wordpress.com/2014/03/25/postfix-opendkim-and-missing-from-header/
Alexander Graf 65133a960a Prevent traceback when using non-email in login
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:

=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 60, in process_bind_param
    localpart, domain_name = value.lower().rsplit('@', 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```

=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "@")
```
Diman0 41f5b43b38 Set nginx logging to level info again.
Diman0 f4cde61148 Make header translatable. More finishing touches.
Florent Daigniere 7d56ed3b70 Merge branch 'master' of https://github.com/Mailu/Mailu into ratelimits
Diman0 fbe0a446b9 Merge branch 'master' of github.com:Mailu/Mailu into fix-sso-1929
Florent Daigniere 1e07b85fa1 doh
Diman0 9894b49cbd Merge/Update with changes from master
Florent Daigniere 24aadf2f52 ensure we log when the rate limiter hits
Florent Daigniere 64bc7972cc Make AUTH_RATELIMIT_IP 60/hour as discussed
Florent Daigniere cab0ce2017 doh
Florent Daigniere a9340e61f5 Log auth attempts on /admin
Florent Daigniere 89ea51d570 Implement rate-limits
Diman0 bf0aad9820 Merge branch 'master' of github.com:Mailu/Mailu into fix-sso-1929
bors[bot] 4c5c6c3b5f
Merge
1966: AdminLTE3 optimizations & compression and caching r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement, bugfix

## What does this PR do?

Optimization and cleanup of styles and javascript code for AdminLTE 3
Adds caching headers, gzip and robots.txt to nginx.

### Related issue(s)

Makes  even better. Thanks to `@DjVinnii` and `@Diman0` for the good work.
Closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] b329971b87
Merge
1971: Updated Polish translation. r=mergify[bot] a=ghostwheel42

## What type of PR?

translation

## What does this PR do?

Update polish translation. Used `pl/LC_MESSAGES/messages.po` from PR  created by `@martys71`
Part of Discussion of 1.9 roadmap 

### Related issue(s)

closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 72e8ec53b7
Merge
1975: Replace traceback with error message when creating initial admin user r=mergify[bot] a=ghostwheel42

## What type of PR?

small enhancement

## What does this PR do?

when creating the admin user via cli a traceback is shown when this user is already present in the database.
This is confusing users. I've replaced the traceback with an error message.

### Related issue(s)



## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 25cf8b5358 better help formatting
Alexander Graf b63081cb48 display error (not exception) when creating admin
repleace misleading python exception (mailu broken)
with error message stating that the admin user is
already present
Alexander Graf 065215d4d1 Merge remote-tracking branch 'upstream/master' into adminlte3_fixes
Alexander Graf 7bec8029a4 strip not necessary anymore
Alexander Graf 05c79b0e3c copy (and not parse) mta sts override config
Alexander Graf b02ceab72f handle DEFER_ON_TLS_ERROR as bool
use /conf/mta-sts-daemon.yml when override is missing
Alexander Graf 1e8b41f731 Merge remote-tracking branch 'upstream/master' into adminlte3_fixes
Alexander Graf b883e3c4a6 duh.
Alexander Graf bb40ccc4b0 normalize HOSTNAMES
should be moved to python lib and normalized in start.py
Alexander Graf 45a2be3766 Updated Polish translation.
Used pl/LC_MESSAGES/messages.po from PR#1751 created by martys71
bors[bot] d464187477
Merge
1964: Alpine3.14.2 r=mergify[bot] a=nextgens

Upgrade to alpine 3.14.2, retry upgrading unbound & switch back to libressl

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf a319ecde29 also precompress static txt files
Alexander Graf b445d9ddd1 set expire headers only for mailu content
also moved robots.txt from config to static folder.
Alexander Graf 698ee4e521 added tiff and webp to list of cached content
Alexander Graf 0094268410 allow to change logo. default color for flash msg
- two new environment variables allow to change logo background color
  and graphic
- flash messages are now green (not cyan)
Alexander Graf d8b4a016af use blue color from https://mailu.io/
bors[bot] 6fe265b548
Merge
1968: optimize handle_authentication r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

catch utf-8 decoding errors and log a warning in handle_authentication instead of writing a traceback into the log.

### Related issue(s)

closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] d8dc765f04
Merge
1967: fix 1789: ensure that nginx resolves ipv4 addresses r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

This fixes ipv6 enabled setup by disabling it. If you were using SUBNET6 in your configuration, odds are it's broken since gunicorn isn't bound on an on an ipv6 enabled socket.

Should we backport this?

### Related issue(s)
- close 
- close 


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 90c96bdddc optimize handle_authentication
- catch decoding of nginx headers (utf-8 exception)
- re-ordered function
Florent Daigniere 7aa403573d no with here
Florent Daigniere 0ee52ba65b Doh
Florent Daigniere 0f0459e9b2 suggestions from @ghostwheel42
Florent Daigniere 9888efe55d Document as suggested on #mailu-dev
Alexander Graf 7bede55fce more verbose cleaning message
Florent Daigniere a9a1b3e55e Reduce the EDNS0 size to 1232
@see
https://github.com/dns-violations/dnsflagday/issues/125
Florent Daigniere 72ba5ca3f9 fix 1789: ensure that nginx resolves ipv4 addresses
Alexander Graf 7fd605cc21 fixed brand link target for normal users
Diman0 b148e41d9b Fix nginx config
Florent Daigniere d8c22db547 Merge remote-tracking branch 'upstream/master' into policyd-mta-sts
Alexander Graf 8cdd7e911d duh. removed debug
Alexander Graf 34df8b3168 AdminLTE3 optimizations & compression and caching
- fixed copy of qemu-arm-static for alpine
- added 'set -eu' safeguard
- silenced npm update notification
- added color to webpack call
- changed Admin-LTE default blue
  (core/admin/Dockerfile)

- AdminLTE 3 style tweaks
  (core/admin/assets/app.css)
  (core/admin/mailu/ui/templates/base.html)
  (core/admin/mailu/ui/templates/sidebar.html)

- localized datatables
  (core/admin/Dockerfile)
  (core/admin/assets/app.js)
  (core/admin/package.json)

- moved external javascript code to vendor.js
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/webpack.config.js)

- added mailu logo
  (core/admin/assets/app.js)
  (core/admin/assets/app.css)
  (core/admin/assets/mailu.png)

- moved all inline javascript to app.js
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/domain/create.html)
  (core/admin/mailu/ui/templates/user/create.html)

- added iframe display of rspamd page
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/views/base.py)
  (core/admin/mailu/ui/templates/sidebar.html)
  (core/admin/mailu/ui/templates/antispam.html)

- updated language-selector to display full language names and use post
  (core/admin/assets/app.js)
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/utils.py)
  (core/admin/mailu/ui/views/languages.py)

- added fieldset to group and en/disable input fields
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/user/settings.html)
  (core/admin/mailu/ui/templates/user/reply.html)

- added clipboard copy buttons
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/domain/details.html)

- cleaned external javascript imports
  (core/admin/assets/vendor.js)

- pre-split first hostname for further use
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/client.html)
  (core/admin/mailu/ui/templates/domain/signup.html)

- cache dns_* properties of domain object (immutable during runtime)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/domain/details.html)

- fixed and splitted dns_dkim property of domain object (space missing)
- added autoconfig and tlsa properties to domain object
  (core/admin/mailu/models.py)

- suppressed extra vertical spacing in jinja2 templates
- improved accessibility for screen reader
  (core/admin/mailu/ui/templates/**.html)

- deleted unused/broken /user/forward route
  (core/admin/mailu/ui/templates/user/forward.html)
  (core/admin/mailu/ui/views/users.py)

- updated gunicorn to 20.1.0 to get rid of buffering error at startup
  (core/admin/requirements-prod.txt)

- switched webpack to production mode
  (core/admin/webpack.config.js)

- added css and javascript minimization
- added pre-compression of assets (gzip)
  (core/admin/webpack.config.js)
  (core/admin/package.json)

- removed obsolte dependencies
- switched from node-sass to dart-sass
  (core/admin/package.json)

- changed startup cleaning message from error to info
  (core/admin/mailu/utils.py)

- move client config to "my account" section when logged in
  (core/admin/mailu/ui/templates/sidebar.html)
Alexander Graf f4e7ce0990 enabled caching, gzip and robots.txt
Alexander Graf 103918ba57 pre-compress assets (*.ico for now)
Alexander Graf 39d7a5c504 pngcrushed images
Diman0 960033525d configure sso in nginx
Diman0 8868aec0dc Merge master. Make sso login working for admin.
Diman0 1cfc9ee1c4 Merge branch 'master' of github.com:Diman0/Mailu into fix-sso-1929
Diman0 9fac3d7ad3 Initial implementation for standalone sso page
bors[bot] 71cc8b0a81
Merge
1800: AdminLTE 3 r=mergify[bot] a=DjVinnii

## What type of PR?

Enhancement

## What does this PR do?

This PR implements AdminLTE 3 for the admin interface. It also includes the implementation of DataTables and a language selector.

### Related issue(s)
- closes: 
- closes:  

## Prerequistes

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Vincent Kling <vincentkling@msn.com>
Co-authored-by: DjVinnii <vincentkling@msn.com>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Tim Foerster 9ec9d4d4fb
postfix/tls_policy: Use lmdb map instead of hash
The alpine postfix package seems to have removed support for btree and hash map type.  
The tls_policy.map stuff has been introduced in  and it has been merged without fixing this before (https://github.com/Mailu/Mailu/pull/1902/#issuecomment-902108080)
Florent Daigniere d7c2b510c7 Give alpine 3.14.2 a shot
Florent Daigniere fe186afb6f Revert "Switch to openssl to workaround alpine #12763"
This reverts commit f8362d04e4.
Florent Daigniere 4abf49edf4 indent
Florent Daigniere c1d94bb725 Ensure that postfix will be able to use the TLSA records
see https://www.huque.com/dane/testsite/ for the testcases
Florent Daigniere ef5f82362c Merge remote-tracking branch 'upstream/master' into policyd-mta-sts
Florent Daigniere 489520f067 forgot about alpine/lmdb
Florent Daigniere 9f66e2672b Use DEFER_ON_TLS_ERROR here too
We just don't know whether the lookup failed because we are under attack
or whether it's a glitch; the safe behaviour is to defer
Florent Daigniere a1da4daa4c Implement the DANE-only lookup policyd
https://github.com/Snawoot/postfix-mta-sts-resolver/issues/67 for
context
Dimitri Huisman 5f18860669 Remove workaround. Remove deprecated url-loader.
Dimitri Huisman 60be06e298 Temporary workaround to get FontAwesome icons working.
Dimitri Huisman 5da7a06675 Resolve webpack.config.js error
Florent Daigniere 67db72d774 Behave like documented
Florent Daigniere 05b57c972e remove the static policy as it will override MTA-STS and DANE