Commit Graph

1055 Commits (8ad8cde0e27b62679e924fc7b4ba64ed7c53cb67)

Author SHA1 Message Date
Florent Daigniere 502affbe66 Use the regexp engine since we have one
Florent Daigniere a349190e52 simplify
Florent Daigniere 10d78a888b Derive a new subkey for SRS
Florent Daigniere 995ce8d437 Remove OUTCLEAN_ADDRESS
I believe that this isn't relevant anymore as we don't use OpenDKIM
anymore

Background on:
https://bofhskull.wordpress.com/2014/03/25/postfix-opendkim-and-missing-from-header/
Alexander Graf 65133a960a Prevent traceback when using non-email in login
There's a traceback when the username used to log via SMTPAUTH
in is not an email address:

=== before ===
```
[...] ERROR in app: Exception on /internal/auth/email [GET]
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1179, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 719, in _init_compiled
    param.append(processors[key](compiled_params[key]))
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1201, in process
    return process_param(value, dialect)
  File "/app/mailu/models.py", line 60, in process_bind_param
    localpart, domain_name = value.lower().rsplit('@', 1)
ValueError: not enough values to unpack (expected 2, got 1)
[...]
[parameters: [{'%(140657157923216 param)s': 'foobar'}]]
```

=== after ===
```
[...] WARNING in nginx: Invalid user 'foobar': (builtins.ValueError) invalid email address (no "@")
```
Florent Daigniere 7d56ed3b70 Merge branch 'master' of https://github.com/Mailu/Mailu into ratelimits
Florent Daigniere 1e07b85fa1 doh
Florent Daigniere 24aadf2f52 ensure we log when the rate limiter hits
Florent Daigniere 64bc7972cc Make AUTH_RATELIMIT_IP 60/hour as discussed
Florent Daigniere cab0ce2017 doh
Florent Daigniere a9340e61f5 Log auth attempts on /admin
Florent Daigniere 89ea51d570 Implement rate-limits
bors[bot] 4c5c6c3b5f
Merge
1966: AdminLTE3 optimizations & compression and caching r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement, bugfix

## What does this PR do?

Optimization and cleanup of styles and javascript code for AdminLTE 3
Adds caching headers, gzip and robots.txt to nginx.

### Related issue(s)

Makes  even better. Thanks to `@DjVinnii` and `@Diman0` for the good work.
Closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
bors[bot] b329971b87
Merge
1971: Updated Polish translation. r=mergify[bot] a=ghostwheel42

## What type of PR?

translation

## What does this PR do?

Update polish translation. Used `pl/LC_MESSAGES/messages.po` from PR  created by `@martys71`
Part of Discussion of 1.9 roadmap 

### Related issue(s)

closes  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] 72e8ec53b7
Merge
1975: Replace traceback with error message when creating initial admin user r=mergify[bot] a=ghostwheel42

## What type of PR?

small enhancement

## What does this PR do?

when creating the admin user via cli a traceback is shown when this user is already present in the database.
This is confusing users. I've replaced the traceback with an error message.

### Related issue(s)



## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Alexander Graf 25cf8b5358 better help formatting
Alexander Graf b63081cb48 display error (not exception) when creating admin
repleace misleading python exception (mailu broken)
with error message stating that the admin user is
already present
Alexander Graf 065215d4d1 Merge remote-tracking branch 'upstream/master' into adminlte3_fixes
Alexander Graf 7bec8029a4 strip not necessary anymore
Alexander Graf 05c79b0e3c copy (and not parse) mta sts override config
Alexander Graf b02ceab72f handle DEFER_ON_TLS_ERROR as bool
use /conf/mta-sts-daemon.yml when override is missing
Alexander Graf 1e8b41f731 Merge remote-tracking branch 'upstream/master' into adminlte3_fixes
Alexander Graf b883e3c4a6 duh.
Alexander Graf bb40ccc4b0 normalize HOSTNAMES
should be moved to python lib and normalized in start.py
Alexander Graf 45a2be3766 Updated Polish translation.
Used pl/LC_MESSAGES/messages.po from PR#1751 created by martys71
bors[bot] d464187477
Merge
1964: Alpine3.14.2 r=mergify[bot] a=nextgens

Upgrade to alpine 3.14.2, retry upgrading unbound & switch back to libressl

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf a319ecde29 also precompress static txt files
Alexander Graf b445d9ddd1 set expire headers only for mailu content
also moved robots.txt from config to static folder.
Alexander Graf 698ee4e521 added tiff and webp to list of cached content
Alexander Graf 0094268410 allow to change logo. default color for flash msg
- two new environment variables allow to change logo background color
  and graphic
- flash messages are now green (not cyan)
Alexander Graf d8b4a016af use blue color from https://mailu.io/
bors[bot] 6fe265b548
Merge
1968: optimize handle_authentication r=mergify[bot] a=ghostwheel42

## What type of PR?

bug-fix

## What does this PR do?

catch utf-8 decoding errors and log a warning in handle_authentication instead of writing a traceback into the log.

### Related issue(s)

closes 

## Prerequistes

Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
bors[bot] d8dc765f04
Merge
1967: fix 1789: ensure that nginx resolves ipv4 addresses r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

This fixes ipv6 enabled setup by disabling it. If you were using SUBNET6 in your configuration, odds are it's broken since gunicorn isn't bound on an on an ipv6 enabled socket.

Should we backport this?

### Related issue(s)
- close 
- close 


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Alexander Graf 90c96bdddc optimize handle_authentication
- catch decoding of nginx headers (utf-8 exception)
- re-ordered function
Florent Daigniere 7aa403573d no with here
Florent Daigniere 0ee52ba65b Doh
Florent Daigniere 0f0459e9b2 suggestions from @ghostwheel42
Florent Daigniere 9888efe55d Document as suggested on #mailu-dev
Alexander Graf 7bede55fce more verbose cleaning message
Florent Daigniere a9a1b3e55e Reduce the EDNS0 size to 1232
@see
https://github.com/dns-violations/dnsflagday/issues/125
Florent Daigniere 72ba5ca3f9 fix 1789: ensure that nginx resolves ipv4 addresses
Alexander Graf 7fd605cc21 fixed brand link target for normal users
Florent Daigniere d8c22db547 Merge remote-tracking branch 'upstream/master' into policyd-mta-sts
Alexander Graf 8cdd7e911d duh. removed debug
Alexander Graf 34df8b3168 AdminLTE3 optimizations & compression and caching
- fixed copy of qemu-arm-static for alpine
- added 'set -eu' safeguard
- silenced npm update notification
- added color to webpack call
- changed Admin-LTE default blue
  (core/admin/Dockerfile)

- AdminLTE 3 style tweaks
  (core/admin/assets/app.css)
  (core/admin/mailu/ui/templates/base.html)
  (core/admin/mailu/ui/templates/sidebar.html)

- localized datatables
  (core/admin/Dockerfile)
  (core/admin/assets/app.js)
  (core/admin/package.json)

- moved external javascript code to vendor.js
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/webpack.config.js)

- added mailu logo
  (core/admin/assets/app.js)
  (core/admin/assets/app.css)
  (core/admin/assets/mailu.png)

- moved all inline javascript to app.js
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/domain/create.html)
  (core/admin/mailu/ui/templates/user/create.html)

- added iframe display of rspamd page
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/views/base.py)
  (core/admin/mailu/ui/templates/sidebar.html)
  (core/admin/mailu/ui/templates/antispam.html)

- updated language-selector to display full language names and use post
  (core/admin/assets/app.js)
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/utils.py)
  (core/admin/mailu/ui/views/languages.py)

- added fieldset to group and en/disable input fields
  (core/admin/assets/app.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/user/settings.html)
  (core/admin/mailu/ui/templates/user/reply.html)

- added clipboard copy buttons
  (core/admin/assets/app.js)
  (core/admin/assets/vendor.js)
  (core/admin/mailu/ui/templates/macros.html)
  (core/admin/mailu/ui/templates/domain/details.html)

- cleaned external javascript imports
  (core/admin/assets/vendor.js)

- pre-split first hostname for further use
  (core/admin/mailu/__init__.py)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/client.html)
  (core/admin/mailu/ui/templates/domain/signup.html)

- cache dns_* properties of domain object (immutable during runtime)
  (core/admin/mailu/models.py)
  (core/admin/mailu/ui/templates/domain/details.html)

- fixed and splitted dns_dkim property of domain object (space missing)
- added autoconfig and tlsa properties to domain object
  (core/admin/mailu/models.py)

- suppressed extra vertical spacing in jinja2 templates
- improved accessibility for screen reader
  (core/admin/mailu/ui/templates/**.html)

- deleted unused/broken /user/forward route
  (core/admin/mailu/ui/templates/user/forward.html)
  (core/admin/mailu/ui/views/users.py)

- updated gunicorn to 20.1.0 to get rid of buffering error at startup
  (core/admin/requirements-prod.txt)

- switched webpack to production mode
  (core/admin/webpack.config.js)

- added css and javascript minimization
- added pre-compression of assets (gzip)
  (core/admin/webpack.config.js)
  (core/admin/package.json)

- removed obsolte dependencies
- switched from node-sass to dart-sass
  (core/admin/package.json)

- changed startup cleaning message from error to info
  (core/admin/mailu/utils.py)

- move client config to "my account" section when logged in
  (core/admin/mailu/ui/templates/sidebar.html)
Alexander Graf f4e7ce0990 enabled caching, gzip and robots.txt
Alexander Graf 103918ba57 pre-compress assets (*.ico for now)
Alexander Graf 39d7a5c504 pngcrushed images
bors[bot] 71cc8b0a81
Merge
1800: AdminLTE 3 r=mergify[bot] a=DjVinnii

## What type of PR?

Enhancement

## What does this PR do?

This PR implements AdminLTE 3 for the admin interface. It also includes the implementation of DataTables and a language selector.

### Related issue(s)
- closes: 
- closes:  

## Prerequistes

- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.


Co-authored-by: Vincent Kling <vincentkling@msn.com>
Co-authored-by: DjVinnii <vincentkling@msn.com>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
Co-authored-by: Diman0 <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Tim Foerster 9ec9d4d4fb
postfix/tls_policy: Use lmdb map instead of hash
The alpine postfix package seems to have removed support for btree and hash map type.  
The tls_policy.map stuff has been introduced in  and it has been merged without fixing this before (https://github.com/Mailu/Mailu/pull/1902/#issuecomment-902108080)
Florent Daigniere d7c2b510c7 Give alpine 3.14.2 a shot
Florent Daigniere fe186afb6f Revert "Switch to openssl to workaround alpine #12763"
This reverts commit f8362d04e4.
Florent Daigniere 4abf49edf4 indent
Florent Daigniere c1d94bb725 Ensure that postfix will be able to use the TLSA records
see https://www.huque.com/dane/testsite/ for the testcases
Florent Daigniere ef5f82362c Merge remote-tracking branch 'upstream/master' into policyd-mta-sts
Florent Daigniere 489520f067 forgot about alpine/lmdb
Florent Daigniere 9f66e2672b Use DEFER_ON_TLS_ERROR here too
We just don't know whether the lookup failed because we are under attack
or whether it's a glitch; the safe behaviour is to defer
Florent Daigniere a1da4daa4c Implement the DANE-only lookup policyd
https://github.com/Snawoot/postfix-mta-sts-resolver/issues/67 for
context
Dimitri Huisman 5f18860669 Remove workaround. Remove deprecated url-loader.
Dimitri Huisman 60be06e298 Temporary workaround to get FontAwesome icons working.
Dimitri Huisman 5da7a06675 Resolve webpack.config.js error
Florent Daigniere 67db72d774 Behave like documented
Florent Daigniere 05b57c972e remove the static policy as it will override MTA-STS and DANE
Florent Daigniere a8142dabbe Introduce DEFER_ON_TLS_ERROR
This will default to True and defer emails that fail even "loose"
validation of DANE or MTA-STS

It should work most of the time but if it doesn't and you would rather
see your emails delivered, you can turn it off.
Florent Daigniere 52d3a33875 Remove the domains that have a valid MTA-STS policy
gmail.com
comcast.net
mail.ru
googlemail.com
wp.pl
Florent Daigniere 4f96e99144 MTA-STS (use rather than publish policies)
Dimitri Huisman 00276d8b70
Merge branch 'master' into AdminLTE-3
Florent Daigniere 394c2fe22c Document REAL_IP_HEADER and REAL_IP_FROM
Fix a security vulnerability whereby we were not clearing other headers
Florent Daigniere 6bba0cecfc Strip the Forwarded header since nothing is compatible with it yet
bors[bot] 6e32092abd
Merge
1873: Completed Hebrew translation r=mergify[bot] a=yarons

The Hebrew translation is incomplete so I've completed it.

Co-authored-by: Yaron Shahrabani <sh.yaron@gmail.com>
Dimitri Huisman 169a540692 Use punycode for HTTP header for radicale and create changelog
Dimitri Huisman 4f5cb0974e Make sure HTTP header only contains ASCII
bors[bot] ecaaf25dcb
Merge
1939: Ensure that we don't do multiple DNS lookups in the sieve script r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It ensures that DNS lookups don't introduce inconsistent state. We may want to go further and actually check the return codes of rspamc too.

I haven't tested it but it should work.

### Related issue(s)
- 



Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Florent Daigniere 368b40b4fd doh
Florent Daigniere 3e676e232a fix
Florent Daigniere ae8db08bdf Ensure that we don't do multiple DNS lookups in the sieve script
Florent Daigniere 65a27b1c7f add additional options to make DANE easier
Florent Daigniere fb8d52ceb2 Merge branch 'master' of https://github.com/Mailu/Mailu into tls_policy_map
Florent Daigniere b4102ba464 doh
Florent Daigniere 9ec7590171 Merge branch 'master' of https://github.com/Mailu/Mailu into wildcard_senders
Florent Daigniere 7252a73e11 WILDCARD_SENDERS can have spaces
bors[bot] b57df78dac
Merge
1916: Ratelimit outgoing emails per user r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

A conflict-free version of  implementing per-user sender limits

### Related issue(s)
- close  
- close 
- close  

## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Dimitri Huisman e5972bd9ec Set default message rate limit to 200/day
Jack Murray dd127f8f06 Change letsencrypt timer from 1h --> 1 day
There's no need to be calling certbot so frequently
Florent Daigniere 6704cb869a Switch to 3072bits dhparam (instead of 4096bits)
We aim for 128bits of security here
Jack Murray e304c352a1 Change letsencrypt timer from 1h --> 1 day
There's no need to be calling certbot so frequently
Florent Daigniere facc4b6427 Allow specific users to send email from any address
Florent Daigniere ee54a615c1 Alpine has removed support for btree and hash
David Fairbrother 24747e33de Add ability to set no WEBROOT_REDIRECT to Nginx
Adds a 'none' env option to WEBROOT_REDIRECT so that no `location /`
configuration is written to nginx.conf.

This is useful for setting up Mailu and Mailman where we override the
root to proxy to the mailing list server instead. Without this change
the nginx container will not start, or for 1.7 users can set their
WEBMAIL_PATH to / with no webmail to get the same results.

This fix means that future users don't have to choose between webmail
and a root override and makes the configuration intention clear.
Florent Daigniere 0b16291153 doh
Florent Daigniere 1db08018da Ensure that we get certificate validation on top90
I have found a list of the top100 email destinations online and ran them
through a script to ensure that all of their MX servers had valid
configuration... this is the result
Florent Daigniere b066a5e2ac add a default tls_policy_map
Florent Daigniere 1df79f8132 give PFS a chance
Florent Daigniere 925105075c this is required in fact
Diman0 5afbf37292 Resolve build issues
Dimitri Huisman df64601b28
Merge branch 'master' into AdminLTE-3
Florent Daigniere 772e5efb7d Disable pipelining to prevent bypass
Florent Daigniere c76a76c0b0 make it optional, add a knob
Florent Daigniere 109a8aa000 Ensure that we always have CERT+INTERMEDIARY CA
Let's encrypt may change things up in the future...
Florent Daigniere dccd8afd51 Thanks @Diman0!
ENEEDSLEEP