1158: Use nginx for kubernetes ingress r=kaiyou a=micw
## What type of PR?
enhancement
## What does this PR do?
Currently, kubernetes uses a complex ingress setting which is not portable across different ingress controllers. This PR simplifies the ingress and delegates everythins special to Mailu to the front container,
### Related issue(s)
- closes#1121
- closes#1117
- closes#1021
- closes#1045
## Prerequistes
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog]
Co-authored-by: Michael Wyraz <michael@wyraz.de>
Before, the ham/spam scripts got the rspamd-ip/port from the environment.
However, when checking the environment of these processes now, it seems
cleared. Maybe the new dovecot version now clears environment? — I couldn’t
find a hint.
In any case, using the common mechanism of injecting the ip/port from where
it’s definately known by the already-used jinja2-mechanism seems reasonably
safe.
1114: Resolve HOST to ADDRESS only if ADDRESS is not already set r=mergify[bot] a=micw
## What type of PR?
bug-fix
## What does this PR do?
~Makes the rsolving from hosts to ips at startup configurable~
I rewrote the pull request after #940 was merged. Now it resolves HOSTs to ADDRESSes only of ADDRESSes are not already set. So on kubernetes we can jsut set the address and have working service discovery.
### Related issue(s)
- closes#1113
## Prerequistes
~Minor change, backward compatible~
Changelog will be added
Co-authored-by: Michael Wyraz <michael@wyraz.de>
1130: Remove unnecessary host variable assignments r=mergify[bot] a=kaiyou
## What type of PR?
Bug fix
## What does this PR do?
### Related issue(s)
- Closes#1126
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
1132: Release Mailu 1.7 r=mergify[bot] a=kaiyou
## What type of PR?
Release
## What does this PR do?
It provides the changelog and migration notes for release 1.7.
### Related issue(s)
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: kaiyou <pierre@jaury.eu>
940: Fix host variables r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
- Fix#884
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
883: Admin create user enhancement r=mergify[bot] a=cr1st1p
## What type of PR?
Enhancement
## What does this PR do?
It allows the admin docker image to also create the admin user.
The idea is that in my kubernetes setup, I do not want to manually do anything, as such, I need a way for the admin user to also be created automatically without me getting inside the pod.
So I had to change the manage.py function that creates the user to allow different 'modes' (me, I'll be using 'ifmissing') and also start.py to call that functionality if appropriate environment variables are present.
So now, in my Deployment, I add 3 more environment variables and I get the admin user created, IF not already present.
### Related issue(s)
none?
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: cristi <cristi.posoiu@gmail.com>
Co-authored-by: cr1st1p <cristi.posoiu@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
1088: Support domain literals (fix#1087) r=mergify[bot] a=hoellen
## What type of PR?
bug-fix
## What does this PR do?
This PR adds error handling for idna enocding. With telnet you now get a "Bad sender address syntax" message.
```
> telnet mail.example.com 25
Connected to example.com.
Escape character is '^]'.
220 mail.example.com ESMTP ready
EHLO dummy.example.com
250-mail.example.com
250 STARTTLS
MAIL FROM: does-not-exist@[116.203.165.200]
250 2.0.0 OK
RCPT TO: some-user@example.com
501 5.1.7 Bad sender address syntax
Connection closed by foreign host.
```
### Related issue(s)
fix#1087
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: hoellen <dev@hoellen.eu>
1052: Upgrade alpine to 3.10 r=mergify[bot] a=Nebukadneza
## What type of PR?
enhancement / bug-fix
## What does this PR do?
Upgrade the alpine base image to 3.10 and clean up ensuing problems. Also directly uses postfix foreground-running with stdout logging.
### Related issue(s)
closes#1049closes#1051
Note: This is a duplicate effort of #1050#1039 …
## Prerequistes
- [X] In case of feature or enhancement: documentation updated accordingly
- [X] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
With these images, I have tested manually:
- Email receive to user
- on main domain
- on additional domain
- via an alias on main-domain
- via an alias on additional domain
- via catchall
- Email reject
- of eicar testfiles
- Email sending
- Fetchmail from legacy POP
- Front LE certificates
- Simple overriding for postfix (only postfix.cf), nginx, dovecot, rspamd
- Creating a CalDAV calendar and CardDAV entry using davx5
I have not (yet) tested:
- certdumper
- databases other than sqlite
Todo:
- [x] Get rid of podop at `git+https://github.com/Nebukadneza/Podop.git@fix_py37` once https://github.com/Mailu/Podop/pull/6 is merged
- [x] Bend Mailustart back to original repository
- [x] Test more (? also help wanted ?)
Co-authored-by: Thomas Sänger <thomas@gecko.space>
Co-authored-by: Dario Ernst <dario@kanojo.de>
Co-authored-by: Reto Glauser <git@blinkeye.ch>
1048: Refactor admin using webpack r=mergify[bot] a=kaiyou
## What type of PR?
Refactoring
## What does this PR do?
This PR brings some refactoring to the admin container :
- remove the dependency to mailustart and replace it with socrate
- remove static assets from the repo and use Webpack for building them
Co-authored-by: hoellen <dev@hoellen.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
931: Resolve webmail in admin r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
Implement mailustart to resolve webmail in admin
### Related issue(s)
Fix#716
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
962: Open some admin sidebar links in a new browser tab r=kaiyou a=mariooos
## What type of PR?
Enhancement
## What does this PR do?
Add target=_blank attribute to html <a href ...> tag in following admin sidebar links:
- Antispam
- Webmail
- Website
- Help
Co-authored-by: mariooos <48351788+mariooos@users.noreply.github.com>
993: Make aliases case-insensitive (too) r=mergify[bot] a=Nebukadneza
Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.
For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …
closes#867
## What type of PR?
bug-fix
## What does this PR do?
Make aliases optionally case-insensitive: After attempting to resolve an alias in its preserved case, also attempt to match it case-insensitively.
This followed after some more thought from #868 …
### Related issue(s)
closes#867
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Dario Ernst <dario@kanojo.de>
945: Add extra newline to main.cf r=mergify[bot] a=flopes89
This should prevent jinja from stripping the newline, which causes overrides to be appended after the comment section
## What type of PR?
Bugfix
## What does this PR do?
Adds a new empty newline a the end of `conf/main.cf` so prevent jinja from stripping it, by which overrides done with `postconf -e` are not appended correctly.
### Related issue(s)
see #941
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
Co-authored-by: Florian Peschka <flopes89@users.noreply.github.com>
1005: distinguish disabled users in user list view by row color r=kaiyou a=zhuangyy
## What type of PR?
enhancement
## What does this PR do?
distinguish disabled users in user list view by row color
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/guide.html#changelog) entry file.
Co-authored-by: Zhuang Yuyao <jssyan@qq.com>
Even though RFC5321 2.4 explains that local-parts are to be case-sensitive,
this does not seem to be how EMail is used today. Thus, instead of reverting
user-emails back to being case sensitive, let’s make aliases case-insensitive
too. Not only more consistent, this also allows users to enjoy receiving EMails
from large airlines or car-rental agencies onto their already existing aliases.
For the rare case of case sensitive aliases existing, let’s query for the
forced-lowercase alias only in the event that the preserved-case one isn’t
found …
closes#867
934: Pass optional arguments to domain model r=mergify[bot] a=ionutfilip
## What type of PR?
bug-fix
## What does this PR do?
Passing optional args to domain model when using cli.
### Related issue(s)
Fixes#933
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
919: Install bash in alpine based images. r=mergify[bot] a=firvida
This fixes#918
Bash shell is used by default in Kubernetes' dashboard console, which is very
useful for admins.
## What type of PR?
bug-fix
## What does this PR do?
### Related issue(s)
- closes#918
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Abel Alfonso Fírvida Donéstevez <abel@merchise.org>
886: Ipv6 support r=mergify[bot] a=muhlemmer
## What type of PR?
(Feature, enhancement, bug-fix, documentation) -> A bit of everything
## What does this PR do?
Document how to use ipv6nat. This, however triggers some kind of flaky behavior with the Docker DNS resolver, resulting in lookup failures between containers. So all resolving needs to be done during container startup/configuration.
In order not to pollute every single start.py file, we've created a small library called [Mailu/MailuStart](https://github.com/Mailu/MailuStart). As an addition, this library also defines the template generation function, including its logging facility.
Note: `docker-compose.yml` downgrade is necessary, as IPv6 settings are not supported by the Docker Compose file format 3 😞
### Related issue(s)
Supersedes PR #844
- Fixes#827
- Hopefully helps with #829 and #834
## No backport yet
This PR directly imports MailuStart from git. This makes it a bit more simple to implement on the short term an do some testing and probably some future improvements. When everything is proved stable, we will create a proper PyPi package with versioning and consider back porting.
## Prerequistes
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.
- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: place entry in the [changelog](CHANGELOG.md), under the latest un-released version.
Co-authored-by: Ionut Filip <ionut.philip@gmail.com>
Co-authored-by: Tim Möhlmann <muhlemmer@gmail.com>
At some places, the string that DOMAIN_REGISTRATION is got used like a boolean
(an easy misassumption to make while in python and dealing with the config
dict), making `DOMAIN_REGISTRATION=False` act as a truthy value. To stop such
future problems from happening, coerce environment config strings to real
bools.
closes#830
Since it’s common for wildcard~ish systems to prefer concrete objects over
wildcards, and aliases can be broad-wildcards (think catchall, %@xxx.tld), it
may be more intuitive for users that user-names rank higher than aliases. This
makes it impossible for user-names to be unreachable, since they can be
completely overridden by a catchall otherwise.
This changes default behavior, and is not configurable.
closes#815
As discussed with hoellen on matrix, since postfix indeed supports including
the recipient delimiter character in a verbatim alias, we should support so too
— and handle its precedence correctly. The clearer and simpler formulation of
the precedence-clauses are credit to @hoellen. Thanks!
This fixes delivery to an alias minus recipient delimiter in cases where a
wildcard alias would also match. For example,
* foo@xxx.tld
* %@xxx.tld
Sending to foo+spam@xxx.tld would get eaten by the catchall before this fix.
Now, the order of alias resolution is made clearer.
closes#813
In the process we found that the previous way of tenacity syntax caused it not to honor any args.
In this commit we've refactored to use the @decorator syntax, in which tenacity seems to behave better.
CVE-2017-18342
Vulnerable versions: < 4.2b1
Patched version: 4.2b1
In PyYAML before 4.1, the yaml.load() API could execute arbitrary code. In other words, yaml.safe_load is not used.
Since postfix now asks us for the complete email over podop, which
includes the recipient-delimiter-and-what-follows not stripped, we need
to attempt to find both the verbatim localpart, as well as the localpart
stripped of the delimited part ….
Fixes#755
The init script was pushing an application context, which maked
flask.g global and persisted across requests. This was evaluated
to have a minimal security impact.
This explains/fixes #738: flask_wtf caches the csrf token in the
application context to have a single token per request, and only
sets the session attribute after the first generation.
Supporting multiple backends requires that specific sqlite
collations are not used, thus lowercase is applied to all non
case-sensitive columns. However, lowercasing the database requires
temporary disabling foreign key constraints, which is not possible
on SQLite and requires we specify the constraint names.
This migration specific to sqlite and postgresql drops every
constraint, whether it is named or not, and recreates all of them
with known names so we can later disable them.
In case of TLS_FLAVOR=[mail,cert], the user supplies their own certificates.
However, since nginx is not aware of changes to these files, it cannot
reload itself e.g. when the certs get renewed.
To solve this, let’s add a small daemon in the place of
`letsencrypt.py`, which uses a flexible file-watching framework and
reloads nginx in the case the certificates change ….
This will make network configuration and host based authentication
more robust, across different deployment platforms.
The options `RELAYNETS` and`POD_ADDRESS_RANGE` are kept for compatibility.
However, their usage have become optional.
To facilitate this, the default redirect at / can be disabled, even if
the default remains at redirecting to the webmailer.
The extensions are within the host scope and are read from
$ROOT/overrides/nginx/*.conf.
At present postfix checks this view for matches in the domain table and is used to accept/deny messages sent into it however it never checks for matches in the alternative table.
Fixes#718
In Docker Swarm mode the services listed below can get stuck in their start script, while they
are waiting for other services become available. Now, with HEALTHCHECK enabled, docker does not resolve
names of services that not pass HEALTHCHECK yet. Meaning that if one of the depenend services is not yet
available, it will create a chain of failing services.
The services below retry to resolve 100 time, with an average of 3.5 seconds. Hence, the --start-time
flag is now set at 350 seconds.
- dovecot (imap)
- postfix (smtp)
- rspamd (antispam)
- Specified seperated /health path in order to allow for healthcheck even if webmail and admin are not seletectd. This also allows healthchecking fom external services like DNS load balancers;
- Make curl not to fail on TLS because localhost is not included in the certificates.
Postfix, after expanding the alias, is not transmitting the original
envelope recipient email address to dovecot and cannot record it in a
Received: header.
The LMTP DSN extension allows postfix to specify an ORCPT= parameter to
the "RCPT TO:" line (in postfix src/smtp/smtp_proto.c). However, dovecot
does not support the DNS extension on the LMTP endpoint. It has
preliminary support of the ORCPT parameter in latest versions but is is
disabled and not working.
The solution taken was to add a sieve script to parse the Received:
header written by postfix and parse the original RCPT TO address from
it. Then add the header through the "editheader" sieve extension. Later
sieve scripts can take this header to perform further filtering.
Allows to use mailu without docker-compose when hostnames are not set up
by docker itself but provided via a separate resolver.
Use case: use mailu using nomad scheduler and consul resolver instead of
docker-compose. Other servers are provided by the DNS resolver that
resolves names like admin.service.consul or webmail.service.consul.
These names needs to be configurable.
See https://www.endpoint.com/blog/2015/05/28/postfix-address-verification
Block client until address verifiction is completed and mail will not be rejected by relaying MTA or smarthost.
If verification fails, mail is rejected.
If verification takes too long, mail is temporaryly rejected and sending client will retry later.
hoellen