Commit Graph

529 Commits (0e8bb507bf8137d1aee258697a03cd7deeb4ac95)

Author SHA1 Message Date
kaiyou b6aaf57be1 Merge branch 'refactor-config' of github.com:kaiyou/mailu into refactor-config
kaiyou d0f07984b0 Merge remote-tracking branch 'upstream/master' into refactor-config
Tim Möhlmann 9dd447e23b
Add login method to smtp_auth under ssl
Fixes 
Patrick Georgi eac4d553a9 nginx: Allow extending config with overrides
To facilitate this, the default redirect at / can be disabled, even if
the default remains at redirecting to the webmailer.

The extensions are within the host scope and are read from
$ROOT/overrides/nginx/*.conf.
mergify[bot] 2d4bac03ad
Merge pull request from usrpro/clean-healthcheck-logs
Admin: Prevent redirects during health checking
mergify[bot] a382f74680
Merge pull request from usrpro/fix-recaptcha
Fix recaptcha
mergify[bot] 37027cfce7
Merge pull request from kaiyou/fix-sender-checks
Improve sender checks
Tim Möhlmann d18cf7cb25
Prevent redirects during health checking
Tim Möhlmann c9df311a0d
Set forward_destination to an empty list
The value of `None` resulted in an error, since a list was expected.
Tim Möhlmann eff6c34632
Catch asterisk before resolve_domain
Asterisk results in IDNA error and a 500 return code.
Ionut Filip 7b8835070d Added tenacity retry fir migrations connection
David Rothera 88c174fb7a Query alternative table for domain matches
At present postfix checks this view for matches in the domain table and is used to accept/deny messages sent into it however it never checks for matches in the alternative table.

Fixes 
Ionut Filip 436055f02c Created function for returning email type
Tim Möhlmann 47a3fd47b5
Fix DB_FLAVOR condition testing for models.py
Tim Möhlmann 0f3c1b9d15
Implement CIText as NOCASE alternative in postgresql
Tim Möhlmann 9b9f3731f6
Make current migrations work with postgresql
Tim Möhlmann 8bdc0c71af
Allow for setting a different DB flavor
Ionut Filip fed7146873 Captcha check on signup form
Tim Möhlmann 4783e61693
Fix password context
Fixes the following error:
```
admin_1      | [2018-11-09 09:44:10,533] ERROR in app: Exception on /internal/auth/email [GET]
admin_1      | Traceback (most recent call last):
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
admin_1      |     response = self.full_dispatch_request()
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
admin_1      |     rv = self.handle_user_exception(e)
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
admin_1      |     reraise(exc_type, exc_value, tb)
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
admin_1      |     raise value
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
admin_1      |     rv = self.dispatch_request()
admin_1      |   File "/usr/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
admin_1      |     return self.view_functions[rule.endpoint](**req.view_args)
admin_1      |   File "/usr/lib/python3.6/site-packages/flask_limiter/extension.py", line 544, in __inner
admin_1      |     return obj(*a, **k)
admin_1      |   File "/app/mailu/internal/views/auth.py", line 18, in nginx_authentication
admin_1      |     headers = nginx.handle_authentication(flask.request.headers)
admin_1      |   File "/app/mailu/internal/nginx.py", line 48, in handle_authentication
admin_1      |     if user.check_password(password):
admin_1      |   File "/app/mailu/models.py", line 333, in check_password
admin_1      |     context = User.pw_context
admin_1      | AttributeError: type object 'User' has no attribute 'pw_context'
```
kaiyou 72e1b444ca Merge alembic migrations
kaiyou 5b769e23da Merge branch 'master' into refactor-config
kaiyou 02995f0a15 Add a mailu command line to flask
kaiyou f9e30bd87c Update the dockerfile and upgrade dependencies
kaiyou 4a7eb1eb6c Explicitely declare flask migrate
kaiyou 2a8808bdec Add the configuration table migration
kaiyou f57d4859f3 Provide an in-context wrapper for getting users
kaiyou f6013aa29f Fix an old migration that was reading configuration before migrating
kaiyou 206cce0b47 Finish the configuration bits
Ionut Filip 1bbf3f235d Using a new class when captcha is enabled
mergify[bot] 12689965bd
Merge pull request from usrpro/fix-admin-bug
Fixed admin_1 errors in the logs
hoellen 680ad4b67a
Catching only ValueError
Co-Authored-By: ionutfilip <ionut.philip@gmail.com>
mergify[bot] e08f3e81d0
Merge pull request from usrpro/feat-startup
Standarize images
Ionut Filip 6dcc33e390 Fixed admin_1 errors in the logs
Fixed errors when trying to log in with an account without domain.
This closes 
Tim Möhlmann 42e2dbe35d
Standarize image by using shared / similair layers
Tim Möhlmann 5fa2aac569
Fix imap login when no webmail selected
Tim Möhlmann 903bb70c5b
Merge remote-tracking branch 'upstream/master' into standarize-images
Scott 56fb74c502 Fix typo (duplicate self). Fixes
Ionut Filip 8a44a44688
Merge branch 'master' into feat-startup
Ionut Filip 1187cac5e1 Finished up switching from .sh to .py
Tim Möhlmann ed81c076f2
Take out "models" path, as we are already in it
Tim Möhlmann aed80a74fa
Rectify decleration of domain_name
Tim Möhlmann 2d382f2d67
Merge branch 'master' into fix-sender-checks
Ionut Filip 0e5606d493 Changed start.sh to start.py
Ionut Filip eb7dfb5771 Cleaning up start.py
Thomas Sänger 603b6e7390
Merge pull request from usrpro/fix-nginx-healthcheck
Fix nginx healthcheck
Tim Möhlmann 81b24f61e8
Merge branch 'master' into feat-healthchecks
Tim Möhlmann a2fea36c79
Increase HEALTHCHECK start time for services that need to wait for host resolving during startup.
In Docker Swarm mode the services listed below can get stuck in their start script, while they
are waiting for other services become available. Now, with HEALTHCHECK enabled, docker does not resolve
names of services that not pass HEALTHCHECK yet. Meaning that if one of the depenend services is not yet
available, it will create a chain of failing services.

The services below retry to resolve 100 time, with an average of 3.5 seconds. Hence, the --start-time
flag is now set at 350 seconds.
- dovecot (imap)
- postfix (smtp)
- rspamd (antispam)
Tim Möhlmann c3e89967fb
Fix front health checking
- Specified seperated /health path in order to allow for healthcheck even if webmail and admin are not seletectd. This also allows healthchecking fom external services like DNS load balancers;
- Make curl not to fail on TLS because localhost is not included in the certificates.
mergify[bot] 90b8c3cc1f
Merge pull request from kaiyou/feat-reply-startdate
Implement a start date filter for autoreply, fixes 
mergify[bot] bce1487338
Merge pull request from hacor/master
Kubernetes fixed for production
kaiyou 1fcaef7c7e
Merge branch 'master' into fix-sender-checks
Paul Williams 78bd5aea1c enable http2, because it's that easy
hoellen 72d4fa2bc9
remove empty line from merge conflict
hoellen 857ad50509
Merge branch 'master' into feat-reply-startdate
mergify[bot] 4a5c0a6d21
Merge pull request from kaiyou/fix-password-performance
Improve password checking performance
mergify[bot] 80658c30da
Merge pull request from hoellen/fix-webmail-root
Fix nginx conf if webmail is on root path
Hans Cornelis f10416e85a Merged with new PRs
mergify[bot] 118ea0f3fb
Merge pull request from ofthesun9/feature-swarm
Enabling swarm deployment on master branch
mergify[bot] 727970514d
Merge pull request from ofthesun9/feat-fuzzyhashes
Trying to enable fuzzy hashes for rspamd
kaiyou 82069ea3f0 Clean most of the refactored code
kaiyou f40fcd7ac0 Use click for the manager command
kaiyou fc24426291 First batch of refactoring, using the app factory pattern
hoellen d4f32c3e7d remove rewrite if webmail is on root
kaiyou 01fa179767 Update the user password in database when needed
kaiyou 988e09e65e Add a profiler in debug mode for improving performance
kaiyou dba8f1810d Do not check the password another time in Dovecot
kaiyou d5162328ec Allow dovecot to write the source configuration directory for compiling sieve scripts
kaiyou ce0bf3366d Learn fuzzy hashes automatically
kaiyou 0a5dbf6230 Re-enable local dovecot sieve scripts
Hans Cornelis ef55ca525c Deleted conflicting merge files
Signed-off-by: Hans Cornelis <hacornelis@gmail.com>
Hans Cornelis e67a0d464b Deleted old folder
Hans Cornelis 3098343360 Merged conflicts
hacor 4ea12deae7 Added kubernetes to Mailu
kaiyou ed3388ed6e Merge branch 'master' into feat-reply-startdate
kaiyou 7c82be904f Merge branch 'master' of github.com:mailu/mailu
Thomas Sänger a412951a30
simpler healthcheck for postfix
Thomas Sänger 0bc901a722
add healthcheck for dovecot
Thomas Sänger 1fc40bf932
add healthcheck for postfix
Thomas Sänger 39272ab05c
add healthcheck for http services
kaiyou e784556330 Fix an edge case with old values containing None for coma separated lists
kaiyou f647d1a0bc Merge branch 'master' into fix-sender-checks
kaiyou 5ada669f43 Rebase reply startdate on master
mergify[bot] bee81d1a54
Merge pull request from HorayNarea/bcrypt
support bcrypt and use it as default
mergify[bot] 9fd7851cb6
Merge pull request from HorayNarea/apk-no-cache
remove apk-warning about cache
kaiyou 15eb2806bf Merge branch 'master' into feat-reply-startdate
kaiyou 5035975c41 Remove Postfix debugging
kaiyou c6846fd8db Merge branch 'master' into feat-reply-startdate
mergify[bot] a91a54b5f1
Merge pull request from usrpro/fix-certbot
Front: move to Alpine:3.8 and fixing 
Tim Möhlmann de43060ef8
Move to Alpine:3.8 and fixing
Thomas Sänger bdfcc5b530
pin alpine-version for 'none'-image
Thomas Sänger 6aafef88bd
remove apk-warning about cache
Thomas Sänger c8b39c5d4a
support bcrypt and use it as default
kaiyou 10ec2f999a Another (embarrassing) fix for a merge typo
kaiyou 2e1aa079c1 Fix one (hopefully) last merge typo
kaiyou 4b9dbf00a8 Fix yet another merge-time typo
kaiyou e8e133b53d Fix a merge typo in postfix build
kaiyou 00b5ae11db
Merge branch 'master' into feat-abstract-db
ofthesun9 13146be57e Merge branch 'master' into feature-swarm
kaiyou 508e519a34 Refactor the postfix views and implement sender checks
kaiyou 8b189ed145 Separate senderaccess and senderlogin maps
ofthesun9 74796201ec Merge branch 'master' into feature-swarm
kaiyou fc99eb7b34 Re-enable sender access check to prevent source spoofing
kaiyou f3f0b98755 Fix relay restrictions so email gets delivered correctly
Tim Möhlmann 0817629869
Increase attempts as it failed on fresh Swarm host
Tim Möhlmann 716ed16f34
Fix typo
Tim Möhlmann 16469d7282
Upgrade to newer pip version
Tim Möhlmann 1bae5968ad
Import tenacy and fix syntax errors
Tim Möhlmann c457ccfa60
Use tenacity for resolver retries
Tim Möhlmann d6c386651d
Merge branch 'master' into fix-swarm-start
ofthesun9 09d77bc2de Handle the case where the variable REJECT_UNLISTED_RECIPIENT is not set
ofthesun9 cc17962c86 fixes
kaiyou 4d70a8737e Expose the data volume for admin container
kaiyou 2cba045013 Explicitely declare required volumes, fixes
kaiyou fcad52b145 Implement a start date filter for autoreply, fixes
kaiyou 82bb8c2fd9 Merge remote-tracking branch 'github/master' into feat-abstract-db
kaiyou f5668dea51 Handle relays as virtual transports through podop
kaiyou 9890e1fb2a Fix the dovecot configuration path
kaiyou 42c6bdb4df Split the internal blueprint into multiple view files
kaiyou dc4b0d21ea Clean the dovecot configuration dir
kaiyou f9c6c98180 Remove fetchmail dependency to the databse
kaiyou 43b6547e1c Lower the loglevel of podop
kaiyou 542793260b Handle wildcard aliases using podop
kaiyou 0d52364eac Fix alias resolution through podop
kaiyou 6d088504bd Adjust podop debug level based on environment
kaiyou 6ba55ee377 Implement the sieve script name resolution pattern
kaiyou 3c725bf634 Only support generating the default sieve script
kaiyou ca6c0bc8fd Fix the user sieve script generation
kaiyou 4d25083847 Move sieve script generation to the admin container
kaiyou cfeaa189f9 Use proper 404 return codes for missing objects
kaiyou d8365bfbcf Use simpler routes for Dovecot
kaiyou 697caaab81 Update podop access and mail restrictions
kaiyou 7143fb8c47 Implement some basic views for podop
kaiyou 39cd0d5034 Upgrade to alpine 3.8 for smtp and imap
kaiyou 68aa797720 Merge branch 'master' into feat-abstract-db
ofthesun9 f5f09fad6e Reverting the patch for dovecot.conf, as it is not needed
Thomas Sänger 0b885548ab
bind to any protocol
kaiyou 73ca5fb3d3 Provide a more generic skeletton for postfix virtual lookups
ofthesun9 23e288aadc Enabling swarm deployment on master branch:
-Extends the usage of POD_ADDRESS_RANGE
-Provides documentation
kaiyou 5dc9ee9516
Merge pull request from hoellen/spam-trash-fix
Dont flag spam as ham if moved to trash
kaiyou d917f60352
Merge pull request from HorayNarea/compress
add optional Maildir-Compression
kaiyou 64269e08c0
Merge pull request from HorayNarea/master
add full-text search support
kaiyou 313b79538e
Merge pull request from mprihoda/feature/better-ratelimit-error
Return correct status codes from auth rate limiter failure.
Tim Möhlmann cc8e15748b Retry 10 times when resolving fails in start.py scripts
Thomas Sänger 0bdb2a16bc
add optional Maildir-Compression
Thomas Sänger fb62e6b5a2
add full-text search support
kaiyou f506966abc Pin Alpine 3.7 to preserve the Postfix version
Tim Möhlmann 9350bb9b9a Use fixed alpine:3.7 tag to prevent postix upgrade
Pierre Jaury 3dca1a834c Pin alpine 3.7 until we fix the certbot issue, see
Pierre Jaury 18fe8cd9f2 Pin alpine:3.7 for Dovecot since extdata was removed from repos, fixes
Pierre Jaury 5ad02ae2e5 Use a more uniform 'Save' for most form submits, fixes
Pierre Jaury c04e58498d Remove unused postfix sqlite files
Pierre Jaury bb73933e1e Switch postfix to Podop
Pierre Jaury 82e738cc53 Remove the old code of postproxy
Pierre Jaury b5d6b93869 Switch to using Podop in Dovecot
Pierre Jaury 809fe78f82 Add dovecot views to the internal API
Pierre Jaury 28001213d4 Remove the redis-based quota code
Pierre Jaury 76617a3c97 Store the quota status in database
Pierre Jaury 2b2ab864d1 Add support for querying the table in Dovecot proxy
Pierre Jaury 70175f8c28 Add postproxy support for Dovecot dict protocol
Pierre Jaury 262e82a367 Add a postfix socketmap to http proxy
Michal Prihoda f5e7751835 Return correct status codes from auth rate limiter failure.
hoellen ca26264d01 Dont flag spam as ham if moved to trash (fix )
kaiyou 75a1bf967c
Merge pull request from hoellen/webmail-messagesize
Use message_size_limit variable from env for webmail client_max_body_size
hoellen c51e1b9eef webmail client_max_body_size with message_size_limit and 8M tolerance
kaiyou 74b72375cb
Merge pull request from MFAshby/user_validation
Made User and UserSignup validation consistent for the local part of
kaiyou 53bf6085dc
Merge pull request from hoellen/webmail-root
Add posibilty to run webmail on root '/'
hoellen 9091e54fda Hide administration header in sidebar for normal users.
hoellen 81a6a7cbf6 Use message_size variable from env for webmail
hoellen a1fb8442e3 Add posibilty to run webmail on root '/'
mfashby 0284b6a8e9 Made User and UserSignup validation consistent for the local part of the email address
Pierre Jaury 14687d09ba Fix announcements for idna domains
Pierre Jaury e543477c2e Revert "Only enable milter for incoming emails"
This reverts commit cfd233039e.
Pierre Jaury cfd233039e Only enable milter for incoming emails
Pierre Jaury 6828231c28 Fix the path of the nginx pid in startup scripts, fixes
Pierre Jaury 1b0b3a2b1e Only check login mismatch for authenticated users, fixes
Pierre Jaury 1371ba5f5e Add the keep field to fetch forms, fixes
Pierre Jaury ea658a174d Fix a typo in the base html template
Pierre Jaury b6c76a5e39 Do not remove openssl when purging build deps, fixes
kaiyou a47ba3474c
Merge pull request from romracer/pod-address
Use POD_ADDRESS_RANGE for Dovecot if it exists
kaiyou 3beceb90ec
Merge pull request from mildred/parametrize-hosts
Add various environment variables to allow running outside of docker-compose
kaiyou a9e41960a1
Merge pull request from dtwardow/flex_tls_filenames
TLS using configurable filenames
kaiyou 91e51a24c8
Merge pull request from sanduhrs/feature/463
Remove services status page
Thomas Sänger 7d661ab80d
don't require BootstrapCDN for FontAwesome (GDPR-compliance)
Mildred Ki'Lya 6bb4c6e2f0 Parametrize front address from dovecot
Mildred Ki'Lya ae8c9f5a6b Add various environment variables to allow running outside of docker-compose
Dennis Twardowsky 50f9f379e9 Flexible filenames for TLS via envvars (flavours 'cert' and 'mail' only)
Stefan Auditor 6177571e4d Remove services status localization
Stefan Auditor b541d4c257 Remove services status sidebar link
Stefan Auditor e89b32a3f4 Remove services route
Stefan Auditor ec8e82aaca Remove services status template
kaiyou f55e5e26cd Update messages.po (POEditor.com)
kaiyou 2b96abbef4 Update messages.po (POEditor.com)
kaiyou af38d5ab0c Update messages.po (POEditor.com)
kaiyou fc89b30e8a Update messages.po (POEditor.com)
kaiyou 791fab688a Update messages.po (POEditor.com)
kaiyou c9b0832899 Update messages.po (POEditor.com)
kaiyou fa1b0ac32c Update messages.po (POEditor.com)
kaiyou 75f0791965 Update messages.po (POEditor.com)
kaiyou c91c5c7493 Update messages.po (POEditor.com)
kaiyou 299a654e97 Update messages.po (POEditor.com)
kaiyou 494e52d8f0 Update messages.po (POEditor.com)
kaiyou 3b7014d563 Add dummy spanish and russion languages
kaiyou be0a0b4ac8 Update translation strings
kaiyou 8bad30cd59 Move the domain MX status to the detail page
kaiyou 7f0447514c Finish storing the user quota to redis
kaiyou 80893be68b Add a missing import to dnspython
kaiyou 091369915b Display the user quota in the admin interface
kaiyou e13593f29a Switch to database 2 for rate limiting
kaiyou d1dbba2d3a Add expose instructions in Dockerfiles, fixes
kaiyou 62d1a0c104 Add a status field to the domain list
kaiyou bb0d7bf6dc Enforce the nocase collation on the email table
kaiyou 186c30d2ac Have the admin listen on ipv6
kaiyou 35276c3101
Merge pull request from Farthen/master
Dovecot: Add SQL iterate_query to config file.
kaiyou dfaedb76f1
Merge pull request from sanduhrs/feature/446
Add a sqlalchemy custom type for unicode to idna conversion of domain names
farthen f75280e4a3 Dovecot: Add SQL iterate_query to config file.
This allows to use doveadm -A to execute maintenance tasks for all users on the server
kaiyou 9968d708f1 Update the prod requirements
kaiyou 381e76511d Add self-service domain registration
Stefan Auditor c688970b32 Respect user enabled flag in admin authentication
Stefan Auditor d3064579f4 Respect user enabled flag in basic authentication
Stefan Auditor 92f4858323 Respect user.enabled status in internal authentication
Stefan Auditor d2c6cecca6 Remove is_enabled method and use the enabled attribute instead
Stefan Auditor 5bfdd75738 Respect user enabled flag on user.login
Stefan Auditor 78f4fa7db9 Add field to ui for user enabled flag
Stefan Auditor 20d6fbae48 Add enabled flag to user model
Scott 2c2a1ed042 Remove stale link to old auto-forward settings. Fixes
Also update a reference to 'smtp' to use HOST_AUTHSMTP
kaiyou a51416a4af
Merge pull request from sanduhrs/feature/449
Add enabled flag to user model
Stefan Auditor e843f7ef1f Respect user enabled flag in admin authentication
Stefan Auditor c8540ddba7 Respect user enabled flag in basic authentication
Stefan Auditor 6fc22e5432 Respect user.enabled status in internal authentication
Stefan Auditor 733b89bff5 Remove is_enabled method and use the enabled attribute instead
Stefan Auditor 3b66fcada7 Respect user enabled flag on user.login
Stefan Auditor 7139a27bf1 Add field to ui for user enabled flag
Stefan Auditor f585197e52 Add enabled flag to user model
Scott e5c25c395f Remove stale link to old auto-forward settings. Fixes
Also update a reference to 'smtp' to use HOST_AUTHSMTP
Stefan Auditor 7f5bd98a2e Add parameters to database field
Stefan Auditor 93d5254b3f Add another type decorator for idna email support
Scott 6018995534 Use POD_ADDRESS_RANGE for Dovecot if it exists
This is required to override allow_nets in a Kubernetes environment where pods are not recreated with the same IP address.
Stefan Auditor 792c720c13 Save user email domain_name as idna representation
Stefan Auditor c40e255f3b Reset relay columns to string
Stefan Auditor d9ea64fac7 Import idna library and move code a bit upwards
Stefan Auditor 5a7272ff12 Replace other occurences of domain names with idna
Stefan Auditor 1b666cd25b Add a sqlalchemy custom type for unicode to idna conersion of domain names
kaiyou db0cd8efb4 Fix the client setup page when not logged in
kaiyou e92113bd57
Merge pull request from mildred/delivered-to-hdr
Add original Delivered-To header to received messages
kaiyou b4134b7774 Add a client setup page, fixes
kaiyou fa0bda7b69 Merge the auto-forward and antispam settings
kaiyou 3ef4e1f6b7 Add support for recaptcha upon signup
kaiyou e02e47c48e
Merge pull request from calebj/patch-1
Add support for sending from alternative domains
Scott b9e67635f4 Use HOST_ADMIN in "Forwarding authentication server". Fixes .
Mildred Ki'Lya 649a4fc9cf Add Delivered-To header to received messages
Postfix, after expanding the alias, is not transmitting the original
envelope recipient email address to dovecot and cannot record it in a
Received: header.

The LMTP DSN extension allows postfix to specify an ORCPT= parameter to
the "RCPT TO:" line (in postfix src/smtp/smtp_proto.c). However, dovecot
does not support the DNS extension on the LMTP endpoint. It has
preliminary support of the ORCPT parameter in latest versions but is is
disabled and not working.

The solution taken was to add a sieve script to parse the Received:
header written by postfix and parse the original RCPT TO address from
it. Then add the header through the "editheader" sieve extension. Later
sieve scripts can take this header to perform further filtering.