1066 Commits (bba6c5bb88017c29f3d40bf327d0165d7da54b59)

Author SHA1 Message Date
bors[bot] bba6c5bb88
Merge #2603
2603: Enable HAPROXY protocol on SUBNET r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

- Enable HAPROXY in between front and imap: With this we avoid running into the limitations of  ``mail_max_userip_connections`` and the logfiles reflect the real IP.
- Enable HAPROXY in between front and smtp: with this postfix and rspamd are aware of whether TLS was used or not on the last hop. In practice this won't work as nginx doesn't send PROTO yet.
- Discard redundant log messages from postfix

With all of this, not only are the logs easier to understand but ``doveadm who`` also works as one would expect.

### Related issue(s)
- closes #894
- #1328
- closes #1364
- #1705

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
bors[bot] 52c17411bd
Merge #2596
2596: db.String without length cause an error in migration for MySQL DB r=mergify[bot] a=csthiang

## What type of PR?

bug-fix

## What does this PR do?

For MySQL, `db.String` requires a length because db.String gets translated to `VARCHAR` in MySQL and `VARCHAR` requires a length. I was considering adding a length to it but since the affected fields were used to store CommaSeparatedList and json-encoded string, I have a feeling it can be quite large in the future. `db.Text` seems to fit into this use case but please correct me if I am wrong.

This actually affects a DB migration with the following error:

```
  File "/app/venv/bin/flask", line 8, in <module>
    sys.exit(main())
  File "/app/venv/lib/python3.10/site-packages/flask/cli.py", line 1047, in main
    cli.main()
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/cli.py", line 149, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/__init__.py", line 98, in wrapped
    f(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/__init__.py", line 185, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/venv/lib/python3.10/site-packages/alembic/command.py", line 322, in upgrade
    script.run_env()
  File "/app/venv/lib/python3.10/site-packages/alembic/script/base.py", line 569, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/app/venv/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/venv/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/migrations/env.py", line 99, in <module>
    run_migrations_online()
  File "/app/migrations/env.py", line 92, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/venv/lib/python3.10/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
    step.migration_fn(**kw)
  File "/app/migrations/versions/f4f0f89e0047_.py", line 18, in upgrade
    with op.batch_alter_table('fetch') as batch:
  File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/app/venv/lib/python3.10/site-packages/alembic/operations/base.py", line 381, in batch_alter_table
    impl.flush()
  File "/app/venv/lib/python3.10/site-packages/alembic/operations/batch.py", line 111, in flush
    fn(*arg, **kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 322, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 195, in _exec
    return conn.execute(construct, multiparams)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
    return connection._execute_ddl(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1469, in _execute_ddl
    compiled = ddl.compile(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 502, in compile
    return self._compiler(dialect, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 32, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 463, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 498, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/ext/compiler.py", line 548, in <lambda>
    lambda *arg, **kw: existing(*arg, **kw),
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/ext/compiler.py", line 604, in __call__
    expr = fn(element, compiler, **kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/base.py", line 190, in visit_add_column
    add_column(compiler, element.column, **kw),
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/base.py", line 330, in add_column
    text = "ADD COLUMN %s" % compiler.get_column_specification(column, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/base.py", line 1714, in get_column_specification
    self.dialect.type_compiler.process(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 532, in process
    return type_._compiler_dispatch(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 82, in _compiler_dispatch
    return meth(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5028, in visit_type_decorator
    return self.process(type_.type_engine(self.dialect), **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 532, in process
    return type_._compiler_dispatch(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 82, in _compiler_dispatch
    return meth(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5006, in visit_string
    return self.visit_VARCHAR(type_, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/base.py", line 2214, in visit_VARCHAR
    raise exc.CompileError(
sqlalchemy.exc.CompileError: VARCHAR requires a length on dialect mysql
[2022-12-22 09:23:12 +0000] [17] [INFO] Starting gunicorn 20.1.0
[2022-12-22 09:23:12 +0000] [17] [INFO] Listening at: http://0.0.0.0:80 (17)
[2022-12-22 09:23:12 +0000] [17] [INFO] Using worker: gthread
[2022-12-22 09:23:12 +0000] [18] [INFO] Booting worker with pid: 18
```

### Related issue(s)
none

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Johnson Thiang <jthiang@pop-os.localdomain>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2 years ago
Florent Daigniere ca44ccbe1c
Use the size other implementations default to 2 years ago
Alexander Graf 6f71ea833b
Update python dependencies as suggested by dependabot 2 years ago
Florent Daigniere 4ae0d7d768 Enable HAPROXY protocol in between front and imap
With this we avoid running into the limitations of
 mail_max_userip_connections (see #894 amd #1364) and the
 logfiles as well as ``doveadm who`` give an accurate picture.
2 years ago
Alexander Graf 84ebab2cb4
Fix creation of deep structures using import in update mode 2 years ago
Florent Daigniere 108958cabb drop privs better 2 years ago
bors[bot] 8461a11ff4
Merge #2588
2588: IMAP folder names may contain characters outside of \w: [a-zA-Z0-9] r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

IMAP folder names may contain characters outside of \w: [a-zA-Z0-9]. Typically it may be subfolders...

I have also simplified the regexp since we strip spaces the line below.

This is used for "external accounts"/fetchmail.

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Johnson Thiang bd20ef04cc change field type to db.text 2 years ago
Alexander Graf c38e6aae4e
Add button to mailu-admin in roundcube task menu 2 years ago
bors[bot] 251db0b1af
Merge #2562
2562: Dynamic address resolution everywhere r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Use dynamic address resolution everywhere.
Derive a new key for admin/SECRET_KEY
Cleanup the environment

This should allow restarting containers.

### Related issue(s)
- closes #1341
- closes #1013
- closes #1430

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere c634b9ac04 IMAP folder names may contain characters outside of \w: [a-zA-Z0-9] 2 years ago
fastlorenzo 135207db3e
fix missing casting to int for SESSION_KEY_BITS
Signed-off-by: fastlorenzo <git@bernardi.be>
2 years ago
Florent Daigniere b630355d03 Autofocus the login form on /sso/login 2 years ago
Florent Daigniere 4e3874b0c1 Enable dynamic resolution of hostnames 2 years ago
bors[bot] 8c3da2815d
Merge #2565
2565: Fix DB downgrade r=mergify[bot] a=nextgens

Fix DB downgrade. This isn't used in normal operations but is wrong nevertheless.

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
bors[bot] cd5e6c896f
Merge #2568
2568: Fix a bug preventing users without IMAP access to access the webmails r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Fix a bug preventing users without IMAP access to access the webmails

### Related issue(s)
- close #2451

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere c565e69a01
as requested 2 years ago
Florent Daigniere b553d025eb
remove newline 2 years ago
Florent Daigniere 00f07ef533 close #2451: prevent an auth-loop on webmails 2 years ago
Florent Daigniere 3e38e7b89d Remove the dependency on pyOpenSSL 2 years ago
Florent Daigniere 98f16b1d47 Fix DB downgrade 2 years ago
Florent Daigniere c1062f3db2 set the umask 2 years ago
bors[bot] 033889dc95
Merge #2542 #2559
2542: Implement header authentication via external proxy r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

Implement header authentication via external proxy

### Related issue(s)
- closes #1972
- closes #2183

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2559: Turns out that php81-ctype is required by roundcube r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It solves:
```
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "NOTICE: PHP message: PHP Fatal error:  Uncaught Error: Call to undefined function Masterminds\HTML5\Parser\ctype_alpha() in /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php:140"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "Stack trace:"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#0 /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php(82): Masterminds\HTML5\Parser\Tokenizer->consumeData()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#1 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(161): Masterminds\HTML5\Parser\Tokenizer->parse()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#2 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(89): Masterminds\HTML5->parse('<html>\n    <hea...', Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#3 /var/www/roundcube/program/lib/Roundcube/rcube_washtml.php(700): Masterminds\HTML5->loadHTML('<html>\n    <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#4 /var/www/roundcube/program/actions/mail/index.php(975): rcube_washtml->wash('<html>\n    <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#5 /var/www/roundcube/program/actions/mail/index.php(1019): rcmail_action_mail_index::wash_html('<!doctype html>...', Array, Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#6 /var/www/roundcube/program/actions/mail/show.php(720): rcmail_action_mail_index::pr..."
```

see https://github.com/roundcube/roundcubemail/issues/7049


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
bors[bot] e0d42cadc0
Merge #2546
2546: Implement a GUI for WILDCARD_SENDERS r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

- Implement a GUI for WILDCARD_SENDERS

### Related issue(s)
- closes #2372

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
Alexander Graf b0990460a4
Fix error display 2 years ago
Alexander Graf 53720876b4
Colorize feature badges 2 years ago
Alexander Graf a5eeab37e1
Add default for column allow_spoofing 2 years ago
Florent Daigniere 3721a6aa02 Merge branch 'master' of https://github.com/Mailu/Mailu into HEAD 2 years ago
bors[bot] 2104c04e3b
Merge #2544
2544: Fix #2242: Make quotas adjustable in 50MiB increments r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Make quotas adjustable in 50MiB increments

### Related issue(s)
- closes #2242

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 19bd9362d3 As suggested by ghost 2 years ago
Florent Daigniere 12117cef37 Reduce the scope of the try: except 2 years ago
Florent Daigniere 546884d10c ghost's requested changes 2 years ago
Florent Daigniere c1144612be
fix sorting 2 years ago
Florent Daigniere 4d8bd210c5
Update run_dev.sh 2 years ago
Florent Daigniere ee512112fb
fix flask db history 2 years ago
bors[bot] 9c6e9b05db
Merge #2543
2543: Fix #2231: make public announcements work r=nextgens a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Ensure public announcements bypass filters.

They can still time-out... but this is already a big improvement that we should be able to backport.

### Related issue(s)
- closes #2231

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere f994c8687e doh 2 years ago
Florent Daigniere 44c47586ea Fix potential permission problems 2 years ago
Florent Daigniere d3d7916b58 Merge remote-tracking branch 'upstream/master' into upgrade-alpine 2 years ago
Florent Daigniere 45b01db9de Fix the language switcher 2 years ago
Florent Daigniere 3fc0a0e7fa Merge branch 'master' of https://github.com/Mailu/Mailu into fetchmail-improvements 2 years ago
Florent Daigniere 4da2db1b0b add comment as requested 2 years ago
Florent Daigniere c79e8d3852 Fix display bug 2 years ago
bors[bot] 553b02fb3d
Merge #2529
2529: Improve fetchmail r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Improve fetchmail:
- allow delivery via LMTP (faster, bypassing the filters)
- allow several folders to be retrieved
- run fetchmail as non-root
- tweak the compose file to ensure we have all the dependencies

### Related issue(s)
- closes #1231 
- closes #2246 
- closes #711

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2 years ago
Florent Daigniere dcf11aea48 Don't force a password reset 2 years ago
Florent Daigniere f802601a08
Update f4f0f89e0047_.py 2 years ago
Florent Daigniere d5ac9199a0
Update 7ac252f2bbbf_.py 2 years ago
Florent Daigniere 7822b41048 same for domains 2 years ago
Florent Daigniere ef9cc3c866 Show spoofing on /admin/user/list too 2 years ago