634 Commits (9803c51d5591cd322dc9f48403818e2402941983)

Author SHA1 Message Date
bors[bot] 179c624116
Merge #2631
2631: Restful api finishing touches r=mergify[bot] a=Diman0

## What type of PR?

enhancement

## What does this PR do?
Some finishing touches for the restful api.

- Make the API configurable via the setup utility.  
  - Configured exactly the same as the ADMIN and WEBMAIL. 
- We have a single config (API) that configures whether it is exposed (via front). Just like ADMIN. The API is always reachable by directly connecting to the admin container.
- API_TOKEN does not enable/disable the API anymore. When it is not configured, an error is returned (via the internet browser) that the API_TOKEN must be configured in mailu.env.
- Fix some small bugs in the setup utility ( selecting none in the dropdown boxes, now correctly changes the config)
- Update Flask-RestX to 1.0.5. This resolves the deprecation warnings introduced by Flask-RestX.

### Related issue(s)

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
Dimitri Huisman d6e7314f05
Make API configurable via the setup utility
Fix some small bugs in the setup utility
Improve documentation on the API.
2 years ago
Alexander Graf c4ca1cffaf
Set default for FETCHMAIL_ENABLED 2 years ago
Dimitri Huisman bcceac359d
Merge branch 'apiv1' of https://github.com/ghostwheel42/Mailu into feature-445-restful-api-ghostwheel 2 years ago
Łukasz Sitarski 202ff8db14
Remove duplicated 'actionstart = ' in fail2ban conf.
In fail2ban example configuration for ipset option, there was a duplicated string which makes the ipset and fail2ban fail to create the set.
Fail2ban will never ban any ip due to this error.
2 years ago
Dimitri Huisman 61d092922c
Process review comments (PR2464) 2 years ago
Dimitri Huisman 46d07ec236
Fix syntax styling api documentation. 2 years ago
Dimitri Huisman 67c423d61f
Add URL for accessing swaggerui to documentation 2 years ago
Dimitri Huisman 5c9cdfe1de
Introduction of the Mailu RESTful API.
Anything that can be configured in the web administration interface,
can also be configured via the Mailu RESTful API.
See the section Advanced configuration in the configuration reference
for the relevant settings in mailu.env for enabling the API.
(API, WEB_API, API_TOKEN).
2 years ago
Dimitri Huisman 6d87fa423c
Mention you must restart rspamd for the changes to take effect. 2 years ago
Dimitri Huisman 33497c8e31
Small extra clarification for new documentation 2 years ago
Florent Daigniere cea533ae57 Merge remote-tracking branch 'upstream/master' into oletools 2 years ago
Florent Daigniere f04be00798 doc 2 years ago
Florent Daigniere 44c064ff38 make it configurable 2 years ago
Florent Daigniere b70be29403 document 2 years ago
Florent Daigniere 4e3874b0c1 Enable dynamic resolution of hostnames 2 years ago
bors[bot] 0bde746610
Merge #2557
2557: Remove Swarm from master r=mergify[bot] a=nextgens

Remove Swarm from master as discussed.

This hasn't been tested

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
2 years ago
bors[bot] 033889dc95
Merge #2542 #2559
2542: Implement header authentication via external proxy r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

Implement header authentication via external proxy

### Related issue(s)
- closes #1972
- closes #2183

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2559: Turns out that php81-ctype is required by roundcube r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

It solves:
```
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "NOTICE: PHP message: PHP Fatal error:  Uncaught Error: Call to undefined function Masterminds\HTML5\Parser\ctype_alpha() in /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php:140"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "Stack trace:"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#0 /var/www/roundcube/vendor/masterminds/html5/src/HTML5/Parser/Tokenizer.php(82): Masterminds\HTML5\Parser\Tokenizer->consumeData()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#1 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(161): Masterminds\HTML5\Parser\Tokenizer->parse()"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#2 /var/www/roundcube/vendor/masterminds/html5/src/HTML5.php(89): Masterminds\HTML5->parse('<html>\n    <hea...', Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#3 /var/www/roundcube/program/lib/Roundcube/rcube_washtml.php(700): Masterminds\HTML5->loadHTML('<html>\n    <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#4 /var/www/roundcube/program/actions/mail/index.php(975): rcube_washtml->wash('<html>\n    <hea...')"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#5 /var/www/roundcube/program/actions/mail/index.php(1019): rcmail_action_mail_index::wash_html('<!doctype html>...', Array, Array)"
[25-Nov-2022 08:19:20] WARNING: [pool php] child 335 said into stderr: "#6 /var/www/roundcube/program/actions/mail/show.php(720): rcmail_action_mail_index::pr..."
```

see https://github.com/roundcube/roundcubemail/issues/7049


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere d0631558c7 Remove Swarm everywhere.
This hasn't been tested
2 years ago
Florent Daigniere 3721a6aa02 Merge branch 'master' of https://github.com/Mailu/Mailu into HEAD 2 years ago
Florent Daigniere 9fa3a3e0c7 doc 2 years ago
bors[bot] 553b02fb3d
Merge #2529
2529: Improve fetchmail r=mergify[bot] a=nextgens

## What type of PR?

enhancement

## What does this PR do?

Improve fetchmail:
- allow delivery via LMTP (faster, bypassing the filters)
- allow several folders to be retrieved
- run fetchmail as non-root
- tweak the compose file to ensure we have all the dependencies

### Related issue(s)
- closes #1231 
- closes #2246 
- closes #711

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [ ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.

Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
2 years ago
Florent Daigniere 38507b2e1b Close #2372: Implement a GUI for WILDCARD_SENDERS 2 years ago
Florent Daigniere e2d4e3eb2e Implement header authentication via external proxy 2 years ago
Florent Daigniere 15b889fac8 Specify that this is optional 2 years ago
bors[bot] f43c8c652e
Merge #2483 #2535
2483: Introduce FETCHMAIL_ENABLED r=mergify[bot] a=DjVinnii

## What type of PR?

Enhancement

## What does this PR do?
Add `FETCHMAIL_ENABLED` to enable/disable the Fetchmail functionality in the Admin UI.

### Related issue(s)
- closes #2127

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


2535: fix the linux/arm/v7 build r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

The arm builder is running aarch64 ... and there is no package for arm/v7


Co-authored-by: Vincent Kling <v.kling@vinniict.nl>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 63513608b9 Close #2533: document SQLALCHEMY_DATABASE_URI 2 years ago
Florent Daigniere 9c7dfbeb24 Doc 2 years ago
bors[bot] 0839490beb
Merge #2479
2479: Rework the anti-spoofing rule r=mergify[bot] a=nextgens

## What type of PR?

Feature

## What does this PR do?

We shouldn't assume that Mailu is the only MTA allowed to send emails on behalf of the domains it hosts.
We should also ensure that it's non-trivial for email-spoofing of hosted domains to happen

Previously we were preventing any spoofing of the envelope from; Now we are preventing spoofing of both the envelope from and the header from unless some form of authentication passes (is a RELAYHOST, SPF, DKIM, ARC)

### Related issue(s)
- close #2475

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Alexander Graf dd3cd1263e
Add development documentation again 2 years ago
Vincent Kling 83fdc07a6f Default FETCHMAIL_ENABLED to True 2 years ago
bors[bot] 12480ccbff
Merge #2328
2328: Feature: Configurable default spam threshold used for new users r=mergify[bot] a=enginefeeder101

## What type of PR?

Feature

## What does this PR do?

This PR adds functionality to set a custom default spam threshold
for new users. The environment variable ``DEFAULT_SPAM_THRESHOLD`` is
used for this purpose. When not set, it defaults back to 80%, as the
default value was before.

If ``DEFAULT_SPAM_THRESHOLD`` is set to a value that Python cannot
parse as an integer, a ValueError is thrown. There is no error handling
for that case built-in. Should that be done?

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: enginefeeder101 <enginefeeder101@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
bors[bot] 5703e97c73
Merge #2460
2460: Switch to a base image containing base tools and the podop and socrate libs r=mergify[bot] a=ghostwheel42

## What type of PR?

enhancement of build process

## What does this PR do?

Changes build.hcl to build core images using a base image.
Also adds a "assets" base image for the admin container.


Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Pierre Jaury <pierre@jaury.eu>
Co-authored-by: kaiyou <pierre@jaury.eu>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
2 years ago
Florent Daigniere 07bf8ce6df Add anti-spoofing to the feature list 2 years ago
Vincent Kling 8a60b658b4 Implement FETCHMAIL_ENABLED 2 years ago
Vincent Kling 23d06a5761 Fix a bunch of typos 2 years ago
Dimitri Huisman 6ea2d84a3c Remove outdated wrong documentation 2 years ago
bors[bot] b5e7cad2d3
Merge #2448
2448: Give a chance to rspamd's bayes classifier r=mergify[bot] a=nextgens

## What type of PR?

bug-fix + documentation

## What does this PR do?

As pointed out in #2442, the bayesian filter of rspamd doesn't get any chance to run as ``min_learns`` is set to 200 and we never teach it any HAM.

This PR enables rspamd's autolearn feature, that will "reinforce" good/bad by learning from the scoring of other modules. It ensures both that we will eventually reach the 200 mark but also that the data stays fresh.

I've also taken this opportunity to update the documentation & FAQ accordingly, to ensure that users teach their HAM & SPAM to both the fuzzy and bayes classifiers.

Thank you to [woj-tek](https://github.com/woj-tek) for doing the ground work on this.

### Related issue(s)
- closes #2442

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
2 years ago
Florent Daigniere 9a38a22df3 typo 2 years ago
Florent Daigniere 6a0e881522 Introduce TLS_PERMISSIVE for port 25
This new advanced setting to harden cipher configuration on port 25. Changing the default is strongly discouraged, please read the documentation before doing so.
2 years ago
Florent Daigniere 7272a99d18 fuzzy matching and bayes are two different things
document accordingly
2 years ago
Vincent Kling bab3f0f5a4 Remove POD_ADDRESS_RANGE 2 years ago
Danny Daemonic ed5e8cce0e Update docs to mention .inc.php for roundcube
A recent patch updated the roundcube overrides to use .inc.php vs .inc,
as it's done in roundcube (and as suggested by roundcube plugin docs).

It corrected the overrides and fixed it's section in the faq, but missed
a section in the configuration docs that to the overrides. This fixes
that oversight.
2 years ago
Danny Daemonic 3eeb7962c2 Correct the extension used for Roundcube overrides
This adds ".inc.php" files to the included overrides while maintaining
support for existing ".inc" files previously included via overrides.

Roundcube itself uses "inc.php" files and these overrides are expected
to match that format. Switching to "inc.php" both tells the user that
these need to be proper php files and conveys they are used for changing
the same settings that Roundcube's inc.php files modify.
2 years ago
Dimitri Huisman 81c9e01d24 finishing touches for PR# 2328
Antispam.rst contained a syntax error.
Move config description to common section which is more fitting.
Fixed wrong assignment of default value for DEFAULT_SPAM_THRESHOLD in models.py.
2 years ago
bors[bot] 53de7b7d60
Merge #2403
2403: Feature: switch CI/CD from build to buildx r=mergify[bot] a=Diman0

## What type of PR?

Feature and enhancement

## What does this PR do?

Switch from docker build to buildx for CI/CD.
    - The main workflow file has been optimised and simplified.
    - Images are built in parallel when building locally resulting in much faster build times.
    - The github action workflow is about 50% faster.
    - Arm images are built as well. These images are not tested due to restrictions of github actions (no arm runners). The tags of the images have -arm appended to it. The arm images are built on merge on master and release branch (x.y). They do not influence the normal CI/CD workflow used for bors (for PR) and real releases (merge on master and branch x.y for x86_64). 
    - Arm images (and normal x86_64 images) can also be built locally.
    - Reusable workflow is introduced for building, testing and deploying the images. This allows the workflow to be reused for other purposes in the future.
    - Workflow can be manually triggered. This allows forked Mailu projects to also use the workflow for building images.

The main workflow makes use of github actions cache to store the cache layer. This layer is used to quickly rebuilt the images in the testing step and deploy step.

Unfortunately the building the arm images fails sometimes due to timeouts. Sometimes the connection to github actions cache is very slow. Restarting the workflow from the last failed step resolves this. I have not observed this with the normal build.

Just as previous time, you can use a forked project for testing the changes (https://github.com/Diman0/Mailu_Fork). You should still have owner access. I have created branch 1.11 for testing. You can see I already push 4 times to branch 1.11 (current version is 1.11.3).

### Related issue(s)
- Mention an issue like: #001
- closes #2383 
- closes #1830
- closes #1200

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
2 years ago
Dimitri Huisman a54baeff00 Fixed error in reverse.rst and remove unneeded example block. 2 years ago
Dimitri Huisman 10c02c0a32 Fix error in reverse proxy example for how to include nginx override.
Also clarified where the file is included in the nginx configuration file.
2 years ago
Dimitri Huisman 28e409f5ea Fix small typo in config-import example template 2 years ago
Dimitri Huisman 3aafecafe7 Merge branch 'master' into feat-switch-buildx 2 years ago