Merge #2253

2253: Workaround the infamous coredns feature r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Ensure that we set the ``DO`` flag on our queries to work around https://github.com/coredns/coredns/issues/5189

Add a FAQ entry to point users in the right direction in other cases (dnsmasq), discourage users from running Mailu without unbound

### Related issue(s)
- closes #2243
- closes #2239
- #2164
- #2163
- #2162
- #2135
- #1988

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>

core/admin/mailu/utils.py

@@ -47,7 +47,7 @@ def handle_needs_login():
 
 # DNS stub configured to do DNSSEC enabled queries
 resolver = dns.resolver.Resolver()
-resolver.use_edns(0, 0, 1232)
+resolver.use_edns(0, dns.flags.DO, 1232)
 resolver.flags = dns.flags.AD | dns.flags.RD
 
 def has_dane_record(domain, timeout=10):
@@ -56,7 +56,6 @@ def has_dane_record(domain, timeout=10):
         if result.response.flags & dns.flags.AD:
             for record in result:
                 if isinstance(record, dns.rdtypes.ANY.TLSA.TLSA):
-                    record.validate()
                     if record.usage in [2,3] and record.selector in [0,1] and record.mtype in [0,1,2]:
                         return True
     except dns.resolver.NoNameservers:

core/admin/start.py

@@ -28,7 +28,7 @@ def test_DNS():
     import time
     # DNS stub configured to do DNSSEC enabled queries
     resolver = dns.resolver.Resolver()
-    resolver.use_edns(0, 0, 1232)
+    resolver.use_edns(0, dns.flags.DO, 1232)
     resolver.flags = dns.flags.AD | dns.flags.RD
     nameservers = resolver.nameservers
     for ns in nameservers:
@@ -37,11 +37,11 @@ def test_DNS():
             try:
                 result = resolver.resolve('example.org', dns.rdatatype.A, dns.rdataclass.IN, lifetime=10)
             except Exception as e:
-                log.critical("Your DNS resolver at %s is not working (%s). Please use another resolver or enable unbound via https://setup.mailu.io.", ns, e);
+                log.critical("Your DNS resolver at %s is not working (%s). Please see https://mailu.io/master/faq.html#the-admin-container-won-t-start-and-its-log-says-critical-your-dns-resolver-isn-t-doing-dnssec-validation", ns, e);
             else:
                 if result.response.flags & dns.flags.AD:
                     break
-                log.critical("Your DNS resolver at %s isn't doing DNSSEC validation; Please use another resolver or enable unbound via https://setup.mailu.io.", ns)
+                log.critical("Your DNS resolver at %s isn't doing DNSSEC validation; Please see https://mailu.io/master/faq.html#the-admin-container-won-t-start-and-its-log-says-critical-your-dns-resolver-isn-t-doing-dnssec-validation.", ns)
             time.sleep(5)
 
 test_DNS()

docs/faq.rst

@@ -493,7 +493,7 @@ The syntax and options are as described in `postfix's documentation`_. Re-creati
 .. _`postfix's documentation`: http://www.postfix.org/postconf.5.html#smtp_tls_policy_maps
 
 403 - Access Denied Errors
----------------------------
+``````````````````````````
 
 While this may be due to several issues, check to make sure your ``DOMAIN=`` entry is the **first** entry in your ``HOSTNAMES=``.
 
@@ -864,4 +864,22 @@ iptables -t nat -A POSTROUTING -o eth0 -p tcp --dport 25 -j SNAT --to <your mx i
 
 A user gets ``Sender address rejected: Access denied. Please check the`` ``message recipient […] and try again`` even though the sender is legitimate?
 ``````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````````
+
 First, check if you are really sure the user is a legitimate sender, i.e. the registered user is authenticated successfully and own either the account or alias he/she is trying to send from. If you are really sure this is correct, then the user might try to errornously send via port 25 insteadof the designated SMTP client-ports. Port 25 is meant for server-to-server delivery, while users should use port 587 or 465.
+
+The admin container won't start and its log says ``Critical: your DNS resolver isn't doing DNSSEC validation``
+``````````````````````````````````````````````````````````````````````````````````````````````````````````````
+Since v1.9, Mailu requires a **validating** DNSSEC enabled DNS resolver. To check whether your DNS resolver (and its upstream) fits the requirements you can use the following command and see whether the **AD** flag is present in the reply:
+
+.. code-block:: bash
+
+  dig @<ip> +adflag example.org A
+
+We recommend that you run your own DNS resolver (enable unbound and update your docker-compose.yml when you update from older versions) instead of relying on publicly available ones. It's better security-wise (you don't have to trust them) and RBLs used by rspamd are known to rate-limit per source-ip address.
+
+We have seen a fair amount of support requests related to the following:
+
+- dnsmasq won't forward DNSSEC results unless instructed to do so. If you are running openwrt or pi-hole, you do need to enable DNSSEC.
+- `coredns has a bug`_ that we have now worked around
+
+.. _`coredns has a bug`: https://github.com/coredns/coredns/issues/5189

setup/static/render.js

@@ -83,4 +83,11 @@ $(document).ready(function() {
 			$("#ipv6").hide();
 		}
 	});
+	$("#resolver_enabled").change(function() {
+		if ($(this).is(":checked")) {
+			$("#unbound").hide();
+		} else {
+			$("#unbound").show();
+		}
+	});
 });

setup/templates/steps/compose/03_expose.html

@@ -40,13 +40,16 @@ avoid generic all-interfaces addresses like <code>0.0.0.0</code> or <code>::</co
   <input class="form-control" type="text" name="subnet6" required value="{{ subnet6 }}:beef::/64">
 </div>
 
-<p>The unbound resolver enables Mailu to do DNSSEC verification, DNS root lookups and caching. This also helps the antispam service not to get blocked by the public or ISP DNS servers.</p>
 <div class="form-check form-check-inline">
+  <p>The unbound resolver enables Mailu to do DNSSEC verification, DNS root lookups and caching. This also helps the antispam service not to get blocked by the public or ISP DNS servers.</p>
   <label class="form-check-label">
-    <input class="form-check-input" type="checkbox" name="resolver_enabled" value="true" checked>
-    Enable unbound resolver
+    <input class="form-check-input" type="checkbox" id="resolver_enabled" name="resolver_enabled" value="true" checked>
+    Enable an internal DNS resolver (unbound)
   </label>
 </div>
+<div class="form-group" id="unbound" style="display: none">
+	<p><span class="label label-danger">Read this:</span> Mailu requires a <b>validating, DNSSEC-enabled</b> DNS resolver</b> to function. Be sure to read our <a href="https://mailu.io/{{ version }}/faq.html#the-admin-container-won-t-start-and-its-log-says-critical-your-dns-resolver-isn-t-doing-dnssec-validation">FAQ entry</a> on the topic.</p>
+</div>
 
 <p>You server will be available under a main hostname but may expose multiple public
 hostnames. Every e-mail domain that points to this server must have one of the

towncrier/newsfragments/2239.bugfix

@@ -0,0 +1 @@
+Work around a bug in coredns: set the DO flag on our DNSSEC queries. Add a new FAQ entry to explain our DNSSEC requirements and ensure that our error message points to it.