Introduce connection string (database url) for roundcube.

Remove database choice from setup.
Remove the old *DB_* database env variables from the documentation.
The env vars are deprecated now. They will be removed after the upcoming
Mailu release.

.github/workflows/build_test_deploy.yml

@@ -154,6 +154,7 @@ jobs:
           shell: bash
           command: |
             set -euxo pipefail \
+              ; /usr/bin/docker info \
               ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
               ; echo "$DOCKER_PASSW" |    docker login --username "$DOCKER_LOGIN" --password-stdin \
               ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -220,6 +221,7 @@ jobs:
           shell: bash
           command: |
             set -euxo pipefail \
+              ; /usr/bin/docker info \
               ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
               ; echo "$DOCKER_PASSW2" |    docker login --username "$DOCKER_LOGIN2" --password-stdin \
               ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -295,6 +297,7 @@ jobs:
           shell: bash
           command: |
             set -euxo pipefail \
+              ; /usr/bin/docker info \
               ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
               ; echo "$DOCKER_PASSW" |   docker login --username "$DOCKER_LOGIN" --password-stdin \
               ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \
@@ -368,6 +371,7 @@ jobs:
           shell: bash
           command: |
             set -euxo pipefail \
+              ; /usr/bin/docker info \
               ; echo "${{ github.token }}" |        docker login --username "${{ github.repository_owner }}" --password-stdin ghcr.io \
               ; echo "$DOCKER_PASSW2" |    docker login --username "$DOCKER_LOGIN2" --password-stdin \
               ; /usr/bin/docker buildx rm builder-${{ env.BUILDER }} \

core/admin/mailu/configuration.py

@@ -27,6 +27,7 @@ DEFAULT_CONFIG = {
     'DB_NAME': 'mailu',
     'SQLITE_DATABASE_FILE': 'data/main.db',
     'SQLALCHEMY_DATABASE_URI': 'sqlite:////data/main.db',
+    'SQLALCHEMY_DATABASE_URI_ROUNDCUBE': 'sqlite:////data/roundcube.db',
     'SQLALCHEMY_TRACK_MODIFICATIONS': False,
     # Statistics management
     'INSTANCE_ID_PATH': '/data/instance',

core/base/requirements-prod.txt

@@ -45,7 +45,7 @@ marshmallow==3.18.0
 marshmallow-sqlalchemy==0.28.1
 msoffcrypto-tool==5.0.0
 multidict==6.0.2
-mysql-connector-python==8.0.29
+mysql-connector-python==8.0.32
 olefile==0.46
 oletools==0.60.1
 packaging==21.3

docs/configuration.rst

@@ -285,26 +285,22 @@ These are used for DNS based service discovery with possibly changing services I
 Database settings
 -----------------
 
+The admin service stores configurations in a database. The roundcube service also stores configurations in a database.
+By default SQLite is used. SQLite is sufficient for practically any deployment.
+The database contains static configuration. It is recommended to use SQLite. However, a different database back-end can be used via a
+`DB URL`_. For Admin, only supported are SQLite, PostgreSQL and MariaDB/MySQL.
 
-The admin service stores configurations in a database.
+- ``SQLALCHEMY_DATABASE_URI`` (default: sqlite:////data/main.db): the SQLAlchemy database URL for accessing the database
+- ``SQLALCHEMY_DATABASE_URI_ROUNDCUBE`` (default: sqlite:////data/roundcube.db): the Roundcube database URL for accessing the Roundcube database
 
-- ``DB_FLAVOR``: the database type for mailu admin service. (``sqlite``, ``postgresql``, ``mysql``)
+For PostgreSQL use driver postgresql (``SQLALCHEMY_DATABASE_URI=postgresql://mailu:mailu_secret_password@database/mailu``).
-- ``DB_HOST``: the database host for mailu admin service. For non-default ports use the notation `host:port`. (when not ``sqlite``)
-- ``DB_PW``: the database password for mailu admin service. (when not ``sqlite``)
-- ``DB_USER``: the database user for mailu admin service. (when not ``sqlite``)
-- ``DB_NAME``: the database name for mailu admin service. (when not ``sqlite``)
 
-Alternatively, if you need more control, you can use a `DB URL`_ : do not set any of the ``DB_`` settings and set ``SQLALCHEMY_DATABASE_URI`` instead.
+For MariaDB/MySQL use driver mysql+mysqlconnector (``SQLALCHEMY_DATABASE_URI= mysql+mysqlconnector://mailu:mailu_secret_password@database/mailu```).
+
+For Roundcube, refer to the `roundcube documentation`_ for the URL specification.
 
 .. _`DB URL`: https://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls
-
+.. _`roundcube documentation`: https://github.com/roundcube/roundcubemail/blob/master/config/defaults.inc.php#L28
-The roundcube service stores configurations in a database.
-
-- ``ROUNDCUBE_DB_FLAVOR``: the database type for roundcube service. (``sqlite``, ``postgresql``, ``mysql``)
-- ``ROUNDCUBE_DB_HOST``: the database host for roundcube service. For non-default ports use the notation `host:port`. (when not ``sqlite``)
-- ``ROUNDCUBE_DB_PW``: the database password for roundcube service. (when not ``sqlite``)
-- ``ROUNDCUBE_DB_USER``: the database user for roundcube service. (when not ``sqlite``)
-- ``ROUNDCUBE_DB_NAME``: the database name for roundcube service. (when not ``sqlite``)
 
 Webmail settings
 ----------------

docs/database.rst

@@ -1,8 +1,10 @@
 Changing the database back-end
 ==============================
 
-By default Mailu uses a SQLite database. We have changed the internals of Mailu
+By default Mailu uses a SQLite database. It is possible to use an alternative database solutions such as PostgreSQL and MySQL/MariaDB.
-to enable the support of alternative database solutions such as PostgreSQL and MySQL/MariaDB.
+
+The Mailu database contains static data. SQLite is sufficient for any deployment scenario of Mailu. There is no need to use a different database system.
+The Mailu development team recommends to use SQLite.
 
 
 Migrating to a different database back-end
@@ -17,7 +19,7 @@ To switch to a different database back-end:
 
 1. Run config-export to export the configuration. E.g.  `docker compose exec admin flask mailu config-export --secrets --output mail-config.yml`
 2. Set up your new database server. Refer to the subsequent sections for tips for creating the database.
-3. Modify the database settings (DB_*) in mailu.env. Refer to the :ref:`configuration guide (link) <db_settings>` for the exact settings.
+3. Modify the database settings (SQLAlchemy database URL) in mailu.env. Refer to the :ref:`configuration guide (link) <db_settings>` for the exact settings.
 4. Start your Mailu deployment.
 5. Run config-import to import the configuration. E.g. `docker exec -i $(docker compose ps -q admin) flask mailu config-import -v < mail-config.yml`
 
@@ -192,10 +194,7 @@ to
 
 .. code-block:: docker
 
-   DB_HOST=database
+   SQLALCHEMY_DATABASE_URI=postgresql://mailu:mailu@database/mailu
-   DB_PORT=5432
-   DB_USER=mailu
-   DB_NAME=mailu
 
 Mailu is now configured to use the official PostgreSQL docker image. Bring your new deployment online
 

setup/flavors/compose/mailu.env

@@ -192,21 +192,3 @@ DEFAULT_SPAM_THRESHOLD=80
 # This is a mandatory setting for using the RESTful API.
 API_TOKEN={{ api_token }}
 
-###################################
-# Database settings
-###################################
-DB_FLAVOR={{ db_flavor }}
-{% if db_flavor == 'postgresql' or db_flavor == 'mysql' %}
-DB_USER={{ db_user }}
-DB_PW={{ db_pw }}
-DB_HOST={{ db_url }}
-DB_NAME={{ db_name }}
-{% endif %}
-
-{% if ((db_flavor_rc == 'postgresql' or db_flavor_rc == 'mysql')) and webmail_type == 'roundcube' %}
-ROUNDCUBE_DB_FLAVOR={{ db_flavor_rc }}
-ROUNDCUBE_DB_USER={{ roundcube_db_user }}
-ROUNDCUBE_DB_PW={{ roundcube_db_pw }}
-ROUNDCUBE_DB_HOST={{ roundcube_db_url }}
-ROUNDCUBE_DB_NAME={{ roundcube_db_name }}
-{% endif %}

setup/templates/steps/database.html

@@ -1,45 +0,0 @@
-{% call macros.panel("info", "Database preferences") %}
-
-<div class="form-group">
-	<label>Which database back end would you like to use?</label>
-	<br/>
-	 <select class="btn btn-primary dropdown-toggle" name="db_flavor" id="database">
-		{% for dbflavor in ["sqlite", "postgresql", "mysql"] %}
-			<option value="{{ dbflavor }}" >{{ dbflavor }}</option>
-		{% endfor %}
-	</select>
-	<div class="form-group" id="external_db" style="display: none">
-		<p>Set external database parameters for <b>Admin UI</b></p>
-		<label>DB User</label>
-		<input class="form-control" type="text" name="db_user" placeholder="Username" id="db_user">
-		<label>Db Password</label>
-		<input class="form-control" type="password" name="db_pw" placeholder="Password" id="db_pw">
-		<label>Db URL (for using a different port use the notation host:port )</label>
-		<input class="form-control" type="text" name="db_url" placeholder="URL" id="db_url">
-		<label>Db Name</label>
-		<input class="form-control" type="text" name="db_name" placeholder="Database Name" id="db_name">
-		<br/>
-		<div class="form-group" id="db_flavor_rc_sel" style="display: none">
-			<label>Which database back end would you like to use for roundcube?</label>
-			<br/>
-			<select class="btn btn-primary dropdown-toggle" name=db_flavor_rc id="database_rc">
-				{% for dbflavor_rc in ["sqlite", "postgresql", "mysql"] %}
-					<option value="{{ dbflavor_rc }}" >{{ dbflavor_rc }}</option>
-				{% endfor %}
-			</select>
-			<div class="form-group" id="roundcube_external_db" style="display: none">
-				<p>Set external database parameters for <b>Roundcube</b></p>
-				<label>DB User</label>
-				<input class="form-control" type="text" name="roundcube_db_user" placeholder="Username" id="roundcube_db_user">
-				<label>DB Password</label>
-				<input class="form-control" type="password" name="roundcube_db_pw" placeholder="Password" id="roundcube_db_pw">
-				<label>DB URL (for using a different port use the notation host:port )</label>
-				<input class="form-control" type="text" name="roundcube_db_url" placeholder="URL" id="roundcube_db_url">
-				<label>DB Name</label>
-				<input class="form-control" type="text" name="roundcube_db_name" placeholder="Database Name" id="roundcube_db_name">
-			</div>
-		</div>
-	</div>
-</div>
-
-{% endcall %}

setup/templates/wizard.html

@@ -14,7 +14,6 @@
   {%for file in steps %}
   {% include "steps/" + flavor + "/" + file %}
   {% endfor %}
-  {% include "steps/database.html" %}
   <input class="btn btn-primary" type="submit" value="Setup Mailu">
   </form>
 {% endblock %}

towncrier/newsfragments/2533.misc

@@ -0,0 +1,17 @@
+Introduce SQLAlchemy database uris for configuring the admin and roundcube database.
+Remove the database configuration option from the setup utility. Using a different
+database system than SQLite is not necessary for Mailu. The Mailu database generally
+contains static data.
+
+The usage of the *DB_* environment variables is deprecated now.
+They can still be used in the release after Mailu 1.9, but will be removed
+after that version. This means it will be removed from master after the upcoming
+Mailu release.
+
+To start using the new environment variables, all *DB_* environment variables must be changed to:
+SQLALCHEMY_DATABASE_URI=<SQLAlchemy database URL>
+SQLALCHEMY_DATABASE_URI_ROUNDCUBE=<Roundcube database URL>
+
+If no URI is specified, SQLite is used with these settings:
+SQLALCHEMY_DATABASE_URI=sqlite:////data/main.db
+SQLALCHEMY_DATABASE_URI_ROUNDCUBE=sqlite:////data/roundcube.db

webmails/start.py

@@ -26,7 +26,8 @@ with open("/etc/resolv.conf") as handle:
     resolver = content[content.index("nameserver") + 1]
     context["RESOLVER"] = f"[{resolver}]" if ":" in resolver else resolver
 
-db_flavor = env.get("ROUNDCUBE_DB_FLAVOR", "sqlite")
+db_uri = env.get("SQLALCHEMY_DATABASE_URI_ROUNDCUBE", "sqlite:////data/roundcube.db")
+db_flavor = env.get("ROUNDCUBE_DB_FLAVOR")
 if db_flavor == "sqlite":
     context["DB_DSNW"] = "sqlite:////data/roundcube.db"
 elif db_flavor == "mysql":
@@ -43,6 +44,8 @@ elif db_flavor == "postgresql":
         env.get("ROUNDCUBE_DB_HOST", "database"),
         env.get("ROUNDCUBE_DB_NAME", "roundcube")
     )
+elif db_uri:
+    context["DB_DSNW"] = db_uri
 else:
     print(f"Unknown ROUNDCUBE_DB_FLAVOR: {db_flavor}", file=sys.stderr)
     exit(1)