Merge #2566

2566: Make it clear that we don't delete users r=mergify[bot] a=nextgens

## What type of PR?

bug-fix

## What does this PR do?

Make it clear that we don't delete users. Users can and should be disabled when not in use anymore.

### Related issue(s)
- closes #1820

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [ ] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>
Co-authored-by: Alexander Graf <ghostwheel42@users.noreply.github.com>
Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>

core/admin/mailu/manage.py

@@ -304,6 +304,7 @@ def config_update(verbose=False, delete_objects=False):
                 if verbose:
                     print(f'Deleting domain: {domain.name}')
                 db.session.delete(domain)
+
     db.session.commit()
 
 
@@ -351,7 +352,7 @@ def config_import(verbose=0, secrets=False, debug=False, quiet=False, color=Fals
             raise click.ClickException(msg) from exc
         raise
 
-    # don't commit when running dry
+    # do not commit when running dry
     if dry_run:
         log.changes('Dry run. Not committing changes.')
         db.session.rollback()
@@ -403,13 +404,16 @@ def config_export(full=False, secrets=False, color=False, dns=False, output=None
 
 @mailu.command()
 @click.argument('email')
+@click.option('-r', '--really', is_flag=True)
 @with_appcontext
-def user_delete(email):
-    """delete user"""
-    user = models.User.query.get(email)
-    if user:
-        db.session.delete(user)
-    db.session.commit()
+def user_delete(email, really=False):
+    """disable or delete user"""
+    if user := models.User.query.get(email):
+        if really:
+            db.session.delete(user)
+        else:
+            user.enabled = False
+        db.session.commit()
 
 
 @mailu.command()
@@ -417,10 +421,9 @@ def user_delete(email):
 @with_appcontext
 def alias_delete(email):
     """delete alias"""
-    alias = models.Alias.query.get(email)
-    if alias:
+    if alias := models.Alias.query.get(email):
         db.session.delete(alias)
-    db.session.commit()
+        db.session.commit()
 
 
 @mailu.command()

core/admin/mailu/ui/templates/user/list.html

@@ -31,7 +31,6 @@
   <tr{% if not user.enabled %} class="warning"{% endif %}>
     <td>
       <a href="{{ url_for('.user_edit', user_email=user.email) }}" title="{% trans %}Edit{% endtrans %}"><i class="fas fa-pencil-alt"></i></a>&nbsp;
-      <a href="{{ url_for('.user_delete', user_email=user.email) }}" title="{% trans %}Delete{% endtrans %}"><i class="fa fa-trash"></i></a>
     </td>
     <td>
       <a href="{{ url_for('.user_settings', user_email=user.email) }}" title="{% trans %}Settings{% endtrans %}"><i class="fa fa-wrench"></i></a>&nbsp;

core/admin/mailu/ui/views/users.py

@@ -80,19 +80,6 @@ def user_edit(user_email):
         domain=user.domain, max_quota_bytes=max_quota_bytes)
 
 
-@ui.route('/user/delete/<path:user_email>', methods=['GET', 'POST'])
-@access.domain_admin(models.User, 'user_email')
-@access.confirmation_required("delete {user_email}")
-def user_delete(user_email):
-    user = models.User.query.get(user_email) or flask.abort(404)
-    domain = user.domain
-    models.db.session.delete(user)
-    models.db.session.commit()
-    flask.flash('User %s deleted' % user)
-    return flask.redirect(
-        flask.url_for('.user_list', domain_name=domain.name))
-
-
 @ui.route('/user/settings', methods=['GET', 'POST'], defaults={'user_email': None})
 @ui.route('/user/usersettings/<path:user_email>', methods=['GET', 'POST'])
 @access.owner(models.User, 'user_email')

docs/cli.rst

@@ -63,13 +63,19 @@ primary difference with simple `user` command is that password is being imported
 
   docker compose run --rm admin flask mailu user-import myuser example.net '$6$51ebe0cb9f1dab48effa2a0ad8660cb489b445936b9ffd812a0b8f46bca66dd549fea530ce' 'SHA512-CRYPT'
 
+
 user-delete
 -----------
 
+Although the action is called "user-delete" the user is only deactivated by default.
+This is due to the fact mailu does not remove user-data (emails and webmail contacts) when a user is deleted.
+Add the flag `-r` to really delete the user after you have deleted user-data manually.
+
 .. code-block:: bash
 
   docker compose exec admin flask mailu user-delete foo@example.net
 
+
 config-update
 -------------
 

docs/faq.rst

@@ -393,6 +393,46 @@ Technical issues
 In this section we are trying to cover the most common problems our users are having.
 If your issue is not listed here, please consult issues with the `troubleshooting tag`_.
 
+.. _delete_users:
+
+How to delete users?
+````````````````````
+
+From the web administration interface, when a user is deleted, the user is only disabled. When a user is not enabled, this user:
+
+* cannot send/receive email
+* cannot access Mailu (admin/webmail)
+* cannot access the email box via pop3/imap
+
+It is not possible to delete users via the Mailu web administration interface. The main reason is to prevent email address reuse. If a user was deleted, it can be recreated and used by someone else. It is not clear that the email address has been used by someone else previously. This new user might receive emails which were meant for the previous user. Disabling the user, prevents the email address to be reused by mistake.
+
+Another reason is that extra post-deletion steps are required after a user has been deleted from the Mailu database. Those additional steps are:
+
+* Delete the dovecot mailbox. If this does not happen, a new user with the same email address reuses the previous user's mailbox.
+* Delete the user from the roundcube database (not required when SnappyMail is used). If this does not happen, a new user with the same email address reuses the previous roundcube data (such as address lists, gpg keys etc).
+
+For safely deleting the user data (and possible the user as well) a script has been introduced. The scripts provides the following information
+
+* commands for deleting mailboxes of unknown users. These users were deleted from Mailu, but still have their mailbox data on the file system.
+* commands for deleting mailboxes and roundcube data for disabled users.
+* commands for deleting users from the Mailu database.
+
+Proceed as following for deleting an user:
+
+1. Disable the to-be-deleted user. This can be done via the Web Administration interface (/admin), the Mailu CLI command user-delete, or the RESTful API. Do **not** delete the user.
+2. Download .\\scripts\\purge_user.sh from the `github project`_. Or clone the Mailu github project.
+3. Copy the script purge_user.sh to the Mailu folder that contains the `docker-compose.yml` file.
+4. Run as root: purge_user.sh
+5. The script will output the commands that can be used for fully purging each disabled user. It will show the instruction for deleting the user from the
+
+   * Dovecot maildir from filesystem (all email data)
+   * Roundcube database (all data saved in roundcube)
+   * Mailu database.
+
+6. Run the commands for deleting all user data for each disabled user.
+
+.. _`github project`: https://github.com/Mailu/Mailu/
+
 Changes in .env don't propagate
 ```````````````````````````````
 
@@ -545,6 +585,7 @@ Below an example how to do so.
 
 If you use a reverse proxy in front of Mailu, it is vital to set the environment variables REAL_IP_HEADER and REAL_IP_FROM.
 Without these environment variables, Mailu will not trust the remote client IP passed on by the reverse proxy and as a result your reverse proxy will be banned.
+
 See the :ref:`configuration reference <reverse_proxy_headers>` for more information.
 
 

docs/webadministration.rst

@@ -314,9 +314,9 @@ This page is also accessible for domain managers. On the users page new users ca
 
 * Edit. For all available options see :ref:`the Add user page <webadministration_add_user>`.
 
-* Delete. Deletes the user. The Admin GUI will ask for confirmation if the user must be really deleted.
+* Delete. Disables the user. For more information on permanently deleting users, refer to the :ref:`How to delete users page<delete_users>`.
 
-* Setting. Access the settings page of the user. See :ref:`the settings page <webadministration_settings>` for more information.
+* Settings. Access the settings page of the user. See :ref:`the settings page <webadministration_settings>` for more information.
 
 * Auto-reply. Access the auto-reply page of the user. See the :ref:`auto-reply page <webadministration_auto-reply>` for more information.
 

scripts/purge_user.sh

@@ -0,0 +1,85 @@
+#!/bin/bash
+
+# get id of running admin container
+admin="$(docker compose ps admin --format=json | jq -r '.[].ID')"
+if [[ -z "${admin}" ]]; then
+	echo "Sorry, can't find running mailu admin container."
+	echo "You need to start this in the path containing your docker-compose.yml."
+	exit 1
+fi
+
+# get storage path
+storage="$(
+	docker inspect "${admin}" \
+	| jq -r '.[].Mounts[] | select(.Destination == "/data") | .Source'
+)/.."
+storage="$(realpath "${storage}")"
+if [[ ! -d "${storage}" ]]; then
+	echo "Sorry, can't find mailu storage path."
+	exit 2
+fi
+
+# fetch list of users from admin
+declare -A users=()
+while read line; do
+	users[${line#* }]="${line/ *}"
+done < <(
+	docker compose exec -T admin \
+		flask mailu config-export -j user.email user.enabled \
+	2>/dev/null | jq -r '.user[] | "\(.enabled) \(.email)"'
+)
+if [[ ${#users[@]} -eq 0 ]]; then
+	echo "mailu config-export returned no users. Aborted."
+	exit 3
+fi
+
+# diff list of users <> storage
+unknown=false
+disabled=false
+for maildir in "${storage}"/mail/*; do
+	[[ -d "${maildir}" ]] || continue
+	email="${maildir/*\/}"
+	enabled="${users[${email}]:-}"
+	if [[ -z "${enabled}" ]]; then
+		unknown=true
+		users[${email}]="unknown"
+	elif ${enabled}; then
+		unset users[${email}]
+	else
+		disabled=true
+		users[${email}]="disabled"
+	fi
+done
+
+if [[ ${#users[@]} -eq 0 ]]; then
+	echo "Nothing to clean up."
+	exit 0
+fi
+
+# is roundcube webmail in use?
+webmail=false
+docker compose exec webmail test -e /data/roundcube.db 2>/dev/null && webmail=true
+
+# output actions
+if ${unknown}; then
+	echo "# To delete maildirs unknown to mailu, run:"
+	for email in "${!users[@]}"; do
+		[[ "${users[${email}]}" == "unknown" ]] || continue
+		echo -n "rm -rf '${storage}/mail/${email}'"
+		${webmail} && \
+			echo -n " && docker compose exec -T webmail su mailu -c \"/var/www/roundcube/bin/deluser.sh --host=front '${email}'\""
+		echo
+	done
+	echo
+fi
+if ${disabled}; then
+	echo "# To purge disabled users, run:"
+	for email in "${!users[@]}"; do
+		[[ "${users[${email}]}" == "disabled" ]] || continue
+		echo -n "docker compose exec -T admin flask mailu user-delete -r '${email}' && rm -rf '${storage}/mail/${email}'"
+		${webmail} && \
+			echo -n " && docker compose exec -T webmail su mailu -c \"/var/www/roundcube/bin/deluser.sh --host=front '${email}'\""
+		echo
+	done
+	echo
+fi

towncrier/newsfragments/2566.misc

@@ -0,0 +1,2 @@
+Remove the ability to delete users via the webui; Disable them instead.
+For more information on deleting users see the entry "How to delete users" in the FAQ.