2596: db.String without length cause an error in migration for MySQL DB r=mergify[bot] a=csthiang

## What type of PR?

bug-fix

## What does this PR do?

For MySQL, `db.String` requires a length because db.String gets translated to `VARCHAR` in MySQL and `VARCHAR` requires a length. I was considering adding a length to it but since the affected fields were used to store CommaSeparatedList and json-encoded string, I have a feeling it can be quite large in the future. `db.Text` seems to fit into this use case but please correct me if I am wrong.

This actually affects a DB migration with the following error:

```
  File "/app/venv/bin/flask", line 8, in <module>
    sys.exit(main())
  File "/app/venv/lib/python3.10/site-packages/flask/cli.py", line 1047, in main
    cli.main()
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/cli.py", line 149, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/__init__.py", line 98, in wrapped
    f(*args, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/flask_migrate/__init__.py", line 185, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/venv/lib/python3.10/site-packages/alembic/command.py", line 322, in upgrade
    script.run_env()
  File "/app/venv/lib/python3.10/site-packages/alembic/script/base.py", line 569, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/app/venv/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/venv/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/migrations/env.py", line 99, in <module>
    run_migrations_online()
  File "/app/migrations/env.py", line 92, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/venv/lib/python3.10/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
    step.migration_fn(**kw)
  File "/app/migrations/versions/f4f0f89e0047_.py", line 18, in upgrade
    with op.batch_alter_table('fetch') as batch:
  File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
    next(self.gen)
  File "/app/venv/lib/python3.10/site-packages/alembic/operations/base.py", line 381, in batch_alter_table
    impl.flush()
  File "/app/venv/lib/python3.10/site-packages/alembic/operations/batch.py", line 111, in flush
    fn(*arg, **kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 322, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/impl.py", line 195, in _exec
    return conn.execute(construct, multiparams)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
    return connection._execute_ddl(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1469, in _execute_ddl
    compiled = ddl.compile(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py", line 502, in compile
    return self._compiler(dialect, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py", line 32, in _compiler
    return dialect.ddl_compiler(dialect, self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 463, in __init__
    self.string = self.process(self.statement, **compile_kwargs)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 498, in process
    return obj._compiler_dispatch(self, **kwargs)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/ext/compiler.py", line 548, in <lambda>
    lambda *arg, **kw: existing(*arg, **kw),
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/ext/compiler.py", line 604, in __call__
    expr = fn(element, compiler, **kw)
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/base.py", line 190, in visit_add_column
    add_column(compiler, element.column, **kw),
  File "/app/venv/lib/python3.10/site-packages/alembic/ddl/base.py", line 330, in add_column
    text = "ADD COLUMN %s" % compiler.get_column_specification(column, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/base.py", line 1714, in get_column_specification
    self.dialect.type_compiler.process(
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 532, in process
    return type_._compiler_dispatch(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 82, in _compiler_dispatch
    return meth(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5028, in visit_type_decorator
    return self.process(type_.type_engine(self.dialect), **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 532, in process
    return type_._compiler_dispatch(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/visitors.py", line 82, in _compiler_dispatch
    return meth(self, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/sql/compiler.py", line 5006, in visit_string
    return self.visit_VARCHAR(type_, **kw)
  File "/app/venv/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/base.py", line 2214, in visit_VARCHAR
    raise exc.CompileError(
sqlalchemy.exc.CompileError: VARCHAR requires a length on dialect mysql
[2022-12-22 09:23:12 +0000] [17] [INFO] Starting gunicorn 20.1.0
[2022-12-22 09:23:12 +0000] [17] [INFO] Listening at: http://0.0.0.0:80 (17)
[2022-12-22 09:23:12 +0000] [17] [INFO] Using worker: gthread
[2022-12-22 09:23:12 +0000] [18] [INFO] Booting worker with pid: 18
```

### Related issue(s)
none

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [x] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Johnson Thiang <jthiang@pop-os.localdomain>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
main
bors[bot] 2 years ago committed by GitHub
commit 52c17411bd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -75,7 +75,7 @@ class CommaSeparatedList(db.TypeDecorator):
""" Stores a list as a comma-separated string, compatible with Postfix.
"""
impl = db.String
impl = db.String(255)
cache_ok = True
python_type = list
@ -96,7 +96,7 @@ class JSONEncoded(db.TypeDecorator):
""" Represents an immutable structure as a json-encoded string.
"""
impl = db.String
impl = db.String(255)
cache_ok = True
python_type = str

Loading…
Cancel
Save