kallithea 0.6.0 under python3: alembic configparser crashes when smtp_password contains % symbols

Issue #375 resolved
Maximilian Kerst created an issue

As the title suggests, running alembic -c my.ini current on Python 3.7.3 fails with a configparser.InterpolationSyntaxError if it encounters a % sign:

(kallithea-venv) xxx@xxx:/usb/kallithea/kallithea $ alembic -c my.ini current
Traceback (most recent call last):
  File "/usb/kallithea/kallithea-venv/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 577, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 571, in main
    self.run_cmd(cfg, options)
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 551, in run_cmd
    **dict((k, getattr(options, k, None)) for k in kwarg)
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/command.py", line 515, in current
    script.run_env()
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/script/base.py", line 489, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
    module = load_module_py(module_id, path)
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/util/compat.py", line 184, in load_module_py
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/kallithea/alembic/env.py", line 108, in <module>
    run_migrations_online()
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/kallithea/alembic/env.py", line 83, in run_migrations_online
    cfg = config.get_section(config.config_ini_section)
  File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/alembic/config.py", line 225, in get_section
    return dict(self.file_config.items(name))
  File "/usr/lib/python3.7/configparser.py", line 857, in items
    return [(option, value_getter(option)) for option in d.keys()]
  File "/usr/lib/python3.7/configparser.py", line 857, in <listcomp>
    return [(option, value_getter(option)) for option in d.keys()]
  File "/usr/lib/python3.7/configparser.py", line 854, in <lambda>
    section, option, d[option], d)
  File "/usr/lib/python3.7/configparser.py", line 394, in before_get
    self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "/usr/lib/python3.7/configparser.py", line 444, in _interpolate_some
    "found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%&CENSORED'

The affected config was smtp_password, which is why i unfortunately cannot provide the complete field.

Fortunately, there is a workaround: the % sign needs to be escaped, e.g. replace ‘%' with '%%’.
After applying the workaround, alembic could successfully upgrade my database.

The bug was found while upgrading from 0.5.2.

System information:
Host: Raspberry Pi 4 (armv71)
OS: Raspbian 10, Kernel 4.19.97
Python: 3.7.3
Kallithea installed via Python venv and pip.

Please excuse my unprofessional report here, for I am still a student and not a professional.

Comments (3)

  1. Mads Kiilerich

    Thanks for the fine report. Don’t think less of being a student.

    I assume you get same failure with gearbox serve? In which case the stack trace shows that it happens in gearbox / paste / configparser, before hitting any Kallithea code.

    True, % is special, as hinted by how %(here)s is special. https://docs.python.org/3/library/configparser.html hints towards that with a single %% example.

    Would it help if the .ini header said something like

    ## The %(here)s variable will generally be replaced with the parent directory of ##
    ## this file. Other use of % must be written as %% .                             ##
    

    ?

  2. Maximilian Kerst reporter

    Yes, your assumption is right - without escaping %, gearbox also crashes and the stacktrace is similar:

    (kallithea-venv) xxx@xxx:/usb/kallithea $ gearbox serve -c kallithea/my.ini
    15:05:15,842 ERROR [gearbox] Failed to load server
    15:05:15,843 ERROR [gearbox] Error in file /usb/kallithea/kallithea/my.ini: '%' must be followed by '%' or '(', found: '%&CENSORED'
    Traceback (most recent call last):
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/main.py", line 172, in _run_subcommand
        return cmd.run(parsed_args)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/command.py", line 31, in run
        self.take_action(parsed_args)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/commands/serve.py", line 269, in take_action
        relative_to=base, global_conf=parsed_vars)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/gearbox/commands/serve.py", line 308, in loadserver
        server_spec, name=name, relative_to=relative_to, **kw)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 261, in loadserver
        return loadobj(SERVER, uri, name=name, **kw)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 277, in loadobj
        global_conf=global_conf)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 302, in loadcontext
        global_conf=global_conf)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 326, in _loadconfig
        return loader.get_context(object_type, name, global_conf)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 415, in get_context
        defaults = self.parser.defaults()
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 74, in defaults
        defaults[key] = self.get('DEFAULT', key) or val
      File "/usr/lib/python3.7/configparser.py", line 799, in get
        d)
      File "/usb/kallithea/kallithea-venv/lib/python3.7/site-packages/paste/deploy/loadwsgi.py", line 101, in before_get
        value, defaults)
      File "/usr/lib/python3.7/configparser.py", line 394, in before_get
        self._interpolate_some(parser, option, L, value, section, defaults, 1)
      File "/usr/lib/python3.7/configparser.py", line 444, in _interpolate_some
        "found: %r" % (rest,))
    configparser.InterpolationSyntaxError: Error in file /usb/kallithea/kallithea/my.ini: '%' must be followed by '%' or '(', found: '%&CENSORED'
    

    I agree with documenting it in the ini header since it is not Kallithea’s code. I also thought about a note in the documentation.

  3. Log in to comment