replace pkg_resources with importlib.resources

[Daniel McCloy]

pybtex/plugin/__init__.py imports pkg_resources which is (according to the warning at the top of their docs)

discouraged in favor of importlib.resources, importlib.metadata, and their backports (importlib_resources, importlib_metadata). Please consider using those libraries instead of pkg_resources.

pybtex’s use of pkg_resources is now causing CI failures in a library I develop (we use sphinxcontrib_bibtex, not pybtex directly). Here’s the traceback we’re seeing; key lines are lines 8-9 (sphinx importing its extensions, including sphinxcontrib_bibtex), lines 26-27 (the pybtex import of pkg_resources) and line 40 (the DeprecationWarning that our CIs fail on).

Traceback (most recent call last):
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinx/application.py", line 223, in __init__
    self.setup_extension(extension)
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinx/application.py", line 398, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinx/registry.py", line 459, in load_extension
    mod = import_module(extname)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinxcontrib/bibtex/__init__.py", line 9, in <module>
    from .domain import BibtexDomain
  File "/home/circleci/python_env/lib/python3.10/site-packages/sphinxcontrib/bibtex/domain.py", line 20, in <module>
    import pybtex_docutils
  File "/home/circleci/python_env/lib/python3.10/site-packages/pybtex_docutils/__init__.py", line 34, in <module>
    from pybtex.backends import BaseBackend
  File "/home/circleci/python_env/lib/python3.10/site-packages/pybtex/backends/__init__.py", line 26, in <module>
    from pybtex.plugin import Plugin
  File "/home/circleci/python_env/lib/python3.10/site-packages/pybtex/plugin/__init__.py", line 26, in <module>
    import pkg_resources
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3257, in <module>
    def _initialize_master_working_set():
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3231, in _call_aside
    f(*args, **kwargs)
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3282, in _initialize_master_working_set
    tuple(dist.activate(replace=False) for dist in working_set)
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 3282, in <genexpr>
    tuple(dist.activate(replace=False) for dist in working_set)
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2803, in activate
    declare_namespace(pkg)
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2297, in declare_namespace
    warnings.warn(msg, DeprecationWarning, stacklevel=2)
DeprecationWarning: Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See <https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages>

Exception occurred:
  File "/home/circleci/python_env/lib/python3.10/site-packages/pkg_resources/__init__.py", line 2297, in declare_namespace
    warnings.warn(msg, DeprecationWarning, stacklevel=2)
DeprecationWarning: Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See <https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages>
The full traceback has been saved in /tmp/sphinx-err-6_80471s.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <<https://github.com/sphinx-doc/sphinx/issues>.> Thanks!
make: *** [Makefile:70: html_dev-noplot] Error 2
make: Leaving directory '/home/circleci/project/doc'

Exited with code exit status 2

‌

[Tomasz Kłoczko]

Here is example how to do that https://github.com/mtkennerly/dunamai/commit/4fdbb9c75a2eb309aeeaf4d43c7238a103ac1ee2

[Tomasz Kłoczko]

gentle ping .. any update? 🤔

[Daniel McCloy]

no update from me… I looked at the example you linked to, but the usage of pkg_resources is much simpler in that example than its usage in this package. I tried for 20-30 minutes but couldn’t sort out if I was doing it correctly. You can see my work-in-progress here, if you want to try to finish it:

https://bitbucket.org/drammock/pybtex/commits/7ced817b3e84e384211250b4b94267ec66438699

I’m not going to have time to chase this down myself

[Tomasz Kłoczko]

looks good ..

[Alan]

On Python 3.12, installing pybtex with pip appears to succeed, but the using pybtex produces the error:
`ModuleNotFoundError: No module named 'pkg_resources',

It seems that setuptools is a missing dependency. After installing setuptools, pybtex appears to be working.

[Stanislav Pankevich]

I can confirm what @Alan says. Installing pybtex with Python 3.12: ``ModuleNotFoundError: No module named 'pkg_resources'`.

Installing setuptools makes the import error go away.

[Philipp A.]

Is this package still maintained? This issue means that pybtex breaks when it’s used in an environment that doesn‘t have an undeclared dependency installed, so it should be very high priority, yet I haven’t seen a maintainer comment in here.

[Alan]

It sounds to me like the developers currently hesitate to replace pkg_resources with importlib.resources, which is the right solution.

In the meantime, they could make setuptools an explicit dependency. Maybe you could open a separate issue for that …

[Maximilian Linhoff]

@drammock I think a much easier and less hacky solution than mutating the entry point dict would be to just have a dict for the runtime registered plugins and check that when plugins are loaded

[Daniel McCloy]

@maxnoe feel free to open a PR with your suggested fix. I'm not a maintainer here, and as I said above:

I’m not going to have time to chase this down myself

[Maximilian Linhoff]

@drammock I did, and it was merged immediately (faster than I was comfortable with to be honest… I expected some amount of discussion / review )

‌

https://bitbucket.org/pybtex-devs/pybtex/pull-requests/46

[Daniel McCloy]

So I guess this can be closed then?

[Philipp A.]

Would be nice to get a release with the fix, but yes.