Percent sign in URL field raises an exception, escaping it produces invalid output

[Former user]

I have several entries in a bibliography file where the URL field contains encoded characters. For example:

@​Article{Anders99, ... Doi = {10.1175/1520-0493(1999)127<2741:amciot>2.0.co;2}, Url = {http://dx.doi.org/10.1175/1520-0493(1999)127<;2741%3AAMCIOT>2.0.CO%3B2} }

When I attempt to build a document with Sphinx, I get the following output:

parsing bibtex file PROJECT/docs/refs.bib... Exception occurred: File "build/bdist.linux-x86_64/egg/pybtex/errors.py", line 78, in report_error raise exception TokenRequired: '}' expected

If I escape the percent characters with a leading slash, the bib file is parsed but the URL includes the slashes and so the link is broken.

In addition, the DOI field includes characters that need to be escaped (':' and ';'). In the link that Sphinx produces, these characters are correctly escaped as '%3A' and '%3B', but the percent signs are again escaped with leading slashes and so the DOI URL is also broken. I am not sure whether this is also a bug in pybtex, or if the problem is elsewhere in my toolchain (sphinx or sphinxcontrib.bibtex).

[Former user]

Sorry, ignore the final paragraph of my original report, the DOI URL is fine.

[Andrey Golovizin]

Can't reproduce the error.

I've copied your snipped to test.bib:

@Article{Anders99,
    Doi = {10.1175/1520-0493(1999)127<2741:amciot>2.0.co;2},
    Url = {http://dx.doi.org/10.1175/1520-0493(1999)127<2741%3AAMCIOT>2.0.CO%3B2}
}

Then run pybtex-convert test.bib test.yaml without getting any syntax errors:

entries:
    anders99:
        doi: 10.1175/1520-0493(1999)127<2741:amciot>2.0.co;2
        type: article
        url: http://dx.doi.org/10.1175/1520-0493(1999)127<2741%3AAMCIOT>2.0.CO%3B2

Could you attach an archive with a complete example that triggers the error?

[Rob Moss]

I've put together a minimal working example. There are two bibliographies, one with escaped '%' ("test-good.bib", which compiles but produces an invalid link) and one with unescaped '%' ("test-bad.bib", which fails to compile).

If I add

:encoding: utf8

to the bibliography directive, sphinx builds the documentation correctly but special characters (e.g., "\'e", "\cc") are displayed verbatim.

[Andrey Golovizin]

This works for me. :)

.. bibliography:: test-bad.bib
    :encoding: utf-8
    :style: unsrt

If this does not help, please report a bug in sphinxcontrib-bibtex. This is not really a Pybtex issue.