# Issues

Issue #1434 resolved

# provide non-minified options for jquery.js, underscore.js, all others

Mike Bayer
created an issue

Projects that wish to include pre-build Sphinx HTML documentation in their source distributions (e.g. SQLAlchemy, Mako, Alembic) are prevented from being packaged in Debian (and possibly Fedora and Redhat) packages, due to the presence of minified .js files. The .js files that are present in Sphinx templates/basic appear to be only the minified versions with no option to install the full source distributions. Debian package maintainers need to untar my source distros, remove the Sphinx documentation, and re-zip the files, and i regularly get pinged from them for me to fix this issue, when IMHO this is an upstream issue.

I could of course rework all of my Sphinx builds to include non-minified versions of these, but then all of my projects would need to track every single .js file that Sphinx uses, and additionally would break again each time Sphinx adds a new .js file. It would be much preferable if Sphinx could provide this feature.

I propose that Sphinx include both standard and minified versions of all .js files which are packaged in its base templates, and provide an option in conf.py which allows selection of "non-minified" .js (and any other minified) files.

1. I think the option to use the non-minified js files is a good idea. I think it would be accomplished in several ways.

A. provide non-minified JS option as a part of html_theme_options. For example, minifiedjs=False option will use non-minified jquery.js and underscore.js files.

B. provide jqueryjs_path and underscorejs_path options as a part of html_theme_options. I think the behavior of the options should fit to the mathjax_path behavior, however we need to keep a compatibility then default behavior is needed to use minified version JS file for the html build. In this case, if user want to use non-minified js, it is necessary to be specified a non-minified js file path (or URL) by user.

C. based upon the B idea. jqueryjs_path (and underscorejs_path) accept 'NONMINIFIED' value that use non-minified JS file.

D. based upon the B idea and drop backward compatibility. The behavior of jqueryjs_path fully synced to the mathjax_path. Sphinx doesn't have jquery.js anymore.

In each case, I think non-selected version of JS file shouldn't be included the build result. If we include both minified/non-minified version JS files in the themes/basic/static directory, they will be copied to the build result.

I think C is the best because it has a backward-compatibility and flexibility and it doesn't require a effort of the user.

Mike Bayer does it come true to your purpose? Georg Brandl how do you think?

2. reporter

I want to be able to say, "minifiedjs=False" in conf.py just once, not having to worry about specific .js files (as sphinx can add new ones any time and I don't want to know about them), and nothing will be minified.

Mike, you are right that minified js is not accepted in Debian in general, but neither are duplicated libraries. I think in case of sphinx all libs should be replaced with symlinks to libs from libjs-sphinxdoc. They have a tool for it: http://manpages.ubuntu.com/manpages/trusty/man1/dh_sphinxdoc.1.html

(I'm not debian developer, but I also happen to maintain a program that uses sphinx and is packaged in Debian)

Marcin, we do not use these JS files in Debian indeed, but we also do not mind shipping them in the tarball (as long as the source is provided and the licence allows distributing them)

3. reporter

Marcin Wojdyr - that would also be impossible as I am shipping a platform-agnostic source .tar.gz file. it again points to the notion that source distributions shipped for Debian would be prevented from including readable documentation that relies upon HTML tools.

Piotr: I'm not sure I follow. Is it a problem for Debian if an upstream tarball includes minified JS in Sphinx-generated docs?

Marcin: minified JS files without original files is a problem, yes. I was referring to your symlink comment (original files only or original files + minified files are OK)

I see that lintian marks searchindex.js, jquery and underscore.js in all packages that use Sphinx as source-is-missing errors: http://lintian.debian.org/tags/source-is-missing.html but to me these are false positive that needs to be fixed in lintian. I guess lintian reports that source is missing when there is no whitespace in js.

The source of documentation are conf.py, RST files, etc and if these are present in the tarball the source is not missing. All the rest can be regenerated using Sphinx. (OTOH if JS files would have white spaces but rst files would not be included - then the source would be missing indeed.)

Even if Takayuki implements option to put non-minified jquery and underscore and all projects start using this option, you will still need to make lintian smarter to handle searchindex.js.

4. reporter

Marcin Wojdyr - I wasn't specific enough. I meant that there are readable HTML files present in the dist that you can point at in your browser, no need to install Sphinx or the many extensions I use and do a build. Our less experienced users would have a hard time with requirements like that just to read the documentation (though I'm sure most people just read the docs online anyway).

It's sounding like the folks who package SQLAlchemy as a debian package will just have to get used to picking apart the files into an acceptable format. The original tar.gz is intended for many platforms and skill levels and can't really contain debian-specific idioms on its own. Not to mention that the test suite includes two binary ".dat" files that are used to test binary reading and writing; these are essentially junk parts of some old .pyc files so they contain unreadable code as well (even though they are not used in the capacity of executable code).

Marcin Wojdyr Thanks for the detail. In that case, if sphinx provides both minified and non-minified JS files, I guess the issue might be resolved, isn't it? Actually, implementing to exclude unchosen JS files is bit complex to the current architecture.

Specifically:

1. Sphinx default themes contain minified and non-minified JS files as:

• jqueryjs
• jquery.min.js
• underscore.js
• underscore.min.js
2. Sphinx make html copies above 4 files under _build/html directory.

3. Sphinx make html also generate searchindex.js files as:

• searchindex.js (newly generated, human-readable, space/LF included)
• searchindex.min.js (renamed from current searchindex.js file)
4. html files reference minified or non-minified JS files that is determined by minifiedjs=False flag. (now it became a supplemental feature).

Takayuki: IMO it doesn't make sense to generate 2 searchindex files. It's clearly a false positive in lintian (Debian tool). It seems to be a new feature of lintian, there were no such problems until recently.

I just had a look at lintian source and it seems that it's already fixed in git: https://github.com/Debian/lintian/commit/4da3a270b80e94eceed36fb6b5cc12733ee991b4 IIUC no action is needed from Sphinx, the issue has been already resolved on Debian side (?).

Marcin: did you read debian-devel threads about this issue (the one from 2012 and the one started this year)? If not (and you have some time), please do. If after reading it you still have doubts, please raise them on debian-legal@l.d.o and not here. TIA

so, again (did I mention I hate this stuff? Our users trust that we'll handle it for them hence my comments here): we are fine with minified versions in the tarball and html files pointing to them if original files are somewhere in the tarball as well. We are fine with pregenerated docs if we can rebuild them. We are fine with docs without sources if user wants it this way (we will not distribute such tarballs, though). If upstream authors do not want to provide sources, that's fine, it's more work for us, but we'll fetch them from git or remove documentation completely... but please give those upstreams that want to make our lives easier (and make our users happy) an option to generate tarballs with all the sources.

(and no, I don't care if searchindex.js is human readable or not as long as all the source files to generate it are in the tarball)

• changed status to open

For now, a generated documentation always contain jquery-1.8.3.js and underscore-1.3.1.js. I hope this change will resolve the original issue.

Thank you! Now the life of Debian packagers will be a lot easier :)

My understanding is:

• non-minified searchindex.js is not necessary.
• including non-minified jquery.js and underscore.js is safer for lintian check.
• it is not a problem even if the document references a minified-js or a non-minified-js.

Right. Lintian does complain about searchindex.js, but that is a false positive.