It was recently brought up in a package review (https://bugzilla.redhat.com/show_bug.cgi?id=2006555) that a Sphinx-generated HTML documentation package contained several JavaScript libraries, and that:
1. their licenses should be accounted for and added to the License field for the -doc subpackage 2. they should be treated as bundled libraries and marked with virtual Provides
Note that the JavaScript is inserted by the Sphinx documentation build system, and is not present in the source tarball. Your “typical” Sphinx documentation has minified and unminified copies of JQuery (js-jquery) and Underscore (js-underscore), both MIT-licensed, plus several unminified utility libraries that belong to Sphinx itself (doctools.js, language_data.js, searchtools.js). The overall Sphinx project is BSD-licensed. The details vary by theme, but this is the most common case.
To me, it seems that this feedback is a reasonable strict reading of the relevant packaging guidelines. If this is the right interpretation of Fedora policy, what should be done about it in general?
-----
Based on looking for installed files named “searchtools.js” in packages named -doc or -docs, there are about 600 documentation packages generated with Sphinx. This might miss some that have exotic themes.
As far as I can tell, only one existing package in the distribution, python-murano-package-check, tries to use virtual Provides to track the libraries that belong to Sphinx itself:
Provides: bundled(js-doctools) Provides: bundled(js-jquery) = 3.1.0 Provides: bundled(js-searchtools) Provides: bundled(js-underscore) = 1.3.1 Provides: bundled(js-websupport)
and this package does not actually build its documentation subpackage.
Only the following packages have virtual Provides for js-jquery and js-underscore. In most cases these lack the version numbers.
- arb-doc - mpdecimal-doc - python-BTrees-doc - python-latexcodec-doc - python-networkx-doc - python3-persistent-doc - sympy-doc
Of the packages in the list above, only mpdecimal-doc has a License field that seems to try to account for the licenses of the JavaScript libraries.
-----
Personally, I can add virtual Provides and adjust the License on the -doc subpackage for my package under review, and it’s even possible for me to replace the Underscore and JQuery libraries with symbolic links to those installed by js-jquery and js-underscore. However, changing one package, or even every package I control, is a tiny drop in a vast ocean of documentation packages. Consider, for example, that Doxygen also includes JQuery and several Doxygen-specific libraries in its output, so the same arguments about licenses and virtual Provides apply to it. A search for “dynsections.js” in packages named *-doc or *-docs gives nearly 400 Doxygen-based packages.
If this is a real problem, it seems like it needs to be handled and/or documented in a way that can potentially scale to the rest of those 600 Sphinx-based packages, to the nearly 400 Doxygen-based packages, and beyond that to the output of other documentation systems. I am not sure what that would look like.
All thoughts are welcome.
– Ben Beasley (FAS: music)
On 24. 09. 21 15:21, Ben Beasley wrote:
It was recently brought up in a package review (https://bugzilla.redhat.com/show_bug.cgi?id=2006555) that a Sphinx-generated HTML documentation package contained several JavaScript libraries, and that:
1. their licenses should be accounted for and added to the License field for the -doc subpackage 2. they should be treated as bundled libraries and marked with virtual Provides
Note that the JavaScript is inserted by the Sphinx documentation build system, and is not present in the source tarball. Your “typical” Sphinx documentation has minified and unminified copies of JQuery (js-jquery) and Underscore (js-underscore), both MIT-licensed, plus several unminified utility libraries that belong to Sphinx itself (doctools.js, language_data.js, searchtools.js). The overall Sphinx project is BSD-licensed. The details vary by theme, but this is the most common case.
To me, it seems that this feedback is a reasonable strict reading of the relevant packaging guidelines. If this is the right interpretation of Fedora policy, what should be done about it in general?
Based on looking for installed files named “searchtools.js” in packages named -doc or -docs, there are about 600 documentation packages generated with Sphinx. This might miss some that have exotic themes.
As far as I can tell, only one existing package in the distribution, python-murano-package-check, tries to use virtual Provides to track the libraries that belong to Sphinx itself:
Provides: bundled(js-doctools) Provides: bundled(js-jquery) = 3.1.0 Provides: bundled(js-searchtools) Provides: bundled(js-underscore) = 1.3.1 Provides: bundled(js-websupport)
and this package does not actually build its documentation subpackage.
Only the following packages have virtual Provides for js-jquery and js-underscore. In most cases these lack the version numbers.
- arb-doc - mpdecimal-doc - python-BTrees-doc - python-latexcodec-doc - python-networkx-doc - python3-persistent-doc - sympy-doc
Of the packages in the list above, only mpdecimal-doc has a License field that seems to try to account for the licenses of the JavaScript libraries.
Personally, I can add virtual Provides and adjust the License on the -doc subpackage for my package under review, and it’s even possible for me to replace the Underscore and JQuery libraries with symbolic links to those installed by js-jquery and js-underscore. However, changing one package, or even every package I control, is a tiny drop in a vast ocean of documentation packages. Consider, for example, that Doxygen also includes JQuery and several Doxygen-specific libraries in its output, so the same arguments about licenses and virtual Provides apply to it. A search for “dynsections.js” in packages named *-doc or *-docs gives nearly 400 Doxygen-based packages.
If this is a real problem, it seems like it needs to be handled and/or documented in a way that can potentially scale to the rest of those 600 Sphinx-based packages, to the nearly 400 Doxygen-based packages, and beyond that to the output of other documentation systems. I am not sure what that would look like.
All thoughts are welcome.
If we fix this: https://github.com/rpm-software-management/rpm/issues/1297
We could adapt the sphinx (themes) package(s) to autogenerate the bundled provides. (We would still need to adapt the license manually unless we can autogenerate that one, which sounds like Sci-Fi at this point.)
I’ve used cairomm as a case study for trying to bring a Doxyen-based HTML documentation package into compliance, and python-autodocsumm as a case study for Sphinx.
-----
For cairomm, I have unbundled js-jquery and altered the License for the -doc subpackage. I am not treating the Doxygen-generated JavaScript files as bundled libraries because they are templated and project-dependent, not simply copied in as fully-reusable libraries.
While Doxygen does not insert a standalone license file for its MIT license, the text is at least present in the JavaScript file headers. I might have missed mentioning some non-JavaScript files inserted by Doxygen that are potentially MIT-licensed but do not reference their license.
Another solution would be to ask Doxygen to build a PDF instead of HTML. The result would have no visible bundled assets, but in my opinion would be less convenient for users.
There are some judgement calls here, but I think the result (https://src.fedoraproject.org/rpms/cairomm/c/948e04f946d95612948d81d9090e2bc...) is reasonable.
-----
The situation for python-autodocsumm’s Sphinx documentation is more dire. I can unbundle the Sphinx-inserted copies of js-jquery and js-underscore. I can add virtual Provides for Sphinx’s libraries doctools.js and searchtools.js:
Provides: bundled(js-sphinx-doctools) Provides: bundled(js-sphinx-searchtools)
although I cannot version them, because the version could only be that of Sphinx itself, and that changes regularly without alterations to my package’s sources; I don’t know a way to handle that automatically, and I could not reasonably keep it synchronized manually. These libraries were never intended to be handled separately from the documentation they support.
I can claim that language_data.js and documentation_options.js are not bundled libraries, because they are generated from templates using project-dependent options.
Now I have the problem that all of the Sphinx-inserted files in _static have the same BSD license as Sphinx itself, but they only reference the Sphinx LICENSE file in their headers; they do not contain license text at all, and BSD is one of the licenses where license text is required.
I could possibly handle this by copying in the license file from the Sphinx package in the build environment. I’d better make sure I don’t miss a hypothetical Sphinx license change, so I should probably grep that license file in %check for a key phrase to make sure its contents are what I expect.
Now I’m left with the theme. The autodocsumm documentation uses sphinx_rtd_theme, which is a very common choice. Theme files are mostly in _static/css/ and _static/js/.
In _static/js/, there are two JavaScript files, badge_only.js and theme.js. Both are minified and contain no license information. Okay, I can check the license of python3-sphinx_rtd_theme, and maybe I can copy in the unminified versions manually to comply with that part of the guidelines. Oh, it turns out only the minified versions are packaged, so there’s nothing I can do. (Well—shipping only minified files isn’t compliant with guidelines, so I could send a PR or file an issue on python-sphinx_rtd_theme and expect the maintainer to do something about it. I’m not trying to call out this particular package or its maintainers, as it’s very hard to find and deal with all the bits of JavaScript and web content that are tucked away in Python packages these days, and such oversights are rampant in existing packages.)
In _static/css/, there are two corresponding CSS files, badge_only.css and theme.css. Both have the same problems (minified only, no license information), but *also*, theme.css has a bundled copy of the CSS from Font Awesome 4.7.0. That’s also MIT-licensed, with only a URL for the license and no included license text.
-----
This is the point at which I say that the burden of trying to package this near-minimal example of Sphinx-generated documentation in strict compliance with the guidelines is unreasonable, and I drop it, along with the documentation in all other Python packages for which I am responsible.
I would be surprised if there is even one case where HTML documentation for a Python package is currently packaged in a way that truly satisfies all of these requirements around license text, web assets, JavaScript, and bundling; and I suspect the broader situation for Doxygen and other documentation generators is not much better.
As a user who has often needed to work offline for various reasons, it would be a great loss if the solution really has to be that HTML documentation should not be packaged in general.
On 9/24/21 09:21, Ben Beasley wrote:
It was recently brought up in a package review (https://bugzilla.redhat.com/show_bug.cgi?id=2006555) that a Sphinx-generated HTML documentation package contained several JavaScript libraries, and that:
1. their licenses should be accounted for and added to the License field for the -doc subpackage 2. they should be treated as bundled libraries and marked with virtual Provides
Note that the JavaScript is inserted by the Sphinx documentation build system, and is not present in the source tarball. Your “typical” Sphinx documentation has minified and unminified copies of JQuery (js-jquery) and Underscore (js-underscore), both MIT-licensed, plus several unminified utility libraries that belong to Sphinx itself (doctools.js, language_data.js, searchtools.js). The overall Sphinx project is BSD-licensed. The details vary by theme, but this is the most common case.
To me, it seems that this feedback is a reasonable strict reading of the relevant packaging guidelines. If this is the right interpretation of Fedora policy, what should be done about it in general?
Based on looking for installed files named “searchtools.js” in packages named -doc or -docs, there are about 600 documentation packages generated with Sphinx. This might miss some that have exotic themes.
As far as I can tell, only one existing package in the distribution, python-murano-package-check, tries to use virtual Provides to track the libraries that belong to Sphinx itself:
Provides: bundled(js-doctools) Provides: bundled(js-jquery) = 3.1.0 Provides: bundled(js-searchtools) Provides: bundled(js-underscore) = 1.3.1 Provides: bundled(js-websupport)
and this package does not actually build its documentation subpackage.
Only the following packages have virtual Provides for js-jquery and js-underscore. In most cases these lack the version numbers.
- arb-doc - mpdecimal-doc - python-BTrees-doc - python-latexcodec-doc - python-networkx-doc - python3-persistent-doc - sympy-doc
Of the packages in the list above, only mpdecimal-doc has a License field that seems to try to account for the licenses of the JavaScript libraries.
Personally, I can add virtual Provides and adjust the License on the -doc subpackage for my package under review, and it’s even possible for me to replace the Underscore and JQuery libraries with symbolic links to those installed by js-jquery and js-underscore. However, changing one package, or even every package I control, is a tiny drop in a vast ocean of documentation packages. Consider, for example, that Doxygen also includes JQuery and several Doxygen-specific libraries in its output, so the same arguments about licenses and virtual Provides apply to it. A search for “dynsections.js” in packages named *-doc or *-docs gives nearly 400 Doxygen-based packages.
If this is a real problem, it seems like it needs to be handled and/or documented in a way that can potentially scale to the rest of those 600 Sphinx-based packages, to the nearly 400 Doxygen-based packages, and beyond that to the output of other documentation systems. I am not sure what that would look like.
All thoughts are welcome.
– Ben Beasley (FAS: music) _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
One last follow-up:
It turns out that even the situation with Doxygen is worse. The file jquery.js inserted by Doxygen was not just a copy of JQuery (js-jquery) to be unbundled; it also contained in the same file:
- JQuery UI (js-jquery-ui) - jquery.ScrollTo (https://github.com/flesler/jquery.scrollTo/) - PowerTip (https://stevenbenner.github.io/jquery-powertip/) - JQuery UI Touch Punch (js-jquery-ui-touch-punch) - SmartMenus jQuery Plugin (http://www.smartmenus.org)
It’s possible to account for all of these, package the missing ones separately, and reconstruct the bundle using unminified sources (or patch the generated HTML to use separate script tags, yuck), but the burden is getting ridiculous again. It looks like dropping all Doxygen-based HTML documentation, or building a less-useful PDF instead, will be the sensible (but disappointing) outcome.
On 9/26/21 10:57, Ben Beasley wrote:
I’ve used cairomm as a case study for trying to bring a Doxyen-based HTML documentation package into compliance, and python-autodocsumm as a case study for Sphinx.
For cairomm, I have unbundled js-jquery and altered the License for the -doc subpackage. I am not treating the Doxygen-generated JavaScript files as bundled libraries because they are templated and project-dependent, not simply copied in as fully-reusable libraries.
While Doxygen does not insert a standalone license file for its MIT license, the text is at least present in the JavaScript file headers. I might have missed mentioning some non-JavaScript files inserted by Doxygen that are potentially MIT-licensed but do not reference their license.
Another solution would be to ask Doxygen to build a PDF instead of HTML. The result would have no visible bundled assets, but in my opinion would be less convenient for users.
There are some judgement calls here, but I think the result (https://src.fedoraproject.org/rpms/cairomm/c/948e04f946d95612948d81d9090e2bc...) is reasonable.
The situation for python-autodocsumm’s Sphinx documentation is more dire. I can unbundle the Sphinx-inserted copies of js-jquery and js-underscore. I can add virtual Provides for Sphinx’s libraries doctools.js and searchtools.js:
Provides: bundled(js-sphinx-doctools) Provides: bundled(js-sphinx-searchtools)
although I cannot version them, because the version could only be that of Sphinx itself, and that changes regularly without alterations to my package’s sources; I don’t know a way to handle that automatically, and I could not reasonably keep it synchronized manually. These libraries were never intended to be handled separately from the documentation they support.
I can claim that language_data.js and documentation_options.js are not bundled libraries, because they are generated from templates using project-dependent options.
Now I have the problem that all of the Sphinx-inserted files in _static have the same BSD license as Sphinx itself, but they only reference the Sphinx LICENSE file in their headers; they do not contain license text at all, and BSD is one of the licenses where license text is required.
I could possibly handle this by copying in the license file from the Sphinx package in the build environment. I’d better make sure I don’t miss a hypothetical Sphinx license change, so I should probably grep that license file in %check for a key phrase to make sure its contents are what I expect.
Now I’m left with the theme. The autodocsumm documentation uses sphinx_rtd_theme, which is a very common choice. Theme files are mostly in _static/css/ and _static/js/.
In _static/js/, there are two JavaScript files, badge_only.js and theme.js. Both are minified and contain no license information. Okay, I can check the license of python3-sphinx_rtd_theme, and maybe I can copy in the unminified versions manually to comply with that part of the guidelines. Oh, it turns out only the minified versions are packaged, so there’s nothing I can do. (Well—shipping only minified files isn’t compliant with guidelines, so I could send a PR or file an issue on python-sphinx_rtd_theme and expect the maintainer to do something about it. I’m not trying to call out this particular package or its maintainers, as it’s very hard to find and deal with all the bits of JavaScript and web content that are tucked away in Python packages these days, and such oversights are rampant in existing packages.)
In _static/css/, there are two corresponding CSS files, badge_only.css and theme.css. Both have the same problems (minified only, no license information), but *also*, theme.css has a bundled copy of the CSS from Font Awesome 4.7.0. That’s also MIT-licensed, with only a URL for the license and no included license text.
This is the point at which I say that the burden of trying to package this near-minimal example of Sphinx-generated documentation in strict compliance with the guidelines is unreasonable, and I drop it, along with the documentation in all other Python packages for which I am responsible.
I would be surprised if there is even one case where HTML documentation for a Python package is currently packaged in a way that truly satisfies all of these requirements around license text, web assets, JavaScript, and bundling; and I suspect the broader situation for Doxygen and other documentation generators is not much better.
As a user who has often needed to work offline for various reasons, it would be a great loss if the solution really has to be that HTML documentation should not be packaged in general.
On 9/24/21 09:21, Ben Beasley wrote:
It was recently brought up in a package review (https://bugzilla.redhat.com/show_bug.cgi?id=2006555) that a Sphinx-generated HTML documentation package contained several JavaScript libraries, and that:
1. their licenses should be accounted for and added to the License field for the -doc subpackage 2. they should be treated as bundled libraries and marked with virtual Provides
Note that the JavaScript is inserted by the Sphinx documentation build system, and is not present in the source tarball. Your “typical” Sphinx documentation has minified and unminified copies of JQuery (js-jquery) and Underscore (js-underscore), both MIT-licensed, plus several unminified utility libraries that belong to Sphinx itself (doctools.js, language_data.js, searchtools.js). The overall Sphinx project is BSD-licensed. The details vary by theme, but this is the most common case.
To me, it seems that this feedback is a reasonable strict reading of the relevant packaging guidelines. If this is the right interpretation of Fedora policy, what should be done about it in general?
Based on looking for installed files named “searchtools.js” in packages named -doc or -docs, there are about 600 documentation packages generated with Sphinx. This might miss some that have exotic themes.
As far as I can tell, only one existing package in the distribution, python-murano-package-check, tries to use virtual Provides to track the libraries that belong to Sphinx itself:
Provides: bundled(js-doctools) Provides: bundled(js-jquery) = 3.1.0 Provides: bundled(js-searchtools) Provides: bundled(js-underscore) = 1.3.1 Provides: bundled(js-websupport)
and this package does not actually build its documentation subpackage.
Only the following packages have virtual Provides for js-jquery and js-underscore. In most cases these lack the version numbers.
- arb-doc - mpdecimal-doc - python-BTrees-doc - python-latexcodec-doc - python-networkx-doc - python3-persistent-doc - sympy-doc
Of the packages in the list above, only mpdecimal-doc has a License field that seems to try to account for the licenses of the JavaScript libraries.
Personally, I can add virtual Provides and adjust the License on the -doc subpackage for my package under review, and it’s even possible for me to replace the Underscore and JQuery libraries with symbolic links to those installed by js-jquery and js-underscore. However, changing one package, or even every package I control, is a tiny drop in a vast ocean of documentation packages. Consider, for example, that Doxygen also includes JQuery and several Doxygen-specific libraries in its output, so the same arguments about licenses and virtual Provides apply to it. A search for “dynsections.js” in packages named *-doc or *-docs gives nearly 400 Doxygen-based packages.
If this is a real problem, it seems like it needs to be handled and/or documented in a way that can potentially scale to the rest of those 600 Sphinx-based packages, to the nearly 400 Doxygen-based packages, and beyond that to the output of other documentation systems. I am not sure what that would look like.
All thoughts are welcome.
– Ben Beasley (FAS: music) _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On Mon, Sep 27, 2021 at 3:35 PM Ben Beasley code@musicinmybrain.net wrote:
One last follow-up:
It turns out that even the situation with Doxygen is worse. The file jquery.js inserted by Doxygen was not just a copy of JQuery (js-jquery) to be unbundled; it also contained in the same file:
- JQuery UI (js-jquery-ui)
- jquery.ScrollTo (https://github.com/flesler/jquery.scrollTo/)
- PowerTip (https://stevenbenner.github.io/jquery-powertip/)
- JQuery UI Touch Punch (js-jquery-ui-touch-punch)
- SmartMenus jQuery Plugin (http://www.smartmenus.org)
It’s possible to account for all of these, package the missing ones separately, and reconstruct the bundle using unminified sources (or patch the generated HTML to use separate script tags, yuck), but the burden is getting ridiculous again. It looks like dropping all Doxygen-based HTML documentation, or building a less-useful PDF instead, will be the sensible (but disappointing) outcome.
Thank you for starting this conversation, I had similar thoughts when I last touched my .spec files for packages that build docs for Python (with sphinx) or C (with doxygen).
In my opinion, building PDFs instead of HTML docs would actually be an improvement in at least some ways: There's no weird bundled JavaScript libraries, and docs can get shipped as a single file (?) instead of lots and lots of small HTML / JS / CSS files (with questionable licenses).
PDF is probably as "standard file format" today as HTML (even most web browsers can open them), and probably all Fedora Editions and Spins ship with a PDF viewer anyway (most of which also have good Ctrl-F and PDF chapter bookmark functionality, which is way nicer than clicking through dozens of hyperlinks in HTML docs).
At any rate, discoverability of pre-built docs from Fedora packages is very poor to start with: - most are hidden away in a separate -doc subpackage that doesn't get installed by default - even if the package is installed, the user has to manually navigate their file browser to /usr/share/doc/foo/html/ (or similar), then open the "index.html" file, and hopefully not forget to add a browser bookmark for it
So, I'm not sure if the benefits of building HTML (or PDF, for that matter) docs actually outweighs the headache they often cause.
Fabio
RDoc generated documentation available in rubygem-*-doc subpackages suffers the similar issues and I honestly don't know how to fix this situation.
Best would be if upstream acknowledged that the generated documentation does not need to contain the copy of template including JS and what not. The template could be shared among the generated documentation.
The other possibility is to grant some exception to generated documentation.
Vít
Dne 28. 09. 21 v 0:54 Fabio Valentini napsal(a):
On Mon, Sep 27, 2021 at 3:35 PM Ben Beasley code@musicinmybrain.net wrote:
One last follow-up:
It turns out that even the situation with Doxygen is worse. The file jquery.js inserted by Doxygen was not just a copy of JQuery (js-jquery) to be unbundled; it also contained in the same file:
- JQuery UI (js-jquery-ui)
- jquery.ScrollTo (https://github.com/flesler/jquery.scrollTo/)
- PowerTip (https://stevenbenner.github.io/jquery-powertip/)
- JQuery UI Touch Punch (js-jquery-ui-touch-punch)
- SmartMenus jQuery Plugin (http://www.smartmenus.org)
It’s possible to account for all of these, package the missing ones separately, and reconstruct the bundle using unminified sources (or patch the generated HTML to use separate script tags, yuck), but the burden is getting ridiculous again. It looks like dropping all Doxygen-based HTML documentation, or building a less-useful PDF instead, will be the sensible (but disappointing) outcome.
Thank you for starting this conversation, I had similar thoughts when I last touched my .spec files for packages that build docs for Python (with sphinx) or C (with doxygen).
In my opinion, building PDFs instead of HTML docs would actually be an improvement in at least some ways: There's no weird bundled JavaScript libraries, and docs can get shipped as a single file (?) instead of lots and lots of small HTML / JS / CSS files (with questionable licenses).
PDF is probably as "standard file format" today as HTML (even most web browsers can open them), and probably all Fedora Editions and Spins ship with a PDF viewer anyway (most of which also have good Ctrl-F and PDF chapter bookmark functionality, which is way nicer than clicking through dozens of hyperlinks in HTML docs).
At any rate, discoverability of pre-built docs from Fedora packages is very poor to start with:
- most are hidden away in a separate -doc subpackage that doesn't get
installed by default
- even if the package is installed, the user has to manually navigate
their file browser to /usr/share/doc/foo/html/ (or similar), then open the "index.html" file, and hopefully not forget to add a browser bookmark for it
So, I'm not sure if the benefits of building HTML (or PDF, for that matter) docs actually outweighs the headache they often cause.
Fabio _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 28. 09. 21 0:54, Fabio Valentini wrote:
On Mon, Sep 27, 2021 at 3:35 PM Ben Beasley code@musicinmybrain.net wrote:
One last follow-up:
It turns out that even the situation with Doxygen is worse. The file jquery.js inserted by Doxygen was not just a copy of JQuery (js-jquery) to be unbundled; it also contained in the same file:
- JQuery UI (js-jquery-ui)
- jquery.ScrollTo (https://github.com/flesler/jquery.scrollTo/)
- PowerTip (https://stevenbenner.github.io/jquery-powertip/)
- JQuery UI Touch Punch (js-jquery-ui-touch-punch)
- SmartMenus jQuery Plugin (http://www.smartmenus.org)
It’s possible to account for all of these, package the missing ones separately, and reconstruct the bundle using unminified sources (or patch the generated HTML to use separate script tags, yuck), but the burden is getting ridiculous again. It looks like dropping all Doxygen-based HTML documentation, or building a less-useful PDF instead, will be the sensible (but disappointing) outcome.
Thank you for starting this conversation, I had similar thoughts when I last touched my .spec files for packages that build docs for Python (with sphinx) or C (with doxygen).
In my opinion, building PDFs instead of HTML docs would actually be an improvement in at least some ways: There's no weird bundled JavaScript libraries, and docs can get shipped as a single file (?) instead of lots and lots of small HTML / JS / CSS files (with questionable licenses).
PDF is probably as "standard file format" today as HTML (even most web browsers can open them), and probably all Fedora Editions and Spins ship with a PDF viewer anyway (most of which also have good Ctrl-F and PDF chapter bookmark functionality, which is way nicer than clicking through dozens of hyperlinks in HTML docs).
At any rate, discoverability of pre-built docs from Fedora packages is very poor to start with:
- most are hidden away in a separate -doc subpackage that doesn't get
installed by default
- even if the package is installed, the user has to manually navigate
their file browser to /usr/share/doc/foo/html/ (or similar), then open the "index.html" file, and hopefully not forget to add a browser bookmark for it
So, I'm not sure if the benefits of building HTML (or PDF, for that matter) docs actually outweighs the headache they often cause.
For Sphinx HTML at least, a "proper" solution might be to create (and maintain) a Public Domain theme that does not bundle any JavaScript etc. and uses shared resources from Fedora if needed (or better yet, does not use any), and ask packagers to manually change the theme to this one. If that works, we could even provide a convenient macro/script to build it instead of requiring each package to patch conf.py.
As for PDF docs, has anybody checked if they bundle fonts?
For Sphinx HTML at least, a "proper" solution might be to create (and maintain) a Public Domain theme that does not bundle any JavaScript etc. and uses shared resources from Fedora if needed (or better yet, does not use any), and ask packagers to manually change the theme to this one. If that works, we could even provide a convenient macro/script to build it instead of requiring each package to patch conf.py.
This seems like a somewhat nontrivial effort, but it could be technically workable.
Some documentation is tightly tied to special features of its preferred Sphinx theme, but most documentation will build acceptably (just as good, or slightly degraded) with any theme.
Patching conf.py in one way or another would still be needed, as it generally contains essential project-specific information, configuration, and plugin setup. However, in most cases the theme could be adjusted by simply appending
html_theme = "my_foo_theme"
to the end of the file to override the upstream setting rather than patching more intelligently.
As for PDF docs, has anybody checked if they bundle fonts?
Well, a PDF generated via Doxygen that I spot-checked by opening it in FontForge certainly does: various flavors of Computer Modern, Nimbus Sans, Nimbus Mono, and DejaVu Sans (all subsetted to include only the glyphs that are actually used). A PDF generated via Sphinx contained various flavors of TeX Gyre Heros, TeX Gyre Termes, Nimbus Roman No. 9, t1xtt, and tcxtt (also subsetted). You can also examine these with “pdffonts” from poppler-utils. But what’s to be done about it?
All the routes I know of to get PDF documentation from either Doxygen or Sphinx go via LaTeX; the documentation system can be asked to output LaTeX, and then pdflatex does the rest, often via a generated helper Makefile. In many cases, like Computer Modern, the t??tt fonts, and the TeX Gyre fonts, the font is originally a Type 1 PostScript font and has no OpenType (TTF/OTF) equivalent in the first place.
I’m not aware of a way to remove embedded fonts from a PDF using free software, nor am I aware of a way to ask pdflatex not to embed them (again, in cases where an OpenType equivalent suitable for packaging as a system font even exists).
If the ban on bundled fonts were interpreted as applying to PDF font embedding, I think that would be tantamount to a practical ban on the PDF file format. It’s hard for me to imagine that outcome is what was intended, but maybe I’m wrong.
-----
The current versions of the cairomm[1] and gi-docgen[2] spec files demonstrate what generating PDF documentation from Doxygen and Sphinx, respectively, can look like. (I have found some packages where Doxygen-generated LaTeX can’t be rendered because it exhausts some fixed TeX resource limit—upstreams commonly test HTML output only—but these techniques do seem to work in most cases.)
[1] https://src.fedoraproject.org/rpms/cairomm/blob/cd53c4ac4f42d38b8f3874a36929... [2] https://src.fedoraproject.org/rpms/gi-docgen/blob/10b9115d8103dd59009c35ab53...
packaging@lists.fedoraproject.org