Hi,
Rdoc generated HTML often includes embedded fonts and javascript which may be problematic to package. Might it be worth suggesting in the packaging documentation[0] to only create Ri files which are accessible on the command line? A related ticket that prompted this is [1].
Regards, Benson
0] https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/ 1] https://bugzilla.redhat.com/show_bug.cgi?id=2097267
Hello Benson,
as of now, this hasn't been an issue and I think there's no obstacle with using the javascript (the library has reported vulnerability AFAIR), but it's not exploitable in the documentation so it's deemed safe (the library should not be used for anything else).
The same goes for the font (if it's shipped in the package itself, it's probably fine to remove it, or state the license).
HIH, Pavel
On Sat, Jun 25, 2022 at 6:35 AM Benson Muite benson_muite@emailplus.org wrote:
Hi,
Rdoc generated HTML often includes embedded fonts and javascript which may be problematic to package. Might it be worth suggesting in the packaging documentation[0] to only create Ri files which are accessible on the command line? A related ticket that prompted this is [1].
Regards, Benson
0] https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/ 1] https://bugzilla.redhat.com/show_bug.cgi?id=2097267 _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Hi Pavel,
Thanks. Is it also worth encouraging packaging of Ri documentation? These are very comandline friendly, Javascript is not. In principle, one might want to have HTML documentation as a separate package from Ri documentation giving the end user a choice of what to install.
Will see if can figure out some way of removing fonts.
Regards, Benson On 6/27/22 13:30, Pavel Valena wrote:
Hello Benson,
as of now, this hasn't been an issue and I think there's no obstacle with using the javascript (the library has reported vulnerability AFAIR), but it's not exploitable in the documentation so it's deemed safe (the library should not be used for anything else).
The same goes for the font (if it's shipped in the package itself, it's probably fine to remove it, or state the license).
HIH, Pavel
On Sat, Jun 25, 2022 at 6:35 AM Benson Muite <benson_muite@emailplus.org mailto:benson_muite@emailplus.org> wrote:
Hi, Rdoc generated HTML often includes embedded fonts and javascript which may be problematic to package. Might it be worth suggesting in the packaging documentation[0] to only create Ri files which are accessible on the command line? A related ticket that prompted this is [1]. Regards, Benson 0] https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/ <https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/> 1] https://bugzilla.redhat.com/show_bug.cgi?id=2097267 <https://bugzilla.redhat.com/show_bug.cgi?id=2097267> _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org <mailto:ruby-sig@lists.fedoraproject.org> To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org <mailto:ruby-sig-leave@lists.fedoraproject.org> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ <https://docs.fedoraproject.org/en-US/project/code-of-conduct/> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines <https://fedoraproject.org/wiki/Mailing_list_guidelines> List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org <https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure <https://pagure.io/fedora-infrastructure>
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Dne 27. 06. 22 v 13:21 Benson Muite napsal(a):
Hi Pavel,
Thanks. Is it also worth encouraging packaging of Ri documentation? These are very comandline friendly, Javascript is not. In principle, one might want to have HTML documentation as a separate package from Ri documentation giving the end user a choice of what to install.
Of course that would be nice in ideal world. If I only had a time to implement this.
Will see if can figure out some way of removing fonts.
There are several options:
1) Keep the status quo,
2) Just drop the fonts and leave the browser to use some substitutes
3) Symlink the system fonts.
I have never experimented with (2) / (3).
Of course ideal would be to have our own template for the RDoc documentation which could be linked instead the copied version. This would save us from these issues as well as saved disk space.
Vít
Regards, Benson On 6/27/22 13:30, Pavel Valena wrote:
Hello Benson,
as of now, this hasn't been an issue and I think there's no obstacle with using the javascript (the library has reported vulnerability AFAIR), but it's not exploitable in the documentation so it's deemed safe (the library should not be used for anything else).
The same goes for the font (if it's shipped in the package itself, it's probably fine to remove it, or state the license).
HIH, Pavel
On Sat, Jun 25, 2022 at 6:35 AM Benson Muite <benson_muite@emailplus.org mailto:benson_muite@emailplus.org> wrote:
Hi,
Rdoc generated HTML often includes embedded fonts and javascript which may be problematic to package. Might it be worth suggesting in the packaging documentation[0] to only create Ri files which are accessible on the command line? A related ticket that prompted this is [1].
Regards, Benson
0] https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/ https://docs.fedoraproject.org/en-US/packaging-guidelines/Ruby/ 1] https://bugzilla.redhat.com/show_bug.cgi?id=2097267 https://bugzilla.redhat.com/show_bug.cgi?id=2097267 _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org mailto:ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org mailto:ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure https://pagure.io/fedora-infrastructure
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Hi,
since it came up in the package review I took a closer look on the state and possibilities.
On 6/27/22 17:47, Vít Ondruch wrote:
Dne 27. 06. 22 v 13:21 Benson Muite napsal(a):
Hi Pavel,
Thanks. Is it also worth encouraging packaging of Ri documentation? These are very comandline friendly, Javascript is not. In principle, one might want to have HTML documentation as a separate package from Ri documentation giving the end user a choice of what to install.
Of course that would be nice in ideal world. If I only had a time to implement this.
Will see if can figure out some way of removing fonts.
There are several options:
Keep the status quo,
Just drop the fonts and leave the browser to use some substitutes
That seems simple enough, from limited local testing, it is absolutely possible, but we would need probably the correct font requirement for ruby-*doc (more on that down in the response...).
However we still have the JS files. We can take those out, in theory. But it breaks code search and code expansion from my limited testing. I'd be in favor of somehow not shipping the HTML at all than to remove the JS. There is a way to not generate it with RDoc and have just the .ri format, however I'd have to look into how it would work with our `%gem_install`.
- Symlink the system fonts.
I'd personally like to dodge symlinking the fonts as the folder names might change in the future. (Or might not, I am not well versed in possibilities of the Font space of world)
Firefox (at least on my machine :) ) and the `fonts.css` generated with the docs are smart enough to take it from the system folders if they are not present in the expected relative location. Tested in one of my Ruby projects like so: ~~~ rdoc rm doc/fonts/* ~~~ When I open the index.html, there are no errors in the browser logs regarding failure to load font files, as long as I have both the required font packages.
So, I'd rather remove them from the package than to have symlinks that might break later on. This approach will require generating `Requires:` for the *-doc subpackages on the `adobe-source-code-pro-fonts` for source code font and the `lato-fonts` for non source code text.
Visually comparing the "pure" generated state right after running `rdoc` and the state after removing the fonts with `rm doc/fonts/*` I observe no loss in quality, so I think that should be the way go for HTML docs in the fonts department as the mechanisms seem to be there.
As to the JS: A) We could also somehow tell the generator where the JS assets live and have it generate the paths into the HTML (I noticed there are some variables around this in the darkfish.rb [0]), but we would have to first find out how to even do that, second figure out where are those JS files going to be as we are running in, IMHO into similar problem as with symlinking, IMHO we do not want to depend on the whole rubygem-rdoc. This approach to me seems time consuming, achieving similar results as simple symlinking in the end.
B) Simply symlinking the files to expected location. Question is what would happen if we will have to move the symlink for whatever reason, but that's more general question around parts of RPM internals I do not explore quite often, so I cannot effectively judge on that.
What form or solution should that take though is a harder question. Why is that harder IMO is what form of requires are needed? Maybe rubygem-rdoc? But that's a lot of files that we do not need when we need just the JS files, probably some subpackage would be better fitting?
Regards, Jarek
[0] `@template_dir` https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L220
I have never experimented with (2) / (3).
Of course ideal would be to have our own template for the RDoc documentation which could be linked instead the copied version. This would save us from these issues as well as saved disk space.
Vít
Dne 27. 06. 22 v 19:51 Jarek Prokop napsal(a):
Hi,
since it came up in the package review I took a closer look on the state and possibilities.
On 6/27/22 17:47, Vít Ondruch wrote:
Dne 27. 06. 22 v 13:21 Benson Muite napsal(a):
Hi Pavel,
Thanks. Is it also worth encouraging packaging of Ri documentation? These are very comandline friendly, Javascript is not. In principle, one might want to have HTML documentation as a separate package from Ri documentation giving the end user a choice of what to install.
Of course that would be nice in ideal world. If I only had a time to implement this.
Will see if can figure out some way of removing fonts.
There are several options:
Keep the status quo,
Just drop the fonts and leave the browser to use some substitutes
That seems simple enough, from limited local testing, it is absolutely possible, but we would need probably the correct font requirement for ruby-*doc (more on that down in the response...).
However we still have the JS files. We can take those out, in theory. But it breaks code search and code expansion from my limited testing. I'd be in favor of somehow not shipping the HTML at all than to remove the JS. There is a way to not generate it with RDoc and have just the .ri format, however I'd have to look into how it would work with our `%gem_install`.
- Symlink the system fonts.
I'd personally like to dodge symlinking the fonts as the folder names might change in the future. (Or might not, I am not well versed in possibilities of the Font space of world)
Firefox (at least on my machine :) ) and the `fonts.css` generated with the docs are smart enough to take it from the system folders if they are not present in the expected relative location. Tested in one of my Ruby projects like so:
rdoc rm doc/fonts/*
When I open the index.html, there are no errors in the browser logs regarding failure to load font files, as long as I have both the required font packages.
So, I'd rather remove them from the package than to have symlinks that might break later on. This approach will require generating `Requires:` for the *-doc subpackages on the `adobe-source-code-pro-fonts` for source code font and the `lato-fonts` for non source code text.
Visually comparing the "pure" generated state right after running `rdoc` and the state after removing the fonts with `rm doc/fonts/*` I observe no loss in quality, so I think that should be the way go for HTML docs in the fonts department as the mechanisms seem to be there.
As to the JS: A) We could also somehow tell the generator where the JS assets live and have it generate the paths into the HTML (I noticed there are some variables around this in the darkfish.rb [0]), but we would have to first find out how to even do that, second figure out where are those JS files going to be as we are running in, IMHO into similar problem as with symlinking, IMHO we do not want to depend on the whole rubygem-rdoc. This approach to me seems time consuming, achieving similar results as simple symlinking in the end.
B) Simply symlinking the files to expected location. Question is what would happen if we will have to move the symlink for whatever reason, but that's more general question around parts of RPM internals I do not explore quite often, so I cannot effectively judge on that.
What form or solution should that take though is a harder question. Why is that harder IMO is what form of requires are needed? Maybe rubygem-rdoc? But that's a lot of files that we do not need when we need just the JS files, probably some subpackage would be better fitting?
Thank you for looking into this.
I agree that the proper font requires should be better and actually enough to be able to drop the fonts and keep the visual output the same.
Please not that we still have js-jquery package around, that could again help with removing the duplicated JS code.
However, doing these two things get us closer to having separate template for documentation. I.e. if we were able to extract the darkfish from RDoc into separate package and symlink the whole template, that would be win. I am pretty sure that RDoc can use custom templates.
Vít
Regards, Jarek
[0] `@template_dir` https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L220
I have never experimented with (2) / (3).
Of course ideal would be to have our own template for the RDoc documentation which could be linked instead the copied version. This would save us from these issues as well as saved disk space.
Vít
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
On 6/28/22 09:54, Vít Ondruch wrote:
Dne 27. 06. 22 v 19:51 Jarek Prokop napsal(a):
Hi,
since it came up in the package review I took a closer look on the state and possibilities.
On 6/27/22 17:47, Vít Ondruch wrote:
Dne 27. 06. 22 v 13:21 Benson Muite napsal(a):
Hi Pavel,
Thanks. Is it also worth encouraging packaging of Ri documentation? These are very comandline friendly, Javascript is not. In principle, one might want to have HTML documentation as a separate package from Ri documentation giving the end user a choice of what to install.
Of course that would be nice in ideal world. If I only had a time to implement this.
Will see if can figure out some way of removing fonts.
There are several options:
Keep the status quo,
Just drop the fonts and leave the browser to use some substitutes
That seems simple enough, from limited local testing, it is absolutely possible, but we would need probably the correct font requirement for ruby-*doc (more on that down in the response...).
However we still have the JS files. We can take those out, in theory. But it breaks code search and code expansion from my limited testing. I'd be in favor of somehow not shipping the HTML at all than to remove the JS. There is a way to not generate it with RDoc and have just the .ri format, however I'd have to look into how it would work with our `%gem_install`.
- Symlink the system fonts.
I'd personally like to dodge symlinking the fonts as the folder names might change in the future. (Or might not, I am not well versed in possibilities of the Font space of world)
Firefox (at least on my machine :) ) and the `fonts.css` generated with the docs are smart enough to take it from the system folders if they are not present in the expected relative location. Tested in one of my Ruby projects like so:
rdoc rm doc/fonts/*
When I open the index.html, there are no errors in the browser logs regarding failure to load font files, as long as I have both the required font packages.
So, I'd rather remove them from the package than to have symlinks that might break later on. This approach will require generating `Requires:` for the *-doc subpackages on the `adobe-source-code-pro-fonts` for source code font and the `lato-fonts` for non source code text.
Visually comparing the "pure" generated state right after running `rdoc` and the state after removing the fonts with `rm doc/fonts/*` I observe no loss in quality, so I think that should be the way go for HTML docs in the fonts department as the mechanisms seem to be there.
As to the JS: A) We could also somehow tell the generator where the JS assets live and have it generate the paths into the HTML (I noticed there are some variables around this in the darkfish.rb [0]), but we would have to first find out how to even do that, second figure out where are those JS files going to be as we are running in, IMHO into similar problem as with symlinking, IMHO we do not want to depend on the whole rubygem-rdoc. This approach to me seems time consuming, achieving similar results as simple symlinking in the end.
B) Simply symlinking the files to expected location. Question is what would happen if we will have to move the symlink for whatever reason, but that's more general question around parts of RPM internals I do not explore quite often, so I cannot effectively judge on that.
What form or solution should that take though is a harder question. Why is that harder IMO is what form of requires are needed? Maybe rubygem-rdoc? But that's a lot of files that we do not need when we need just the JS files, probably some subpackage would be better fitting?
Thank you for looking into this.
I agree that the proper font requires should be better and actually enough to be able to drop the fonts and keep the visual output the same.
Please not that we still have js-jquery package around, that could again help with removing the duplicated JS code.
Not by much :/ those files are still present, with JQuery they were just loaded in darkfish.js instead of the HTML `<header>` section. It would perhaps even worsen the situation with additional JS dependency. (see: https://github.com/ruby/rdoc/commit/b6206b943f95026872c9b690ef59f04ee2f54fec)
However, doing these two things get us closer to having separate template for documentation. I.e. if we were able to extract the darkfish from RDoc into separate package and symlink the whole template, that would be win. I am pretty sure that RDoc can use custom templates.
I agree with this form, we could have a template like that with darkfish as source. I can try whipping something up, but I cannot promise anything concrete ATM.
Jarek
Vít
Regards, Jarek
[0] `@template_dir` https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L220
I have never experimented with (2) / (3).
Of course ideal would be to have our own template for the RDoc documentation which could be linked instead the copied version. This would save us from these issues as well as saved disk space.
Vít
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4...
Just a few notes from the limited time I spend trying to understand the approach.
1) The template can't be subpackage of the rubygem-rdoc. It needs to live in completely separate project to enable us to decouple RDoc updates from the template updates.
2) You are using the `--format darkfish`, but wouldn't it be better to use `--template=NAME` instead?
3) I don't think we necessarily need to use the `%{_webassetdir}` or `%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
4) I would not mind to keep the original template including the bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
5) There is too much changes into the RDoc. If possible, I think it would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
~~~
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
~~~
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
I guess that RPM can't preserve them.
Vít
Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs to
live in completely separate project to enable us to decouple RDoc updates from the template updates.
- You are using the `--format darkfish`, but wouldn't it be better to
use `--template=NAME` instead?
- I don't think we necessarily need to use the `%{_webassetdir}` or
`%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
And just another anecdote, the hardlinks are there per my request:
https://github.com/ruby/rdoc/issues/186
The only issue is that they probably have almost 0 effect, since to be effective, the owner of the original file have to be the same as the owner of the target file. In our case, the origin is typically owned by "root", while the build system is using the "mockbuild" user. The hardlinks can't work even for user installed gem. The only case when they works is when RDoc is installed via `gem install` as well as the other gems (OTOH, this is always the case for people compiling their own Rubies).
Not sure if I should propose to use just plain old copy instead 🤔
Vít
Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
I guess that RPM can't preserve them.
Vít
Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs to
live in completely separate project to enable us to decouple RDoc updates from the template updates.
- You are using the `--format darkfish`, but wouldn't it be better
to use `--template=NAME` instead?
- I don't think we necessarily need to use the `%{_webassetdir}` or
`%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
So here is my proof of concept:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/
https://koji.fedoraproject.org/koji/taskinfo?taskID=93399794
It can be executed as follows:
~~~
$ RUBYOPT="-I/usr/share/darkfish-rdoc/ -rfedora_darkfish" gem install gem2rpm --doc=ri,rdoc Fetching gem2rpm-1.0.2.gem WARNING: You don't have /builddir/bin in your PATH, gem executables will not run. Successfully installed gem2rpm-1.0.2 Parsing documentation for gem2rpm-1.0.2 Installing ri documentation for gem2rpm-1.0.2 Installing darkfish documentation for gem2rpm-1.0.2 Done installing documentation for gem2rpm after 0 seconds 1 gem installed
$ ruby -run -e httpd ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/ -p 8000 [2022-10-24 17:17:05] INFO WEBrick 1.7.0 [2022-10-24 17:17:05] INFO ruby 3.1.2 (2022-04-12) [x86_64-linux] [2022-10-24 17:17:05] INFO WEBrick::HTTPServer#start: pid=34 port=8000 [2022-10-24 17:17:05] INFO To access this server, open this URL in a browser: [2022-10-24 17:17:05] INFO http://127.0.0.1:8000 [2022-10-24 17:17:05] INFO http://%5B::1%5D:8000 127.0.0.1 - - [24/Oct/2022:17:17:08 CEST] "GET / HTTP/1.1" 200 3964 - -> / 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/fonts.css HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /css/fonts.css 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/rdoc.css HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /css/rdoc.css 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/search.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/searcher.js HTTP/1.1" 304 0 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search_index.js HTTP/1.1" 200 19687 http://127.0.0.1:8000/ -> /js/search_index.js http://127.0.0.1:8000/ -> /js/searcher.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/darkfish.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/darkfish.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/navigation.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/navigation.js 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /Gem2Rpm/Helpers.html HTTP/1.1" 200 13074 http://127.0.0.1:8000/ -> /Gem2Rpm/Helpers.html 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /js/search_index.js HTTP/1.1" 304 0 http://127.0.0.1:8000/Gem2Rpm/Helpers.html -> /js/search_index.js 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /images/zoom.png HTTP/1.1" 304 0 http://127.0.0.1:8000/css/rdoc.css -> /images/zoom.png ^C[2022-10-24 17:17:17] INFO going to shutdown ... [2022-10-24 17:17:17] INFO WEBrick::HTTPServer#start done.
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/js/ total 36 lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 darkfish.js -> /usr/share/darkfish-rdoc/darkfish/js/darkfish.js lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 navigation.js -> /usr/share/darkfish-rdoc/json_index/js/navigation.js lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 search.js -> /usr/share/darkfish-rdoc/darkfish/js/search.js -rw-r--r--. 1 mockbuild mock 19687 Oct 24 17:17 search_index.js lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 searcher.js -> /usr/share/darkfish-rdoc/json_index/js/searcher.js
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/css/ total 8 lrwxrwxrwx. 1 mockbuild mock 47 Oct 24 17:17 fonts.css -> /usr/share/darkfish-rdoc/darkfish/css/fonts.css lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 rdoc.css -> /usr/share/darkfish-rdoc/darkfish/css/rdoc.css
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/images/ total 100 lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 add.png -> /usr/share/darkfish-rdoc/darkfish/images/add.png lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 arrow_up.png -> /usr/share/darkfish-rdoc/darkfish/images/arrow_up.png lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 brick.png -> /usr/share/darkfish-rdoc/darkfish/images/brick.png lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 brick_link.png -> /usr/share/darkfish-rdoc/darkfish/images/brick_link.png lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 bug.png -> /usr/share/darkfish-rdoc/darkfish/images/bug.png lrwxrwxrwx. 1 mockbuild mock 57 Oct 24 17:17 bullet_black.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_black.png lrwxrwxrwx. 1 mockbuild mock 64 Oct 24 17:17 bullet_toggle_minus.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_minus.png lrwxrwxrwx. 1 mockbuild mock 63 Oct 24 17:17 bullet_toggle_plus.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_plus.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 date.png -> /usr/share/darkfish-rdoc/darkfish/images/date.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 delete.png -> /usr/share/darkfish-rdoc/darkfish/images/delete.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 find.png -> /usr/share/darkfish-rdoc/darkfish/images/find.png lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 loadingAnimation.gif -> /usr/share/darkfish-rdoc/darkfish/images/loadingAnimation.gif lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 macFFBgHack.png -> /usr/share/darkfish-rdoc/darkfish/images/macFFBgHack.png lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 package.png -> /usr/share/darkfish-rdoc/darkfish/images/package.png lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 page_green.png -> /usr/share/darkfish-rdoc/darkfish/images/page_green.png lrwxrwxrwx. 1 mockbuild mock 60 Oct 24 17:17 page_white_text.png -> /usr/share/darkfish-rdoc/darkfish/images/page_white_text.png lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 page_white_width.png -> /usr/share/darkfish-rdoc/darkfish/images/page_white_width.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 plugin.png -> /usr/share/darkfish-rdoc/darkfish/images/plugin.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 ruby.png -> /usr/share/darkfish-rdoc/darkfish/images/ruby.png lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 tag_blue.png -> /usr/share/darkfish-rdoc/darkfish/images/tag_blue.png lrwxrwxrwx. 1 mockbuild mock 54 Oct 24 17:17 tag_green.png -> /usr/share/darkfish-rdoc/darkfish/images/tag_green.png lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 transparent.png -> /usr/share/darkfish-rdoc/darkfish/images/transparent.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 wrench.png -> /usr/share/darkfish-rdoc/darkfish/images/wrench.png lrwxrwxrwx. 1 mockbuild mock 58 Oct 24 17:17 wrench_orange.png -> /usr/share/darkfish-rdoc/darkfish/images/wrench_orange.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 zoom.png -> /usr/share/darkfish-rdoc/darkfish/images/zoom.png
~~~
And now a few remarks and questions:
1) I hoped that any modifications or code won't be needed. But unfortunately, due to hardlinks, that does not seems to be the case :/
2) I am completely unsure how to name this. So far, I incidentally went with `darkfish`, `darkfish-rdoc` or `fedora-darkfish` and I am using the naming scheme inconsistently. It could possibly be `rdoc-darkfish`, which would probably reflect that this was extracted from RDoc.
Also there is the -assets subpackage. Is it the right name? It could be e.g. -static.
3) Yes, this template was extracted from RDoc and therefore goes with the RDoc version. Not sure if this is right approach.
4) Speaking about the version, I am not sure if the package should not have the version embedded in the name and we could possibly have multiple parallel version installed side by side. I am not sure how compatible the templates are between each other, so this could allow us to update the template without rebuilding everything. But maybe rebuild of everything due to the template change is the right thing.
5) The template version used to generate to the documentation should be probably hard required in the -doc subpackages. And of course every -doc subpackage will need to grow the dependency on the darkfish-assets subpackage.
6) The -assets package will pull in two fonts. Source Code Pro and Lato. (Not)surprigly, the Lato is probably not installed by default, at least I don't have it available on my system.
Not surprisingly, the fonts are completely removed from the package and I have also removed the license notices. If the fonts are not included, it would just provide false information.
7) The fedora_darkfish.rb file monkeypatches the Darkfish generator (as well as json_index generator, which is internally used by Darkfish). The changes are not unit tested or does not have integration suite. Maybe we should have some code coverage. That could help with answers to (4).
8) The json_index used to generate the .js files, but it also provided their .js.gz counterparts. We have never removed the "duplication". Since this was inconsistent anyway, I have disable generating the gzipped files altogether. I don't think that browser can pick up the gzipped version if the documentation is opened directly from the file system, but I might be wron.
9) We probably don't want to execute the command I have used above (although it is certainly one option). Therefore I am thinking about integrating with operating_system.rb. There already is `Gem.rpmbuild?` used to detect RPM build environment. But how to leverage this? Modification to operating_system.rb will likely impact performance of every Ruby execution. Is there different approach? E.g. RubyGems plugin? Or some games with the file name and order of loading, so the monkeypatch would be loaded instead of some other file?
I am probably at the end of list of issues here. Please give it a try and share your ideas.
Cheers,
Vít
Dne 21. 10. 22 v 12:53 Vít Ondruch napsal(a):
And just another anecdote, the hardlinks are there per my request:
https://github.com/ruby/rdoc/issues/186
The only issue is that they probably have almost 0 effect, since to be effective, the owner of the original file have to be the same as the owner of the target file. In our case, the origin is typically owned by "root", while the build system is using the "mockbuild" user. The hardlinks can't work even for user installed gem. The only case when they works is when RDoc is installed via `gem install` as well as the other gems (OTOH, this is always the case for people compiling their own Rubies).
Not sure if I should propose to use just plain old copy instead 🤔
Vít
Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
I guess that RPM can't preserve them.
Vít
Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs to
live in completely separate project to enable us to decouple RDoc updates from the template updates.
- You are using the `--format darkfish`, but wouldn't it be better
to use `--template=NAME` instead?
- I don't think we necessarily need to use the `%{_webassetdir}` or
`%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Dne 24. 10. 22 v 17:42 Vít Ondruch napsal(a):
So here is my proof of concept:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/
https://koji.fedoraproject.org/koji/taskinfo?taskID=93399794
It can be executed as follows:
$ RUBYOPT="-I/usr/share/darkfish-rdoc/ -rfedora_darkfish" gem install gem2rpm --doc=ri,rdoc Fetching gem2rpm-1.0.2.gem WARNING: You don't have /builddir/bin in your PATH, gem executables will not run. Successfully installed gem2rpm-1.0.2 Parsing documentation for gem2rpm-1.0.2 Installing ri documentation for gem2rpm-1.0.2 Installing darkfish documentation for gem2rpm-1.0.2 Done installing documentation for gem2rpm after 0 seconds 1 gem installed $ ruby -run -e httpd ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/ -p 8000 [2022-10-24 17:17:05] INFO WEBrick 1.7.0 [2022-10-24 17:17:05] INFO ruby 3.1.2 (2022-04-12) [x86_64-linux] [2022-10-24 17:17:05] INFO WEBrick::HTTPServer#start: pid=34 port=8000 [2022-10-24 17:17:05] INFO To access this server, open this URL in a browser: [2022-10-24 17:17:05] INFO http://127.0.0.1:8000 [2022-10-24 17:17:05] INFO http://[::1]:8000 127.0.0.1 - - [24/Oct/2022:17:17:08 CEST] "GET / HTTP/1.1" 200 3964 - -> / 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/fonts.css HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /css/fonts.css 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/rdoc.css HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /css/rdoc.css 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/search.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/searcher.js HTTP/1.1" 304 0 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search_index.js HTTP/1.1" 200 19687 http://127.0.0.1:8000/ -> /js/search_index.js http://127.0.0.1:8000/ -> /js/searcher.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/darkfish.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/darkfish.js 127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/navigation.js HTTP/1.1" 304 0 http://127.0.0.1:8000/ -> /js/navigation.js 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /Gem2Rpm/Helpers.html HTTP/1.1" 200 13074 http://127.0.0.1:8000/ -> /Gem2Rpm/Helpers.html 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /js/search_index.js HTTP/1.1" 304 0 http://127.0.0.1:8000/Gem2Rpm/Helpers.html -> /js/search_index.js 127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /images/zoom.png HTTP/1.1" 304 0 http://127.0.0.1:8000/css/rdoc.css -> /images/zoom.png ^C[2022-10-24 17:17:17] INFO going to shutdown ... [2022-10-24 17:17:17] INFO WEBrick::HTTPServer#start done. $ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/js/ total 36 lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 darkfish.js -> /usr/share/darkfish-rdoc/darkfish/js/darkfish.js lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 navigation.js -> /usr/share/darkfish-rdoc/json_index/js/navigation.js lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 search.js -> /usr/share/darkfish-rdoc/darkfish/js/search.js -rw-r--r--. 1 mockbuild mock 19687 Oct 24 17:17 search_index.js lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 searcher.js -> /usr/share/darkfish-rdoc/json_index/js/searcher.js $ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/css/ total 8 lrwxrwxrwx. 1 mockbuild mock 47 Oct 24 17:17 fonts.css -> /usr/share/darkfish-rdoc/darkfish/css/fonts.css lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 rdoc.css -> /usr/share/darkfish-rdoc/darkfish/css/rdoc.css $ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/images/ total 100 lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 add.png -> /usr/share/darkfish-rdoc/darkfish/images/add.png lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 arrow_up.png -> /usr/share/darkfish-rdoc/darkfish/images/arrow_up.png lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 brick.png -> /usr/share/darkfish-rdoc/darkfish/images/brick.png lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 brick_link.png -> /usr/share/darkfish-rdoc/darkfish/images/brick_link.png lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 bug.png -> /usr/share/darkfish-rdoc/darkfish/images/bug.png lrwxrwxrwx. 1 mockbuild mock 57 Oct 24 17:17 bullet_black.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_black.png lrwxrwxrwx. 1 mockbuild mock 64 Oct 24 17:17 bullet_toggle_minus.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_minus.png lrwxrwxrwx. 1 mockbuild mock 63 Oct 24 17:17 bullet_toggle_plus.png -> /usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_plus.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 date.png -> /usr/share/darkfish-rdoc/darkfish/images/date.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 delete.png -> /usr/share/darkfish-rdoc/darkfish/images/delete.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 find.png -> /usr/share/darkfish-rdoc/darkfish/images/find.png lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 loadingAnimation.gif -> /usr/share/darkfish-rdoc/darkfish/images/loadingAnimation.gif lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 macFFBgHack.png -> /usr/share/darkfish-rdoc/darkfish/images/macFFBgHack.png lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 package.png -> /usr/share/darkfish-rdoc/darkfish/images/package.png lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 page_green.png -> /usr/share/darkfish-rdoc/darkfish/images/page_green.png lrwxrwxrwx. 1 mockbuild mock 60 Oct 24 17:17 page_white_text.png -> /usr/share/darkfish-rdoc/darkfish/images/page_white_text.png lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 page_white_width.png -> /usr/share/darkfish-rdoc/darkfish/images/page_white_width.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 plugin.png -> /usr/share/darkfish-rdoc/darkfish/images/plugin.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 ruby.png -> /usr/share/darkfish-rdoc/darkfish/images/ruby.png lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 tag_blue.png -> /usr/share/darkfish-rdoc/darkfish/images/tag_blue.png lrwxrwxrwx. 1 mockbuild mock 54 Oct 24 17:17 tag_green.png -> /usr/share/darkfish-rdoc/darkfish/images/tag_green.png lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 transparent.png -> /usr/share/darkfish-rdoc/darkfish/images/transparent.png lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 wrench.png -> /usr/share/darkfish-rdoc/darkfish/images/wrench.png lrwxrwxrwx. 1 mockbuild mock 58 Oct 24 17:17 wrench_orange.png -> /usr/share/darkfish-rdoc/darkfish/images/wrench_orange.png lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 zoom.png -> /usr/share/darkfish-rdoc/darkfish/images/zoom.png
And now a few remarks and questions:
- I hoped that any modifications or code won't be needed. But
unfortunately, due to hardlinks, that does not seems to be the case :/
- I am completely unsure how to name this. So far, I incidentally
went with `darkfish`, `darkfish-rdoc` or `fedora-darkfish` and I am using the naming scheme inconsistently. It could possibly be `rdoc-darkfish`, which would probably reflect that this was extracted from RDoc.
Also there is the -assets subpackage. Is it the right name? It could be e.g. -static.
- Yes, this template was extracted from RDoc and therefore goes with
the RDoc version. Not sure if this is right approach.
- Speaking about the version, I am not sure if the package should not
have the version embedded in the name and we could possibly have multiple parallel version installed side by side. I am not sure how compatible the templates are between each other, so this could allow us to update the template without rebuilding everything. But maybe rebuild of everything due to the template change is the right thing.
- The template version used to generate to the documentation should
be probably hard required in the -doc subpackages. And of course every -doc subpackage will need to grow the dependency on the darkfish-assets subpackage.
- The -assets package will pull in two fonts. Source Code Pro and
Lato. (Not)surprigly, the Lato is probably not installed by default, at least I don't have it available on my system.
Not surprisingly, the fonts are completely removed from the package and I have also removed the license notices. If the fonts are not included, it would just provide false information.
- The fedora_darkfish.rb file monkeypatches the Darkfish generator
(as well as json_index generator, which is internally used by Darkfish).
The other subthread with Jarek reminded me that one of the options could be to extract/fork the whole Darkfish generator instead of monkey patching. But Darkfish is pretty complex. We would probably not avoided any issues.
Vít
The changes are not unit tested or does not have integration suite. Maybe we should have some code coverage. That could help with answers to (4).
- The json_index used to generate the .js files, but it also provided
their .js.gz counterparts. We have never removed the "duplication". Since this was inconsistent anyway, I have disable generating the gzipped files altogether. I don't think that browser can pick up the gzipped version if the documentation is opened directly from the file system, but I might be wron.
- We probably don't want to execute the command I have used above
(although it is certainly one option). Therefore I am thinking about integrating with operating_system.rb. There already is `Gem.rpmbuild?` used to detect RPM build environment. But how to leverage this? Modification to operating_system.rb will likely impact performance of every Ruby execution. Is there different approach? E.g. RubyGems plugin? Or some games with the file name and order of loading, so the monkeypatch would be loaded instead of some other file?
I am probably at the end of list of issues here. Please give it a try and share your ideas.
Cheers,
Vít
Dne 21. 10. 22 v 12:53 Vít Ondruch napsal(a):
And just another anecdote, the hardlinks are there per my request:
https://github.com/ruby/rdoc/issues/186
The only issue is that they probably have almost 0 effect, since to be effective, the owner of the original file have to be the same as the owner of the target file. In our case, the origin is typically owned by "root", while the build system is using the "mockbuild" user. The hardlinks can't work even for user installed gem. The only case when they works is when RDoc is installed via `gem install` as well as the other gems (OTOH, this is always the case for people compiling their own Rubies).
Not sure if I should propose to use just plain old copy instead 🤔
Vít
Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
I guess that RPM can't preserve them.
Vít
Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs
to live in completely separate project to enable us to decouple RDoc updates from the template updates.
- You are using the `--format darkfish`, but wouldn't it be better
to use `--template=NAME` instead?
- I don't think we necessarily need to use the `%{_webassetdir}`
or `%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
The other subthread with Jarek reminded me that one of the options could be to extract/fork the whole Darkfish generator instead of monkey patching. But Darkfish is pretty complex. We would probably not avoided any issues.
Maybe it would be possible to just provide the subclass of Darkfish generator as a single file, perhaps monkey patch the JSON index while we're at it.
One can provide a different template to a generator and a different generator to a template it seems from messing around with the RDoc CLI and the extracting of the Darkfish template. That leads me to believe that there is a way to do that from code. It would work kind of like this: set the Darkfish template as default, modify the behavior to use symlinks and whatnot. We wouldn't need to keep forked JS/HTML files in a new package.
This would probably minimize required code to get it running.
IOW create a glue between darkfish, json index and RDoc that would make it do what we want to instead of copying it all.
We would then create a tight dependency on those template files, but we can write tests and then check for failures that would crop up via koschei.
Jarek
Vít
Dne 25. 10. 22 v 13:12 Jarek Prokop napsal(a):
The other subthread with Jarek reminded me that one of the options could be to extract/fork the whole Darkfish generator instead of monkey patching. But Darkfish is pretty complex. We would probably not avoided any issues.
Maybe it would be possible to just provide the subclass of Darkfish generator as a single file, perhaps monkey patch the JSON index while we're at it.
Sub-clasing is interesting idea. But that would mean we needed to provide our own generator.
One can provide a different template to a generator and a different generator to a template it seems from messing around with the RDoc CLI and the extracting of the Darkfish template. That leads me to believe that there is a way to do that from code. It would work kind of like this: set the Darkfish template as default, modify the behavior to use symlinks and whatnot. We wouldn't need to keep forked JS/HTML files in a new package.
Please note that my proof of concept provides symlinks.
This would probably minimize required code to get it running.
IOW create a glue between darkfish, json index and RDoc that would make it do what we want to instead of copying it all.
We would then create a tight dependency on those template files, but we can write tests and then check for failures that would crop up via koschei.
Vít
Jarek
Vít
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
On 10/25/22 14:46, Vít Ondruch wrote:
Dne 25. 10. 22 v 13:12 Jarek Prokop napsal(a):
The other subthread with Jarek reminded me that one of the options could be to extract/fork the whole Darkfish generator instead of monkey patching. But Darkfish is pretty complex. We would probably not avoided any issues.
Maybe it would be possible to just provide the subclass of Darkfish generator as a single file, perhaps monkey patch the JSON index while we're at it.
Sub-clasing is interesting idea. But that would mean we needed to provide our own generator.
We will either monkey patch it, requiring us to come up with an idea of how to bring the monkey patches into the default behavior or provide other generator, this seems main 2 options we are working with.
Our own generator with subclassing seems as a "purer" idea, leaving the behavior of the shipped darkfish generator intact. When executing the %gem_install macro, we might be able to invoke the custom generator outright without worrying if the monkey patches are loaded correctly.
One can provide a different template to a generator and a different generator to a template it seems from messing around with the RDoc CLI and the extracting of the Darkfish template. That leads me to believe that there is a way to do that from code. It would work kind of like this: set the Darkfish template as default, modify the behavior to use symlinks and whatnot. We wouldn't need to keep forked JS/HTML files in a new package.
Please note that my proof of concept provides symlinks.
See other email that used your PoC but combined with my approach of subclassing and completely skipping shipping the darfish template (html, js,...) files, the symlinking functionality is preserved (but requires fine tuning as to what will get symlinked).
Jarek
This would probably minimize required code to get it running.
IOW create a glue between darkfish, json index and RDoc that would make it do what we want to instead of copying it all.
We would then create a tight dependency on those template files, but we can write tests and then check for failures that would crop up via koschei.
Vít
Jarek
Vít
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
Hi,
Sub-clasing is interesting idea. But that would mean we needed to provide our own generator.
I am working on a kind of a prototype on this.
Sources with specfile are available here: https://fedorapeople.org/cgit/jackorp/public_git/fedora_darkfish.git/ I also created copr repo that contains the newest build: https://copr.fedorainfracloud.org/coprs/jackorp/fedora_darkfish/packages/
I have been successful with hijacking Rubygems documentation generation via the plugin installed from the copr:
``` [root@33809b4da933 result]# gem install -V --force --document=ri,rdoc c HEAD https://index.rubygems.org/ 200 OK GET https://index.rubygems.org/info/c 200 OK GET https://index.rubygems.org/quick/Marshal.4.8/c-0.0.3.gemspec.rz 200 OK Downloading gem c-0.0.3.gem GET https://index.rubygems.org/gems/c-0.0.3.gem Fetching c-0.0.3.gem 200 OK This is pre_install /usr/local/share/gems/gems/c-0.0.3/.gitignore /usr/local/share/gems/gems/c-0.0.3/Gemfile /usr/local/share/gems/gems/c-0.0.3/LICENSE /usr/local/share/gems/gems/c-0.0.3/README.md /usr/local/share/gems/gems/c-0.0.3/Rakefile /usr/local/share/gems/gems/c-0.0.3/c.gemspec /usr/local/share/gems/gems/c-0.0.3/lib/c.rb /usr/local/share/gems/gems/c-0.0.3/lib/c/version.rb This is post_build This is post_install Successfully installed c-0.0.3 Parsing documentation for c-0.0.3 Parsing sources... 100% [ 2/ 2] lib/c/version.rb Installing ri documentation for c-0.0.3 Installing fedora documentation for c-0.0.3 Done installing documentation for c after 0 seconds 1 gem installed [root@33809b4da933 result]# ls -al /usr/local/share/gems/doc/ total 0 drwxr-xr-x. 1 root root 14 Oct 26 09:43 . drwxr-xr-x. 1 root root 106 Oct 26 09:43 .. drwxr-xr-x. 1 root root 12 Oct 26 09:43 c-0.0.3 [root@33809b4da933 result]# ls -al /usr/local/share/gems/doc/c-0.0.3/rdoc/* -rw-r--r--. 1 root root 2533 Oct 26 09:43 /usr/local/share/gems/doc/c-0.0.3/rdoc/C.html -rw-r--r--. 1 root root 2329 Oct 26 09:43 /usr/local/share/gems/doc/c-0.0.3/rdoc/index.html -rw-r--r--. 1 root root 1178 Oct 26 09:43 /usr/local/share/gems/doc/c-0.0.3/rdoc/table_of_contents.html
/usr/local/share/gems/doc/c-0.0.3/rdoc/css: total 8 drwxr-xr-x. 1 root root 34 Oct 26 09:43 . drwxr-xr-x. 1 root root 98 Oct 26 09:43 .. lrwxrwxrwx. 1 root root 82 Oct 26 09:43 fonts.css -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/css/fonts.css lrwxrwxrwx. 1 root root 81 Oct 26 09:43 rdoc.css -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/css/rdoc.css
/usr/local/share/gems/doc/c-0.0.3/rdoc/images: total 100 drwxr-xr-x. 1 root root 656 Oct 26 09:43 . drwxr-xr-x. 1 root root 98 Oct 26 09:43 .. lrwxrwxrwx. 1 root root 83 Oct 26 09:43 add.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/add.png lrwxrwxrwx. 1 root root 88 Oct 26 09:43 arrow_up.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/arrow_up.png lrwxrwxrwx. 1 root root 85 Oct 26 09:43 brick.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/brick.png lrwxrwxrwx. 1 root root 90 Oct 26 09:43 brick_link.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/brick_link.png lrwxrwxrwx. 1 root root 83 Oct 26 09:43 bug.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bug.png lrwxrwxrwx. 1 root root 92 Oct 26 09:43 bullet_black.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_black.png lrwxrwxrwx. 1 root root 99 Oct 26 09:43 bullet_toggle_minus.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_toggle_minus.png lrwxrwxrwx. 1 root root 98 Oct 26 09:43 bullet_toggle_plus.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_toggle_plus.png lrwxrwxrwx. 1 root root 84 Oct 26 09:43 date.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/date.png lrwxrwxrwx. 1 root root 86 Oct 26 09:43 delete.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/delete.png lrwxrwxrwx. 1 root root 84 Oct 26 09:43 find.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/find.png lrwxrwxrwx. 1 root root 96 Oct 26 09:43 loadingAnimation.gif -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/loadingAnimation.gif lrwxrwxrwx. 1 root root 91 Oct 26 09:43 macFFBgHack.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/macFFBgHack.png lrwxrwxrwx. 1 root root 87 Oct 26 09:43 package.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/package.png lrwxrwxrwx. 1 root root 90 Oct 26 09:43 page_green.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_green.png lrwxrwxrwx. 1 root root 95 Oct 26 09:43 page_white_text.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_white_text.png lrwxrwxrwx. 1 root root 96 Oct 26 09:43 page_white_width.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_white_width.png lrwxrwxrwx. 1 root root 86 Oct 26 09:43 plugin.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/plugin.png lrwxrwxrwx. 1 root root 84 Oct 26 09:43 ruby.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/ruby.png lrwxrwxrwx. 1 root root 88 Oct 26 09:43 tag_blue.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/tag_blue.png lrwxrwxrwx. 1 root root 89 Oct 26 09:43 tag_green.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/tag_green.png lrwxrwxrwx. 1 root root 91 Oct 26 09:43 transparent.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/transparent.png lrwxrwxrwx. 1 root root 86 Oct 26 09:43 wrench.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/wrench.png lrwxrwxrwx. 1 root root 93 Oct 26 09:43 wrench_orange.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/wrench_orange.png lrwxrwxrwx. 1 root root 84 Oct 26 09:43 zoom.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/zoom.png
/usr/local/share/gems/doc/c-0.0.3/rdoc/js: total 20 drwxr-xr-x. 1 root root 118 Oct 26 09:43 . drwxr-xr-x. 1 root root 98 Oct 26 09:43 .. lrwxrwxrwx. 1 root root 83 Oct 26 09:43 darkfish.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/js/darkfish.js lrwxrwxrwx. 1 root root 87 Oct 26 09:43 navigation.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/json_index/js/navigation.js lrwxrwxrwx. 1 root root 81 Oct 26 09:43 search.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/js/search.js -rw-r--r--. 1 root root 106 Oct 26 09:43 search_index.js lrwxrwxrwx. 1 root root 85 Oct 26 09:43 searcher.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/json_index/js/searcher.js ```
I am afraid we won't dodge monkey patching here due to the lack of parametrization of the documentation generator options.
Following, I can uncomment the following line: https://fedorapeople.org/cgit/jackorp/public_git/fedora_darkfish.git/tree/li... which would hopefully enable this only when we are in an RPM build. I do not expect this package to be installed outside of an RPM buildroot for a regular user.
There are some debug prints and leftover code, I'll get to the cleanup later, they help me with at least seeing if the plugin was required and the hooks were registered.
Interestingly enough, I was not successful in monkey patching outside of the Gem hooks (pre_install etc...).
Next possible steps are: * Patch rdoc with this patch https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rdoc-6.4... However I am afraid we might break something...
* Require fonts for rdoc and/or add font requires to doc subpackages. We can maybe add implicit font requires via some already used macros, but that is just an idea so far. * Attempt to unbundle fonts from rubygem-rdoc
* And most importantly: have successful rubygem builds that use symlink in the doc subpackage.
Another thing I am concerned about are the symlinks. Maybe we should split the static darkfish template files into a subpackage, add font requires on that, and have -doc subpackages depend on that, so that we don't pull the whole rdoc when we want just the specific static files.
Feedback is welcomed.
Regards, Jarek
Dne 26. 10. 22 v 11:59 Jarek Prokop napsal(a):
Hi,
Sub-clasing is interesting idea. But that would mean we needed to provide our own generator.
I am working on a kind of a prototype on this.
Sources with specfile are available here: https://fedorapeople.org/cgit/jackorp/public_git/fedora_darkfish.git/ I also created copr repo that contains the newest build: https://copr.fedorainfracloud.org/coprs/jackorp/fedora_darkfish/packages/
I have been successful with hijacking Rubygems documentation generation via the plugin
Interesting. I was against making gem out of this, but the RubyGems plugin mechanism seems to be convincing. This would give us a way to control the behavior just by installing the plugin (and the pluging would be presumably pulled in just via the rubygems-devel). So if the plugin is installed, the custom generator is used, if it is not installed, then the default generator is used.
That seems convincing comparing to modifications of operating_system.rb I was considering previously. This would avoid the (negligible) runtime impact.
I am afraid we won't dodge monkey patching here due to the lack of parametrization of the documentation generator options.
I will need to experiment with this idea a bit.
Following, I can uncomment the following line: https://fedorapeople.org/cgit/jackorp/public_git/fedora_darkfish.git/tree/li... which would hopefully enable this only when we are in an RPM build. I do not expect this package to be installed outside of an RPM buildroot for a regular user.
That is probably not necessary at all. Either it is installed and it does its job or it is not installed. I would not bother with this. KISS.
Interestingly enough, I was not successful in monkey patching outside of the Gem hooks (pre_install etc...).
This might get quickly complex and patch the right thing might not me easy.
Next possible steps are:
- Patch rdoc with this patch
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rdoc-6.4... However I am afraid we might break something...
In worst case, the fonts will be broken in a way that there will be used some different font. Browsers are ready for this.
- Require fonts for rdoc and/or add font requires to doc subpackages.
We can maybe add implicit font requires via some already used macros, but that is just an idea so far.
Well, the whole -doc subpackage is pretty static piece of code. So it could be replaced by some macro, which in turn would need to modify gem2rpm and all packages.
The other option might be custom RPM generator. That might be naively checking path and if there is path such as something like `/usr/share/gems/doc/gem2rpm-1.0.2/rdoc`, just put there the font requires.
Actually, I still prefer if the template was extracted (as in my example), because it is not just about the fonts. So there should not be dependency on the fonts but on the template-asset package.
Another thing I am concerned about are the symlinks. Maybe we should split the static darkfish template files into a subpackage, add font requires on that, and have -doc subpackages depend on that,
Right.
so that we don't pull the whole rdoc when we want just the specific static files.
You are thinking in the right direction ;)
Vít
Dne 26. 10. 22 v 16:50 Vít Ondruch napsal(a):
Dne 26. 10. 22 v 11:59 Jarek Prokop napsal(a):
I am afraid we won't dodge monkey patching here due to the lack of parametrization of the documentation generator options.
I will need to experiment with this idea a bit.
One option to avoid the monkey patching is to convince RubyGems to load the appropriate rubygems/rdoc.rb (but this is done via require_relative, so this probably won't fly) or provide our own rdoc/rubygems_hook and ensure it is loaded earlier then the original one (but this could mean keeping our own copy :/).
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Another option might be to provide additional generator, which would override the 'rdoc' directory generated by Darkfish.
Just ideas .....
Vít
On 10/26/22 18:32, Vít Ondruch wrote:
Dne 26. 10. 22 v 16:50 Vít Ondruch napsal(a):
Dne 26. 10. 22 v 11:59 Jarek Prokop napsal(a):
I am afraid we won't dodge monkey patching here due to the lack of parametrization of the documentation generator options.
I will need to experiment with this idea a bit.
One option to avoid the monkey patching is to convince RubyGems to load the appropriate rubygems/rdoc.rb (but this is done via require_relative, so this probably won't fly) or provide our own rdoc/rubygems_hook and ensure it is loaded earlier then the original one (but this could mean keeping our own copy :/).
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Another option might be to provide additional generator, which would override the 'rdoc' directory generated by Darkfish.
Leaving RDoc as it is and applying patches to the generated documentation might be easier to maintain. The prototype code on cgit does this well, but rather than changing RDoc, it may be better to change the Fedora Ruby documentation build process. In particular:
a) Fonts can be symlinked, there are 652K for each install. b) Standard images should also be symlinked 104K for each install. One can use a script to remove standard images and link them to images from a supplementary Fedora RDoc package. c) Css files similarly can be symlinked. 20K per install.
Finally, it may be good to split the documentation sub package into a commandline sub package with RI files, and web doc sub package with RDoc files. Have a few packages for review where have
Just ideas .....
Vít
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Dne 01. 11. 22 v 5:51 Benson Muite napsal(a):
On 10/26/22 18:32, Vít Ondruch wrote:
Dne 26. 10. 22 v 16:50 Vít Ondruch napsal(a):
Dne 26. 10. 22 v 11:59 Jarek Prokop napsal(a):
I am afraid we won't dodge monkey patching here due to the lack of parametrization of the documentation generator options.
I will need to experiment with this idea a bit.
One option to avoid the monkey patching is to convince RubyGems to load the appropriate rubygems/rdoc.rb (but this is done via require_relative, so this probably won't fly) or provide our own rdoc/rubygems_hook and ensure it is loaded earlier then the original one (but this could mean keeping our own copy :/).
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Another option might be to provide additional generator, which would override the 'rdoc' directory generated by Darkfish.
Leaving RDoc as it is and applying patches to the generated documentation might be easier to maintain. The prototype code on cgit does this well, but rather than changing RDoc, it may be better to change the Fedora Ruby documentation build process. In particular:
a) Fonts can be symlinked, there are 652K for each install.
In theory, this should not be needed if the fonts are installed into system location. In that case, browser should be able to pick them up. This just needs minor .css modification IMHO. However, I am not CSS expert. But I am quite confident anyway ;)
b) Standard images should also be symlinked 104K for each install. One can use a script to remove standard images and link them to images from a supplementary Fedora RDoc package.
Right. However, on top of that, I have my doubts the images are actually used, at least most of them. I think I have found traces about usage of four of them.
c) Css files similarly can be symlinked. 20K per install.
If we don't install the fonts, the license information from .css files could be removed and the size possibly reduced.
Finally, it may be good to split the documentation sub package into a commandline sub package with RI files, and web doc sub package with RDoc files. Have a few packages for review where have
Right, -ri and -rdoc subpackages would be nice as well as having them autogenerated. Keeping the -doc just for doc stuff and possibly have -test subpackage (if I am not mistaken, Perl started to ship -test subpackages for great benefit).
As you can see, if you start to dig into this, you can quickly find more issues which would deserve attention to make things better (including ideas such as [1]). It is not easy to pick a spot where to start to make things better, which results in paralysis and keeping status quo.
But hopefully, we are moving somewhere finally. So thank you for your patience and nudging us to make the improvements.
Vít
[1] https://github.com/rubygems/rubygems/discussions/6021
Just ideas .....
Vít
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
a) Fonts can be symlinked, there are 652K for each install.
In theory, this should not be needed if the fonts are installed into system location. In that case, browser should be able to pick them up. This just needs minor .css modification IMHO. However, I am not CSS expert. But I am quite confident anyway ;)
Ok. Can check this. Then we can just create a post install rubygem specific macro that removes the fonts.
b) Standard images should also be symlinked 104K for each install. One can use a script to remove standard images and link them to images from a supplementary Fedora RDoc package.
Right. However, on top of that, I have my doubts the images are actually used, at least most of them. I think I have found traces about usage of four of them.
Maybe manage fonts first.
c) Css files similarly can be symlinked. 20K per install.
If we don't install the fonts, the license information from .css files could be removed and the size possibly reduced.
Ok. Seems fonts are the first thing to solve.
Finally, it may be good to split the documentation sub package into a commandline sub package with RI files, and web doc sub package with RDoc files. Have a few packages for review where have
Right, -ri and -rdoc subpackages would be nice as well as having them autogenerated. Keeping the -doc just for doc stuff and possibly have -test subpackage (if I am not mistaken, Perl started to ship -test subpackages for great benefit).
Perhaps we can update the packaging guidelines and example template? Then can automate some of this, perhaps starting with gem2rpm is probably.
As you can see, if you start to dig into this, you can quickly find more issues which would deserve attention to make things better (including ideas such as [1]). It is not easy to pick a spot where to start to make things better, which results in paralysis and keeping status quo.
Improving RubyGems RDoc behavior is great, but since this will likely not change the files that are produced, maybe it is easier to start with Fedora?
But hopefully, we are moving somewhere finally. So thank you for your patience and nudging us to make the improvements.
Vít
[1] https://github.com/rubygems/rubygems/discussions/6021
Just ideas .....
Vít
Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a):
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method.
Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO.
BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic.
Vít
[1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36...
On 11/1/22 20:53, Vít Ondruch wrote:
Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a):
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method.
Thanks for this. Is it ok to build upon this incrementally? License information is needed for all included components, even fonts. So an extra license tag should be added to all Ruby packages that produce rdoc documentation if it will be troublesome to unbundle fonts relatively soon. Impact on css is primarily where the fonts are, but css and javascript files are much smaller than the font files.
Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO.
BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic.
Vít
[1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36...
Dne 22. 11. 22 v 9:21 Benson Muite napsal(a):
On 11/1/22 20:53, Vít Ondruch wrote:
Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a):
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method.
Thanks for this. Is it ok to build upon this incrementally?
This would be just distraction. I'll rather finally spend time trying to find the ultimate solution then fixing licenses where this is know issue since forever, but was never really pressing.
And if you really want some quick and dirty solution, then it would be better to just drop the fonts instead of adding license tags. This would result in slightly broken documentation, which would frankly nobody noticed.
Less dirty solution would be to drop the fonts and require them into the -doc packages.
There is even possible to not generate the documentation at all or drop it and not ship it.
As you can see, there is many opportunities for bad workarounds and I don't really want to dig this rabbit hole.
Vít
License information is needed for all included components, even fonts. So an extra license tag should be added to all Ruby packages that produce rdoc documentation if it will be troublesome to unbundle fonts relatively soon. Impact on css is primarily where the fonts are, but css and javascript files are much smaller than the font files.
Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO.
BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic.
Vít
[1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36...
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Building upon this, here is another attempt:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/commit/?h=raw...
https://koji.fedoraproject.org/koji/taskinfo?taskID=94460961
While I have reverted back to non gem approach, the RubyGems plugin hook can still be utilized. I have reverted from the gem approach, because it allows to have the template outside of the RubyGems directory structure, while it does not prohibit to install RubyGems hook.
I have also tried different approaches, such as having full featured RDoc extension, but the RDoc templates are loaded quite late in the process [1], so they would not be available for the RubyGems generator.
This iteration also removes monkey patching in favor of inheritance.
Last but not least, this iteration helps me a bit with clarification of the name of the project. The generators has to have `RDoc::Generator::` prefix, therefore the class is named `RDoc::Generator::Fedora::Darkfish` (and there is also `RDoc::Generator::Fedora::JsonIndex`), therefore I think the package/project should be named fedora-darkfish. I'm still not clear if `rdoc` suffix/prefix would help. While long, maybe `rdoc-generator-fedora-darkfish` prefix would correlate with the class name the best and it would hint to RDoc and Darkfish. Should there be another generators packaged, they could possibly follow similar scheme (while in theory the should have rubygem- prefix 🫣).
PTAL and let me know what do you think about this approach. Thx
Vít
[1] https://github.com/ruby/rdoc/blob/45259f9024b7d08ae2ba119c9a02983bf672f10c/l...
Dne 01. 11. 22 v 18:53 Vít Ondruch napsal(a):
Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a):
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method.
Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO.
BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic.
Vít
[1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36...
Hi,
looks great :), I'll comment more inline.
nit regarding the spec license, "GPL-2.0" is not valid SPDX (according to license-validate at least), however, there is "GPL-2.0-only" or "GPL-2.0-or-later". I am maybe coming too soon with this comment, but since we are also on the SPDX topic in parallel ;).
On 11/23/22 17:47, Vít Ondruch wrote:
Building upon this, here is another attempt:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/commit/?h=raw...
https://koji.fedoraproject.org/koji/taskinfo?taskID=94460961
Would you consider using COPR for these proof-of-concept builds? It makes testing the builds a small bit easier.
I am having trouble making it work, there already is registered generator in the "done_installing_hooks", resulting in hitting both the raises (the second one I hit after commenting out the first one). Secondly (if I ignore both exceptions), I am met with a trace when trying to generate `ri` (default on my `gem install --local`): ~~~ $ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=ri,rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options` Parsing documentation for chronic-0.10.2 ERROR: While executing gem ... (OptionParser::InvalidArgument) invalid argument: Invalid output formatter fedora::ri /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/options.rb:1220:in `setup_generator' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:121:in `document' /usr/share/darkfish-rdoc/rubygems_plugin.rb:30:in `document' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:195:in `generate' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:56:in `block in generation_hook' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:55:in `each' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:55:in `generation_hook' /usr/share/darkfish-rdoc/rubygems_plugin.rb:25:in `generation_hook' /usr/share/rubygems/rubygems/request_set.rb:311:in `block in install_hooks' /usr/share/rubygems/rubygems/request_set.rb:310:in `each' /usr/share/rubygems/rubygems/request_set.rb:310:in `install_hooks' /usr/share/rubygems/rubygems/request_set.rb:209:in `install' /usr/share/rubygems/rubygems/commands/install_command.rb:210:in `install_gem' /usr/share/rubygems/rubygems/commands/install_command.rb:226:in `block in install_gems' /usr/share/rubygems/rubygems/commands/install_command.rb:219:in `each' /usr/share/rubygems/rubygems/commands/install_command.rb:219:in `install_gems' /usr/share/rubygems/rubygems/commands/install_command.rb:167:in `execute' /usr/share/rubygems/rubygems/command.rb:323:in `invoke_with_build_args' /usr/share/rubygems/rubygems/command_manager.rb:185:in `process_args' /usr/share/rubygems/rubygems/command_manager.rb:149:in `run' /usr/share/rubygems/rubygems/gem_runner.rb:51:in `run' /usr/bin/gem:13:in `<main>'
~~~
From `invalid argument: Invalid output formatter fedora::ri` present in the log, I'd say that following line https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rubygems... is too aggressive. We can also provide the corresponding RDoc generator (even if it would be empty subclass).
This succeeds with only generating rdoc, ignoring ri format: ~~~ $ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options` Parsing documentation for chronic-0.10.2 Installing fedora::darkfish documentation for chronic-0.10.2 Done installing documentation for chronic after 1 seconds 1 gem installed ~~~
In regards to tests (%check): What we want to test is: We can generate the documentation in context of Fedora (IOW, `gem install` passes with `--document=rdoc,ri`) and then validate that anything that is an image, or JS file that is not "search_index.js" is symlinked to the template and that there are no fonts present, as those are the places where we differ from the tested upstream behavior.
This would help us catch any major regression that I can think of, at build time, hopefully being a step ahead of accidentally breaking rubygem builds.
Or maybe in addition there should a new sanity test case for the general Ruby integration tests, that we can successfully run the mentioned gem install, maybe even some simple rubygem package build, for the case that the RDoc darkfish template changes in some breaking way during a Ruby update.
While I have reverted back to non gem approach, the RubyGems plugin hook can still be utilized. I have reverted from the gem approach, because it allows to have the template outside of the RubyGems directory structure, while it does not prohibit to install RubyGems hook.
I have also tried different approaches, such as having full featured RDoc extension, but the RDoc templates are loaded quite late in the process [1], so they would not be available for the RubyGems generator.
This iteration also removes monkey patching in favor of inheritance.
Nit to the code, if it is using inheritance, maybe replace the method aliasing with `super' in this file: https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/fedora_d... ?
It seems like it can be simply replaced if I didn't miss anything. That was one of my reasons for inheritance, call to `super' seems simpler.
Last but not least, this iteration helps me a bit with clarification of the name of the project. The generators has to have `RDoc::Generator::` prefix, therefore the class is named `RDoc::Generator::Fedora::Darkfish` (and there is also `RDoc::Generator::Fedora::JsonIndex`), therefore I think the package/project should be named fedora-darkfish. I'm still not clear if `rdoc` suffix/prefix would help. While long, maybe `rdoc-generator-fedora-darkfish` prefix would correlate with the class name the best and it would hint to RDoc and Darkfish. Should there be another generators packaged, they could possibly follow similar scheme (while in theory the should have rubygem- prefix 🫣).
I am a fan of the "rdoc-generator-fedora-darkfish".
RDoc generators could have rubygem-rdoc-generator-* if we would want to mandate prefixes, but that would come with new packaging guidelines, etc... This iteration does not have rubygem-* prefix as it is not a gem, so I would say, this is the odd case :).
PTAL and let me know what do you think about this approach. Thx
I'd say we are on the right path with this.
Regards, Jarek
Vít
[1] https://github.com/ruby/rdoc/blob/45259f9024b7d08ae2ba119c9a02983bf672f10c/l...
Dne 01. 11. 22 v 18:53 Vít Ondruch napsal(a):
Dne 26. 10. 22 v 17:32 Vít Ondruch napsal(a):
We could also try to somehow modify the Gem.done_installing hooks and remove the RDoc hook and replace it with ours.
Building upon this idea, I am attaching an proof of concept. This loads the original RubyGems RDoc hook, removes it an replaces by a custom hook. In this custom hook (which is inherited from the original code), the generator name can be modified from "darkfish" to something completely different and hence load the custom generator (which can inherit from Darkfish). The options could also be potentially modified to change the #setup_generator method.
Another option could be leave the generator as it is and have additional hook, which would run after the default generators and update the generated content. While tempting, I still think the generator/template should be extracted from RDoc and therefore this is less appealing option IMHO.
BTW we could also use the hook to remove the `--document=ri,rdoc` option from `%gem_install` macro [1]. But that would be probably too much magic.
Vít
[1] https://src.fedoraproject.org/rpms/ruby/blob/588a4ae9f02928d7bedbcf46a739d36...
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
Dne 24. 11. 22 v 13:12 Jarek Prokop napsal(a):
Hi,
looks great :), I'll comment more inline.
nit regarding the spec license, "GPL-2.0" is not valid SPDX (according to license-validate at least), however, there is "GPL-2.0-only" or "GPL-2.0-or-later".
I think that it used to be valid identifier at some point in time. This should be fixed in rubygem-rdoc then:
https://src.fedoraproject.org/rpms/rubygem-rdoc/c/5e165eed4822757b49dd3b1219...
but mainly upstream:
https://github.com/ruby/rdoc/issues/924
I am maybe coming too soon with this comment
Maybe just a bit ;)
, but since we are also on the SPDX topic in parallel ;).
Yeah, IC.
On 11/23/22 17:47, Vít Ondruch wrote:
Building upon this, here is another attempt:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/commit/?h=raw...
https://koji.fedoraproject.org/koji/taskinfo?taskID=94460961
Would you consider using COPR for these proof-of-concept builds? It makes testing the builds a small bit easier.
I am having trouble making it work, there already is registered generator in the "done_installing_hooks", resulting in hitting both the raises (the second one I hit after commenting out the first one). Secondly (if I ignore both exceptions), I am met with a trace when trying to generate `ri` (default on my `gem install --local`):
$ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=ri,rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options`
Eh, have I left the message ^^ there? :)
Parsing documentation for chronic-0.10.2 ERROR: While executing gem ... (OptionParser::InvalidArgument) invalid argument: Invalid output formatter fedora::ri /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/options.rb:1220:in `setup_generator' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:121:in `document' /usr/share/darkfish-rdoc/rubygems_plugin.rb:30:in `document' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:195:in `generate' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:56:in `block in generation_hook' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:55:in `each' /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/rubygems_hook.rb:55:in `generation_hook' /usr/share/darkfish-rdoc/rubygems_plugin.rb:25:in `generation_hook' /usr/share/rubygems/rubygems/request_set.rb:311:in `block in install_hooks' /usr/share/rubygems/rubygems/request_set.rb:310:in `each' /usr/share/rubygems/rubygems/request_set.rb:310:in `install_hooks' /usr/share/rubygems/rubygems/request_set.rb:209:in `install' /usr/share/rubygems/rubygems/commands/install_command.rb:210:in `install_gem' /usr/share/rubygems/rubygems/commands/install_command.rb:226:in `block in install_gems' /usr/share/rubygems/rubygems/commands/install_command.rb:219:in `each' /usr/share/rubygems/rubygems/commands/install_command.rb:219:in `install_gems' /usr/share/rubygems/rubygems/commands/install_command.rb:167:in `execute' /usr/share/rubygems/rubygems/command.rb:323:in `invoke_with_build_args' /usr/share/rubygems/rubygems/command_manager.rb:185:in `process_args' /usr/share/rubygems/rubygems/command_manager.rb:149:in `run' /usr/share/rubygems/rubygems/gem_runner.rb:51:in `run' /usr/bin/gem:13:in `<main>'
From `invalid argument: Invalid output formatter fedora::ri` present in the log, I'd say that following line https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rubygems_plugin.rb?h=rawhide&id=487234ef5f64f78291ce767a8a989649ee941c73#n29 is too aggressive. We can also provide the corresponding RDoc generator (even if it would be empty subclass). This succeeds with only generating rdoc, ignoring ri format:
$ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options` Parsing documentation for chronic-0.10.2 Installing fedora::darkfish documentation for chronic-0.10.2 Done installing documentation for chronic after 1 seconds 1 gem installed
Good catch. I was testing plain `--document=rdoc`.
Hm, there should be probably done some early matching, if the `fedora::` prefixed generator is there. Otherwise just move on with whatever was specified.
In regards to tests (%check): What we want to test is: We can generate the documentation in context of Fedora (IOW, `gem install` passes with `--document=rdoc,ri`) and then validate that anything that is an image, or JS file that is not "search_index.js" is symlinked to the template and that there are no fonts present, as those are the places where we differ from the tested upstream behavior.
This would help us catch any major regression that I can think of, at build time, hopefully being a step ahead of accidentally breaking rubygem builds.
I have not spend too much time thinking about %check section yet, but something along these lines seems reasonable.
Or maybe in addition there should a new sanity test case for the general Ruby integration tests, that we can successfully run the mentioned gem install, maybe even some simple rubygem package build, for the case that the RDoc darkfish template changes in some breaking way during a Ruby update.
While I have reverted back to non gem approach, the RubyGems plugin hook can still be utilized. I have reverted from the gem approach, because it allows to have the template outside of the RubyGems directory structure, while it does not prohibit to install RubyGems hook.
I have also tried different approaches, such as having full featured RDoc extension, but the RDoc templates are loaded quite late in the process [1], so they would not be available for the RubyGems generator.
This iteration also removes monkey patching in favor of inheritance.
Nit to the code, if it is using inheritance, maybe replace the method aliasing with `super' in this file: https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/fedora_d... ?
It seems like it can be simply replaced if I didn't miss anything. That was one of my reasons for inheritance, call to `super' seems simpler.
Yes, that was probably just oversight. Will fix this.
Last but not least, this iteration helps me a bit with clarification of the name of the project. The generators has to have `RDoc::Generator::` prefix, therefore the class is named `RDoc::Generator::Fedora::Darkfish` (and there is also `RDoc::Generator::Fedora::JsonIndex`), therefore I think the package/project should be named fedora-darkfish. I'm still not clear if `rdoc` suffix/prefix would help. While long, maybe `rdoc-generator-fedora-darkfish` prefix would correlate with the class name the best and it would hint to RDoc and Darkfish. Should there be another generators packaged, they could possibly follow similar scheme (while in theory the should have rubygem- prefix 🫣).
I am a fan of the "rdoc-generator-fedora-darkfish".
Thx 👍
RDoc generators could have rubygem-rdoc-generator-* if we would want to mandate prefixes, but that would come with new packaging guidelines, etc... This iteration does not have rubygem-* prefix as it is not a gem, so I would say, this is the odd case :).
Lets leave future problems for future us. So far, there is no other RDoc generator and I don't see any other generator coming any time soon, so lets ignore the `rubygem-` prefix for now. This still might end up as a gem after all :)
PTAL and let me know what do you think about this approach. Thx
I'd say we are on the right path with this.
Thx. Next thing to consider is how to enforce the dependency on the -assets. There should probably be included some generator, otherwise we would need to update gem2rpm as well as every gem.
Also, the other question is how to pull this into the build root. Should this be pulled in by rubygems-devel? Shouldn't it even be part of rubygems-devel? Or shouldn't this be opt-in, where one needs to specify this explicitly? Maybe just from the beginning unless we are sure this does not cause more troubles then it solves?
Vít
On 11/24/22 14:20, Vít Ondruch wrote:
Dne 24. 11. 22 v 13:12 Jarek Prokop napsal(a):
nit regarding the spec license, "GPL-2.0" is not valid SPDX (according to license-validate at least), however, there is "GPL-2.0-only" or "GPL-2.0-or-later".
I think that it used to be valid identifier at some point in time. This should be fixed in rubygem-rdoc then:
https://src.fedoraproject.org/rpms/rubygem-rdoc/c/5e165eed4822757b49dd3b1219...
but mainly upstream:
Right
On 11/23/22 17:47, Vít Ondruch wrote:
FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options`
Eh, have I left the message ^^ there? :)
It seems so https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rubygems... :)
This succeeds with only generating rdoc, ignoring ri format:
$ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options` Parsing documentation for chronic-0.10.2 Installing fedora::darkfish documentation for chronic-0.10.2 Done installing documentation for chronic after 1 seconds 1 gem installed
Good catch. I was testing plain `--document=rdoc`.
Hm, there should be probably done some early matching, if the `fedora::` prefixed generator is there. Otherwise just move on with whatever was specified.
Something along those lines. Maybe this: ~~~
|generator = "fedora::#{generator}" if generator == "rdoc"|
~~~
In regards to tests (%check): What we want to test is: We can generate the documentation in context of Fedora (IOW, `gem install` passes with `--document=rdoc,ri`) and then validate that anything that is an image, or JS file that is not "search_index.js" is symlinked to the template and that there are no fonts present, as those are the places where we differ from the tested upstream behavior.
This would help us catch any major regression that I can think of, at build time, hopefully being a step ahead of accidentally breaking rubygem builds.
I have not spend too much time thinking about %check section yet, but something along these lines seems reasonable.
RDoc generators could have rubygem-rdoc-generator-* if we would want to mandate prefixes, but that would come with new packaging guidelines, etc... This iteration does not have rubygem-* prefix as it is not a gem, so I would say, this is the odd case :).
Lets leave future problems for future us. So far, there is no other RDoc generator and I don't see any other generator coming any time soon, so lets ignore the `rubygem-` prefix for now. This still might end up as a gem after all :)
Agreed.
PTAL and let me know what do you think about this approach. Thx
I'd say we are on the right path with this.
Thx. Next thing to consider is how to enforce the dependency on the -assets. There should probably be included some generator, otherwise we would need to update gem2rpm as well as every gem.
I vote for a generator.
Also, the other question is how to pull this into the build root. Should this be pulled in by rubygems-devel? Shouldn't it even be part of rubygems-devel?
To this point: https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec#_319
Line 324: `# Needed for RDoc documentation format generation.`
So perhaps we want it as part of the rubygems-devel. Theoretically, we can sweep it under the rug and add a conditional depending on some RPM build specific env variable in the RubyGems plugin.
Or shouldn't this be opt-in, where one needs to specify this explicitly? Maybe just from the beginning unless we are sure this does not cause more troubles then it solves?
I think we could provide some macro options, but I'd make them opt-out for gems where it does indeed cause troubles?
Or in gems where one wouldn't want documentation.
Something like `%gem_document_options` containing `rdoc,ri` and then we can do `gem install ... --document=%{gem_document_options}` . I think packager should then be able to `%define gem_document_options %{nil}`, which disables documentation as `gem install --document=` does nothing but create the gem's doc directory documentation-wise.
Just throwing ideas right now.
Jarek
Vít
I am back with some updates. You can see the latest version here:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/
and I have also prepared Copr build:
https://copr.fedorainfracloud.org/coprs/vondruch/darkfish/
I have incorporated most of the feedback, except the %check section and I'm still not sure how to pull it into the buildroot. On top of that, I have added the RPM dependency generator.
One caveat is that manually installing some gem via `gem install`, it uses the fedora-darkfish generator. Not sure how big concern this is. I am probably fine with that given that 1) the generator is assumed to be installed just during build 2) the RDoc documentation is not generated by default 3) less duplication might be considered advantage for local documentation.
Vít
Dne 24. 11. 22 v 15:46 Jarek Prokop napsal(a):
On 11/24/22 14:20, Vít Ondruch wrote:
Dne 24. 11. 22 v 13:12 Jarek Prokop napsal(a):
nit regarding the spec license, "GPL-2.0" is not valid SPDX (according to license-validate at least), however, there is "GPL-2.0-only" or "GPL-2.0-or-later".
I think that it used to be valid identifier at some point in time. This should be fixed in rubygem-rdoc then:
https://src.fedoraproject.org/rpms/rubygem-rdoc/c/5e165eed4822757b49dd3b1219...
but mainly upstream:
Right
On 11/23/22 17:47, Vít Ondruch wrote:
FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options`
Eh, have I left the message ^^ there? :)
It seems so https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/tree/rubygems... :)
This succeeds with only generating rdoc, ignoring ri format:
$ gem uninstall chronic; gem install ./chronic-0.10.2.gem --document=rdoc --local Successfully uninstalled chronic-0.10.2 Successfully installed chronic-0.10.2 FedoraRDoc.generation_hook - the `specs` could be modified to include additional `rdoc_options` Parsing documentation for chronic-0.10.2 Installing fedora::darkfish documentation for chronic-0.10.2 Done installing documentation for chronic after 1 seconds 1 gem installed
Good catch. I was testing plain `--document=rdoc`.
Hm, there should be probably done some early matching, if the `fedora::` prefixed generator is there. Otherwise just move on with whatever was specified.
Something along those lines. Maybe this:
|generator = "fedora::#{generator}" if generator == "rdoc"|
In regards to tests (%check): What we want to test is: We can generate the documentation in context of Fedora (IOW, `gem install` passes with `--document=rdoc,ri`) and then validate that anything that is an image, or JS file that is not "search_index.js" is symlinked to the template and that there are no fonts present, as those are the places where we differ from the tested upstream behavior.
This would help us catch any major regression that I can think of, at build time, hopefully being a step ahead of accidentally breaking rubygem builds.
I have not spend too much time thinking about %check section yet, but something along these lines seems reasonable.
RDoc generators could have rubygem-rdoc-generator-* if we would want to mandate prefixes, but that would come with new packaging guidelines, etc... This iteration does not have rubygem-* prefix as it is not a gem, so I would say, this is the odd case :).
Lets leave future problems for future us. So far, there is no other RDoc generator and I don't see any other generator coming any time soon, so lets ignore the `rubygem-` prefix for now. This still might end up as a gem after all :)
Agreed.
PTAL and let me know what do you think about this approach. Thx
I'd say we are on the right path with this.
Thx. Next thing to consider is how to enforce the dependency on the -assets. There should probably be included some generator, otherwise we would need to update gem2rpm as well as every gem.
I vote for a generator.
Also, the other question is how to pull this into the build root. Should this be pulled in by rubygems-devel? Shouldn't it even be part of rubygems-devel?
To this point: https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec#_319
Line 324: `# Needed for RDoc documentation format generation.`
So perhaps we want it as part of the rubygems-devel. Theoretically, we can sweep it under the rug and add a conditional depending on some RPM build specific env variable in the RubyGems plugin.
Or shouldn't this be opt-in, where one needs to specify this explicitly? Maybe just from the beginning unless we are sure this does not cause more troubles then it solves?
I think we could provide some macro options, but I'd make them opt-out for gems where it does indeed cause troubles?
Or in gems where one wouldn't want documentation.
Something like `%gem_document_options` containing `rdoc,ri` and then we can do `gem install ... --document=%{gem_document_options}` . I think packager should then be able to `%define gem_document_options %{nil}`, which disables documentation as `gem install --document=` does nothing but create the gem's doc directory documentation-wise.
Just throwing ideas right now.
Jarek
Vít
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
On 10/24/22 17:42, Vít Ondruch wrote:
So here is my proof of concept:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/
https://koji.fedoraproject.org/koji/taskinfo?taskID=93399794
It can be executed as follows:
<...snip...>
And now a few remarks and questions:
- I hoped that any modifications or code won't be needed. But
unfortunately, due to hardlinks, that does not seems to be the case :/
- I am completely unsure how to name this. So far, I incidentally
went with `darkfish`, `darkfish-rdoc` or `fedora-darkfish` and I am using the naming scheme inconsistently. It could possibly be `rdoc-darkfish`, which would probably reflect that this was extracted from RDoc.
IMO fedora-darkfish. `rdoc-darkfish` makes me think its just the darkfish.
Also there is the -assets subpackage. Is it the right name? It could be e.g. -static.
- Yes, this template was extracted from RDoc and therefore goes with
the RDoc version. Not sure if this is right approach.
We could go with a separate gem that uses darkfish as a base. Following versions is not a bad idea, but it also does not feel necessary.
- Speaking about the version, I am not sure if the package should not
have the version embedded in the name and we could possibly have multiple parallel version installed side by side. I am not sure how compatible the templates are between each other, so this could allow us to update the template without rebuilding everything. But maybe rebuild of everything due to the template change is the right thing.
Not sure I understand fully, what different templates are you talking about that could be different/incompatible?
To the incompatibility I would be careful if there is a chance of accidentally loading fedora-darkfish. With the symlinks, unless there is a change to the JS index generation I do not think we will even need a rebuild. We do not rebuild everything now anyway, no? Also, the templates might be somewhat compatible.
- The template version used to generate to the documentation should
be probably hard required in the -doc subpackages. And of course every -doc subpackage will need to grow the dependency on the darkfish-assets subpackage.
What about a require to the `rubygems-devel`? It is my understanding that package is responsible for bringing in everything needed for not only RPM build (https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec#_312 ).
- The -assets package will pull in two fonts. Source Code Pro and
Lato. (Not)surprigly, the Lato is probably not installed by default, at least I don't have it available on my system.
Not surprisingly, the fonts are completely removed from the package and I have also removed the license notices. If the fonts are not included, it would just provide false information.
Makes sense.
- The fedora_darkfish.rb file monkeypatches the Darkfish generator
(as well as json_index generator, which is internally used by Darkfish). The changes are not unit tested or does not have integration suite. Maybe we should have some code coverage. That could help with answers to (4).
I am doubtful about the test suite. We can try also getting inspired by the Darkfish test suite. From my testing, it is mostly testing that the files are placed correctly AFAICT.
Also, I'd dodge the monkeypatching with this "core" a component, perhaps subclassing is a better idea here.
- The json_index used to generate the .js files, but it also provided
their .js.gz counterparts. We have never removed the "duplication". Since this was inconsistent anyway, I have disable generating the gzipped files altogether. I don't think that browser can pick up the gzipped version if the documentation is opened directly from the file system, but I might be wron.
We can skip shipping .gz files IMO.
- We probably don't want to execute the command I have used above
(although it is certainly one option). Therefore I am thinking about integrating with operating_system.rb. There already is `Gem.rpmbuild?` used to detect RPM build environment. But how to leverage this? Modification to operating_system.rb will likely impact performance of every Ruby execution. Is there different approach? E.g. RubyGems plugin? Or some games with the file name and order of loading, so the monkeypatch would be loaded instead of some other file?
Maybe replace the monkeypatch with subclassing first (the affected scope seems limited, but still...) and go with: 1) gem plugin, 2) get inspired by other plugins: https://github.com/mislav/hanna the README here gives me hope that rdoc can load it from normal loadpaths.
I am not sure what it looks for though, just putting it in some of the load paths would be enough, perhaps we will have to put it into a gem.
As to how to introduce it into the builds, we can adjust gem install's CLI opts for RPM build in macros.rubygems. That is IMHO the correct way to propagate this change into the RPM ecosystem. The documentation subpackage is an artifact of running the gem install the way we run it in the macro.
I am probably at the end of list of issues here. Please give it a try and share your ideas.
Just a note: one of the positives to approach with web-assets and friends that I have been messing around with is that the browser is configured to look there IIRC, giving us the options to skip symlinking for the large part. But maybe if one would like to host it remotely, then that approach won't work.
Is there someone who hosts Fedora provided rubygem documentation on a web server? Is that a use case we care about and want to consider?
Jarek
Cheers,
Vít
Dne 21. 10. 22 v 12:53 Vít Ondruch napsal(a):
And just another anecdote, the hardlinks are there per my request:
https://github.com/ruby/rdoc/issues/186
The only issue is that they probably have almost 0 effect, since to be effective, the owner of the original file have to be the same as the owner of the target file. In our case, the origin is typically owned by "root", while the build system is using the "mockbuild" user. The hardlinks can't work even for user installed gem. The only case when they works is when RDoc is installed via `gem install` as well as the other gems (OTOH, this is always the case for people compiling their own Rubies).
Not sure if I should propose to use just plain old copy instead 🤔
Vít
Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
I guess that RPM can't preserve them.
Vít
Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs
to live in completely separate project to enable us to decouple RDoc updates from the template updates.
- You are using the `--format darkfish`, but wouldn't it be better
to use `--template=NAME` instead?
- I don't think we necessarily need to use the `%{_webassetdir}`
or `%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Vít
Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,
I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.
For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_stat... And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template
To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).
To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory. Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.
I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]). I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.
If you are interested in more details of the implementation, scroll down for a more detailed explanation.
Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.
JFTR, for now I have disabled the test suite as moving the files breaks it. Copying the static files to proper directories and then removing them would be better in general, but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.
Regards, Jarek
[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_stat...
PS: Implementation:
Package: All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem. The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.
Fonts. To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).
However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory. I'll need to take a closer look to see how to solve this.
Images and CSS stylesheets: These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.
Javascript: This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.
Everything is then moved to proper directories to be symlinked later.
RDoc template: Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements. Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.
There is one WIP item which is extraction the paths into a variable in the ERB template [4]. The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.
[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/ [2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ [3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... [4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4... _______________________________________________ ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
Hi,
I took a different approach using Vít's script as a base.
the -T could be specified in code, that is subject for more refinement in the next iteration.
Also, it seems the fonts are symlinked, a `.reject` on the right place might help with that.
~~~
[vagrant@fedora test]$ cat ./hello.rb class Hello; end [vagrant@fedora test]$ RUBYOPT="-I/usr/share/darkfish-rdoc/ -rfedora" rdoc --fmt fedora -T darkfish Parsing sources... 100% [ 1/ 1] hello.rb
Generating Fedora format into /vagrant/result/test/doc...
Files: 1
Classes: 1 (1 undocumented) Modules: 0 (0 undocumented) Constants: 0 (0 undocumented) Attributes: 0 (0 undocumented) Methods: 0 (0 undocumented)
Total: 1 (1 undocumented) 0.00% documented
Elapsed: 0.0s
[vagrant@fedora test]$ cat /usr/share/darkfish-rdoc/fedora.rb require 'rdoc' require 'rdoc/generator/darkfish' require 'rdoc/generator/json_index'
class RDoc::Generator::Fedora < RDoc::Generator::Darkfish RDoc::RDoc.add_generator self BUILTIN_STYLE_ITEMS = BUILTIN_STYLE_ITEMS.reject { |str| str =~ /font/ }.freeze
DESCRIPTION = 'HTML generator, fedora'
def initialize store, options super store, options @json_index = RDoc::Generator::FedoraJsonIndex.new self,options
end
def install_rdoc_static_file source, destination, options # :nodoc: return unless source.exist?
begin FileUtils.mkdir_p File.dirname(destination), **options FileUtils.ln_sf source, destination, **options rescue end end end
class RDoc::Generator::FedoraJsonIndex < RDoc::Generator::JsonIndex def generate super
out_dir = @base_dir + @options.op_dir
Dir.chdir @template_dir do Dir['**/*.js'].each do |source| dest = File.join out_dir, source source = File.realpath source FileUtils.ln_sf source, dest, :verbose => $DEBUG_RDOC end end end
# Don't generate gzipped content. Because we have always shipped also the # original, it just increases the size. def generate_gzipped end end [vagrant@fedora test]$ ls -al doc/* -rw-r--r--. 1 vagrant vagrant 73 Oct 25 12:03 doc/created.rid -rw-r--r--. 1 vagrant vagrant 2465 Oct 25 12:03 doc/Hello.html -rw-r--r--. 1 vagrant vagrant 2331 Oct 25 12:03 doc/index.html -rw-r--r--. 1 vagrant vagrant 1179 Oct 25 12:03 doc/table_of_contents.html
doc/css: total 8 drwxr-xr-x. 1 vagrant vagrant 34 Oct 25 12:03 . drwxr-xr-x. 1 vagrant vagrant 138 Oct 25 12:03 .. lrwxrwxrwx. 1 vagrant vagrant 82 Oct 25 12:03 fonts.css -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/css/fonts.css lrwxrwxrwx. 1 vagrant vagrant 81 Oct 25 12:03 rdoc.css -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/css/rdoc.css
doc/fonts: total 24 drwxr-xr-x. 1 vagrant vagrant 238 Oct 25 12:03 . drwxr-xr-x. 1 vagrant vagrant 138 Oct 25 12:03 .. lrwxrwxrwx. 1 vagrant vagrant 95 Oct 25 12:03 Lato-LightItalic.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/Lato-LightItalic.ttf lrwxrwxrwx. 1 vagrant vagrant 89 Oct 25 12:03 Lato-Light.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/Lato-Light.ttf lrwxrwxrwx. 1 vagrant vagrant 97 Oct 25 12:03 Lato-RegularItalic.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/Lato-RegularItalic.ttf lrwxrwxrwx. 1 vagrant vagrant 91 Oct 25 12:03 Lato-Regular.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/Lato-Regular.ttf lrwxrwxrwx. 1 vagrant vagrant 97 Oct 25 12:03 SourceCodePro-Bold.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/SourceCodePro-Bold.ttf lrwxrwxrwx. 1 vagrant vagrant 100 Oct 25 12:03 SourceCodePro-Regular.ttf -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/fonts/SourceCodePro-Regular.ttf
doc/images: total 100 drwxr-xr-x. 1 vagrant vagrant 656 Oct 25 12:03 . drwxr-xr-x. 1 vagrant vagrant 138 Oct 25 12:03 .. lrwxrwxrwx. 1 vagrant vagrant 83 Oct 25 12:03 add.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/add.png lrwxrwxrwx. 1 vagrant vagrant 88 Oct 25 12:03 arrow_up.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/arrow_up.png lrwxrwxrwx. 1 vagrant vagrant 90 Oct 25 12:03 brick_link.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/brick_link.png lrwxrwxrwx. 1 vagrant vagrant 85 Oct 25 12:03 brick.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/brick.png lrwxrwxrwx. 1 vagrant vagrant 83 Oct 25 12:03 bug.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bug.png lrwxrwxrwx. 1 vagrant vagrant 92 Oct 25 12:03 bullet_black.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_black.png lrwxrwxrwx. 1 vagrant vagrant 99 Oct 25 12:03 bullet_toggle_minus.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_toggle_minus.png lrwxrwxrwx. 1 vagrant vagrant 98 Oct 25 12:03 bullet_toggle_plus.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/bullet_toggle_plus.png lrwxrwxrwx. 1 vagrant vagrant 84 Oct 25 12:03 date.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/date.png lrwxrwxrwx. 1 vagrant vagrant 86 Oct 25 12:03 delete.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/delete.png lrwxrwxrwx. 1 vagrant vagrant 84 Oct 25 12:03 find.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/find.png lrwxrwxrwx. 1 vagrant vagrant 96 Oct 25 12:03 loadingAnimation.gif -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/loadingAnimation.gif lrwxrwxrwx. 1 vagrant vagrant 91 Oct 25 12:03 macFFBgHack.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/macFFBgHack.png lrwxrwxrwx. 1 vagrant vagrant 87 Oct 25 12:03 package.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/package.png lrwxrwxrwx. 1 vagrant vagrant 90 Oct 25 12:03 page_green.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_green.png lrwxrwxrwx. 1 vagrant vagrant 95 Oct 25 12:03 page_white_text.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_white_text.png lrwxrwxrwx. 1 vagrant vagrant 96 Oct 25 12:03 page_white_width.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/page_white_width.png lrwxrwxrwx. 1 vagrant vagrant 86 Oct 25 12:03 plugin.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/plugin.png lrwxrwxrwx. 1 vagrant vagrant 84 Oct 25 12:03 ruby.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/ruby.png lrwxrwxrwx. 1 vagrant vagrant 88 Oct 25 12:03 tag_blue.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/tag_blue.png lrwxrwxrwx. 1 vagrant vagrant 89 Oct 25 12:03 tag_green.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/tag_green.png lrwxrwxrwx. 1 vagrant vagrant 91 Oct 25 12:03 transparent.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/transparent.png lrwxrwxrwx. 1 vagrant vagrant 93 Oct 25 12:03 wrench_orange.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/wrench_orange.png lrwxrwxrwx. 1 vagrant vagrant 86 Oct 25 12:03 wrench.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/wrench.png lrwxrwxrwx. 1 vagrant vagrant 84 Oct 25 12:03 zoom.png -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/images/zoom.png
doc/js: total 20 drwxr-xr-x. 1 vagrant vagrant 118 Oct 25 12:03 . drwxr-xr-x. 1 vagrant vagrant 138 Oct 25 12:03 .. lrwxrwxrwx. 1 vagrant vagrant 83 Oct 25 12:03 darkfish.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/js/darkfish.js lrwxrwxrwx. 1 vagrant vagrant 87 Oct 25 12:03 navigation.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/json_index/js/navigation.js lrwxrwxrwx. 1 vagrant vagrant 85 Oct 25 12:03 searcher.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/json_index/js/searcher.js -rw-r--r--. 1 vagrant vagrant 122 Oct 25 12:03 search_index.js lrwxrwxrwx. 1 vagrant vagrant 81 Oct 25 12:03 search.js -> /usr/share/gems/gems/rdoc-6.4.0/lib/rdoc/generator/template/darkfish/js/search.js ~~~
Hi,
On 7/20/22 12:28, Vít Ondruch wrote:
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs to
live in completely separate project to enable us to decouple RDoc updates from the template updates.
But then we will have to keep up with updates to darkfish and update it by hand should there be some CVE in the template if we won't want to have it part of rubygem-rdoc. Not sure how much of an issue would that be, if that is an acceptable tradeoff, there isn't much of a problem with keeping it as a separate package IMO.
- You are using the `--format darkfish`, but wouldn't it be better to
use `--template=NAME` instead?
As I return to it, I understand the difference between RDoc generator vs template a bit better :).
`--template` is probably a usable approach as long as we don't want to customize some other assets or their paths in the generator's internals too much.
- I don't think we necessarily need to use the `%{_webassetdir}` or
`%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
AFAICT it does not seem to be using JQuery for some time, Darkfish contains its own JS (and CSS) now, which I view as another issue to solve, since we are essentially not only bundling fonts but also the darkfish's JS.
That is the motivation for splitting out the files into these directories. As there are JS/CSS files that come with a doc subpackage, and our motivation is to have HTML viewable in the browser, I followed these guidelines: For CSS: https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ And for JS: https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
So the question is, what are darkfish JS files, that would probably decide which directory will we keep them in.
But it gets even more "interesting". Only some files that are like that can be simply factored out into a single reusable file, since the Darkfish generates a search index for the specific package for the functions, modules and whatnot, which deserves its own directory.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
I feel we would not resolve the issue at hand fully. Moreover, the generator can copy out the fonts provided by RPMs, provided they are installed.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
I think rdoc uses the name of template it pulls from somewhere and specifying path like this does not work. At least not for me locally.
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Hmm... this might either just work with only edits to templates (which would be great) or we will have to modify the generator.
I plan to create a copr (hopefully) this week with some possible approaches. I'll share it here once I have it at least partially running.
Regards, Jarek
Dne 24. 10. 22 v 18:00 Jarek Prokop napsal(a):
Hi,
On 7/20/22 12:28, Vít Ondruch wrote:
Just a few notes from the limited time I spend trying to understand the approach.
- The template can't be subpackage of the rubygem-rdoc. It needs to
live in completely separate project to enable us to decouple RDoc updates from the template updates.
But then we will have to keep up with updates to darkfish
Of course. But on the +side, we will have the freedom to lag behind upstream and not update at all.
and update it by hand should there be some CVE in the template if we won't want to have it part of rubygem-rdoc.
There used to be jQuery, but that was removed several releases ago. I don't think there is high change of CVEs.
Not sure how much of an issue would that be, if that is an acceptable tradeoff, there isn't much of a problem with keeping it as a separate package IMO.
The issue I am afraid is that we can't tell if update of RDoc will break the documentation or not. I'd rather keep the old template and have working documentation then the other way around.
- You are using the `--format darkfish`, but wouldn't it be better
to use `--template=NAME` instead?
As I return to it, I understand the difference between RDoc generator vs template a bit better :).
`--template` is probably a usable approach as long as we don't want to customize some other assets or their paths in the generator's internals too much.
Darkfish is quite huge and it even depend on another generator (json_index). Therefore probably the less we differ, the better.
- I don't think we necessarily need to use the `%{_webassetdir}` or
`%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.
AFAICT it does not seem to be using JQuery for some time
Right. I discovered that afterwards. That is good thing :)
, Darkfish contains its own JS (and CSS) now, which I view as another issue to solve, since we are essentially not only bundling fonts but also the darkfish's JS.
This brings the question if in my proof of concept, there should be some bundled provides or not. But what would they be if Darkfish is not exactly standalone project anymore.
That is the motivation for splitting out the files into these directories. As there are JS/CSS files that come with a doc subpackage, and our motivation is to have HTML viewable in the browser, I followed these guidelines: For CSS: https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/ And for JS: https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
So the question is, what are darkfish JS files, that would probably decide which directory will we keep them in.
The JS files are part of the template and I don't see them being useful for anything else. Although I have not investigated their history ....
But it gets even more "interesting". Only some files that are like that can be simply factored out into a single reusable file, since the Darkfish generates a search index for the specific package for the functions, modules and whatnot, which deserves its own directory.
Yes, right, this one is certainly part of the generated documentation.
- I would not mind to keep the original template including the
bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.
I feel we would not resolve the issue at hand fully. Moreover, the generator can copy out the fonts provided by RPMs, provided they are installed.
Do I understand correctly that your concern apart of the generated documentation is the RDoc package containing itself the copy of the fonts. That is fair concern.
- There is too much changes into the RDoc. If possible, I think it
would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:
/some/path/to/our/template/ - This patch contains our version of Darkfish
/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.
$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/
I think rdoc uses the name of template it pulls from somewhere and specifying path like this does not work. At least not for me locally.
Yes, RDoc are doing some assumptions. Basically the template must be also placed on Ruby `$LOAD_PATH`.
https://github.com/ruby/rdoc/blob/master/lib/rdoc/options.rb#L1241-L1249
And the end result should be:
/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js
But this might not be realistic at all ....
Hmm... this might either just work with only edits to templates (which would be great) or we will have to modify the generator.
This won't work due to RDoc using hard links:
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L584...
Basically, the `FileUtils.ln source, destination, **options` fails when the hardlink is attempted to be created from file owned by root to file owned by mockbuild and therefore it falls back to `FileUtils.cp source, destination, **options`
See the other parts of this thread where I have already elaborated about this.
Vít
I plan to create a copr (hopefully) this week with some possible approaches. I'll share it here once I have it at least partially running.
Regards, Jarek
ruby-sig mailing list --ruby-sig@lists.fedoraproject.org To unsubscribe send an email toruby-sig-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.o... Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
ruby-sig@lists.fedoraproject.org