On 10/24/22 17:42, Vít Ondruch wrote:
So here is my proof of concept:

https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/

https://koji.fedoraproject.org/koji/taskinfo?taskID=93399794


It can be executed as follows:
<...snip...>


And now a few remarks and questions:

1) I hoped that any modifications or code won't be needed. But unfortunately, due to hardlinks, that does not seems to be the case :/

2) I am completely unsure how to name this. So far, I incidentally went with `darkfish`, `darkfish-rdoc` or `fedora-darkfish` and I am using the naming scheme inconsistently. It could possibly be `rdoc-darkfish`, which would probably reflect that this was extracted from RDoc.
IMO fedora-darkfish. `rdoc-darkfish` makes me think its just the darkfish.

Also there is the -assets subpackage. Is it the right name? It could be e.g. -static.

3) Yes, this template was extracted from RDoc and therefore goes with the RDoc version. Not sure if this is right approach.
We could go with a separate gem that uses darkfish as a base. Following versions is not a bad idea, but it also does not feel necessary.

4) Speaking about the version, I am not sure if the package should not have the version embedded in the name and we could possibly have multiple parallel version installed side by side. I am not sure how compatible the templates are between each other, so this could allow us to update the template without rebuilding everything. But maybe rebuild of everything due to the template change is the right thing.
Not sure I understand fully, what different templates are you talking about that could be different/incompatible?

To the incompatibility I would be careful if there is a chance of accidentally loading fedora-darkfish. With the symlinks, unless there is a change to the JS index generation I do not think we will even need a rebuild. We do not rebuild everything now anyway, no? Also, the templates might be somewhat compatible.

5) The template version used to generate to the documentation should be probably hard required in the -doc subpackages. And of course every -doc subpackage will need to grow the dependency on the darkfish-assets subpackage.
What about a require to the `rubygems-devel`? It is my understanding that package is responsible for bringing in everything needed for not only RPM build (https://src.fedoraproject.org/rpms/ruby/blob/rawhide/f/ruby.spec#_312 ).

6) The -assets package will pull in two fonts. Source Code Pro and Lato. (Not)surprigly, the Lato is probably not installed by default, at least I don't have it available on my system.

Not surprisingly, the fonts are completely removed from the package and I have also removed the license notices. If the fonts are not included, it would just provide false information.
Makes sense.

7) The fedora_darkfish.rb file monkeypatches the Darkfish generator (as well as json_index generator, which is internally used by Darkfish). The changes are not unit tested or does not have integration suite. Maybe we should have some code coverage. That could help with answers to (4).

I am doubtful about the test suite. We can try also getting inspired by the Darkfish test suite. From my testing, it is mostly testing that the files are placed correctly AFAICT.

Also, I'd dodge the monkeypatching with this "core" a component, perhaps subclassing is a better idea here.


8) The json_index used to generate the .js files, but it also provided their .js.gz counterparts. We have never removed the "duplication". Since this was inconsistent anyway, I have disable generating the gzipped files altogether. I don't think that browser can pick up the gzipped version if the documentation is opened directly from the file system, but I might be wron.
We can skip shipping .gz files IMO.

9) We probably don't want to execute the command I have used above (although it is certainly one option). Therefore I am thinking about integrating with operating_system.rb. There already is `Gem.rpmbuild?` used to detect RPM build environment. But how to leverage this? Modification to operating_system.rb will likely impact performance of every Ruby execution. Is there different approach? E.g. RubyGems plugin? Or some games with the file name and order of loading, so the monkeypatch would be loaded instead of some other file?

Maybe replace the monkeypatch with subclassing first (the affected scope seems limited, but still...) and go with: 1) gem plugin, 2) get inspired by other plugins: https://github.com/mislav/hanna the README here gives me hope that rdoc can load it from normal loadpaths.

I am not sure what it looks for though, just putting it in some of the load paths would be enough, perhaps we will have to put it into a gem.

As to how to introduce it into the builds, we can adjust gem install's CLI opts for RPM build in macros.rubygems. That is IMHO the correct way to propagate this change into the RPM ecosystem.
The documentation subpackage is an artifact of running the gem install the way we run it in the macro.



I am probably at the end of list of issues here. Please give it a try and share your ideas.

Just a note: one of the positives to approach with web-assets and friends that I have been messing around with is that the browser is configured to look there IIRC, giving us the options to skip symlinking for the large part.
But maybe if one would like to host it remotely, then that approach won't work.

Is there someone who hosts Fedora provided rubygem documentation on a web server? Is that a use case we care about and want to consider?

Jarek



Cheers,


Vít


Dne 21. 10. 22 v 12:53 Vít Ondruch napsal(a):
And just another anecdote, the hardlinks are there per my request:

https://github.com/ruby/rdoc/issues/186

The only issue is that they probably have almost 0 effect, since to be effective, the owner of the original file have to be the same as the owner of the target file. In our case, the origin is typically owned by "root", while the build system is using the "mockbuild" user. The hardlinks can't work even for user installed gem. The only case when they works is when RDoc is installed via `gem install` as well as the other gems (OTOH, this is always the case for people compiling their own Rubies).

Not sure if I should propose to use just plain old copy instead 🤔


Vít



Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
Just playing with this, the funny thing is that RDoc does not copy the template, but they use hardlinks:

https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591


I guess that RPM can't preserve them.


Vít


Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
Just a few notes from the limited time I spend trying to understand the approach.

1) The template can't be subpackage of the rubygem-rdoc. It needs to live in completely separate project to enable us to decouple RDoc updates from the template updates.

2) You are using the `--format darkfish`, but wouldn't it be better to use `--template=NAME` instead?

3) I don't think we necessarily need to use the `%{_webassetdir}` or `%{_jsdir}` dirs. OTOH, the template is using jQuery, which resides in the `%{_jsdir}`, isn't it? But I can't see any reference to jQuery.

4) I would not mind to keep the original template including the bundled fonts around. The original functionality including usage of the original template for generating the documentation should be completely preserved.

5) There is too much changes into the RDoc. If possible, I think it would be better to keep the RDoc in its pristine form. I'd rather seems some RPM macro to do some replacements if needed. But I still think that if the template was done smart and it would e.g. already contained symlinks, so copying of files would be in reality copying of symlinks, that could save us from the modifications. IOW my naive idea is:

/some/path/to/our/template/ - This patch contains our version of Darkfish

/some/path/to/template/which/contains/just/symlinks/to/template/ - As the path says, this on the first look appears to be the template, but in reality these are just symlinks.

$ rdoc --template=/some/path/to/template/which/contains/just/symlinks/to/template/

And the end result should be:

~~~

/usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js -> /some/path/to/our/template/js/darkfish.js

~~~

But this might not be realistic at all ....


Vít


Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
Hi all,

I did some initial work in unbundling the static files and adjusting the darkfish template that we can then copy out and use for generating Fedora's HTML documentation.

For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.

You can check the spec at: https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_static_files
And the RDoc sources at: https://github.com/jackorp/rdoc/tree/fedora_doc_template

To build the RDoc ruby package, you need to build the gem from my RDoc fork's branch (which is based on RDoc tag v6.4.0).

To test it, pick any project that makes use of Darkfish for documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in the project's directory.
Note that the `RPMBUILD` env variable has to be non empty to force symlinking, this is a WIP feature to allow for comparing results.

I used the symlink approach, but it has a few shortcomings, that need to be worked around (That is WIP see specfile comment [0]).
I was able to get the docs to load properly, though, the search index generation is broken due to the symlink.

If you are interested in more details of the implementation, scroll down for a more detailed explanation.

Also, If you have any notes regarding my approach, feel free to reach out and we can discuss.

JFTR, for now I have disabled the test suite as moving the files breaks it.
Copying the static files to proper directories and then removing them would be better in general,
but I use this approach for now to ensure I don't have some files in improper locations, during the development of the custom template, which could hide some bugs.

Regards,
Jarek

[0] https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_static_files/f/rubygem-rdoc.spec#_109

PS:
Implementation:

Package:
All static files were moved to rubygem-rdoc-darkfish-static, which has more dependencies, such as the replaced fonts and the web assets filesystem.
The %{_jsdir} and %{_webassetdir} are used to comply with Fedora Packaging guidelines [1] [2] which seem relevant to this case.

  Fonts.
  To unbundle fonts, I removed them completely. Browsers can load them if they are present on other system paths (which is taken care of by the `Requires:`).

  However, this approach has a shortcoming. If a user generates documentation using system RDoc, then fonts will be missing from the result directory.
  I'll need to take a closer look to see how to solve this.

  Images and CSS stylesheets:
  These are moved to %{_webassetdir} and symlinked to the result folder with generated documentation.

  Javascript:
  This is more complicated, since RDoc uses gzip compression. We know it is going to do that, so it is done ahead of time and the gzip files are added to gemspec.

  Everything is then moved to proper directories to be symlinked later.

RDoc template:
Nothing too complicated. The generation and installation (actually symlinking) of static files is basically rewritten to satisfy our requirements.
Most of note are the darkfish.rb lines 631 to 663 that contain most of the symlinking logic that is then used [3] when the env variable is set.

There is one WIP item which is extraction the paths into a variable in the ERB template [4].
The paths are hardcoded now, but there should be a way to make it prettier and not hardcoded, but I pushed that back onto TODO, as it is not as pressing for developing the template.

[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
[2] https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/
[3] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4425#diff-ddeb55726f649cf3320f376f40db7ca34e75cc0f53765ca8fcef20ca35161bf6R631
[4] https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51fa4425#diff-75d0842deb485ab1e0116eb9e2d8257ab952026dac7e541380a254539eb04fa8L10
_______________________________________________
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org
To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

_______________________________________________
ruby-sig mailing list -- ruby-sig@lists.fedoraproject.org
To unsubscribe send an email to ruby-sig-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraproject.org
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue