Dne 24. 10. 22 v 17:42 Vít Ondruch napsal(a):
So here is my proof of concept:
https://fedorapeople.org/cgit/vondruch/public_git/darkfish.git/
https://koji.fedoraproject.org/koji/taskinfo?taskID=93399794
It can be executed as follows:
~~~
$ RUBYOPT="-I/usr/share/darkfish-rdoc/ -rfedora_darkfish" gem install
gem2rpm --doc=ri,rdoc
Fetching gem2rpm-1.0.2.gem
WARNING: You don't have /builddir/bin in your PATH,
gem executables will not run.
Successfully installed gem2rpm-1.0.2
Parsing documentation for gem2rpm-1.0.2
Installing ri documentation for gem2rpm-1.0.2
Installing darkfish documentation for gem2rpm-1.0.2
Done installing documentation for gem2rpm after 0 seconds
1 gem installed
$ ruby -run -e httpd ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/
-p 8000
[2022-10-24 17:17:05] INFO WEBrick 1.7.0
[2022-10-24 17:17:05] INFO ruby 3.1.2 (2022-04-12) [x86_64-linux]
[2022-10-24 17:17:05] INFO WEBrick::HTTPServer#start: pid=34 port=8000
[2022-10-24 17:17:05] INFO To access this server, open this URL in a
browser:
[2022-10-24 17:17:05] INFO
http://127.0.0.1:8000
[2022-10-24 17:17:05] INFO http://[::1]:8000
127.0.0.1 - - [24/Oct/2022:17:17:08 CEST] "GET / HTTP/1.1" 200 3964
- -> /
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/fonts.css
HTTP/1.1" 304 0
http://127.0.0.1:8000/ -> /css/fonts.css
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /css/rdoc.css HTTP/1.1"
304 0
http://127.0.0.1:8000/ -> /css/rdoc.css
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search.js HTTP/1.1"
304 0
http://127.0.0.1:8000/ -> /js/search.js
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/searcher.js
HTTP/1.1" 304 0
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/search_index.js
HTTP/1.1" 200 19687
http://127.0.0.1:8000/ -> /js/search_index.js
http://127.0.0.1:8000/ -> /js/searcher.js
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/darkfish.js
HTTP/1.1" 304 0
http://127.0.0.1:8000/ -> /js/darkfish.js
127.0.0.1 - - [24/Oct/2022:17:17:09 CEST] "GET /js/navigation.js
HTTP/1.1" 304 0
http://127.0.0.1:8000/ -> /js/navigation.js
127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /Gem2Rpm/Helpers.html
HTTP/1.1" 200 13074
http://127.0.0.1:8000/ -> /Gem2Rpm/Helpers.html
127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /js/search_index.js
HTTP/1.1" 304 0
http://127.0.0.1:8000/Gem2Rpm/Helpers.html -> /js/search_index.js
127.0.0.1 - - [24/Oct/2022:17:17:14 CEST] "GET /images/zoom.png
HTTP/1.1" 304 0
http://127.0.0.1:8000/css/rdoc.css -> /images/zoom.png
^C[2022-10-24 17:17:17] INFO going to shutdown ...
[2022-10-24 17:17:17] INFO WEBrick::HTTPServer#start done.
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/js/
total 36
lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 darkfish.js ->
/usr/share/darkfish-rdoc/darkfish/js/darkfish.js
lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 navigation.js ->
/usr/share/darkfish-rdoc/json_index/js/navigation.js
lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 search.js ->
/usr/share/darkfish-rdoc/darkfish/js/search.js
-rw-r--r--. 1 mockbuild mock 19687 Oct 24 17:17 search_index.js
lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 searcher.js ->
/usr/share/darkfish-rdoc/json_index/js/searcher.js
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/css/
total 8
lrwxrwxrwx. 1 mockbuild mock 47 Oct 24 17:17 fonts.css ->
/usr/share/darkfish-rdoc/darkfish/css/fonts.css
lrwxrwxrwx. 1 mockbuild mock 46 Oct 24 17:17 rdoc.css ->
/usr/share/darkfish-rdoc/darkfish/css/rdoc.css
$ ll ~/.local/share/gem/ruby/doc/gem2rpm-1.0.2/rdoc/images/
total 100
lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 add.png ->
/usr/share/darkfish-rdoc/darkfish/images/add.png
lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 arrow_up.png ->
/usr/share/darkfish-rdoc/darkfish/images/arrow_up.png
lrwxrwxrwx. 1 mockbuild mock 50 Oct 24 17:17 brick.png ->
/usr/share/darkfish-rdoc/darkfish/images/brick.png
lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 brick_link.png ->
/usr/share/darkfish-rdoc/darkfish/images/brick_link.png
lrwxrwxrwx. 1 mockbuild mock 48 Oct 24 17:17 bug.png ->
/usr/share/darkfish-rdoc/darkfish/images/bug.png
lrwxrwxrwx. 1 mockbuild mock 57 Oct 24 17:17 bullet_black.png ->
/usr/share/darkfish-rdoc/darkfish/images/bullet_black.png
lrwxrwxrwx. 1 mockbuild mock 64 Oct 24 17:17 bullet_toggle_minus.png
-> /usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_minus.png
lrwxrwxrwx. 1 mockbuild mock 63 Oct 24 17:17 bullet_toggle_plus.png ->
/usr/share/darkfish-rdoc/darkfish/images/bullet_toggle_plus.png
lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 date.png ->
/usr/share/darkfish-rdoc/darkfish/images/date.png
lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 delete.png ->
/usr/share/darkfish-rdoc/darkfish/images/delete.png
lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 find.png ->
/usr/share/darkfish-rdoc/darkfish/images/find.png
lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 loadingAnimation.gif ->
/usr/share/darkfish-rdoc/darkfish/images/loadingAnimation.gif
lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 macFFBgHack.png ->
/usr/share/darkfish-rdoc/darkfish/images/macFFBgHack.png
lrwxrwxrwx. 1 mockbuild mock 52 Oct 24 17:17 package.png ->
/usr/share/darkfish-rdoc/darkfish/images/package.png
lrwxrwxrwx. 1 mockbuild mock 55 Oct 24 17:17 page_green.png ->
/usr/share/darkfish-rdoc/darkfish/images/page_green.png
lrwxrwxrwx. 1 mockbuild mock 60 Oct 24 17:17 page_white_text.png ->
/usr/share/darkfish-rdoc/darkfish/images/page_white_text.png
lrwxrwxrwx. 1 mockbuild mock 61 Oct 24 17:17 page_white_width.png ->
/usr/share/darkfish-rdoc/darkfish/images/page_white_width.png
lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 plugin.png ->
/usr/share/darkfish-rdoc/darkfish/images/plugin.png
lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 ruby.png ->
/usr/share/darkfish-rdoc/darkfish/images/ruby.png
lrwxrwxrwx. 1 mockbuild mock 53 Oct 24 17:17 tag_blue.png ->
/usr/share/darkfish-rdoc/darkfish/images/tag_blue.png
lrwxrwxrwx. 1 mockbuild mock 54 Oct 24 17:17 tag_green.png ->
/usr/share/darkfish-rdoc/darkfish/images/tag_green.png
lrwxrwxrwx. 1 mockbuild mock 56 Oct 24 17:17 transparent.png ->
/usr/share/darkfish-rdoc/darkfish/images/transparent.png
lrwxrwxrwx. 1 mockbuild mock 51 Oct 24 17:17 wrench.png ->
/usr/share/darkfish-rdoc/darkfish/images/wrench.png
lrwxrwxrwx. 1 mockbuild mock 58 Oct 24 17:17 wrench_orange.png ->
/usr/share/darkfish-rdoc/darkfish/images/wrench_orange.png
lrwxrwxrwx. 1 mockbuild mock 49 Oct 24 17:17 zoom.png ->
/usr/share/darkfish-rdoc/darkfish/images/zoom.png
~~~
And now a few remarks and questions:
1) I hoped that any modifications or code won't be needed. But
unfortunately, due to hardlinks, that does not seems to be the case :/
2) I am completely unsure how to name this. So far, I incidentally
went with `darkfish`, `darkfish-rdoc` or `fedora-darkfish` and I am
using the naming scheme inconsistently. It could possibly be
`rdoc-darkfish`, which would probably reflect that this was extracted
from RDoc.
Also there is the -assets subpackage. Is it the right name? It could
be e.g. -static.
3) Yes, this template was extracted from RDoc and therefore goes with
the RDoc version. Not sure if this is right approach.
4) Speaking about the version, I am not sure if the package should not
have the version embedded in the name and we could possibly have
multiple parallel version installed side by side. I am not sure how
compatible the templates are between each other, so this could allow
us to update the template without rebuilding everything. But maybe
rebuild of everything due to the template change is the right thing.
5) The template version used to generate to the documentation should
be probably hard required in the -doc subpackages. And of course every
-doc subpackage will need to grow the dependency on the
darkfish-assets subpackage.
6) The -assets package will pull in two fonts. Source Code Pro and
Lato. (Not)surprigly, the Lato is probably not installed by default,
at least I don't have it available on my system.
Not surprisingly, the fonts are completely removed from the package
and I have also removed the license notices. If the fonts are not
included, it would just provide false information.
7) The fedora_darkfish.rb file monkeypatches the Darkfish generator
(as well as json_index generator, which is internally used by Darkfish).
The other subthread with Jarek reminded me that one of the options could
be to extract/fork the whole Darkfish generator instead of monkey
patching. But Darkfish is pretty complex. We would probably not avoided
any issues.
Vít
The changes are not unit tested or does not have integration suite.
Maybe we should have some code coverage. That could help with answers
to (4).
8) The json_index used to generate the .js files, but it also provided
their .js.gz counterparts. We have never removed the "duplication".
Since this was inconsistent anyway, I have disable generating the
gzipped files altogether. I don't think that browser can pick up the
gzipped version if the documentation is opened directly from the file
system, but I might be wron.
9) We probably don't want to execute the command I have used above
(although it is certainly one option). Therefore I am thinking about
integrating with operating_system.rb. There already is `Gem.rpmbuild?`
used to detect RPM build environment. But how to leverage this?
Modification to operating_system.rb will likely impact performance of
every Ruby execution. Is there different approach? E.g. RubyGems
plugin? Or some games with the file name and order of loading, so the
monkeypatch would be loaded instead of some other file?
I am probably at the end of list of issues here. Please give it a try
and share your ideas.
Cheers,
Vít
Dne 21. 10. 22 v 12:53 Vít Ondruch napsal(a):
> And just another anecdote, the hardlinks are there per my request:
>
>
https://github.com/ruby/rdoc/issues/186
>
> The only issue is that they probably have almost 0 effect, since to
> be effective, the owner of the original file have to be the same as
> the owner of the target file. In our case, the origin is typically
> owned by "root", while the build system is using the "mockbuild"
> user. The hardlinks can't work even for user installed gem. The only
> case when they works is when RDoc is installed via `gem install` as
> well as the other gems (OTOH, this is always the case for people
> compiling their own Rubies).
>
> Not sure if I should propose to use just plain old copy instead 🤔
>
>
> Vít
>
>
>
> Dne 21. 10. 22 v 12:08 Vít Ondruch napsal(a):
>> Just playing with this, the funny thing is that RDoc does not copy
>> the template, but they use hardlinks:
>>
>>
https://github.com/ruby/rdoc/blob/master/lib/rdoc/generator/darkfish.rb#L591
>>
>>
>>
>> I guess that RPM can't preserve them.
>>
>>
>> Vít
>>
>>
>> Dne 20. 07. 22 v 12:28 Vít Ondruch napsal(a):
>>> Just a few notes from the limited time I spend trying to understand
>>> the approach.
>>>
>>> 1) The template can't be subpackage of the rubygem-rdoc. It needs
>>> to live in completely separate project to enable us to decouple
>>> RDoc updates from the template updates.
>>>
>>> 2) You are using the `--format darkfish`, but wouldn't it be better
>>> to use `--template=NAME` instead?
>>>
>>> 3) I don't think we necessarily need to use the `%{_webassetdir}`
>>> or `%{_jsdir}` dirs. OTOH, the template is using jQuery, which
>>> resides in the `%{_jsdir}`, isn't it? But I can't see any reference
>>> to jQuery.
>>>
>>> 4) I would not mind to keep the original template including the
>>> bundled fonts around. The original functionality including usage of
>>> the original template for generating the documentation should be
>>> completely preserved.
>>>
>>> 5) There is too much changes into the RDoc. If possible, I think it
>>> would be better to keep the RDoc in its pristine form. I'd rather
>>> seems some RPM macro to do some replacements if needed. But I still
>>> think that if the template was done smart and it would e.g. already
>>> contained symlinks, so copying of files would be in reality copying
>>> of symlinks, that could save us from the modifications. IOW my
>>> naive idea is:
>>>
>>> /some/path/to/our/template/ - This patch contains our version of
>>> Darkfish
>>>
>>> /some/path/to/template/which/contains/just/symlinks/to/template/ -
>>> As the path says, this on the first look appears to be the
>>> template, but in reality these are just symlinks.
>>>
>>> $ rdoc
>>> --template=/some/path/to/template/which/contains/just/symlinks/to/template/
>>>
>>> And the end result should be:
>>>
>>> ~~~
>>>
>>> /usr/share/gems/doc/rdoc-6.3.2/rdoc/js/darkfish.js ->
>>> /some/path/to/our/template/js/darkfish.js
>>>
>>> ~~~
>>>
>>> But this might not be realistic at all ....
>>>
>>>
>>> Vít
>>>
>>>
>>> Dne 04. 07. 22 v 21:23 Jarek Prokop napsal(a):
>>>> Hi all,
>>>>
>>>> I did some initial work in unbundling the static files and
>>>> adjusting the darkfish template that we can then copy out and use
>>>> for generating Fedora's HTML documentation.
>>>>
>>>> For this I used rubygem-rdoc and rdoc v6.4.0 for protyping.
>>>>
>>>> You can check the spec at:
>>>>
https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/tree/dedup_s...
>>>> And the RDoc sources at:
>>>>
https://github.com/jackorp/rdoc/tree/fedora_doc_template
>>>>
>>>> To build the RDoc ruby package, you need to build the gem from my
>>>> RDoc fork's branch (which is based on RDoc tag v6.4.0).
>>>>
>>>> To test it, pick any project that makes use of Darkfish for
>>>> documentation and run `RPMBUILD=TRUE rdoc --format darkfish` in
>>>> the project's directory.
>>>> Note that the `RPMBUILD` env variable has to be non empty to force
>>>> symlinking, this is a WIP feature to allow for comparing results.
>>>>
>>>> I used the symlink approach, but it has a few shortcomings, that
>>>> need to be worked around (That is WIP see specfile comment [0]).
>>>> I was able to get the docs to load properly, though, the search
>>>> index generation is broken due to the symlink.
>>>>
>>>> If you are interested in more details of the implementation,
>>>> scroll down for a more detailed explanation.
>>>>
>>>> Also, If you have any notes regarding my approach, feel free to
>>>> reach out and we can discuss.
>>>>
>>>> JFTR, for now I have disabled the test suite as moving the files
>>>> breaks it.
>>>> Copying the static files to proper directories and then removing
>>>> them would be better in general,
>>>> but I use this approach for now to ensure I don't have some files
>>>> in improper locations, during the development of the custom
>>>> template, which could hide some bugs.
>>>>
>>>> Regards,
>>>> Jarek
>>>>
>>>> [0]
>>>>
https://src.fedoraproject.org/fork/jackorp/rpms/rubygem-rdoc/blob/dedup_s...
>>>>
>>>> PS:
>>>> Implementation:
>>>>
>>>> Package:
>>>> All static files were moved to rubygem-rdoc-darkfish-static, which
>>>> has more dependencies, such as the replaced fonts and the web
>>>> assets filesystem.
>>>> The %{_jsdir} and %{_webassetdir} are used to comply with Fedora
>>>> Packaging guidelines [1] [2] which seem relevant to this case.
>>>>
>>>> Fonts.
>>>> To unbundle fonts, I removed them completely. Browsers can load
>>>> them if they are present on other system paths (which is taken
>>>> care of by the `Requires:`).
>>>>
>>>> However, this approach has a shortcoming. If a user generates
>>>> documentation using system RDoc, then fonts will be missing from
>>>> the result directory.
>>>> I'll need to take a closer look to see how to solve this.
>>>>
>>>> Images and CSS stylesheets:
>>>> These are moved to %{_webassetdir} and symlinked to the result
>>>> folder with generated documentation.
>>>>
>>>> Javascript:
>>>> This is more complicated, since RDoc uses gzip compression. We
>>>> know it is going to do that, so it is done ahead of time and the
>>>> gzip files are added to gemspec.
>>>>
>>>> Everything is then moved to proper directories to be symlinked
>>>> later.
>>>>
>>>> RDoc template:
>>>> Nothing too complicated. The generation and installation (actually
>>>> symlinking) of static files is basically rewritten to satisfy our
>>>> requirements.
>>>> Most of note are the darkfish.rb lines 631 to 663 that contain
>>>> most of the symlinking logic that is then used [3] when the env
>>>> variable is set.
>>>>
>>>> There is one WIP item which is extraction the paths into a
>>>> variable in the ERB template [4].
>>>> The paths are hardcoded now, but there should be a way to make it
>>>> prettier and not hardcoded, but I pushed that back onto TODO, as
>>>> it is not as pressing for developing the template.
>>>>
>>>> [1]
>>>>
https://docs.fedoraproject.org/en-US/packaging-guidelines/JavaScript/
>>>> [2]
>>>>
https://docs.fedoraproject.org/en-US/packaging-guidelines/Web_Assets/
>>>> [3]
>>>>
https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51...
>>>> [4]
>>>>
https://github.com/jackorp/rdoc/commit/de73f599ae44ad95fa2a404caf3a6bdb51...
>>>> _______________________________________________
>>>> ruby-sig mailing list -- ruby-sig(a)lists.fedoraproject.org
>>>> To unsubscribe send an email to
>>>> ruby-sig-leave(a)lists.fedoraproject.org
>>>> Fedora Code of Conduct:
>>>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>> List Guidelines:
>>>>
https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>> List Archives:
>>>>
https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fedoraprojec...
>>>> Do not reply to spam on the list, report it:
>>>>
https://pagure.io/fedora-infrastructure