Hi and very much thank you for looking into this issue.
Dne 10. 08. 23 v 18:24 Jarek Prokop napsal(a):
Hi,
I recently had an idea of faking load paths [0] to satisfy bundler's
loading and help bring our behavior closer to upstream while still
unbundling the libraries.
To automatically test for the usual user's workflow with bundler and a
default gem (only RDoc so far),
I have created small TMT test suite [1]. It pulls the Ruby RPMs from
my copr repo
https://copr.fedorainfracloud.org/coprs/jackorp/ruby-rdoc-default-gem/ .
It is adjustable to fit your own copr if you wish to do your own
changes [2], TMT can also install RPMs from a local directory, but
that's going off-topic.
One of my TODO item here is to validate this against some rbenv Ruby
install.
To the loading itself and my examination.
Default gems seem to skip bundler somewhere and go their own path
which is more similar to regular `Kernel.require` without bundler.
Therefore, just having it on the default load path, where default gems
are expected to exist is enough.
However, regular Ruby loading also means that it expects a certain
layout that it won't search for itself if it can't find a subfile, see
Rake and `rdoc/task` below.
So, if we place `rdoc.rb` into `/usr/share/ruby/` that adjusts the
load path accordingly and then just passes on the require, that's
enough if the code is either
~~~
$ cat foo.rb
require 'rdoc'
p RDoc::VERSION # Demonstrate that RDoc is available
~~~
and run with bundler `bundle exec ruby ./foo.rb`.
or in CLI like this: `bundle exec ruby -rrdoc -e 'p RDoc::VERSION'`.
In both cases, the associated Gemfile is empty. No gems are recored in
the file.
My approach, at this point in time, falls apart with Rake and RDoc
using `require rdoc/task`, as there is no
/usr/share/ruby/rdoc/task.rb. I am unable to track down the exact code
path,
but it seems to skip some places inside bundler, even rubygems, that
could otherwise make it possible to work (with a few minor adjustments
of course).
At this place it is necessary to understand what Bundler actually does.
Please note that it is possible to run not only `bundle exec rdoc`, but
it also is possible to run `bundle exec bash` and there are several
interesting environment variables injected by Bundler:
~~~
$ bundle exec bash
[mockbuild@fbbdeb5a66084b7eae24c24f33923368 ~]$ env | sort
BUNDLER_ORIG_BUNDLER_SETUP=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_BUNDLER_VERSION=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_BUNDLE_BIN_PATH=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_BUNDLE_GEMFILE=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_GEM_HOME=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_GEM_PATH=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_MANPATH=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
BUNDLER_ORIG_RB_USER_INSTALL=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_RUBYLIB=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_ORIG_RUBYOPT=BUNDLER_ENVIRONMENT_PRESERVER_INTENTIONALLY_NIL
BUNDLER_SETUP=/usr/share/gems/gems/bundler-2.4.10/lib/bundler/setup
BUNDLER_VERSION=2.4.10
BUNDLE_BIN_PATH=/usr/share/gems/gems/bundler-2.4.10/libexec/bundle
BUNDLE_GEMFILE=/builddir/Gemfile
... snip ...
RUBYLIB=/usr/share/gems/gems/bundler-2.4.10/lib
RUBYOPT=-r/usr/share/gems/gems/bundler-2.4.10/lib/bundler/setup
... snip ...
~~~
As you can see, Bundler places itself on Ruby `$LOAD_PATH` via
`$RUBYLIB` and it loads itself via `$RUBYOPT`. Now should also be clear,
that you actually don't have to run `bundle exec`, it is enough to setup
the right env variables.
If I am not mistaken, historically Bundler already at this stage
injected the load paths via `RUBYLIB`. Apparently, it does not doing so
anymore (I think that is probably because they are handling too many
corner cases).
Anyway, what `bundler/setup` does at this stage is that it populates the
Ruby `$LOAD_PATH` (and loads the libraries, unless there is `:require =>
false`, but this is not really important). As a last step, Bundler
actually unloads the customized `require` method and resorts back to the
plain Ruby (!!!) version of require. Not even the RubyGems loader is
active. Nothing. So what is not on load path is not loaded.
These are references to the key steps:
https://github.com/rubygems/rubygems/blob/556349fd863ba586930804140fbcdd6...
https://github.com/rubygems/rubygems/blob/556349fd863ba586930804140fbcdd6...
This demostrated a problem, we'd have to fake out the whole
structure
of any gem we'd like to unbundle this way.
That, of course, is a possibility.
Given the above, I think this is necessity.
So the pros / cons of this approach currently seem:
+ we'd unbundle default gems away
+ no more symlinks.
+ closer to upstream behavior for the users
- We'd have to retain the structure and provide each "fake" path to a
file.
* Though we are doing something a small bit similar with the symlinks.
- added complexity of writing correct paths into the files on their
correct places.
One cons is that still, the `$LOAD_PATH` with this approach will be
different to what would be the default.
Alternatively, this is just an academic idea, populate the default
load path with default gems.
I, however, do not know where would the right place for that be, or
what would be the cons of this approach
(apart from potentially breaking expectations set accross the whole
Ruby environment for loadables).
Not sure about this. We would need to ensure that the paths has lower
priority in `$LOAD_PATH`.
Also to adjust the Kernel.require, but that feels very wrong (also
potentially unsafe, as the code is in C AFAICT, and very divergent
from upstream).
Just given the two links above, this is asking for even more troubles.
But I have quickly dismissed these as I have high doubts such patches
would ever be accepted upstream and maintainability of such approaches
is quetionable at best.
Additionally, I am not sure what should be the behaviour with the
plain rdoc executable and bundler executed like this: `bundle exec rdoc`
From the `env` output you can see, that at minimum, Bundler preserves
some of the original env variables, such as `BUNDLER_ORIG_PATH`, because
it obviously can modify `$PATH` env variable. It can also influence
`MANPATH`:
https://github.com/rubygems/rubygems/blob/556349fd863ba586930804140fbcdd6...
I was not aware of this behavior and it seems to complicate the matter :/
BTW you are using `find` and what not to create the stubs, but it could
be possible to use the already built Ruby and use them to e.g. evaluate
the .gemspec files to get the list of gem files. Or even try to identify
the unbundled default gems.
Vít
I will try to apply the approach of faking the whole gem directory
and
file tree in /usr/share/ruby for RDoc and record my findings.
I am open to discussion, should you want to share your thoughts or
ideas regarding this topic.
Regards,
Jarek Prokop
[0]
https://src.fedoraproject.org/fork/jackorp/rpms/ruby/c/1f660500090ad6d176...
[1]
https://gitlab.com/jackorp/test_default_gem_fedora_ruby
[2]
https://gitlab.com/jackorp/test_default_gem_fedora_ruby/-/blob/7658195644...
_______________________________________________
ruby-sig mailing list --ruby-sig(a)lists.fedoraproject.org
To unsubscribe send an email toruby-sig-leave(a)lists.fedoraproject.org
Fedora Code of
Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List
Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines
List
Archives:https://lists.fedoraproject.org/archives/list/ruby-sig@lists.fed...
Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue