On Thu, Mar 16, 2017 at 6:58 AM, Tomasz Kłoczko
<kloczko.tomasz(a)gmail.com> wrote:
On 16 March 2017 at 04:50, Nico Kadel-Garcia <nkadel(a)gmail.com> wrote:
[..]
>
> > And one more clarification: remove static libraries from glibc distro
> > packages does not blocks static linking.
> > It will only removes possibility linking against static glibc libraries.
>
> Yes. This is a blocker for some people, who want *completely* static
> binaries to have complete binary control of their active package.
> glibc changing from a random update and possibly introducing a
> regression problem is anathema to some critical software developers,
> and the compilation of completely static binaries has been helpful for
> cross-platform compatibility, for building chroot cages, and for
> building well managed containers of various sorts.
There are few typical scenarios when someone may want to have 100% static
binaries:
1) to have binary independent from ABI changes on distribution level in any
used shared libraries.
And regression errors not present in the version of the library they
built and tested the software with. This is a *very big deal*. Been
there, done that, got the T-shirt. Having provenance for your
software, and certainty that library updates will not affect anything
but the expected software is critical. Not protecting critical,
high-availability from software patches or updates to the rest of the
operating system isa very, very big deal for some of us.
So here is kind contradiction because my past experience that such
binaries
are used so long (+6 years) that it causes silent issues with conflicts on
kernel<->user space and sooner or later initial intention turns into
disaster as no one remembers who and how initially such binary was made.
That is also a problem. It's also not really solved by dynamic
libraries. if the ABI for the libraries is changing that much, it's
likely necessary to recompile the base software *anyway*.
2) some bootstrapping scenarios like for example static linking grub
binaries
In such cases binaries are will be regenerated with every small change in
non-public ABI/API changes will be followed by immediate recompilation of
such binaries so ris here is effective null and such limited number binaries
should be accepted and carefully maintained.
And... we're to build them without glibc-static.... how?
Scenario when such special binaries been crafted for initrd already
are
nullified because today even smallest systems have enough memory to use
regular shared libraries. Simple one one needs today to fit such initrds on
3.5' floppy disk with 1.44MB available storage.
Boot media are one of those cases where binary stability are *most* critical.
3) some people are thinking that static linking make sense from
performance
or resource consumption perspective.
Here is the issue that such binaries would be typically moved/propagated
around to different location executed in multiple instances. What was
initial intention of saving memory typically turns into higher memory
consumption. As long as such binaries will be detached from original
locations installed by regular locations risk that those binaries will be
used longer to expose them to risk created by kernel<->user space ABI
changes and internal glibc is relatively high.
Straw man argument, 5 yard penalty.
The relocation of binaries to different places in the same operating
system is not the problem. Moving it from one operating sytem or
container to another, *that* is the problem. And running exactly the
version you expect, even if it's inside a heavily modified container
or distinct operating system of some sort is a powerful goal. I don't
run into it as much as I used to, but I certainly used to have to
compile components statically in various environments for use
elsewhere.
> You seem to pointing out that NSS is a stability problem.
You're quite
> right. Saying that "NSS is unstable, therefore glibc should be forced
> to dynamic libraries only" does not follow. The underlying API for
> nsswitch.conf and NSS does not change that quickly, it's the feature
> churn for add-ons that are being tied into NSS. For high stability
> software, *who cared*? You won't use the most recent NSS changes, and
> if you do, they're quite likely to be available the the glibc static
> library at the time you compile them. Statically.
You are wrong that this is abut messiness/stability of the NSS interface.
We are talking about internal glibc ABI/API on which none of the
system/distribution binaries should rely on such internal interfaces which
any project maintainer has freedom to change without even noticing this in
changelogs.
If they're dinking with it that much, it's likely to lead to changes
in the ABI as well and force recompilation. That level of change is
not normally permitted within a single Fedora release, and packages
for Fedora are rebuilt for new releases.
Again, straw man argument, and another 5 yeard penalty.
Using statically linked binaries creates RISK here if those binaries
will be
not refreshed.
Now, *that* is real. However, see above, to wit:
* Packages for Fedora are rebuilt for new releases.
If you install a package from a previous Fedora release, or run it on
your newer Fedora release, well, you have a risk of trouble. Mind you,
I've *done* so when build environments for older software didn't work
for new releases. It's more common with highly proprietary software
whose authors tended to "tune" their performance at the cost of
portability.
As long as number of such binaries will be limited and will be under
regular
distro hood control I have no problems with such binaries.
As long as whole distribution consumer may start using such internal ABI
interface we are entering on area where sooner or later initial honest
intention will turn against someone who had such intention.
Here is really end of the story if you will really accept meaning,
consequences and existence of internal libraries APIs/ABIs. This is why
executable binaries have public symbols tables. NSS is not part of the
public interface.
Risk of not to be exposed on internal NSS ABI changes can be very easy
nullified by using in such rare cases static linking with libc like uClibc
which has no NSS interface.
And if I walk on only my right foot, I can save wear and tear on my
left shoe. Doesn't seem worth the effort. uClibc is a cool idea, but
unlikely to work well for a broad variety of complex software which
has ever been compiled or tested with it.
In last decade bash already created enough number CVEs to start
thinking
about moving away from bash as /bin/sh provider.
Sun/Oracle already done careful ksh security review which bash never had
done. Because ksh is very small and his main goal is provide /bin/sh number
of any future changes will be very well known and limited to cleanups and
bug fixes.
And we can get C++ programmers to going back to C, to avoid the
function overloading problem.
I'm afraid it's not gonna happen.
Changing /bin/sh to real SH interpreter IMO should be kind of long
term
Fedora target. It will be not easy and maybe even painful but minimize
/bin/sh dependencies and minimize security risk is IMO worth to start
thinking about some preparations to be opened on such change in the future.
Other issue is that bash is not the fastest /bin/sh interpreter :)
It's a lot of work with a profound loss of shell functionality and
features, with a nebulous security benefit. Sorry, but I don't see it
happening anytime in the foreseeable future.
Again: minimize /bin/sh dependencies. Minimize number of other
dependencies.
Here are laying biggest deposits of minimize such risks (waaay bigger than
those related to static linking).
Small example.
Long time ago I've been able to gain minimize number of dependencies by
injecting LDFLAGS="-Wl,--as-needed" into %configure macros. As on mean time
cmake emerged this move will be not so effective as it was decade ago. Today
I think that better solution could apply small change in ld default behavior
to use by default ---as-need (and make -fno-as-needed optional).
This could be done in one few lines patch (few lines .. because it would be
good IMO add printing warning that linking with some libraries was dropped).
Result would be reduction whole distribution rpm REQUIRES entries by at
least 10-20%.
Now, *THAT* is an interesting point. I like it. Want to toss in a
feature request for it?
For example Fedora rawhide still is in the middle openssl migration
to
1.1.x. If before this migration all packages would be linked with --as-need
it will be necessary to rebuild much less packages.
Printing by ld warnings may allow fix original build code by remove
redundant -l<foo> in those frameworks implementations.
Few weeks ago I've done short experiment with injecting
LDFLAGS="-Wl,--as-need" into %configure parameters of the gnome-shell.spec.
After producing two binary packages with and without -Wl,--as-need here is
the diff of the output of the queries:
$ rpm -qp --qf "[%{REQUIRENAME} %{REQUIREFLAGS:depflags}
%{REQUIREVERSION}\n]" gnome-shell-3.2*.fc26.x86_64.rpm
--- gnome-shell.cur 2016-09-18 12:51:51.496412774 +0100
+++ gnome-shell.new 2016-09-19 20:16:59.613362311 +0100
@@ -62,10 +62,6 @@
libgthread-2.0.so.0()(64bit)
libgtk-3.so.0()(64bit)
libical.so.2()(64bit)
-libicalss.so.2()(64bit)
-libicalvcal.so.2()(64bit)
-libicui18n.so.57()(64bit)
-libicuuc.so.57()(64bit)
Ye gods, yes, this kind of dependency improvement is invaluable.
libinput.so.10()(64bit)
libjson-glib-1.0.so.0()(64bit)
libm.so.6()(64bit)
@@ -96,7 +92,6 @@
libsecret-1.so.0()(64bit)
libsoup-2.4.so.1()(64bit)
libstartup-notification-1.so.0()(64bit)
-libstdc++.so.6()(64bit)
That it includes libstdc++ is fascinating.
Back to linker optimisations.
I know that in some extreme packages cases using linker with --as-need can
halven number of SONAME package dependencies.
I *like* it!!!
In other words all those goals which you mention are not strictly
related to
static linking and it is possible to do a lot (or much more) to minimize
such risks without even thinking about provide glibc-static. Static linking
Umm. No. You're mistaking "reduction of library dependencies" with
"discaarding static library compilation".
with glibc touches at the moment only few packages. Reduction of
SONAME
dependencies will be related probably to ~3/4 of all Fedora packages so we
are talking about ~15k packages.
Do you see how few lines change cay change few magnitude orders more? (if
you are really care about regression problems/risks coming from
dependencies).
As I said, i *like* this.
Spreading ld with a bit different behavior printing warnings about
overuse
some -l<foo> in longer consequences will not only improve Fedora binaries
health but as long as source code maintainers of the packages used by Fedora
will start use changed ld they will see that they can improve linker
settings of own projects.
Such subjects should be consulted with binutils maintainers because they
would able to asses risk of doing such change.
As long as I've done many times full recompilation on the scale of whole
distribution in the past I know that sometimes is necessary to add small
linker fix when library X is linked with other libraries A and B and only A
is used and linked executable is using libX and libB ABI but has no -lB in
linker options, but majority of those changes have been already merged to
most of the packages source trees.
*Cool*. And you've my sympathies, those kinds of subtle undetected
dependencies can be tricky to resolve.
> kloczek
> --
> Tomasz Kłoczko | LinkedIn:
http://lnkd.in/FXPWxH
>
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>