Forgot to add this was measured on my Lenovo T440s, with SSD and 12 GB RAM, running F21 with mock 1.2.3, configured to use LVM plugin and DNF (see attached config files). And network? Who knows, probably fast. Should be 1Gb/s if I am not mistaken.

Not sure how similar/different the tests could look on Fedoras Koji builders.


Vít



Dne 13.1.2015 v 15:56 Vít Ondruch napsal(a):
Dne 13.1.2015 v 14:02 Richard W.M. Jones napsal(a):
On Tue, Jan 13, 2015 at 01:31:06PM +0100, Sven Lankes wrote:
On Tue, Jan 13, 2015 at 12:16:42PM +0000, Richard W.M. Jones wrote:

That's simply an idiotic thing to say.  What is the progress involved
in adding a new BR to thousands of packages?
Am I wrong assuming that it would be rather easy to add the BR to all
packages semi-automatically previous to a mass rebuild so that people
who want to get rid of gcc for their packages can easily remove it?

What am I missing?
What is missing is there is no quantifcation yet of the benefit.
How much faster will builds be?

Since you are asking I did 5 times:

$ mock -r minimal --scrub all && time mock -r minimal --init

Where the "Total" is DNF's output of package download speed and amount of downloaded data obtained from mock output. Other DNF information are shown on top of each section as well.


== with current minimal build root, i.e. "install @buildsys-build"

Install  164 Packages

Total download size: 119 M
Installed size: 445 M

1)
Total                                            26 MB/s | 119 MB     00:04
real    0m51.364s
user    0m33.050s
sys    0m6.454s

2)
Total                                            13 MB/s | 119 MB     00:09
real    0m51.479s
user    0m33.024s
sys    0m3.400s

3)
Total                                            19 MB/s | 119 MB     00:06
real    0m52.373s
user    0m33.171s
sys    0m7.121s

4)
Total                                            21 MB/s | 119 MB     00:05
real    0m54.213s
user    0m32.999s
sys    0m7.122s

5)
Total                                            13 MB/s | 119 MB     00:09
real    0m56.093s
user    0m32.708s
sys    0m5.745s


== without gcc, gcc-c++ and make, i.e. "install rpm-build shadow-utils util-linux which"

Install  153 Packages

Total download size: 81 M
Installed size: 350 M

1)
Total                                            14 MB/s |  81 MB     00:05
real    0m49.789s
user    0m28.926s
sys    0m6.864s

2)
Total                                            17 MB/s |  81 MB     00:04
real    0m44.010s
user    0m28.965s
sys    0m3.434s

3)
Total                                            18 MB/s |  81 MB     00:04
real    0m45.860s
user    0m29.032s
sys    0m5.730s

4)
Total                                            19 MB/s |  81 MB     00:04
real    0m42.047s
user    0m28.944s
sys    0m3.099s

5)
Total                                            14 MB/s |  81 MB     00:05
real    0m47.079s
user    0m29.839s
sys    0m3.978s


== without gcc, gcc-c++, make and without perl (rhbz#1158860, note that the rpm packages are local in this case), i.e. "install rpm-build shadow-utils util-linux which"

Install  124 Packages

Total size: 69 M
Total download size: 68 M
Installed size: 310 M

1)
Total                                            11 MB/s |  68 MB     00:06
real    0m44.725s
user    0m27.644s
sys    0m6.126s

2)
Total                                            12 MB/s |  68 MB     00:05
real    0m46.253s
user    0m27.624s
sys    0m7.056s

3)
Total                                           6.5 MB/s |  68 MB     00:10
real    0m50.296s
user    0m27.567s
sys    0m6.064s

4)
Total                                            12 MB/s |  68 MB     00:05
real    0m46.078s
user    0m27.581s
sys    0m7.698s

5)
Total                                            16 MB/s |  68 MB     00:04
real    0m44.307s
user    0m27.830s
sys    0m5.293s

== Only rpmbuild install "install rpm-build shadow-utils"

Install  111 Packages

Total size: 61 M
Total download size: 60 M
Installed size: 286 M

1)
Total                                            20 MB/s |  60 MB     00:03
real    0m37.848s
user    0m26.158s
sys    0m2.676s

2)
Total                                            13 MB/s |  60 MB     00:04
real    0m44.756s
user    0m26.442s
sys    0m5.185s

3)
Total                                            16 MB/s |  60 MB     00:03
real    0m42.703s
user    0m26.118s
sys    0m3.174s

4)
Total                                            13 MB/s |  60 MB     00:04
real    0m42.365s
user    0m26.439s
sys    0m4.193s

5)
Total                                           5.9 MB/s |  60 MB     00:10
real    0m48.940s
user    0m26.608s
sys    0m4.576s

== Minimal root I can shell in, i.e. "install bash shadow-utils"

Install  48 Packages

Total download size: 31 M
Installed size: 176 M

1)
Total                                           8.3 MB/s |  31 MB     00:03
real    0m36.745s
user    0m22.019s
sys    0m4.936s

2)
Total                                           9.5 MB/s |  31 MB     00:03
real    0m32.608s
user    0m21.805s
sys    0m2.719s

3)
Total                                            11 MB/s |  31 MB     00:02
real    0m34.111s
user    0m22.170s
sys    0m3.651s

4)
Total                                            12 MB/s |  31 MB     00:02
real    0m41.141s
user    0m22.595s
sys    0m5.535s

5)
Total                                           8.4 MB/s |  31 MB     00:03
real    0m34.303s
user    0m21.851s
sys    0m3.839s


And several times during the test I encountered bug similar to:

Downloading Packages:
Error: Error downloading packages:
  Curl error: Failure when receiving data from the peer for ftp://mirror.slu.cz/fedora/linux/development/rawhide/x86_64/os/Packages/e/elfutils-libelf-0.161-1.fc22.x86_64.rpm [response reading failed]

Less packages to download, less possible issues.


You can interpret these data yourself, but with less packages in build root, I can see:

1) Saved build time
2) Saved network bandwidth
3) Saved storage
4) Less things to break


  Versus how long will it take to
either modify all these packages / write and run the script to do
that?  Is A > B?

There are things which are hard to quantify. If you want to prohibit any change, this is probably right question to ask. Of course answering such questions also takes some time, I hope you will count it into the final equation at the end, although hard to say on which side it should belong.


Vít