Hi.
I might have an idea how to make building Perl packages faster and their buildroot a little bit smaller.
perl-devel depends on systemtap-sdt-devel and that package contains a single script written in Python and using pycparser. The single script bring python3-pycparser and therefore the whole Python with its standard library to the buildroot of all perl packages although (according to my testing) none of the packages needs it.
I've selected all packages build-requiring perl-devel but don't build-requiring python-devel directly - 520 in total. And from that number:
7 faild to build for unrelated reasons 3 packages have python3-devel in buildroot (different reasons than systempat-sdt-devel) 81 packages have python3-libs but not python3-devel (different reasons than systempat-sdt-devel)
and finally, the rest - 436 packages - builds fine without the python script in systemtap-sdt-devel and therefore without Python at all.
My idea is to split systemtap-sdt-devel into two packages: one with all the content but without the python script (/usr/bin/dtrace) and a new one containing only the mentioned script.
That would make buildroots for many packages smaller and their builds faster.
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
What do you think about this idea? Is it worth writing a Fedora change for it?
Have a nice day. Lumír
V Fri, May 10, 2024 at 01:13:53PM +0200, Lumír Balhar napsal(a):
I might have an idea how to make building Perl packages faster and their buildroot a little bit smaller.
perl-devel depends on systemtap-sdt-devel and that package contains a single script written in Python and using pycparser. The single script bring python3-pycparser and therefore the whole Python with its standard library to the buildroot of all perl packages although (according to my testing) none of the packages needs it.
I've selected all packages build-requiring perl-devel but don't build-requiring python-devel directly - 520 in total. And from that number:
7 faild to build for unrelated reasons 3 packages have python3-devel in buildroot (different reasons than systempat-sdt-devel) 81 packages have python3-libs but not python3-devel (different reasons than systempat-sdt-devel)
and finally, the rest - 436 packages - builds fine without the python script in systemtap-sdt-devel and therefore without Python at all.
My idea is to split systemtap-sdt-devel into two packages: one with all the content but without the python script (/usr/bin/dtrace) and a new one containing only the mentioned script.
That would make buildroots for many packages smaller and their builds faster.
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
What do you think about this idea? Is it worth writing a Fedora change for it?
That looks like a great optimization from Perl point of view. It also seems that only 10 components will need adjustments, so a self-contained Fedora change should be enough.
Unanswered question is run-time dependencies. There might be packages which run-require systemtap-sdt-devel because of dtrace executable:
# dnf -q -C repoquery --whatrequires systemtap-sdt-devel lttng-ust-devel-0:2.13.8-1.fc41.x86_64 perl-devel-4:5.38.2-507.fc41.x86_64 systemtap-testsuite-0:5.1~pre17062192g5fd8daba-1.fc40.x86_64
But probably the most important part is systemtap maintainer (CCed). Is he fine with this change? Isn't this incompatible change too obstrusive for dtrace users? I believe it isn't, otherwise dtrace would not be packaged in a -devel package.
-- Petr
Hi -
[...]
My idea is to split systemtap-sdt-devel into two packages: one with all the content but without the python script (/usr/bin/dtrace) and a new one containing only the mentioned script.
No objection here.
[...] I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
(The logistic challenge there will be side-tag rebuilding all those after a systemtap subrpm split.)
How much time did excluding the python bits from the perl buildroot actually save during your tests?
Unanswered question is run-time dependencies. There might be packages which run-require systemtap-sdt-devel because of dtrace executable:
# dnf -q -C repoquery --whatrequires systemtap-sdt-devel lttng-ust-devel-0:2.13.8-1.fc41.x86_64 perl-devel-4:5.38.2-507.fc41.x86_64 systemtap-testsuite-0:5.1~pre17062192g5fd8daba-1.fc40.x86_64
By default, these packages could inherit dependencies on both of the split subrpms, until their respective packagers decide to analyze and narrow it further down.
- FChE
On 10. 05. 24 17:16, Frank Ch. Eigler wrote:
[...] I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
(The logistic challenge there will be side-tag rebuilding all those after a systemtap subrpm split.)
I don't understand why that would be necessary. Could you please explain why do you believe it would?
Hi -
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: [...]
(The logistic challenge there will be side-tag rebuilding all those after a systemtap subrpm split.)
I don't understand why that would be necessary. Could you please explain why do you believe it would?
OK, build-time dependency changes may not need the side tag but do need a spec update to prevent a FTBFS at next build.
- FChE
V Mon, May 13, 2024 at 11:11:05AM -0400, Frank Ch. Eigler napsal(a):
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: [...]
(The logistic challenge there will be side-tag rebuilding all those after a systemtap subrpm split.)
I don't understand why that would be necessary. Could you please explain why do you believe it would?
OK, build-time dependency changes may not need the side tag but do need a spec update to prevent a FTBFS at next build.
I was more worried about these races: If you first add BuildRequies and then the new package, there will be a window when the packages won't resolve build dependencies. If you first add the new package and then the BuildRequiers, then there will a window when the packages will fail to build.
This race can be solved by two separate changes in systemtap: First add the new package and run-require it from systemtap-sdt-devel. Then add the BuildRequires. And finally remove the run-time dependency from systemtap-sdt-devel.
-- Petr
On 13. 05. 24 17:11, Frank Ch. Eigler wrote:
Hi -
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: [...]
(The logistic challenge there will be side-tag rebuilding all those after a systemtap subrpm split.)
I don't understand why that would be necessary. Could you please explain why do you believe it would?
OK, build-time dependency changes may not need the side tag but do need a spec update to prevent a FTBFS at next build.
Only those packages that actually need dtrace would require changes. Such changes could land gradually as needed.
Hi -
OK, build-time dependency changes may not need the side tag but do need a spec update to prevent a FTBFS at next build.
Only those packages that actually need dtrace would require changes. Such changes could land gradually as needed.
They'd have to land at the next respin of each of those packages.
- FChE
Dne 10. 05. 24 v 14:16 Petr Pisar napsal(a):
V Fri, May 10, 2024 at 01:13:53PM +0200, Lumír Balhar napsal(a):
I might have an idea how to make building Perl packages faster and their buildroot a little bit smaller.
perl-devel depends on systemtap-sdt-devel and that package contains a single script written in Python and using pycparser. The single script bring python3-pycparser and therefore the whole Python with its standard library to the buildroot of all perl packages although (according to my testing) none of the packages needs it.
I've selected all packages build-requiring perl-devel but don't build-requiring python-devel directly - 520 in total. And from that number:
7 faild to build for unrelated reasons 3 packages have python3-devel in buildroot (different reasons than systempat-sdt-devel) 81 packages have python3-libs but not python3-devel (different reasons than systempat-sdt-devel)
and finally, the rest - 436 packages - builds fine without the python script in systemtap-sdt-devel and therefore without Python at all.
My idea is to split systemtap-sdt-devel into two packages: one with all the content but without the python script (/usr/bin/dtrace) and a new one containing only the mentioned script.
That would make buildroots for many packages smaller and their builds faster.
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
What do you think about this idea? Is it worth writing a Fedora change for it?
That looks like a great optimization from Perl point of view. It also seems that only 10 components will need adjustments, so a self-contained Fedora change should be enough.
Unanswered question is run-time dependencies. There might be packages which run-require systemtap-sdt-devel because of dtrace executable:
# dnf -q -C repoquery --whatrequires systemtap-sdt-devel lttng-ust-devel-0:2.13.8-1.fc41.x86_64 perl-devel-4:5.38.2-507.fc41.x86_64 systemtap-testsuite-0:5.1~pre17062192g5fd8daba-1.fc40.x86_64
What about BuildRequires? Shouldn't they be included in your query?
Vít
But probably the most important part is systemtap maintainer (CCed). Is he fine with this change? Isn't this incompatible change too obstrusive for dtrace users? I believe it isn't, otherwise dtrace would not be packaged in a -devel package.
-- Petr
-- _______________________________________________ devel mailing list --devel@lists.fedoraproject.org To unsubscribe send an email todevel-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it:https://pagure.io/fedora-infrastructure/new_issue
V Mon, May 13, 2024 at 04:14:53PM +0200, Vít Ondruch napsal(a):
Dne 10. 05. 24 v 14:16 Petr Pisar napsal(a):
V Fri, May 10, 2024 at 01:13:53PM +0200, Lumír Balhar napsal(a):
I also did a test rebuild of all packages directly build-requiring systemtap-sdt-devel and identified these packages that really need the dtrace script: glib2, sssd, qemu, python2.7, postgresql15, postgresql16, perl, php, mariadb10.11, and libvirt. Those would newly depend on a new package where we move the script to.
[...]
Unanswered question is run-time dependencies. There might be packages which run-require systemtap-sdt-devel because of dtrace executable:
# dnf -q -C repoquery --whatrequires systemtap-sdt-devel lttng-ust-devel-0:2.13.8-1.fc41.x86_64 perl-devel-4:5.38.2-507.fc41.x86_64 systemtap-testsuite-0:5.1~pre17062192g5fd8daba-1.fc40.x86_64
What about BuildRequires? Shouldn't they be included in your query?
The original poster already checked the BuildRequires with a rebuild.
-- Petr
On Fri, May 10, 2024 at 01:13:53PM +0200, Lumír Balhar wrote:
My idea is to split systemtap-sdt-devel into two packages: one with all the content but without the python script (/usr/bin/dtrace) and a new one containing only the mentioned script.
+1
I think it's weird that a user program (/usr/bin/dtrace) is installed in a package called *-devel. So I think it'd good to split it out to a separate package even for that.
Zbyszek
-
Frank Ch. Eigler -
Lumír Balhar -
Miro Hrončok -
Petr Pisar -
Vít Ondruch -
Zbigniew Jędrzejewski-Szmek