Hi,
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
However, I feel that step belongs in %prep for two reasons:
1. It's a preparatory step for the Python build that follows
The `ant` job provides codec mappings (from Java Charsets), which are subsequently used in the Python package. The process is described in the CodecMapper README [2].
2. It would be (needlessly?) more complicated to do that in %build
I need to prep the sources and move stuff around for the Python package to live at the top level, before calling `%pyproject_buildrequires`, which is done before %build.
The actual Python package, from which the PyPI sdist and wheel are produced lives in a sub directory of the CodecMapper repository [3]. Unfortunately, the PyPI sdist is incomplete. That's why I'm using the GitHub sources.
I'd appreciate any guidance with regards to permissibility and/or alternative approaches.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2244402 [2] https://github.com/roskakori/CodecMapper#codecmapper [3] https://github.com/roskakori/CodecMapper/tree/master/ebcdic
Cheers,
On Sun, Oct 29, 2023 at 2:15 PM Sandro lists@penguinpee.nl wrote:
Hi,
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
However, I feel that step belongs in %prep for two reasons:
- It's a preparatory step for the Python build that follows
The `ant` job provides codec mappings (from Java Charsets), which are subsequently used in the Python package. The process is described in the CodecMapper README [2].
- It would be (needlessly?) more complicated to do that in %build
I need to prep the sources and move stuff around for the Python package to live at the top level, before calling `%pyproject_buildrequires`, which is done before %build.
The actual Python package, from which the PyPI sdist and wheel are produced lives in a sub directory of the CodecMapper repository [3]. Unfortunately, the PyPI sdist is incomplete. That's why I'm using the GitHub sources.
I'd appreciate any guidance with regards to permissibility and/or alternative approaches.
Usually, steps like calling `%configure` or `%meson` SHOULD (or must?) happen in %build, not in `prep`. However, this is kind of a special case, where you need to prepare the Python package in order to run `%pyproject_buildrequires`, so by definition, it will need to happen in `%prep`. I don't really see a way around that, other than ... not using dynamically generated BuildRequires.
Fabio
On 10/30/23 07:09, Fabio Valentini wrote:
Usually, steps like calling `%configure` or `%meson` SHOULD (or must?) happen in %build, not in `prep`. However, this is kind of a special case, where you need to prepare the Python package in order to run `%pyproject_buildrequires`, so by definition, it will need to happen in `%prep`. I don't really see a way around that, other than ... not using dynamically generated BuildRequires.
Fabio
To add to that, one of the main reasons for running things like %configure or %meson in %build rather than %prep is to minimize the number of dependencies for building the source RPM. It’s inconvenient to have to install a long list of BuildRequires (on the host system rather than just the build chroot) in order to be able to run “fedpkg srpm”.
However, I agree that anything that affects the generated BuildRequires certainly must be run in %prep. I personally think that the disadvantages of doing without generated BuildRequires are generally worse than the disadvantages of having significant dependencies in %prep, although I suppose this is open to debate.
Dne 30. 10. 23 v 13:30 Ben Beasley napsal(a):
On 10/30/23 07:09, Fabio Valentini wrote:
Usually, steps like calling `%configure` or `%meson` SHOULD (or must?)
Or newly in `%conf` section:
https://rpm-software-management.github.io/rpm/manual/spec.html#conf-since-rp...
Vít
happen in %build, not in `prep`. However, this is kind of a special case, where you need to prepare the Python package in order to run `%pyproject_buildrequires`, so by definition, it will need to happen in `%prep`. I don't really see a way around that, other than ... not using dynamically generated BuildRequires.
Fabio
To add to that, one of the main reasons for running things like %configure or %meson in %build rather than %prep is to minimize the number of dependencies for building the source RPM. It’s inconvenient to have to install a long list of BuildRequires (on the host system rather than just the build chroot) in order to be able to run “fedpkg srpm”.
However, I agree that anything that affects the generated BuildRequires certainly must be run in %prep. I personally think that the disadvantages of doing without generated BuildRequires are generally worse than the disadvantages of having significant dependencies in %prep, although I suppose this is open to debate. _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On pondělí 30. října 2023 13:30:56 CET Ben Beasley wrote:
On 10/30/23 07:09, Fabio Valentini wrote:
Usually, steps like calling `%configure` or `%meson` SHOULD (or must?) happen in %build, not in `prep`. However, this is kind of a special case, where you need to prepare the Python package in order to run `%pyproject_buildrequires`, so by definition, it will need to happen in `%prep`. I don't really see a way around that, other than ... not using dynamically generated BuildRequires.
Fabio
To add to that, one of the main reasons for running things like %configure or %meson in %build rather than %prep is to minimize the number of dependencies for building the source RPM. It’s inconvenient to have to install a long list of BuildRequires (on the host system rather than just the build chroot) in order to be able to run “fedpkg srpm”.
This is a confusing argument. You don't need to run %prep to generate SRPM.
Pavel
However, I agree that anything that affects the generated BuildRequires certainly must be run in %prep. I personally think that the disadvantages of doing without generated BuildRequires are generally worse than the disadvantages of having significant dependencies in %prep, although I suppose this is open to debate. _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 10/31/23 10:05, Pavel Raiskup wrote:
On pondělí 30. října 2023 13:30:56 CET Ben Beasley wrote:
On 10/30/23 07:09, Fabio Valentini wrote:
Usually, steps like calling `%configure` or `%meson` SHOULD (or must?) happen in %build, not in `prep`. However, this is kind of a special case, where you need to prepare the Python package in order to run `%pyproject_buildrequires`, so by definition, it will need to happen in `%prep`. I don't really see a way around that, other than ... not using dynamically generated BuildRequires.
Fabio
To add to that, one of the main reasons for running things like %configure or %meson in %build rather than %prep is to minimize the number of dependencies for building the source RPM. It’s inconvenient to have to install a long list of BuildRequires (on the host system rather than just the build chroot) in order to be able to run “fedpkg srpm”.
This is a confusing argument. You don't need to run %prep to generate SRPM.
Pavel
However, I agree that anything that affects the generated BuildRequires certainly must be run in %prep. I personally think that the disadvantages of doing without generated BuildRequires are generally worse than the disadvantages of having significant dependencies in %prep, although I suppose this is open to debate.
If the dependencies are specified, ant can be moved to the build portion. Packaging guidelines do indicate that automatically generated BuildRequires should be done when possible, but not a must. Running commands such as %configure or %autogen in the prep section can sometimes fail. Java guidelines also indicate that ant should be run in the build section. The dependencies in this case are not to large.
The package is on pypi but does not have a source distribution[1]
The java program[2] used could be converted to a Python/Perl/bash script so that compilation is not required in the prep stage. If it is common to generate Python scripts for a package, probably the new %conf section should be used provided the package is only needed on newer Fedora releases. One might also want to package the java program first, and then add it in the build requires as a bootstrap dependency.
1) https://pypi.org/project/ebcdic/#files 2) https://github.com/roskakori/CodecMapper/blob/master/source/at/roskakori/cod...
Sandro lists@penguinpee.nl writes:
However, I feel that step belongs in %prep for two reasons:
I agree that it should not be in %prep. %prep is for unpacking and patching sources. Not much else should be done there.
- It would be (needlessly?) more complicated to do that in %build
Certainly you have to do what is required to build the package; if RPM ordering oddities make it so much easier to do that in %prep then I guess you have to do what you have to do. I think this situation is why %conf was created, but I don't know if the ordering works out and I don't know if it is present on all the releases where you would like to have the package (it was added in RPM 4.18).
- J<
Thank you to everyone who replied. It seems the main takeaway so far is that what goes into the different stages is more in the order of SHOULD, rather than MUST or MUST NOT.
In other words, if one can argue why certain steps are executed in a certain stage and it doesn't violate the packaging guidelines, it is permissible.
On 30-10-2023 16:52, Jason L Tibbitts III wrote:
- It would be (needlessly?) more complicated to do that in %build
Certainly you have to do what is required to build the package; if RPM ordering oddities make it so much easier to do that in %prep then I guess you have to do what you have to do. I think this situation is why %conf was created, but I don't know if the ordering works out and I don't know if it is present on all the releases where you would like to have the package (it was added in RPM 4.18).
I tried moving the execution of `ant` and the clean up of the sources into %conf (before I looked at the RPM documentation).
The result was a failure in %generate_buildrequires, which, as the RPM docs confirm, is ordered before %conf. So, unfortunately, using %conf is not an option. It would have been too easy, I guess. The change is literally splitting %prep:
diff --git a/python-ebcdic.spec b/python-ebcdic.spec index 0b3fb5b..8dde457 100644 --- a/python-ebcdic.spec +++ b/python-ebcdic.spec @@ -48,6 +48,8 @@ Summary: %{summary} %prep %forgeautosetup -p1
+ +%conf # Generate codecs with local python3 sed -i 's|value=".*python"|value="python3"|' build.xml ant ebcdic
So, with %build and %conf not being an option, I guess I have to do it in %prep. Or I could ask upstream to ship all the extracted codecs. Not sure if that's an option, though.
-- Sandro
On Sun, 2023-10-29 at 14:15 +0100, Sandro wrote:
Hi,
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
This is correct IMHO ,
if you run `fedpkg prep` on directory where you heave the spec file of the package (in a git-dist), you wont be able to get "Local test rpmbuild prep" if you haven't ant installed on the system.
for exemplify : fedpkg clone eclipse-swt cd eclipse-swt fedpkg prep
However, I feel that step belongs in %prep for two reasons:
- It's a preparatory step for the Python build that follows
The `ant` job provides codec mappings (from Java Charsets), which are subsequently used in the Python package. The process is described in the CodecMapper README [2].
- It would be (needlessly?) more complicated to do that in %build
I need to prep the sources and move stuff around for the Python package to live at the top level, before calling `%pyproject_buildrequires`, which is done before %build.
The actual Python package, from which the PyPI sdist and wheel are produced lives in a sub directory of the CodecMapper repository [3]. Unfortunately, the PyPI sdist is incomplete. That's why I'm using the GitHub sources.
I'd appreciate any guidance with regards to permissibility and/or alternative approaches.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2244402 [2] https://github.com/roskakori/CodecMapper#codecmapper [3] https://github.com/roskakori/CodecMapper/tree/master/ebcdic
Cheers,
-- Sandro FAS: gui1ty IRC: Penguinpee Elsewhere: [Pp]enguinpee _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 05-11-2023 16:00, Sérgio Basto wrote:
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
This is correct IMHO ,
if you run `fedpkg prep` on directory where you heave the spec file of the package (in a git-dist), you wont be able to get "Local test rpmbuild prep" if you haven't ant installed on the system.
And why is that an issue I should be concerned about?
It's an honest question. When running the full build with `fedpkg mockbuild`, all BRs from the spec file will be installed first, before %prep is executed.
for exemplify : fedpkg clone eclipse-swt cd eclipse-swt fedpkg prep
I know plenty of Python packages that need to apply some git foo in %prep. These would also fail the exercise if git-core were not installed on the host system.
I'm not sure why `fedpkg prep` just fails, while `fedpkg install` will happily tell you what dependencies are missing.
-- Sandro
On 11/6/23 02:00, Sandro wrote:
On 05-11-2023 16:00, Sérgio Basto wrote:
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
This is correct IMHO ,
if you run `fedpkg prep` on directory where you heave the spec file of the package (in a git-dist), you wont be able to get "Local test rpmbuild prep" if you haven't ant installed on the system.
And why is that an issue I should be concerned about?
It's an honest question. When running the full build with `fedpkg mockbuild`, all BRs from the spec file will be installed first, before %prep is executed.
for exemplify : fedpkg clone eclipse-swt cd eclipse-swt fedpkg prep
I know plenty of Python packages that need to apply some git foo in %prep. These would also fail the exercise if git-core were not installed on the host system.
I'm not sure why `fedpkg prep` just fails, while `fedpkg install` will happily tell you what dependencies are missing.
The idea is that %prep only unpacks (which is useful to looking at the sources) so it doesn't need dependencies beyond those of rpmbuild itself. Once upon a time rpm was changed to ignore buildrequires for prep with this rationale, but it's a flawed one: technically, packages are free to use whatever tooling they need to do what they need in %prep as much as any other section. And to do that they'd must be able to express those dependencies.
- Panu -
On 06-11-2023 12:32, Panu Matilainen wrote:
On 11/6/23 02:00, Sandro wrote:
On 05-11-2023 16:00, Sérgio Basto wrote:
During review of a package I submitted [1], the reviewer asked me to move running an `ant` script from %prep to %build.
This is correct IMHO ,
if you run `fedpkg prep` on directory where you heave the spec file of the package (in a git-dist), you wont be able to get "Local test rpmbuild prep" if you haven't ant installed on the system.
And why is that an issue I should be concerned about?
It's an honest question. When running the full build with `fedpkg mockbuild`, all BRs from the spec file will be installed first, before %prep is executed.
for exemplify : fedpkg clone eclipse-swt cd eclipse-swt fedpkg prep
I know plenty of Python packages that need to apply some git foo in %prep. These would also fail the exercise if git-core were not installed on the host system.
I'm not sure why `fedpkg prep` just fails, while `fedpkg install` will happily tell you what dependencies are missing.
The idea is that %prep only unpacks (which is useful to looking at the sources) so it doesn't need dependencies beyond those of rpmbuild itself. Once upon a time rpm was changed to ignore buildrequires for prep with this rationale, but it's a flawed one: technically, packages are free to use whatever tooling they need to do what they need in %prep as much as any other section. And to do that they'd must be able to express those dependencies.
Thank you, Panu. I conclude from that, that it is okay to have (justified) BRs that are needed for %prep, just like git-core is frequently required for Python packages, needing to tag trees in %prep for dynamic versioning tools like Versioneer.
-- Sandro
packaging@lists.fedoraproject.org
-
Ben Beasley -
Benson Muite -
Fabio Valentini -
Jason L Tibbitts III -
Panu Matilainen -
Pavel Raiskup -
Sandro -
Sérgio Basto -
Vít Ondruch