On Thu, Dec 17, 2020 at 8:06 PM Ben Cotton <bcotton(a)redhat.com> wrote:
https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing
== Summary ==
This change should enable an opt-in spec file preprocessor in Fedora
infrastructure for the benefit of packagers. The preprocessor allows
some very neat tricks that were impossible before, for example
generate changelog and release automatically from git metadata or pack
the entire dist-git repository into an rpm-source tarball (effectively
allowing unpacked repos to live in DistGit).
As far as I can tell, this is the third implementation of generated
changelogs ... did the autorelease / autochangelog work that was even
already deployed in staging not go anywhere?
== Owner ==
* Name: [[User:clime| Michal Novotný]]
* Email: clime(a)fedoraproject.org
== Detailed Description ==
There is a recently added feature into mock:
[
https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preproce...
the rpkg preprocessor] which, if enabled, introduces an intermediate
step just before srpm building. This step consists of running the spec
file through a text preprocessing engine that includes an already
present library of macros designed specifically for rpm spec file
generation from git metadata. This library is called
[
https://docs.pagure.org/rpkg-util/v3/macro_reference.html
rpkg-macros]. The macros there allow packagers to have their
`%changelog`, `Release`, `Version`, `VCS` tag, or even `Source` fields
automatically generated from dist-git repository data and metadata.
The library can be easily extended in future to support more packager
use-cases or even a completely new library can be developed that
doesn't look at git metadata at all and instead, for example, analyses
already present tarball content to render spec file based on upstream
information. This doesn't mean it will happen but the framework is
generic enough to support that. There is also support for user-defined
macros that are loaded on-demand from a file placed alongside the
package sources, maintained by packager. This feature wouldn't be
enabled by this change from start but it's an example of freedom that
the preprocessing framework is able to provide. Enabling this change
should be very easy, basically adding:
<pre>
config_opts['plugin_conf']['rpkg_preprocessor_enable'] = True
</pre>
into mock configuration of Koji builders and using at least mock 2.7.
Some very minor change may be also needed in Koji regarding the spec
file lookup.
Even if the change is enabled on the infrastructure level like this,
the packager will still need to opt-in to use the preprocessor. The
opting-in is done by placing `rpkg.conf` file into the package
top-level directory with the following content:
<pre>
[rpkg]
preprocess_spec = True
</pre>
When this is done by a packager, the preprocessor will be finally
enabled for the given package.
Alongside, there is an ongoing work to add the preprocessor support
into the `rpkg` python library so that a packager can easily work with
the spec files containing the preprocessor (rpkg) macros:
https://pagure.io/rpkg/pull-request/530
The macros are supported since epel6 until the most latest Fedora
(preproc, rpkg-macros, and rpm-git-tag-sort packages are needed). The
spec preprocessing step in mock happens in a target chroot just before
the srpm build.
== Benefit to Fedora ==
This change offers solution to some long-standing issues in Fedora
around packaging (i.e. automatic release and changelog generation)
while also offering some interesting future options (for example
unpacked dist-git repos). The big advantage of this approach is that
it is explicit. Spec file stays the source of truth and by looking
inside one, you will be able to determine how the text will expand for
a certain git repository state.
What does "unpacked dist-git repos" mean? Is this a euphemism for source-git?
== Scope ==
* Proposal owners:
For the very basic support, probably a small patch in Koji is needed
to be able to lookup not only `.spec` files but also `.spec.rpkg`
files (the `.spec.rpkg` extension explicitly states that the spec file
is a template). Also the `rpmdevtools/rpmdev-bumpspec` script should
be tweaked to be compatible with spec files using the macros.
Is the Change owner going to submit patches for fedpkg and rpmdevtools?
* Other developers:
Some optional help with `rpmdevtools/rpmdev-bumpspec` changes would be welcome.
rpmdev-bumpspec is a tangled mess of spaghetti code and I'd rather not
touch it or make it even more complicated.
I also think this can't be optional. releng uses rpmdev-bumpspec as
part of scripted mass rebuilds, so it must work with all packages.
* Release engineering: [
https://pagure.io/releng/issue/9910 #9910]
(a
check of an impact with Release Engineering is needed)
Enabling the rpkg_preprocessor plugin in mock config for Koji builders.
* Policies and guidelines:
The new macro support should be mentioned or even described in the
packaging guidelines. We should decide if the full power of the
rpkg-macros library should be allowed from start (i.e. even unpacked
repos).
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
Because of the opt-in nature on packager side, there should be no
compatibility issues.
== How To Test ==
Once the feature is enabled, one can test it by providing the
`rpkg.conf` file with the required content in a package repository and
use some rpkg macro in the spec file: e.g.
<pre>
Name: {{{ git_dir_name }}}
</pre>
to generate the name of the package from the repository name (this
should actually produce the original text as package names should be
the same as the repository basenames).
Not sure I understand, but what's the benefit of making the Name a
macro determined by the repository name?
As stated, they must always be the same anyway, so why make this a
dynamic templated value?
> To try it out first without committing to dist-git, one can use `rpkg`
> command-line tool from
>
https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ or even
> fedpkg's koji scratch build after
> [
https://pagure.io/rpkg/pull-request/530 the work in the pyrpkg]
> library is finished.
>
> One can also currently use Copr's SCM "rpkg" build method where the
> macros are enabled but the rpkg-macros there are in version 2 whereas
> this change is about introducing the
> [
https://docs.pagure.org/rpkg-util/v3/macro_reference.html version 3
> rpkg-macros]. However, while there are some differences between v2 and
> v3, the idea and most of the working is the same.
>
> == User Experience ==
> This change is intended for packagers. It should help to make a bit of
> their work easier and offer them some new interesting options.
>
> == Dependencies ==
> N/A
>
> == Contingency Plan ==
>
> Packagers can opt-out on individual basis by removing the `rpkg.conf`
> file or just setting the `preproces_spec` property to `False`. On
> infrastructure level, the rpkg_preprocessor plugin could be disabled
> again.
>
> == Documentation ==
> - [
https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preproce...
> Mock's rpkg_preprocessor plugin]
>
> - [
https://docs.pagure.org/rpkg-util/v3/macro_reference.html
> rpkg-macros reference (the library of macros ready to be used in spec
> files)]
>
> == Release Notes ==
> Currently N/A
>
>
> --
> Ben Cotton
> He / Him / His
> Senior Program Manager, Fedora & CentOS Stream
> Red Hat
> TZ=America/Indiana/Indianapolis
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org