Hi folks,
Since 2010 we have defined VCS tag in RPM, but we're not using it actually. No guidelines for it, nothing other. Just tag.
============ Introduction ============ Currently we are using "Source0" for "%setup" and related macroes. All mesa builds in Fedora are done from git (master and other stable branches) + some patches. Kernel also built on top of upstream git tree (tags or master) + patches (from ML, backported or fedora specific). Many of packages comes from github built from git tags.
======= Problem ======= Now we need to have fork of upstream repo, commit our patches, rebase on top of needed branch/tag, do git format-patch, add those patches to fedora repo and to spec file. If we're building from git -- git should have webUI for downloading snapshot or we should make tarballs via script. Apply patches if needed and etc. It all takes time and knowledge.
In case if we want to build package from git we should follow naming guidelines[0], and the same. Do archive, patches, etc.
======== Proposal ======== With having something like below we can simplify maintaining RPMs. VCS: git://github.com/foo/bar.git#commit_or_tag which will add all do we need in naming and archiving instead of Source0: https://github.com/foo/bar/archive/commit_or_tag/%%7Bname%7D-%%7Bshortcommit... and some RPM macro playing around commits and in case of non-github/bitbucket/etc. to have shell scripts like my[1].
Probably we'd want to add some logic for picking up patches from that repo. For example, I had to backport 10+ patches from upstream in beignet[2]. Here I don't have ideas - allow commit sha in PatchX tags if VCS defined? Have new tags?
Sure, usage of VCS tag is optional. You still can use Source0 instead of VCS.
rpm-side: if VCS specified -- clone, checkout, make git archive and use generated archive as source to use/unpack
dist-git-side: we wil upload rpm-generated source
====== Syntax ====== As we don't have any guidelines/syntax for VCS tag I'd propose: NOTE: Don't allow builds without optional mark in fedora - scm:git:git://github.com/foo/bar.git#commit #commit is optional for RPM itself, if it doesn't exist -- we will get package with version 0.0.1-1.DATEgitCOMMIT%{?dist}. Or something like this.
I've inspired by syntax from Peter's erlang package in Fedora[3].
===== Notes ===== When I'm talking about git -- the same could be done for hg/cvs/svn/etc.
======= Outline =======
Appstream-GLib -------------- appstream-glib has reporting it in metadata[4], I've planned to use it as filter to know is project is dead in upstream or no longer developed.
createrepo_c ------------ createrepo_c has issue[5] about vcs tag reporting in metadata, but nothing done due to lacks of use-cases.
[0] https://fedoraproject.org/wiki/Packaging:SourceURL#Git_Hosting_Services [1] http://pkgs.fedoraproject.org/cgit/mesa.git/tree/Makefile [2] http://pkgs.fedoraproject.org/cgit/beignet.git/commit/?id=49c7a3c15b9985ab3f... [3] http://pkgs.fedoraproject.org/cgit/erlang.git/tree/erlang.spec#n20 [4] https://github.com/hughsie/appstream-glib/issues/27 [5] https://github.com/Tojaj/createrepo_c/pull/1 -- -Igor Gnatenko
On Mon, Aug 3, 2015 at 9:50 AM, Igor Gnatenko ignatenkobrain@fedoraproject.org wrote:
Hi folks,
Since 2010 we have defined VCS tag in RPM, but we're not using it actually. No guidelines for it, nothing other. Just tag.
============ Introduction ============ Currently we are using "Source0" for "%setup" and related macroes. All mesa builds in Fedora are done from git (master and other stable branches) + some patches. Kernel also built on top of upstream git tree (tags or master) + patches (from ML, backported or fedora specific). Many of packages comes from github built from git tags.
The kernel isn't built from git. It's built from tarballs and patches.
======= Problem ======= Now we need to have fork of upstream repo, commit our patches, rebase on top of needed branch/tag, do git format-patch, add those patches to fedora repo and to spec file. If we're building from git -- git should have webUI for downloading snapshot or we should make tarballs via script. Apply patches if needed and etc. It all takes time and knowledge.
In case if we want to build package from git we should follow naming guidelines[0], and the same. Do archive, patches, etc.
======== Proposal ======== With having something like below we can simplify maintaining RPMs. VCS: git://github.com/foo/bar.git#commit_or_tag which will add all do we need in naming and archiving instead of Source0: https://github.com/foo/bar/archive/commit_or_tag/%%7Bname%7D-%%7Bshortcommit... and some RPM macro playing around commits and in case of non-github/bitbucket/etc. to have shell scripts like my[1].
Probably we'd want to add some logic for picking up patches from that repo. For example, I had to backport 10+ patches from upstream in beignet[2]. Here I don't have ideas - allow commit sha in PatchX tags if VCS defined? Have new tags?
Sure, usage of VCS tag is optional. You still can use Source0 instead of VCS.
rpm-side: if VCS specified -- clone, checkout, make git archive and use generated archive as source to use/unpack
dist-git-side: we wil upload rpm-generated source
I'm not really sure I'm following your proposal at all. If it is to allow the Fedora buildsystem to clone a specified repo during RPM build for a package, I don't believe that will work. Koji doesn't allow outside network access during builds.
====== Syntax ====== As we don't have any guidelines/syntax for VCS tag I'd propose: NOTE: Don't allow builds without optional mark in fedora
- scm:git:git://github.com/foo/bar.git#commit
#commit is optional for RPM itself, if it doesn't exist -- we will get package with version 0.0.1-1.DATEgitCOMMIT%{?dist}. Or something like this.
I've inspired by syntax from Peter's erlang package in Fedora[3].
===== Notes ===== When I'm talking about git -- the same could be done for hg/cvs/svn/etc.
Even if koji did allow outside network access, I'm not sure we would want to allow this for CVS. It doesn't have the concept of tree-wide commits and is generally terrible.
Ideally you'd want your VCS to specify an immutable commit, like a git sha1 hash. That way if you build foo-1.1 based on commit 1234567 in August and someone builds the exact same package in January, they are using the same source. If you're using branch names or HEADs, then those are movable and may result in different packages.
josh
Dne 3.8.2015 v 16:10 Josh Boyer napsal(a):
On Mon, Aug 3, 2015 at 9:50 AM, Igor Gnatenko ignatenkobrain@fedoraproject.org wrote:
Hi folks,
Since 2010 we have defined VCS tag in RPM, but we're not using it actually. No guidelines for it, nothing other. Just tag.
============ Introduction ============ Currently we are using "Source0" for "%setup" and related macroes. All mesa builds in Fedora are done from git (master and other stable branches) + some patches. Kernel also built on top of upstream git tree (tags or master) + patches (from ML, backported or fedora specific). Many of packages comes from github built from git tags.
The kernel isn't built from git. It's built from tarballs and patches.
======= Problem ======= Now we need to have fork of upstream repo, commit our patches, rebase on top of needed branch/tag, do git format-patch, add those patches to fedora repo and to spec file. If we're building from git -- git should have webUI for downloading snapshot or we should make tarballs via script. Apply patches if needed and etc. It all takes time and knowledge.
In case if we want to build package from git we should follow naming guidelines[0], and the same. Do archive, patches, etc.
======== Proposal ======== With having something like below we can simplify maintaining RPMs. VCS: git://github.com/foo/bar.git#commit_or_tag which will add all do we need in naming and archiving instead of Source0: https://github.com/foo/bar/archive/commit_or_tag/%%7Bname%7D-%%7Bshortcommit... and some RPM macro playing around commits and in case of non-github/bitbucket/etc. to have shell scripts like my[1].
Probably we'd want to add some logic for picking up patches from that repo. For example, I had to backport 10+ patches from upstream in beignet[2]. Here I don't have ideas - allow commit sha in PatchX tags if VCS defined? Have new tags?
Sure, usage of VCS tag is optional. You still can use Source0 instead of VCS.
rpm-side: if VCS specified -- clone, checkout, make git archive and use generated archive as source to use/unpack
dist-git-side: we wil upload rpm-generated source
I'm not really sure I'm following your proposal at all. If it is to allow the Fedora buildsystem to clone a specified repo during RPM build for a package, I don't believe that will work. Koji doesn't allow outside network access during builds.
Actually, I can see this more useful for fedpkg. E.g.:
VCS: git://github.com/foo/bar.git#%{shortcommit} Source0: %{name}-%{shortcommit}.tar.gz
Later, you could call:
$ fedpkg sources
or something similar (sources is actually in use, not sure if that could cover this scenario or we would need different command), which would prepare the archive for you. From here, you can continue as you always did.
Vít
On Mon, Aug 3, 2015, at 09:50 AM, Igor Gnatenko wrote:
Since 2010 we have defined VCS tag in RPM, but we're not using it actually. No guidelines for it, nothing other. Just tag.
Yeah...I pushed for this then dropped it - sorry about that.
which will add all do we need in naming and archiving instead of
The "git instead of tarballs" is quite an interesting and detailed topic. I have been working on one portion of the problem space here: https://github.com/cgwalters/git-evtag
rpm-side: if VCS specified -- clone, checkout, make git archive and use generated archive as source to use/unpack
This is another project of mine in this space: https://github.com/cgwalters/rpmdistro-gitoverlay
It handles some important subtleties, such as doing recursive mirroring and archiving for git submodules.
On Mon, 2015-08-03 at 16:50 +0300, Igor Gnatenko wrote:
Probably we'd want to add some logic for picking up patches from that repo. For example, I had to backport 10+ patches from upstream in beignet[2]. Here I don't have ideas - allow commit sha in PatchX tags if VCS defined? Have new tags?
I'm not sure I'd use a feature like that, given that I could just as easily set up a git branch containing the patches I want:
VCS: git://people.freedesktop.org/~ajax/xserver#server-1.17-fedora-23-branch
- ajax
-
Adam Jackson -
Colin Walters -
Igor Gnatenko -
Josh Boyer -
Vít Ondruch