Hi,
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close "https://repos.fedorapeople.org/" ? It seems very old...
Regards, Casper
Dne 29. 12. 22 v 10:41 Casper napsal(a):
Hi,
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close"https://repos.fedorapeople.org/" ? It seems very old...
Icon Name https://repos.fedorapeople.org/?C=N;O=A Last modified https://repos.fedorapeople.org/?C=M;O=A Size https://repos.fedorapeople.org/?C=S;O=A Description https://repos.fedorapeople.org/?C=D;O=A ------------------------------------------------------------------------------------------------------------------------ [DIR] thl/ https://repos.fedorapeople.org/thl/ 2022-11-28 06:41 - [DIR] openstack/ https://repos.fedorapeople.org/openstack/ 2022-11-14 13:15 - [DIR] leo/ https://repos.fedorapeople.org/leo/ 2022-02-25 20:26 -
These three seems to be last users.
But otherwise +1 from me.
M.
On Thu, Dec 29, 2022 at 10:31:16PM +0100, Miroslav Suchý wrote:
Dne 29. 12. 22 v 10:41 Casper napsal(a):
Hi,
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close"https://repos.fedorapeople.org/" ? It seems very old...
Icon Name https://repos.fedorapeople.org/?C=N;O=A Last modified https://repos.fedorapeople.org/?C=M;O=A Size https://repos.fedorapeople.org/?C=S;O=A Description https://repos.fedorapeople.org/?C=D;O=A
[DIR] thl/ https://repos.fedorapeople.org/thl/ 2022-11-28 06:41 - [DIR] openstack/ https://repos.fedorapeople.org/openstack/ 2022-11-14 13:15 - [DIR] leo/ https://repos.fedorapeople.org/leo/ 2022-02-25 20:26 -
These three seems to be last users.
But otherwise +1 from me.
Added them to CC here. :)
I'm not sure thl can do his kernel-vanilla builds in copr, but perhaps? leo doesn't have anything there currently. I know openstack uses this space for some of their releases, but not sure the details.
kevin
[sending this again after subscribing to the infra list]
On 08.01.23 23:03, Kevin Fenzi wrote:
On Thu, Dec 29, 2022 at 10:31:16PM +0100, Miroslav Suchý wrote:
Dne 29. 12. 22 v 10:41 Casper napsal(a):
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close"https://repos.fedorapeople.org/" ? It seems very old...
Ugh :-/
Icon Name https://repos.fedorapeople.org/?C=N;O=A Last modified https://repos.fedorapeople.org/?C=M;O=A Size https://repos.fedorapeople.org/?C=S;O=A Description https://repos.fedorapeople.org/?C=D;O=A
[DIR] thl/ https://repos.fedorapeople.org/thl/ 2022-11-28 06:41 - [DIR] openstack/ https://repos.fedorapeople.org/openstack/ 2022-11-14 13:15 - [DIR] leo/ https://repos.fedorapeople.org/leo/ 2022-02-25 20:26 -
These three seems to be last users.
But otherwise +1 from me.
Added them to CC here. :)
thx
I'm not sure thl can do his kernel-vanilla builds in copr, but perhaps?
I tried a few (three? four?) years ago and there were iirc three showstoppers back then. I'm sure I wrote them down somewhere, but I can't find it right now. :-( But I think I remember one:
It iirc (and afaik back then) was not possible to build packages and test them before their publication. Is that possible these days? I occasionally do something stupid, that's why I fire up the x86_64 builds in qemu locally these days before publishing a build.
/me looks to answer that question
Seems it's somehow possible these days: https://docs.pagure.org/copr.copr/createrepo.html Is that scriptable? I'd prefer not to do this manually for each build.
Not sure what the other two problems where. One might have been speed, but I might misremember/mix something up.
/me give copr a try to check how suitable it would be these days
Ciao, Thorsten
On 09.01.23 07:12, Thorsten Leemhuis wrote:
On 08.01.23 23:03, Kevin Fenzi wrote:
On Thu, Dec 29, 2022 at 10:31:16PM +0100, Miroslav Suchý wrote:
Dne 29. 12. 22 v 10:41 Casper napsal(a):
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close"https://repos.fedorapeople.org/" ? It seems very old...
Ugh :-/
Icon Name https://repos.fedorapeople.org/?C=N;O=A Last modified https://repos.fedorapeople.org/?C=M;O=A Size https://repos.fedorapeople.org/?C=S;O=A Description https://repos.fedorapeople.org/?C=D;O=A
[DIR] thl/ https://repos.fedorapeople.org/thl/ 2022-11-28 06:41 - [DIR] openstack/ https://repos.fedorapeople.org/openstack/ 2022-11-14 13:15 - [DIR] leo/ https://repos.fedorapeople.org/leo/ 2022-02-25 20:26 -
These three seems to be last users.
But otherwise +1 from me.
Added them to CC here. :)
thx
I'm not sure thl can do his kernel-vanilla builds in copr, but perhaps?
I tried a few (three? four?) years ago and there were iirc three showstoppers back then. I'm sure I wrote them down somewhere, but I can't find it right now. :-( But I think I remember one:
It iirc (and afaik back then) was not possible to build packages and test them before their publication. Is that possible these days? I occasionally do something stupid, that's why I fire up the x86_64 builds in qemu locally these days before publishing a build.
/me looks to answer that question
Seems it's somehow possible these days: https://docs.pagure.org/copr.copr/createrepo.html Is that scriptable? I'd prefer not to do this manually for each build.
Not sure what the other two problems where. One might have been speed, but I might misremember/mix something up.
/me give copr a try to check how suitable it would be these days
That made me remember the biggest showstopper:
I regularly push the results of one build to two different repositories (which ones depends on the phase of the kernel devel cycle and if Fedora is shipping the latest stable series).
To explain: currently I build one vanilla mainline package (6.2-rc...) and push it to the kernel-vanilla-mainline and kernel-vanilla-mainline-wo-mergew repos (rpms hardlinked of course). But within the merge window (e.g. after 6.1 was released and before 6.2-rc1 came out) I didn't do that; back then I pushed the latest stable releases to the kernel-vanilla-stable and kernel-vanilla-mainline-wo-mergew repos, as some people want to avoid mainline builds during that phase (even some kernel devs are careful with mainline builds during that time frame).
And right now kernel-vanilla-stable and kernel-vanilla-fedora ship different builds (6.1.y and 6.0.y to be precise); but that will change again in a few days, when Fedora will jump to 6.1 in updates-testing, then those repos will start shipping the same packages again.
See https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories#Configure_the_rep... to get a brief idea.
I assume that's not possible with copr, unless I build everything twice I now simply hardlink. Which would require quite a build of compute time and storage space without a strong reason.
Ciao, Thorsten
Dne 09. 01. 23 v 7:38 Thorsten Leemhuis napsal(a):
That made me remember the biggest showstopper:
I regularly push the results of one build to two different repositories (which ones depends on the phase of the kernel devel cycle and if Fedora is shipping the latest stable series).
To explain: currently I build one vanilla mainline package (6.2-rc...) and push it to the kernel-vanilla-mainline and kernel-vanilla-mainline-wo-mergew repos (rpms hardlinked of course). But within the merge window (e.g. after 6.1 was released and before 6.2-rc1 came out) I didn't do that; back then I pushed the latest stable releases to the kernel-vanilla-stable and kernel-vanilla-mainline-wo-mergew repos, as some people want to avoid mainline builds during that phase (even some kernel devs are careful with mainline builds during that time frame).
And right now kernel-vanilla-stable and kernel-vanilla-fedora ship different builds (6.1.y and 6.0.y to be precise); but that will change again in a few days, when Fedora will jump to 6.1 in updates-testing, then those repos will start shipping the same packages again.
See https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories#Configure_the_rep... to get a brief idea.
I assume that's not possible with copr, unless I build everything twice I now simply hardlink. Which would require quite a build of compute time and storage space without a strong reason.
1) Create project "kernel-vanilla-mainline" in Copr
2) Create project "kernel-vanilla-mainline-wo-mergew" in Copr and in settings set:
* external repositories: copr://user/kernel-vanilla-mainline
* Runtime dependencies: copr://user/kernel-vanilla-mainline
3) When you build into "kernel-vanilla-mainline" then the build will be available in both repositories. Available both as build requires and even as run-time because:
4) when you run 'dnf copr enable kernel-vanilla-mainline-wo-mergew" then it will enable "kernel-vanilla-mainline" too. So all build from both project will be available for users of "kernel-vanilla-mainline-wo-mergew"
Miroslav
On 09.01.23 13:10, Miroslav Suchý wrote:
Dne 09. 01. 23 v 7:38 Thorsten Leemhuis napsal(a):
That made me remember the biggest showstopper:
I regularly push the results of one build to two different repositories (which ones depends on the phase of the kernel devel cycle and if Fedora is shipping the latest stable series).
To explain: currently I build one vanilla mainline package (6.2-rc...) and push it to the kernel-vanilla-mainline and kernel-vanilla-mainline-wo-mergew repos (rpms hardlinked of course). But within the merge window (e.g. after 6.1 was released and before 6.2-rc1 came out) I didn't do that; back then I pushed the latest stable releases to the kernel-vanilla-stable and kernel-vanilla-mainline-wo-mergew repos, as some people want to avoid mainline builds during that phase (even some kernel devs are careful with mainline builds during that time frame).
And right now kernel-vanilla-stable and kernel-vanilla-fedora ship different builds (6.1.y and 6.0.y to be precise); but that will change again in a few days, when Fedora will jump to 6.1 in updates-testing, then those repos will start shipping the same packages again.
See https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories#Configure_the_rep... to get a brief idea.
I assume that's not possible with copr, unless I build everything twice I now simply hardlink. Which would require quite a build of compute time and storage space without a strong reason.
Create project "kernel-vanilla-mainline" in Copr
Create project "kernel-vanilla-mainline-wo-mergew" in Copr and in
settings set:
* external repositories: copr://user/kernel-vanilla-mainline
* Runtime dependencies: copr://user/kernel-vanilla-mainline
3) When you build into "kernel-vanilla-mainline" then the build will be available in both repositories. Available both as build requires and even as run-time because:
- when you run 'dnf copr enable kernel-vanilla-mainline-wo-mergew" then
it will enable "kernel-vanilla-mainline" too. So all build from both project will be available for users of "kernel-vanilla-mainline-wo-mergew"
Thx for outlining thing. Hmmm, that sounds doable, but one question: Will dnf do the right thing automatically (and not bother users with any questions!) when I change the setting outlined in "2" at a later point (e.g. when the next merge window starts I would point it to kernel-vanilla-stable instead; after two weeks, when the merge window ended, I would switch back)?
Ciao, Thorsten
Dne 09. 01. 23 v 13:33 Thorsten Leemhuis napsal(a):
Create project "kernel-vanilla-mainline" in Copr
Create project "kernel-vanilla-mainline-wo-mergew" in Copr and in
settings set:
* external repositories: copr://user/kernel-vanilla-mainline
* Runtime dependencies: copr://user/kernel-vanilla-mainline
3) When you build into "kernel-vanilla-mainline" then the build will be available in both repositories. Available both as build requires and even as run-time because:
- when you run 'dnf copr enable kernel-vanilla-mainline-wo-mergew" then
it will enable "kernel-vanilla-mainline" too. So all build from both project will be available for users of "kernel-vanilla-mainline-wo-mergew"
Thx for outlining thing. Hmmm, that sounds doable, but one question: Will dnf do the right thing automatically (and not bother users with any questions!) when I change the setting outlined in "2" at a later point (e.g. when the next merge window starts I would point it to kernel-vanilla-stable instead; after two weeks, when the merge window ended, I would switch back)?
Hmm, no.
The runtime dependency is evaluated when user runs:
dnf copr enable
The dependency can be transitive, but should not change any time later.
But reading https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories#Configure_the_rep... again... it should work the other way around:
1) project "kernel-vanilla-stable"
2) project "kernel-vanilla-mainline-wo-mergew" which depends (both BR and runtime) on "kernel-vanilla-stable"
3) project "kernel-vanilla-mainline" which depends on "kernel-vanilla-mainline-wo-mergew".
You build stable kernel to "kernel-vanilla-stable" .
You build a git snapshots are built to "kernel-vanilla-mainline-wo-mergew" when there it is NOT merge window
You build a git snapshots to "kernel-vanilla-mainline" when there IS a merge window.
This way the dependency does not change.
And users of kernel-vanilla-mainline-wo-mergew will get git snapshots all the time except the merge window.
Users of kernel-vanilla-mainline will get git snapshots all the time. Because it is either build directly to this project during merge window or inherited from "kernel-vanilla-mainline-wo-mergew" when there is not a merge window.
You just need to flip your script at the beginning and end of merge window. Which I guess, you have to do anyway.
I hope I understand your structure correctly :)
Miroslav
On 09.01.23 14:36, Miroslav Suchý wrote:
But reading https://fedoraproject.org/wiki/Kernel_Vanilla_Repositories#Configure_the_rep... again... it should work the other way around:
Took me a moment to understand the approach, but yeah, that should work. Nice idea. :-D
There is one question on my mind currently before looking closer at this:
- project "kernel-vanilla-mainline-wo-mergew" which depends (both BR
and runtime) on "kernel-vanilla-stable"
From reading...
https://pagure.io/copr/copr/issue/1556
...and...
https://github.com/fedora-copr/copr/blob/main/frontend/coprs_frontend/coprs/...
...it sounds like there is no GPG check for runtime dependency repos? That sounds … well, quite unfortunate. Or is that template only used for external repos and thus not for any specified using copr://user/project?
You just need to flip your script at the beginning and end of merge window. Which I guess, you have to do anyway.
Yeah, sure! thx for your help
/me meanwhile wonders if he should go straight to packit or if he should get copr running first and later decide if using packit is a good idea.
Ciao, Thorsten
Dne 09. 01. 23 v 7:12 Thorsten Leemhuis napsal(a):
I tried a few (three? four?) years ago and there were iirc three showstoppers back then. I'm sure I wrote them down somewhere, but I can't find it right now. 🙁 But I think I remember one:
It iirc (and afaik back then) was not possible to build packages and test them before their publication. Is that possible these days? I occasionally do something stupid, that's why I fire up the x86_64 builds in qemu locally these days before publishing a build.
TIMWTDI
1) If you are using pull request then you can use Packit https://packit.dev/ Packit build the code in PR in new Copr project and links results from PR.
2) If you are not using PR and want to share a results before you push it, you can create project, e.g., thorsten-testing, and configure it that is requires some other repo (your upstream project) and all build in thorsten-testing can use package from some-other-repo and your builds in thorsten-testing will not impact some-other-repo.
Generally speaking: creating project is cheap task and you can create/delete project on daily basis.
Seems it's somehow possible these days: https://docs.pagure.org/copr.copr/createrepo.html Is that scriptable? I'd prefer not to do this manually for each build.
copr-cli modify--disable_createrepo true
copr-cli regenerate-repos
But I would use one of these options above. The createrepo is originally inteded for larges projects like KDE, Gnome. Where you have to builds lots of package and it can takes days it you get all packages built. And during that rebuild the repo can be broken.
Miroslav
On 09.01.23 12:39, Miroslav Suchý wrote:
Dne 09. 01. 23 v 7:12 Thorsten Leemhuis napsal(a):
I tried a few (three? four?) years ago and there were iirc three showstoppers back then. I'm sure I wrote them down somewhere, but I can't find it right now. 🙁 But I think I remember one:
It iirc (and afaik back then) was not possible to build packages and test them before their publication. Is that possible these days? I occasionally do something stupid, that's why I fire up the x86_64 builds in qemu locally these days before publishing a build.
TIMWTDI
- If you are using pull request then you can use Packit
https://packit.dev/ Packit build the code in PR in new Copr project and links results from PR.
- If you are not using PR and want to share a results before you push
it, you can create project, e.g., thorsten-testing, and configure it that is requires some other repo (your upstream project) and all build in thorsten-testing can use package from some-other-repo and your builds in thorsten-testing will not impact some-other-repo.
Generally speaking: creating project is cheap task and you can create/delete project on daily basis.
Either I'm missing something or your explanation is missing one important detail: can I rename a project once I tested the newly built packages so that users of my repos get the latest packages with the next "dnf update" without doing anything?
Ciao, Thorsten
Dne 09. 01. 23 v 13:12 Thorsten Leemhuis napsal(a):
Either I'm missing something or your explanation is missing one important detail: can I rename a project once I tested the newly built packages so that users of my repos get the latest packages with the next "dnf update" without doing anything?
No, you cannot rename it. The closest we have is "fork project" but that is for different use case.
Copr does cannot move package - the way you may know from Satellite and Pulp.
Workflow of Copr's users is usually: build a build from PR code in temporary project. Test it. Merge it. Build it from main git branche (together with other commit that got there in meantime) into "project-nightly" project. Once tagged as a release build it into "project-stable" project.
While moving package between project can safe few CPU cycles (even hours in kernel case) it break reproducible builds. And makes history less obvious.
If you worry about rebuilding kernel twice - it does not cause a problem for us.
Miroslav
On pondělí 9. ledna 2023 14:13:54 CET Miroslav Suchý wrote:
Dne 09. 01. 23 v 13:12 Thorsten Leemhuis napsal(a):
Either I'm missing something or your explanation is missing one important detail: can I rename a project once I tested the newly built packages so that users of my repos get the latest packages with the next "dnf update" without doing anything?
No, you cannot rename it. The closest we have is "fork project" but that is for different use case.
Copr does cannot move package - the way you may know from Satellite and Pulp.
Actually you can use "fork" again and again - which makes me think that "fork" is not a good name for this feature.
Even though both say projects 'A' and 'B' already exist, you may peform "fork" action for package 'Foo' from 'A' to 'B' anytime you consider 'Foo' is "stabilized". Copr then basically just copies the corresponding RPMs and runs createrepo_c.
Pavel
Workflow of Copr's users is usually: build a build from PR code in temporary project. Test it. Merge it. Build it from main git branche (together with other commit that got there in meantime) into "project-nightly" project. Once tagged as a release build it into "project-stable" project.
While moving package between project can safe few CPU cycles (even hours in kernel case) it break reproducible builds. And makes history less obvious.
If you worry about rebuilding kernel twice - it does not cause a problem for us.
Miroslav _______________________________________________ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedorapro... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Sun, 8 Jan 2023 at 17:03, Kevin Fenzi kevin@scrye.com wrote:
On Thu, Dec 29, 2022 at 10:31:16PM +0100, Miroslav Suchý wrote:
Dne 29. 12. 22 v 10:41 Casper napsal(a):
Hi,
COPR is awesome and provides online repositories, with a perfect integration in dnf.
How about to close"https://repos.fedorapeople.org/" ? It seems very old...
Icon Name https://repos.fedorapeople.org/?C=N;O=A
Last modified https://repos.fedorapeople.org/?C=M;O=A Size < https://repos.fedorapeople.org/?C=S;O=A%3E Description < https://repos.fedorapeople.org/?C=D;O=A%3E
[DIR] thl/ https://repos.fedorapeople.org/thl/
2022-11-28 06:41 -
[DIR] openstack/ https://repos.fedorapeople.org/openstack/
2022-11-14 13:15 -[DIR] leo/ https://repos.fedorapeople.org/leo/
2022-02-25 20:26 -
These three seems to be last users.
But otherwise +1 from me.
Added them to CC here. :)
I'm not sure thl can do his kernel-vanilla builds in copr, but perhaps? leo doesn't have anything there currently. I know openstack uses this space for some of their releases, but not sure the details.
The top 20 repositories of repos.fedorapeople.org per day are in order of unique ip addresses
9662 /repos/dchen/apache-maven/epel-6/x86_64/repodata/repomd.xml 3515 /repos/dchen/apache-maven/epel-7/x86_64/repodata/repomd.xml 927 /repos/slaanesh/bacula7/epel-7/x86_64/repodata/repomd.xml 249 /repos/jkaluza/httpd24/epel-6/x86_64/repodata/repomd.xml 235 /repos/pulp/pulp/stable/2/7/x86_64/repodata/repomd.xml 226 /repos/pulp/pulp/stable/2.21/7/x86_64/repodata/repomd.xml 224 /repos/dchen/apache-maven/epel-7Server/x86_64/repodata/repomd.xml 200 /repos/sic/qt48/epel-6/x86_64/repodata/repomd.xml 199 /repos/candlepin/subscription-manager/epel-7/x86_64/repodata/repomd.xml 158 /repos/slaanesh/bacula/epel-6/x86_64/repodata/repomd.xml 124 /repos/dchen/apache-maven/epel-6/SRPMS/repodata/repomd.xml 122 /repos/slaanesh/bacula/epel-7/x86_64/repodata/repomd.xml 114 /repos/thl/kernel-vanilla-stable/fedora-37/x86_64/repodata/repomd.xml 82 /repos/pulp/pulp/stable/2/7Server/x86_64/repodata/repomd.xml 77 /repos/candlepin/subscription-manager/epel-6/x86_64/repodata/repomd.xml 66 /repos/pulp/pulp/stable/latest/6/x86_64/repodata/repomd.xml 65 /repos/thl/kernel-vanilla-stable/fedora-36/x86_64/repodata/repomd.xml 57 /repos/candlepin/subscription-manager/epel-7Server/x86_64/repodata/repomd.xml 55 /repos/slaanesh/bacula7/epel-6/x86_64/repodata/repomd.xml 42 /repos/jkaluza/httpd24/epel-6Server/x86_64/repodata/repomd.xml
The EPEL-6 and EPEL-7 are mainly coming from Amazon NAT gateways so the numbers are cut a bit. Going by unique requests per day
46369 /repos/dchen/apache-maven/epel-6/x86_64/repodata/repomd.xml 30676 /repos/dchen/apache-maven/epel-7/x86_64/repodata/repomd.xml 6744 /repos/slaanesh/bacula7/epel-7/x86_64/repodata/repomd.xml 4856 /repos/pulp/pulp/stable/2/7/x86_64/repodata/repomd.xml 3338 /repos/slaanesh/bacula/epel-7/x86_64/repodata/repomd.xml 3204 /repos/pulp/pulp/stable/2.21/7/x86_64/repodata/repomd.xml 2169 /repos/slaanesh/bacula/epel-6/x86_64/repodata/repomd.xml 1955 /repos/dchen/apache-maven/epel-7Server/x86_64/repodata/repomd.xml 1401 /repos/candlepin/subscription-manager/epel-7/x86_64/repodata/repomd.xml 1053 /repos/jkaluza/httpd24/epel-6/x86_64/repodata/repomd.xml 853 /repos/thl/kernel-vanilla-stable/fedora-37/x86_64/repodata/repomd.xml 782 /repos/pulp/pulp/stable/2.16/7/x86_64/repodata/repomd.xml 682 /repos/candlepin/subscription-manager/epel-7Server/x86_64/repodata/repomd.xml 587 /repos/pulp/pulp/stable/latest/6Server/x86_64/repodata/repomd.xml 490 /repos/pulp/pulp/stable/2/7Server/x86_64/repodata/repomd.xml 418 /repos/candlepin/subscription-manager/epel-6/x86_64/repodata/repomd.xml 318 /repos/dchen/apache-maven/epel-6/SRPMS/repodata/repomd.xml 300 /repos/sic/qt48/epel-6/x86_64/repodata/repomd.xml 300 /repos/pulp/pulp/stable/2.21/7Server/x86_64/repodata/repomd.xml 296 /repos/pulp/pulp/stable/2.18/7/x86_64/repodata/repomd.xml
My guess is that we would need to put out an announcement for the 50k server admins to find that apache-maven will need to be found elsewhere on any reinstalls
infrastructure@lists.fedoraproject.org
-
Casper -
Kevin Fenzi -
Miroslav Suchý -
Pavel Raiskup -
Stephen Smoogen -
Thorsten Leemhuis