Greetings.
A group of folks are looking at getting modular content in the non modular buildroot in fedora. The current plan for this has us creating a list of modules (starting with the default modules and adding a few) that we want to be in the buildroot, doing a pungi compose to gather them into a repo (all marked default in that repo) and then adding them as a external repo the buildroot inherits.
The question has come up how we want to regenerate this repo, and I wondered if perhaps we could expand kojira to do it? Is that too much scope creep for kojira? I know the above plan has external repos, but what if we didn't do external and could 'teah' kojira how to detect when this modular buildroot repo was out of date and fire off a pungi compose to update it? basically making kojira more generic and handling rpm buildroots and regenerating them as a single case.
Is this something that sounds workable? Or should we look at doing something completely external to koji here?
kevin
My first thoughts would be to see if pulp-project yum repos could be leveraged for this.
Pat
On 9/17/19 12:37 PM, Kevin Fenzi wrote:
Greetings.
A group of folks are looking at getting modular content in the non modular buildroot in fedora. The current plan for this has us creating a list of modules (starting with the default modules and adding a few) that we want to be in the buildroot, doing a pungi compose to gather them into a repo (all marked default in that repo) and then adding them as a external repo the buildroot inherits.
The question has come up how we want to regenerate this repo, and I wondered if perhaps we could expand kojira to do it? Is that too much scope creep for kojira? I know the above plan has external repos, but what if we didn't do external and could 'teah' kojira how to detect when this modular buildroot repo was out of date and fire off a pungi compose to update it? basically making kojira more generic and handling rpm buildroots and regenerating them as a single case.
Is this something that sounds workable? Or should we look at doing something completely external to koji here?
kevin
koji-devel mailing list -- koji-devel@lists.fedorahosted.org To unsubscribe send an email to koji-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.o...
I work on Pulp and would be glad to assist with an integration. I would recommend integrating with Pulp 3. The documentation for the RPM plugin can be found here: https://pulp-rpm.readthedocs.io/
What use cases would this integration involve?
On 9/19/19 9:01 AM, Dennis Kliban wrote:
I work on Pulp and would be glad to assist with an integration. I would recommend integrating with Pulp 3. The documentation for the RPM plugin can be found here: https://pulp-rpm.readthedocs.io/
What use cases would this integration involve?
We currently don't run pulp, and don't have any plans to. :)
So, this won't help us any, but of course it might be worthwhile and help others.
kevin
On 9/19/19 11:29 AM, Kevin Fenzi wrote:
On 9/19/19 9:01 AM, Dennis Kliban wrote:
I work on Pulp and would be glad to assist with an integration. I would recommend integrating with Pulp 3. The documentation for the RPM plugin can be found here: https://pulp-rpm.readthedocs.io/
What use cases would this integration involve?
We currently don't run pulp, and don't have any plans to. :)
So, this won't help us any, but of course it might be worthwhile and help others.
kevin
Long term I'd love to see the repo management in koji move to pulp. In theory that might simplify the overall work that kojira needs to do.
Pat
On Thu, Sep 19, 2019 at 12:38 PM Pat Riehecky riehecky@fnal.gov wrote:
On 9/19/19 11:29 AM, Kevin Fenzi wrote:
On 9/19/19 9:01 AM, Dennis Kliban wrote:
I work on Pulp and would be glad to assist with an integration. I
would recommend integrating with Pulp 3. The documentation for the RPM plugin can be found here: https://pulp-rpm.readthedocs.io/
What use cases would this integration involve?
We currently don't run pulp, and don't have any plans to. :)
So, this won't help us any, but of course it might be worthwhile and help others.
kevin
Long term I'd love to see the repo management in koji move to pulp. In theory that might simplify the overall work that kojira needs to do.
That is serious overkill, and I really don't want Koji to rely on such a large and complicated system.
On 9/18/19 12:07 AM, Miroslav Suchý wrote:
Dne 17. 09. 19 v 19:37 Kevin Fenzi napsal(a):
A group of folks are looking at getting modular content in the non modular buildroot in fedora.
I wonder... Why?
To allow non modular things to keep building when things they need to build are only available as modules?
For example, the java stack has a java-buildtools module that many java packages could use to build, but have no way to do so currently.
kevin
Dne 19. 09. 19 v 18:30 Kevin Fenzi napsal(a):
On 9/18/19 12:07 AM, Miroslav Suchý wrote:
Dne 17. 09. 19 v 19:37 Kevin Fenzi napsal(a):
A group of folks are looking at getting modular content in the non modular buildroot in fedora.
I wonder... Why?
To allow non modular things to keep building when things they need to build are only available as modules?
For example, the java stack has a java-buildtools module that many java packages could use to build, but have no way to do so currently.
Can we instead of:
The current plan for this has us creating a list of modules (starting with the default modules and adding a few) that we want to be in the buildroot, doing a pungi compose to gather them into a repo (all marked default in that repo)
teach Koji(ra) to consume modules as modules and put into Mock config: config_opts['module_enable'] = ['list', 'of', 'modules'] config_opts['module_install'] = ['module1/profile', 'module2/profile'] ? I have to admit that I do not know what is responsible to create final module artifact (rpm repo with modulemd metadata). Is it Pungi or MBS?
On Fri, Sep 20, 2019 at 09:32:09AM +0200, Miroslav Suchý wrote:
Can we instead of:
The current plan for this has us creating a list of modules (starting with the default modules and adding a few) that we want to be in the buildroot, doing a pungi compose to gather them into a repo (all marked default in that repo)
teach Koji(ra) to consume modules as modules and put into Mock config: config_opts['module_enable'] = ['list', 'of', 'modules'] config_opts['module_install'] = ['module1/profile', 'module2/profile'] ?
Well, the problem there is that we need a place to download and install and enable the modules from (in the buildroot). We could add the fedora-modular repo as a extenal repo, but then we loose some reproducability, and we can't update it as much (so mbs builds needing a chain would fail or wait for a day).
We could have a script that adds modules we wants tags into the buildroot, but thats basically ursa major that we don't want to use.
I have to admit that I do not know what is responsible to create final module artifact (rpm repo with modulemd metadata). Is it Pungi or MBS?
Well, MBS builds everything and creates a repo with the finished module, pungi gathers up all the modules from their tags and makes a fedora-modular repo from it that is the thing that is pushed out.
kevin
On Tue, Sep 17, 2019 at 1:38 PM Kevin Fenzi kevin@scrye.com wrote:
A group of folks are looking at getting modular content in the non modular buildroot in fedora. The current plan for this has us creating a list of modules (starting with the default modules and adding a few) that we want to be in the buildroot, doing a pungi compose to gather them into a repo (all marked default in that repo) and then adding them as a external repo the buildroot inherits.
The question has come up how we want to regenerate this repo, and I wondered if perhaps we could expand kojira to do it? Is that too much scope creep for kojira? I know the above plan has external repos, but what if we didn't do external and could 'teah' kojira how to detect when this modular buildroot repo was out of date and fire off a pungi compose to update it? basically making kojira more generic and handling rpm buildroots and regenerating them as a single case.
Kojira doesn't create repos.
Kojira triggers repo regenerations by creating tasks in Koji. Those tasks are run by the Koji builders. We could say that the builders create the repos (or at least the ones in the createrepo channel), but really it's best to just say that "Koji does it" since the process involves many parts of Koji.
Anyway, the point is that would not really be just a kojira change. In fact kojira might not need to change at all -- it doesn't really need to know the details of repo construction.
Pedantry aside, it's reasonable to change how Koji generates repos. We've done it before. The key things to keep in mind are:
1) performance -- adding more complexity to regens slows them down and you know how frequently that happens 2) sanity -- koji repos must be determined by the contents of a tag at an event. Any information that tells Koji to do a different thing with the repo must be part of that data.
Is this something that sounds workable? Or should we look at doing
something completely external to koji here?
kevin
koji-devel mailing list -- koji-devel@lists.fedorahosted.org To unsubscribe send an email to koji-devel-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahosted.o...
koji-devel@lists.fedorahosted.org