I have added a request for koji in Fedora infrastructure to add
configuration to enable all of this. Could you folks check if I forgot
anything and/or clarify what I requested?
Thanks!
On Mon 15 May 2017 12:21:17 PM CEST Stanislav Ochotnicky <sochotnicky(a)redhat.com>
wrote:
I did make tweaks to the code representing these changes in:
https://pagure.io/fm-orchestrator/pull-request/554
Example output:
https://fedorapeople.org/~sochotni/mbs-content-generator-v2.json
Summary of changes from original:
* Added modulemd version to tools
* Added all installed rpms from mbs host into the buildroots.0.components
* Changed output.0.type from "modulemd" to "file"
* Added empty extra.typeinfo.module into output.0
Few more comments inline
On Fri 12 May 2017 07:43:46 PM CEST Mike McLean wrote:
> On 05/12/2017 01:07 PM, Ralph Bean wrote:
>> Just pinging on this. I think stano is blocked waiting for an ack
>> from a koji-devel and (possibly) fedora-releng here.
>
> Overall thrust is fine. Particulars of the data need work. We need to
> make sure to distinguish between:
>
> btype: the build type specified for builds and archives (e.g. rpm,
> maven, win, image, module). That is the data that goes into typeinfo.
OK, this should be better now
> output type: this unfortunately named field in the metadata should be
> either "file", "rpm", or "log".
Changed the output type to file
> archivetype: not specified in the metadata, but reported in the api
> based on file extension (e.g. jar, zip, pom, iso, qcow2)
So the file is a yaml metadata file. Not an archive really. Which is
what my initial question around this was - we use this metadata file as
a place to link those components described by the metadata file. Is that
fine? I am guessing we'll need to add "yaml" extension to archivetype in
the db?
>> On Tue, May 09, 2017 at 01:45:55PM +0200, Stanislav Ochotnicky wrote:
>>> On Fri 05 May 2017 07:23:18 PM CEST Ralph Bean <rbean(a)redhat.com>
wrote:
>>>> On Mon, Apr 24, 2017 at 05:06:47PM +0200, Stanislav Ochotnicky wrote:
>>>>> On Mon 24 Apr 2017 04:56:05 PM CEST Jan Kaluža
<jkaluza(a)redhat.com> wrote:
>>>>>> On 04/24/2017 04:20 PM, Stanislav Ochotnicky wrote:
>>>>>>> On Mon 24 Apr 2017 03:25:48 PM CEST Ralph Bean
<rbean(a)redhat.com> wrote:
>>>>>>>> On Mon, Apr 24, 2017 at 01:51:39PM +0200, Stanislav
Ochotnicky wrote:
>>>>>>>>> This is the first stab and what the mbs CG metadata
output might look like:
>>>>>>>>>
https://paste.fedoraproject.org/paste/5ix-EZVVRkmqpSvCQKOw715M1UNdIGYhyRL...
>>>>>>>>
>>>>>>>> Just to clarify - in the "build" entry here,
this is a build of the
>>>>>>>> "nginx module". The "version" (which
is 1) and the "release" (which is
>>>>>>>> 2) are in koji terms, not MBS terms.
>>>>>>>>
>>>>>>>> The mapping between MBS and koji terms should look
something like:
>>>>>>>>
>>>>>>>> MBS -> Koji
>>>>>>>> ---------------
>>>>>>>> name -> name
>>>>>>>> stream -> version
>>>>>>>> version -> release
>>>>>>>>
>>>>>>>> The only kicker I can see at the moment is the datatype
of stream(MBS)
>>>>>>>> versus version(Koji). The MBS stream is a string
(master, or f26, or
>>>>>>>> whatever). The Koji version... does it have to be an
integer?
>>>>>>>
>>>>>>> Well the version even for rpms is a string. Usually semantic
versioning
>>>>>>> style thing - but not necessarily. Of course if you use
something lime
>>>>>>> rpmdev-vercmp on these mbs builds EVRs it might make things
weird. I.e.:
>>>>>>> rpmdev-vercmp master-20 version2-10
>>>>>>> master-20 < version2-10
>>>>>>>
>>>>>>> But I think that's OK.
>>>>>>>
>>>>>>> One thing however - can stream contain dash? Because version
string
>>>>>>> cannot. I'd be tempted to just replace it with underscore
hoping there
>>>>>>> won't be both :-)
>>>>>>
>>>>>> We currently set the the dist-git branch as a stream value. From
MBS
>>>>>> point of view, there is no problem with dash in a stream name and
I
>>>>>> presume there will dashes in a stream name if we don't decide
you cannot
>>>>>> use dash in dist-git branch name for modules. I would not like
to
>>>>>> restrict that if there is no real reason to do that.
>>>>>>
>>>>>> How much user-visible will the content generator data be? Can
for
>>>>>> example user search for module in Koji by typing its
>>>>>> name-stream-version? In that case, I would start thinking about
>>>>>> restricting the branch so it does not contain dash, because
people will
>>>>>> not replace dash with underscore when searching Koji.
>>>>>
>>>>> Well this part of content generator is very visible - it's part
of the
>>>>> build NVR after all. So things like API calls getBuild(nvr) would be
>>>>> affected by it. And the NVR will need to be input as part of
shipping
>>>>> update in Bodhi for example.
>>>>>
>>>>> I would guess people *will* use dash in the stream name unless they
are
>>>>> prevented from doing it?
>>>>
>>>> From discussion in IRC today (paraphrasing others):
>>>>
>>>> We can use a new "build type" for this ("module").
With a new build
>>>> type (btype) we can:
>>>>
>>>> - store the original MBS name-stream-version in koji as extra
information, preserving dashes
>>>> - while the build has a koji-reformatted name-version-release with dashes
converted to underscores.
>>>>
>>>> There is precedent for this in the way Maven GAV is mapped to NVR.
>>>>
>>>> In the CG metadata provided by MBS, btype specific data goes into
>>>> extra->typeinfo->BTYPENAME for build and archive and this how koji
>>>> knows the btype. So, the MBS would need to define the mapping from NSV
>>>> to NVR (hyphens become underscores, etc.) and use the mapped NVR to
>>>> identify the build. The original, unmodified NSV would be recorded in
>>>> the "extra" field of the build.
>>>>
>>>> (Archives can only have one btype. builds btypes must match their
>>>> component archives. btype info can technically be {})
>>>>
>>>> Is that sufficient to solve this mapping problem? Are there any other
>>>> outstanding questions you need resolved, stano?
>>>>
>>>> (Sorry for this getting splintered across multiple lists...)
>>>
>>> So:
>>> * I'll change "modulemd" btype for the build to just
"module"
>>> * I can do the tweak for n-v-r dashes fairly easily (i.e. convert - to _
>>> in the version/stream field) and provide an mbs stream/version metadata
>>> in the extra/typeinfo/module space
>>> * As mentioned above - the output archives need the type too. Right now
>>> the "type" is "module" and there's an empty
typeinfo for it. Should
>>> we use some generic type (i.e. "archive" or whatever makes
sense) and
>>> just provide the "extra/typeinfo" piece? This is a little
confusing
>>> since OSBS doesn't seem to be currently using this structure so I had
>>> no example to base from and the docs[1] are not making it
>>> clear. Basically the above is based on experimentation based on
>>> errors I was getting from koji when I was importing. I'd appreciate
>>> someone from koji folks to have a look and tell me whether I can
>>> likely simplify/adjust something.
>>>
>>> Sorry about the fpaste expiring. I didn't realize the TTL is so short
>>> there. I uploaded the latest version example updated with changes
>>> proposed here:
>>>
https://fedorapeople.org/~sochotni/mbs-content-generator.json
>>>
>>> If the above content generator example seems OK I can do the final
>>> tweaks pretty quickly. I'd appreciate an ACK on the format from koji
>>> devs before I go ahead with that tweak though.
>>>
>>>
>>> _______________________________________________
>>> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
>>> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> _______________________________________________
> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Business System Analyst, PnT DevOps - Brno
PGP: F434 2286 27DC 7D9B 2B64 0866 BCBD 752E 7B08 7241
Red Hat Inc.
http://www.redhat.com
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Software Engineer, PnT DevOps - Brno
PGP: F434 2286 27DC 7D9B 2B64 0866 BCBD 752E 7B08 7241
Red Hat Inc.