I did make tweaks to the code representing these changes in:
https://pagure.io/fm-orchestrator/pull-request/554
Example output:
https://fedorapeople.org/~sochotni/mbs-content-generator-v2.json
Summary of changes from original:
* Added modulemd version to tools
* Added all installed rpms from mbs host into the buildroots.0.components
* Changed output.0.type from "modulemd" to "file"
* Added empty extra.typeinfo.module into output.0
Few more comments inline
On Fri 12 May 2017 07:43:46 PM CEST Mike McLean wrote:
On 05/12/2017 01:07 PM, Ralph Bean wrote:
> Just pinging on this. I think stano is blocked waiting for an ack
> from a koji-devel and (possibly) fedora-releng here.
Overall thrust is fine. Particulars of the data need work. We need to
make sure to distinguish between:
btype: the build type specified for builds and archives (e.g. rpm,
maven, win, image, module). That is the data that goes into typeinfo.
OK, this should be better now
output type: this unfortunately named field in the metadata should be
either "file", "rpm", or "log".
Changed the output type to file
archivetype: not specified in the metadata, but reported in the api
based on file extension (e.g. jar, zip, pom, iso, qcow2)
So the file is a yaml metadata file. Not an archive really. Which is
what my initial question around this was - we use this metadata file as
a place to link those components described by the metadata file. Is that
fine? I am guessing we'll need to add "yaml" extension to archivetype in
the db?
> On Tue, May 09, 2017 at 01:45:55PM +0200, Stanislav Ochotnicky
wrote:
>> On Fri 05 May 2017 07:23:18 PM CEST Ralph Bean <rbean(a)redhat.com> wrote:
>>> On Mon, Apr 24, 2017 at 05:06:47PM +0200, Stanislav Ochotnicky wrote:
>>>> On Mon 24 Apr 2017 04:56:05 PM CEST Jan Kaluža <jkaluza(a)redhat.com>
wrote:
>>>>> On 04/24/2017 04:20 PM, Stanislav Ochotnicky wrote:
>>>>>> On Mon 24 Apr 2017 03:25:48 PM CEST Ralph Bean
<rbean(a)redhat.com> wrote:
>>>>>>> On Mon, Apr 24, 2017 at 01:51:39PM +0200, Stanislav
Ochotnicky wrote:
>>>>>>>> This is the first stab and what the mbs CG metadata
output might look like:
>>>>>>>>
https://paste.fedoraproject.org/paste/5ix-EZVVRkmqpSvCQKOw715M1UNdIGYhyRL...
>>>>>>>
>>>>>>> Just to clarify - in the "build" entry here, this
is a build of the
>>>>>>> "nginx module". The "version" (which is
1) and the "release" (which is
>>>>>>> 2) are in koji terms, not MBS terms.
>>>>>>>
>>>>>>> The mapping between MBS and koji terms should look something
like:
>>>>>>>
>>>>>>> MBS -> Koji
>>>>>>> ---------------
>>>>>>> name -> name
>>>>>>> stream -> version
>>>>>>> version -> release
>>>>>>>
>>>>>>> The only kicker I can see at the moment is the datatype of
stream(MBS)
>>>>>>> versus version(Koji). The MBS stream is a string (master, or
f26, or
>>>>>>> whatever). The Koji version... does it have to be an
integer?
>>>>>>
>>>>>> Well the version even for rpms is a string. Usually semantic
versioning
>>>>>> style thing - but not necessarily. Of course if you use something
lime
>>>>>> rpmdev-vercmp on these mbs builds EVRs it might make things
weird. I.e.:
>>>>>> rpmdev-vercmp master-20 version2-10
>>>>>> master-20 < version2-10
>>>>>>
>>>>>> But I think that's OK.
>>>>>>
>>>>>> One thing however - can stream contain dash? Because version
string
>>>>>> cannot. I'd be tempted to just replace it with underscore
hoping there
>>>>>> won't be both :-)
>>>>>
>>>>> We currently set the the dist-git branch as a stream value. From MBS
>>>>> point of view, there is no problem with dash in a stream name and I
>>>>> presume there will dashes in a stream name if we don't decide you
cannot
>>>>> use dash in dist-git branch name for modules. I would not like to
>>>>> restrict that if there is no real reason to do that.
>>>>>
>>>>> How much user-visible will the content generator data be? Can for
>>>>> example user search for module in Koji by typing its
>>>>> name-stream-version? In that case, I would start thinking about
>>>>> restricting the branch so it does not contain dash, because people
will
>>>>> not replace dash with underscore when searching Koji.
>>>>
>>>> Well this part of content generator is very visible - it's part of
the
>>>> build NVR after all. So things like API calls getBuild(nvr) would be
>>>> affected by it. And the NVR will need to be input as part of shipping
>>>> update in Bodhi for example.
>>>>
>>>> I would guess people *will* use dash in the stream name unless they are
>>>> prevented from doing it?
>>>
>>> From discussion in IRC today (paraphrasing others):
>>>
>>> We can use a new "build type" for this ("module"). With
a new build
>>> type (btype) we can:
>>>
>>> - store the original MBS name-stream-version in koji as extra information,
preserving dashes
>>> - while the build has a koji-reformatted name-version-release with dashes
converted to underscores.
>>>
>>> There is precedent for this in the way Maven GAV is mapped to NVR.
>>>
>>> In the CG metadata provided by MBS, btype specific data goes into
>>> extra->typeinfo->BTYPENAME for build and archive and this how koji
>>> knows the btype. So, the MBS would need to define the mapping from NSV
>>> to NVR (hyphens become underscores, etc.) and use the mapped NVR to
>>> identify the build. The original, unmodified NSV would be recorded in
>>> the "extra" field of the build.
>>>
>>> (Archives can only have one btype. builds btypes must match their
>>> component archives. btype info can technically be {})
>>>
>>> Is that sufficient to solve this mapping problem? Are there any other
>>> outstanding questions you need resolved, stano?
>>>
>>> (Sorry for this getting splintered across multiple lists...)
>>
>> So:
>> * I'll change "modulemd" btype for the build to just
"module"
>> * I can do the tweak for n-v-r dashes fairly easily (i.e. convert - to _
>> in the version/stream field) and provide an mbs stream/version metadata
>> in the extra/typeinfo/module space
>> * As mentioned above - the output archives need the type too. Right now
>> the "type" is "module" and there's an empty typeinfo
for it. Should
>> we use some generic type (i.e. "archive" or whatever makes sense)
and
>> just provide the "extra/typeinfo" piece? This is a little confusing
>> since OSBS doesn't seem to be currently using this structure so I had
>> no example to base from and the docs[1] are not making it
>> clear. Basically the above is based on experimentation based on
>> errors I was getting from koji when I was importing. I'd appreciate
>> someone from koji folks to have a look and tell me whether I can
>> likely simplify/adjust something.
>>
>> Sorry about the fpaste expiring. I didn't realize the TTL is so short
>> there. I uploaded the latest version example updated with changes
>> proposed here:
>>
https://fedorapeople.org/~sochotni/mbs-content-generator.json
>>
>> If the above content generator example seems OK I can do the final
>> tweaks pretty quickly. I'd appreciate an ACK on the format from koji
>> devs before I go ahead with that tweak though.
>>
>>
>> _______________________________________________
>> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
>> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
_______________________________________________
koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
--
Stanislav Ochotnicky <sochotnicky(a)redhat.com>
Business System Analyst, PnT DevOps - Brno
PGP: F434 2286 27DC 7D9B 2B64 0866 BCBD 752E 7B08 7241
Red Hat Inc.
http://www.redhat.com