On Tue, 24 Sep 2013 12:38:44 +0800
Mathieu Bridon <bochecha(a)fedoraproject.org> wrote:
Hi,
Dennis Gilmore has been talking for some time about how he'd like to
run Mash in Koji tasks, instead of the set of scripts releng
currently uses.
So I started working on this, and I got to the point where it actually
works, but I have a few questions before I can finish it and submit
the patches.
1. What would be an appropriate weight for this new task?
At the moment I'm just using the default, but given that mashing can
be quite intensive, it might be better to increase that. I have no
idea on what would be the right number, though.
Yeah, not sure either... but I wonder if it wouldn't make sense to make
this it's own channel so it could go to specific builders.
In the Fedora case we would want that as our mashes are very memory
intensive (mostly due to deltarpm generation), so if somehow an arm
builder got it, not sure how long it would take it to finish (if ever).
2. What would be the appropriate permissions to run this task?
Right now, the task requires the user to be an admin. Should I
introduce a new 'mash' permission? Is there an existing 'releng'
permission I should use?
Admin would probibly be ok for a first cut.
3. What to do with the output?
Mash will create the repositories for all configured architectures,
corresponding to a certain Koji tag.
As such, its output is much bigger that, say, a build task.
The task could upload everything back to the Hub, but that could take
a significant time and might hammer the Hub?
Dennis was saying that another possibility was to mount /mnt/koji as
read-write on the builder, and inside the chroot, so that the task
could write its output directly there.
What would be the preferred option?
Not sure. For some data to toss in tho, a current rawhide mash takes
up about 163GB. Branched f20 is currently around 153GB.
So, in addition to the question of if it should try and upload to the
hub or not, it needs to make sure and have enough local working space
to hold all that (unless it's writing to a rw /mnt/koji for that).
kevin