On Thu, 16 Oct 2014 11:14:01 +0200
Mikolaj Izdebski <mizdebsk(a)redhat.com> wrote:
On 10/15/2014 09:31 PM, Kevin Fenzi wrote:
> ok, some general questions, please excuse me if they are dumb. ;)
>
> high level:
>
> * How well does it keep up currently? I know you are careful not to
> overload koji, but I wonder if that means things like perl builds
> are often behind because there are so many of them?
Koji has more than enough resources to sustain current Koschei load
(~3000 packages). Storage might become problematic if more packages
are added (scratch build results are kept for some period of time),
but we have a solution for that (see [2] or [3]). If more Koji
builders are ever need then I think it sould be quite easy to add
them, as long as there is budget for that.
Great. I wasn't sure if it was keeping up or not. ;)
Interesting idea on the immediate purge.
> * right now the service is opt-in right? Someone adds a group
and
> packages in that group and then when one of them changes it
> scratch rebuilds the rest. Do you see a time/case when we could
> just make it operate on all builds? that is, build foo is made, and
> it just does all the things that buildrequire foo?
For now only some subset of all packages is tracked by Koschei, but
the ultimate goal is to track all packages - they would be added
automatically after first build appears on Koji and removed when they
are blocked. What would be up to individuals is maintainig package
groups. (One package can be in any number of groups.)
ok. Groups are only managed manually by maintainers?
And deps are then checked when something updates in a group and the
rest of the group rebuilt? or can you explain when a scratch build is
fired?
> * The notifications of failed builds currently are via fedmsg?
We
> should investigate adding this to FMN if it's not already there,
> so anyone interested could be notified via that.
fedmsg publishing is already operational as can be seen on [1]. FMN
rule has been recently added. The new FMN is not yet in production,
but in (hopefully near) future users will be able to enable email or
IRC notifications for buildability status of packages they are
interested in.
Great!
> todo's/ideas:
>
> * Could this ever be a koji plugin? Or does it do too much on top of
> that to ever be a plugin?
Koschei has its own architecture and converting it to Koji plugin
would require substantial amount of work. In other words, it should
be possible, but I don't see any good reason to do so.
Fair enough.
> * Might it be possible to run on all the broken deps packages
in
> rawhide/branched? This would depend I guess on the compose process
> generating fedmsgs with those package names, but if so it could
> tell maintainers "hey, your package is broken in rawhide, but a
> simple rebuild will fix it" (or any other group that just wants to
> go fix them).
This is an interesting idea.
A simillar feature was planned for future. The idea was that Koschei
could be resolving runtime dependencies of all packages besides just
build dependencies. Users would be able to see whether package is
installable and if yes, see its installation size with dependencies
(in terms of MB to download, MB installed size and package count).
There would be graphs showing how this changes in time. (We had a
simillar POC service runnig for a few months, see [4].)
We could extend this and make Koschei resolve runtime dependencies of
successful scratch builds it runs. In case scratch build would fix
broken package in offcial repo, a fedmsg would be emited.
Yeah, something to consider...
> * boost is another group of packages I could see this being
useful
> for. Perhaps it woul<d be worth reaching out to the boost
> maintainers?
I don't know specifics of boost packages, but we'll cosider any
feature request.
ok. Boost often updates once a cycle or so, and lots of dependent
packages need rebuilding. If we could see which of those fail it could
be helpfull. But this is up to boost maintainers I suppose.
> * Could this be used to scratch build packages that are
> ExcludeArch/ExclusiveArch with that removed? ie, to tell
> maintainers, "hey, you exclude arm, but it builds ok, are you sure
> thats fine?"
This would require generating a new SRPM with
ExcludeArch/ExclusiveArch removed, which requires installing all
build dependencies, so it should be done by Koji as buildSRPMfromSCM
task. This in turn requires Koschei having ability to push to some
branch in SCM or maintaining separate git repo and changing Koji
policy to allow scratch builds from it. And of course this would have
to be implemented in Koschei. Not impossible, but looks like a lot of
work for something that could be done manually by running some script
from time to time.
Yeah, agreed.
> technical:
>
> * Can this application be load balanced any? Ie, if we have two of
> them could they operate against the same db at the same time?
To answer this question I need to elaborate more about Koschei
architecture. tl;dr yes, it can be load balanced well.
...snip...
To sum up, all components either don't need load balancing or can
be
load-balanced.
ok. And I suspect this service would be ok just being one instance as
well, since it's not critical. ie, if we had to update/reboot it we
could just do so and it would go on, no need to keep everything up 100%?
> * Are there any common sysadmin tasks we need to know about with the
> instance? Is there any special process to start/stop/reinstall
> it?
Installing Koschei is done by installing RPM package from Fedora or
EPEL repositories and coping a single config file.
Managing all services (starting, stopping, viewing logs etc.) is done
using standard system tools (systemctl, journalctl and so on).
There is nothing special to be done besides standard sysadmin stuff
(updating packages, viewing logs, backing up database and so on).
Excellent.
> * When there is koji maint, should we stop this service? How do
we
> gracefully do that and start it again?
In this case you can stop Koschei services that communicate with Koji
(using systemctl stop koschei-<servicename>) and start them when
maintenance is over. Web UI will remain functional during that time,
but there will be no new build scheduled.
> Thats all I can think of right now. :)
I hope this pretty long email answers your questions.
It does indeed. Thanks for taking the time to do so. :)
kevin