Hey ARM community,
I'm playing with some new scripts to try and report useful stats from the builders so that team
members can better diagnose why they might be down. Below is the URL I'm using to
track the various builder machines and any suggestions (like certain config #s or stats)
that I should add would be useful. I just wanted know what your opinions or thoughts are basically!
Thanks muchly for all your time,
We have arranged a sync-up meeting for tomorrow (Wed) Feb 1, 2012 at 4pm
EST (16:00), which is 9pm UK time. This is the new time slot, so please
let me know if there is a reason that it will no longer work for you.
Here is a first cut agenda we can use:
0). Update on current build failures
- What have we fixed?
- What remains broken?
- Who has the ball on what issues?
- Which issues need someone to own them?
1). Plan for remainder of Fedora 17 work
2). Build system updates.
3). Any other business
- PA planning
- Raspberry Pi
I'd like to propose bringing back the regular daily sync at the same
time (1pm PST, 3pm CST, 4pm EST, 9pm UK), beginning on Thursday. If we
can ensure we sync once a day on what needs fixing, I think it'll help.
f15stats.html :: Tue Jan 31 13:05:01 UTC 2012 -- Wed Feb 1 13:05:01 UTC 2012
Errors Unbuilt Different Working Built
564 [-30] 652 [-9] 297 [+3] 1 [-1] 8901 [+37]
f17stats.html :: Tue Jan 31 13:05:02 UTC 2012 -- Wed Feb 1 13:05:01 UTC 2012
Errors Unbuilt Different Working Built
9  5588 [+4] 117 [+46] 2 [-1] 5154 [-37]
* "Different" contains packages that are built but have a varying ENVR to that of PA-Koji
* "Working" lists any packages currently being built
* The numbers in the brackets show the difference recorded in the counts since the script was last run
I had a very enjoyable phonecall with Peter this evening to catch up on
the process, and to augment some of our IRC conversations. Here are a
few things that we would like to remind everyone around Fedora 17. I
know this stuff can be confusing, so I hope this helps! :)
Currently, we are working on completing a number of the core build
dependencies for F17. Brendan, David, Peter, Dennis, and others have all
performed excellent feats in fixing stuff, which is awesome. It will be
good to continue to focus our energies on identifying the cause of build
failures that Peter has posted on the wiki, based on koji-shadow output.
At the same time that we work on fixing stuff, we need to be very
careful not to kick off unnecessary builds, especially out of our shared
sense of frustration at having idle builders! I know the builders are
often idle at the moment, and we did perform many builds by hand in the
past, but for F17 we need to ensure we build only the minimum by hand,
and then allow koji-shadow to handle the dependencies for us (which
Peter has a handle on and is running). If we don't do this, we can have
problems later with sonames (library versioning) because if library X
has the wrong version, and is then used to build package (or library) Y,
which is used to build package Z, we can end up with three packages that
need to be fixed later! There are different problems we can see, too. It
turns out that koji-shadow was written precisely to solve this problem!
I know it's a pain to feel like we're twiddling our thumbs sometimes,
but the best thing we can do right now is help to isolate build problems
that Peter has identified from running koji-shadow, on the wiki. For
those who aren't programmers, even identifying where a problem is by
finding the test case that fails and having a simple reproducer could be
helpful. In the coming days, Peter is going to setup publicly viewable
logs from the script runs so that we can, in addition, collectively help
to populate the wiki with problems from those koji-shadow runs.
Meanwhile, we would be best served in trying to figure out why packages
are failing. Both Peter and I agree that, in the interest of expediently
having a base F17, those packages where the problem is caused by tests
that are bogus on ARM, and wherein we have filed a bug and received no
feedback, can have their tests disabled on ARM (with blocker BZ applied)
in the short term. We can fix those test cases to work on ARM later. It
isn't the most elegant, but I think it is a reasonable compromise idea?
Peter has built a "kernel" package that contains only a qemu target. We
don't need a real kernel at this stage, but we do need more than merely
kernel-headers. For the moment, this kernel package approach should be
fine to facilitate building F17 packages. We can get a real kernel built
later, and the kernel NVR changes frequently enough that this will not
be a problem. Meanwhile, David and others have produced many good kernel
packages that we can continue to use on real hardware platforms.
My own goal is to help grab tricky packages this week and fix them up.
Currently, I am trying to fix one of the heavybuilders in HSV but then I
want to stop poking at these and focus my own energy on fixing stuff. If
we can work together in the coming days, I agree with Peter that we can
have an F17 bare essentials image in alpha timeframe. It looks like a
few thousand package dep problems will soon go away, facilitating the
commencement of the real "mass" rebuild in koji-shadow, too. So, let's
all hang in there and look forward to reaching the end goal soon!
That's more than enough rambling from me. Everyone is doing an excellent
job here, and I am really happy with the progress overall. It might look
like we're some way off, but in reality I think we can have a good set
of packages built within the next few weeks, once we get over this one.