Dear Tim,
On Thu, Feb 12, 2015 at 10:31:23AM +0100, Tim Flink wrote:
> [...]
> I assume, running rpmgrill in Taskotron could mean, that each rpmgrill
> test result translates to an entry in Resultsdb as an outcome (PASSED,
> FAILED). Yet the question for the developer as to why it failed is
> important.
I don't think that desire is unique to rpmgrill and something that we
decided to leave out of the initial Taskotron system.
My initial thought is that that a discrete state (PASS, FAIL,
NEEDS_INSPECTION etc.) a one-line-or-so summary and link to output
from rpmgrill would work well. Are there other things that you'd like
to see in terms of output?
From what Kamil pointed out, there is no state missing.
> 1. Ed Santiago is currently running RPMGrill here:
>
>
http://rpmgrill-fc20.edsantiago.com:5000/recent
>
> which in terms of presenting results to the user provides a
> different experience. The results are grouped by test and provide
> more insight as to why a test failed. Test results in Resultsdb
> currently seem to have only one outcome and technical details are
> left on the build master.
This was less of a desired final outcome for us than it was the easiest
way to get started.
I think that our current method of having gigantic text-only log files
is far from optimal (and something that we did years ago in AutoQA
before making the logs more easily accessible) but we haven't improved
the log accessibility/readability in taskotron yet.
Good to know. I guess this is something to discuss and work on?
> How does a representation like RPMGrill translate into
Resultsdb?
> Are there plans/ideas on how to provide a more diversified result
> representation in Resultsdb (e.g. error reason, hint on how to solve
> an error).
I think that having rpmgrill output static html would likely be the
best approach here unless you're planning to continue supporting the
webapp that you linked to (and from previous conversations, I'm under
the impression that was never intended to be a long-term thing).
If we'd have a way to turn the TAP output into details retrievable from
resultsdb, it'll be better than rpmgrill printing HTML?
> 2. Are there plans/ideas about implementing a (fulltext-) search
over
> results?
Nothing written down, no but that's been on my wishlist since before
Taskotron really got started. Unless there's more demand for the
fulltext indexing than I think there is, we'd probably start with
something less complicated like indexing the summary statements.
I'm open to other ideas, though. Just concerned about what fulltext
indexing of all logs might entail.
True. We've had quite big logs if some big packages contain a lot of
regressions. Good to know that it is on the wish list.
> 3. Are there plans/ideas on implementing a waiver mechanism?
Yeah, it'll have to happen before we start gating any
rawhide/branched/stable builds or updates for Fedora. We don't have a
design yet but it'll happen at some point.
Ok. This also seems like something we might be able to contribute to.
Perhaps better to discuss this in another thread?
> [1] -
https://git.fedorahosted.org/git/rpmgrill.git
>
> PS: Yes - I'm aware that there is already a bitbucket repository of
> rpmgrill for Taskotron, but have only had a quick glimpse at it.
Someone was planning to work on getting rpmgrill working in taskotron
and that bitbucket repo was meant to hold the task that's executed in
taskotron.
From our end, that is being tracked as:
https://phab.qadevel.cloud.fedoraproject.org/T281
Let us know if you have any other questions or concerns.
Thanks. I'll have a look at the tasks. I don't want to make any
promises, but I hope we'll find some time to get the rpmgrill tasks in a
'done' state.
Thanks and Regards,
--
Róman Joost
Software Engineer, HSS - Software Engineering and Development (Brisbane)
email: rjoost(a)redhat.com | tz: UTC+10
irc: #hss #rpmdiff