Dear Kamil,
On Thu, Feb 12, 2015 at 03:41:38AM -0500, Kamil Paral wrote:
> [...]
> I assume, running rpmgrill in Taskotron could mean, that each rpmgrill
> test result translates to an entry in Resultsdb as an outcome (PASSED,
> FAILED).
If you are interested, we support more outcome keywords, like INFO or NEEDS_INSPECTION:
https://docs.qadevel.cloud.fedoraproject.org/libtaskotron/latest/library....
Great. Thanks for the pointer.
> Yet the question for the developer as to why it failed is
> important.
>
> I have mainly three main questions around Taskatron and specifically
> Resultsdb:
>
> 1. Ed Santiago is currently running RPMGrill here:
>
>
http://rpmgrill-fc20.edsantiago.com:5000/recent
>
> which in terms of presenting results to the user provides a different
> experience. The results are grouped by test and provide more insight
> as to why a test failed. Test results in Resultsdb currently seem to
> have only one outcome and technical details are left on the build
> master.
>
> How does a representation like RPMGrill translate into Resultsdb? Are
> there plans/ideas on how to provide a more diversified result
> representation in Resultsdb (e.g. error reason, hint on how to solve
> an error).
At the moment, each result contains a link to the task log, which you can inspect, e.g.:
https://taskotron.fedoraproject.org/resultsdb/results?job_id=45746
points to
https://taskotron.fedoraproject.org/taskmaster//builders/x86_64/builds/36...
That log contains both taskotron execution info and above statements and task output. If
you know an undocumented trick, you can also strip down the path a bit arrive to the
buildbot job info page:
https://taskotron.fedoraproject.org/taskmaster//builders/x86_64/builds/36...
and there you have access to taskotron.log, which contains full debug info, which we use
for debugging libtaskotron issues. This is usually not needed by package maintainers, so
we don't advertise this too much yet.
Thanks.
I agree it would be great if taskotron could execute rpmgrill, and
then upload the results to your special application and nicely display
them. Or alternatively generate the html report locally. Unfortunately
we don't have a support for storing and displaying artifacts (e.g. a
html page) at the moment -- but we plan to do it in the future.
If you want to have something working right now, I see several ways to
go:
1. You could print out a text representation of the rpmgrill results
into the log. You could also post the results into your web
applications, and then include a hyperlink in the log in order to
point people to a better visualization.
2. I guess we could change the "Log →" hyperlink in resultsdb to point
to your web application directly, rather than into our buildbot log.
However, I see two concerns with that, one is that accessing the task
log itself would be non-trivial, and second that displaying any
results at all would be relying on your web app availability. If it
went down, nobody could see any results at all.
In the future, generating the html result locally and then showing it
from resultsdb interface is probably the way to go.
That's were we would like to help to make this happen. Before discussing
any technical details, my first e-mail was aimed to figure out if this
is a desirable feature or not. If that's something which looks like it
is part of where resultsdb is going, I could start a technical
discussion in another thread?
> 3. Are there plans/ideas on implementing a waiver mechanism?
We will definitely need this, for most of the checks. But we haven't
started yet designing that feature, so it won't be available any time
soon.
Thanks for sharing.
BTW, I talked to Miroslav Vadkerti on DevConf about rpmgrill. We
should meet up again in the following weeks and try to cook some
actionable plan for rpmgrill in Taskotron.
Wow great!
Would it make sense to create a roadmap document or is there one
already?
Kind Regards,
--
Róman Joost
Software Engineer, HSS - Software Engineering and Development (Brisbane)
email: rjoost(a)redhat.com | tz: UTC+10
irc: #hss #rpmdiff