On Mon, 8 Feb 2016 06:54:17 -0500 (EST)
Josef Skladanka <jskladan(a)redhat.com> wrote:
This is an initial take on stuff that was discussed in person during
Tim's stay in Brno. Sending to the list for additional
discussion/fine-tuning.
= What =
Talking rpmgrill-like checks, there will be a need to be able to
facilitate some kind of structure for representing that a check is
composed of multiple subchecks, for example:
check - FAILED
subcheck1 - PASSED
subcheck2 - PASSED
subcheck3 - FAILED
subcheck4 - PASSED
!IMPORTANT: ResultsDB will not be responsible for computing the
result value for an "upper level" Result from the subchecks - this is
the check's (check developer's) responsibility.
Are we going to require an overall status for the grouped subchecks?
Would ResultsDB do any of the grouping outside of facilitating the
namespaces?
This could (should?) be done on two levels:
* physicall nesting the Results as such in the database structure
* namespacing Testcases
For the start, we decided to go with the simplistic approach of
nesting the Testcases via a simple namespacing - thus allowing a
frontend/query tool to reconstruct the structure at least to some
extent e.g. by relying on a premise, that Results that are a part of
one Job can be converted to a tree-like structure, based on the
Testcase namespacing, at least to some extent, if needed.
== Namespace structure ==
We'll be providing some top-level namespaces (list not yet final):
* app
* fedoraqa
* package
* scratch (?)
These will the further split to facilitate for a finer level of
granularity, e.g.:
app
testdays
powermanagement
pm-suspendr
fedoraqa
depcheck
rpmgrill
package
<pkgname>
unit
func
Everything below the top-level will be 100% user defined. We might
have recommendations for specific namespaces (like
package.<pkgname>), but we won't be enforcing them.
The structure will be implemented (at least in the initial
implementation) just via the Testcase.name attribute in the DB, using
dots as a separator. Later on, we can easily add an easy way of using
wildcards for searching (e.g. app.testdays.*.pm-suspendr)
!IMPORTANT: the namespaces are not to be used to represent
"additional data" about the underlying result such as architecture,
item under test, etc. This is what the Result's extra-data
(ResultData) is there for.
NOTE: Although we do not encourage to store the results to the finest
granularity "just because" (e.g. individual results of a unittest
testsuite), we leave it to the check-developer's judgement. If there
is a usecase for it, let them do it, we don't care, as long as the DB
is not extremely overloaded.
To nitpick a bit, I had to read the first part of that a couple times
before understanding.
Maybe something like:
Although the check developer has the final say over the granularity of
results stored, we do not suggest storing results simply for the sake
of having them
That being said, how many results are we talking about here if the DB
would get overloaded?
== Authentication/Authorization ==
We'll be continuing with the "expect no malice" approach we have
right now. There will be just a simple limitation in libtaskotron:
check git clone
if cloned: only allow non-pkg namespace if __our__ repo
else: do whatever, don't care
in libtaskotron:
check the git checkout like listed above
have whitelisted napespace repos in config
!FIXME: the mechanism above is just copied from tflink's notes, I
can't remember the details :/
My memory of the idea is:
The check that we'd do for non-pkg namespaces would be to look at the
git checkout during task execution - if that repo was cloned from a set
of whitelisted repos (kept in config), it would be allowed to use
non-pkg namespaces when reporting to resultsdb.
It's certainly not perfect and could be worked around without too much
trouble but I think it'll be enough so long as we have sane behavior in
the case where we don't even find a repo to look at. Putting the
repo-checking code into libtaskotron should decrease the amount of work
and error cases that we have to deal with (assuming non-malicious
users).
Tim
== TODOs ==
* Change our checks to use the fedoraqa namespace
* Implement repo checking in libtaskotron
* Write docs for how to report stuff to ResultsDB
* Come up with root nodes for namespaces
_______________________________________________
qa-devel mailing list
qa-devel(a)lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/qa-devel@lists.fedoraproject.org