On Wed, Nov 30, 2016 at 11:14 AM, Adam Williamson <adamwill@fedoraproject.org> wrote:
On Wed, 2016-11-30 at 02:10 -0800, Adam Williamson wrote:
> On Wed, 2016-11-30 at 09:38 +0100, Josef Skladanka wrote:
> > So if this is what you wanted to do (data validation), it might be a good
> > idea to have that submitter middleware.
>
> Yeah, that's really kind of the key 'job' of that layer. Remember,
> we're dealing with *manual* testing here. We can't really just have a
> webapp that forwards whatever the hell people manage to stuff through
> its input fields into ResultsDB.

I guess another way you could look at it is, this would be the layer
where we actually define what kinds of manual test results we want to
store in ResultsDB, and what the format for each type should be. I
kinda like the idea that we could use the same middleware to do that
job for various different frontends for submitting and viewing results,
e.g. the webUI part of this project, a CLI app like relval, and a
different webUI like testdays...

Yes, that IMO makes a lot of sense. Especially if we want to target multiple "input tools". Then it might make sense to have what I was discussing in the previous post (and what you have been, I think talking about)  - a format (two of them, actually) that defines:
1) what testcases are relevant for X (where X is, say Rawhide nightly testing, Testday for translations, foobar)
2) required structure (fields, types of the field) of the response

The question here is, whether the "required structure" is better off "per testcase" (i.e. "this testcase always requires these fields") or "per context" (i.e. results for this "thing" always require these fields) or event those combined ("this testcase, in this context, requires X, Y and Z, but in this other context, it only needs FOOBAR")

I would try not to go the third way, because that is really prone to erros IMO, and I'm not sure that "per context" is always right. So for me, the "TCMS" part of the data, should be:
1) testcases (with required fields/types of the fields in the "result response"
2) testplans - which testcases, possibly organized into groups. Maybe even dependencies + saying "I need testcase X to pass, Y can be pass or warn, Z can be whatever when A passes, for the testplan to pass"

But this is fairly complex thing, to be honest, and it would be the first and only useable TCMS in the world (from my point of view).

Let's do it!