On Tue, 2017-02-14 at 14:46 -0500, Kamil Paral wrote:
> The only problem with this kind of testing is, that we still
don't really
> have a good way to test trigger, as it is tied to external events. My idea
> here was that I could add something like wiki edit consumer, and trigger
> tasks off of that, making that one "triggering" edit from inside the
> testsuite.
In my experience, some fedmsg notifications are quite delayed from
time to time, easily by several hours. But I don't know if the delay
is in actual fedmsg bus, or just in FMN (which I use to get notified
on IRC). I suspect rather FMN. But if that turned out to be a fedmsg
delay, that might make this approach impractical (or at least less
reliable). So something to consider and possibly investigate.
I believe it's FMN. I've noticed the same delay with FMN notifications,
but I've never noticed any delay with the actual fedmsg notifications
used by all the stuff we've built around release validation (compose
complete fedmsgs, openQA test complete fedmsgs etc.)
I think it would be better to fake the input data. Either by having
our own fedmsg producer and/or bus (I don't know precisely how fedmsg
works and how hard is to set up something like that),
Are you aware of fedmsg-dg-replay? It's a fairly easy way to 'replay'
fedmsgs for testing. All you need (IIRC) is the fedmsg-relay service
running on the same system, and you can run
'fedmsg-dg-replay --msg-id <somemsgid>' and the message will be
replayed on the local bus (if the message is .prod. or .stg. , this
will be changed to .dev. , and the message will not be signed).
I don't remember exactly how the taskotron trigger code works, but it
shouldn't be too hard to configure a trigger to accept unsigned
messages with .dev. topics, for the purpose of testing with -replay. In
the fedmsg consumers I've written I have a pattern of providing three
consumers, one which listens for .prod. messages, one for .stg.
messages, and one which listens for for .dev. messages and is intended
for testing with fedmsg-dg-replay.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net