This is an interesting problem. We want to post comments from AutoQA into Bodhi. But some of our tests are designed in such a way that they test a particular package several (dozen) times. For example depcheck will test all packages in -updates-pending every time this tag is updated (new package is added/removed).
If some package fails depcheck, we probably don't want to add 10 comments to its Bodhi page saying "depcheck failed" every single day.
So we devised a solution where we won't post duplicate comments. If the first comment says "depcheck fails", we won't post any new comment until it says something else.
But that has a catch - we also want to add a URL to the full results of the test. If we don't post duplicate comments, the maintainer will see just the original URL. But the reason of the failure might have changed. How do we tell the maintainer what's wrong when we don't post updated URL?
One silly idea I've come up with is that we will post duplicate comments, but only once in a few days. For example we will post "depcheck failed (URL)" again only after at least two days from the previous "depcheck failed (URL)".
Ideas?
On Mon, 2010-11-15 at 12:16 -0500, Kamil Paral wrote:
This is an interesting problem. We want to post comments from AutoQA into Bodhi. But some of our tests are designed in such a way that they test a particular package several (dozen) times. For example depcheck will test all packages in -updates-pending every time this tag is updated (new package is added/removed).
If some package fails depcheck, we probably don't want to add 10 comments to its Bodhi page saying "depcheck failed" every single day.
So we devised a solution where we won't post duplicate comments. If the first comment says "depcheck fails", we won't post any new comment until it says something else.
But that has a catch - we also want to add a URL to the full results of the test. If we don't post duplicate comments, the maintainer will see just the original URL. But the reason of the failure might have changed. How do we tell the maintainer what's wrong when we don't post updated URL?
One silly idea I've come up with is that we will post duplicate comments, but only once in a few days. For example we will post "depcheck failed (URL)" again only after at least two days from the previous "depcheck failed (URL)".
That sounds much better than posting a comment each time that update fails (which would be *many* times a day until the issue is resolved).
Ideas?
I would need to see what the test output for depcheck looks like ... I have no idea right now. However, one thing I'm using now might be helpful. I wrote a wrapper around the 'repoclosure' command that auto-files bugs into bugzilla for repoclosure issues. The wrapper script hashes the repoclosure output for each failure and attempts to make it unique enough to prevent filing duplicate bugs. It works well enough that I've been happy with the result. Using this hashing, we could detect when the failure reason has changed, and add a new bodhi comment accordingly.
You can see bugs that were filed by the script at http://bit.ly/cTpTLt . I don't have the script published anywhere, I'm still working on how best to communicate this sample script (autoqa-devel or a blog post etc...).
Thanks, James
autoqa-devel@lists.fedorahosted.org