Hi Amit,
thanks for the reaction.
On 03/02/2015 01:42 AM, Amit Saha wrote:
Hi Jaroslav,
----- Original Message -----
> From: "Jaroslav Kortus" <jkortus(a)redhat.com>
> To: "Beaker development" <beaker-devel(a)lists.fedorahosted.org>
> Sent: Saturday, February 28, 2015 7:00:07 AM
> Subject: [Beaker-devel] Integration Tests results
>
> Hi there,
>
> I've just executed full integration tests run with the following result
> (patched develop
> branch):
> Ran 1345 tests in 6731.735s
>
> FAILED (SKIP=3, errors=70, failures=55)
>
> real 112m14.756s
> user 101m11.856s
> sys 7m42.248s
>
> Quite some errors then. Much more than I expected :). Looking at the results
> I have mixed
> feelings.
That's a lot of errors. What distro are you using? We run our pre-merge tests on RHEL
6.6
and we get a near 100% pass all the time. I personally use Fedora 21 and I get a few
failures
mainly due to incompatibilities of third-party libraries between releases. If you have
docker installed,
you can try this approach of running your tests in a F21 container - it will install all
the dependencies
required/database setup, etc:
https://github.com/amitsaha/docker_files/tree/master/dev_workflow/beaker-...
I was running them on 6.5GA, have upgraded now to 6.6, but the issues remain. However, the
clean develop branch does not have that many errors by far, so I suspect I patched
something that looks innocent but causes mayhem in the tests :). Will dig into it deeper
to find it out...
>
> Some selenium tests were failing (almost all in my case). When I ran the same
> test alone
> it passed with no problems.
Can you please paste the error messages? Perhaps fpaste it?
That was my problem - there are no error messages in the normal run-tests.sh run. If I run
the test separately it succeeds.
> FAIL: test_all_tables_use_innodb
> (bkr.inttest.server.test_model.SchemaSanityTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
>
"/root/beaker-private/IntegrationTests/src/bkr/inttest/server/test_model.py",
> line
> 55, in test_all_tables_use_innodb
> table), 'InnoDB')
> AssertionError: u'MyISAM' != 'InnoDB'
>
> I've not created any new tables there (only added columns to existing
> tables). So how can
> this happen?
Sorry, not sure what you mean by that. Beaker uses the InnoDB storage engine, so
MariaDB/MySQL
should be configured appropriately [1].
Hmm, I configured it only following the developer part of patch testing [1]. There is only
a snip about creating extra databases (which I think are created by the tests anyway).
These two parts describing DB config are probably a good candidates for merging :).
https://beaker-project.org/dev/guide/writing-a-patch.html#testing-your-patch
[1]
https://beaker-project.org/dev/guide/getting-started.html
>
> So the main question here is, are all the tests expected to pass on the
> develop branch? Or
> what are the usual results I can compare my patches to? Are there any other
> requirements
> besides the deps of integration tests RPM and
>
https://beaker-project.org/dev/guide/writing-a-patch.html#testing-your-pa...
On RHEL 6.6/ CentOS, they should all pass. On Fedora 20/21, you may have a few failures.
We have a script to setup everything needed for running the tests:
https://git.beaker-project.org/cgit/beaker/tree/Misc/beaker_test_setup.sh...
But, it needs an update. The part where it downloads the Selenium JAR is no longer
relevant.
Thanks, now I have a better idea what to expect.
>
> Thank you,
> Jaroslav.
>
> PS: are the results saved somewhere or is stdout the only option?
> "--debug-log" looks more
> like the framework debugging than the actual tests' debug output...
I think stdout is the only option.
:(
--
Jaroslav.