----- Original Message -----
From: "Jaroslav Kortus" <jkortus(a)redhat.com>
To: "Beaker development" <beaker-devel(a)lists.fedorahosted.org>
Sent: Saturday, February 28, 2015 7:00:07 AM
Subject: [Beaker-devel] Integration Tests results
I've just executed full integration tests run with the following result
Ran 1345 tests in 6731.735s
FAILED (SKIP=3, errors=70, failures=55)
Quite some errors then. Much more than I expected :). Looking at the results
I have mixed
That's a lot of errors. What distro are you using? We run our pre-merge tests on RHEL
and we get a near 100% pass all the time. I personally use Fedora 21 and I get a few
mainly due to incompatibilities of third-party libraries between releases. If you have
you can try this approach of running your tests in a F21 container - it will install all
required/database setup, etc:
Some selenium tests were failing (almost all in my case). When I ran the same
it passed with no problems.
Can you please paste the error messages? Perhaps fpaste it?
To get more data I ran the suite with "-sv" switch. Looking at the output
there are some
things that I'm not sure are errors or not.
bkr.server.mail: ERROR: Exception thrown when trying to send mail
error: [Errno 111] Connection refused
bkr.server.tests.data_setup: DEBUG: Marked R:706 as complete with result Pass
Yeah, you can safely ignore this. It needs a local SMTP daemon running iirc and ignores
it if it doesn't find it.
Traceback (most recent call last):
55, in test_all_tables_use_innodb
AssertionError: u'MyISAM' != 'InnoDB'
I've not created any new tables there (only added columns to existing
tables). So how can
Sorry, not sure what you mean by that. Beaker uses the InnoDB storage engine, so
should be configured appropriately .
So the main question here is, are all the tests expected to pass on the
develop branch? Or
what are the usual results I can compare my patches to? Are there any other
besides the deps of integration tests RPM and
On RHEL 6.6/ CentOS, they should all pass. On Fedora 20/21, you may have a few failures.
We have a script to setup everything needed for running the tests:
But, it needs an update. The part where it downloads the Selenium JAR is no longer
PS: are the results saved somewhere or is stdout the only option?
"--debug-log" looks more
like the framework debugging than the actual tests' debug output...
I think stdout is the only option.
Hope that helps,
SED, Hosted & Shared Services
Red Hat, Inc.