On Wed, May 02, 2012 at 08:08:07PM +0800, wenchao xia wrote:
于 2012-5-2 19:53, Dan Kenigsberg 写道:
On Wed, May 02, 2012 at 08:36:18AM +0800, wenchao xia wrote:
于 2012-4-30 17:26, Dan Kenigsberg 写道:
On Fri, Apr 27, 2012 at 11:19:21AM +0800, wenchao xia wrote:
于 2012-4-26 21:57, Adam Litke 写道:
On Thu, Apr 26, 2012 at 05:24:29PM +0800, wenchao xia wrote: >Hello, > vdsm now have UT suits for developer, but sometimes building and >installation machine is not the same one, and additional check is need >which is ignored at building time, so I think some test cases should be >also run on target machine to check potential errors, Then I want to >introduce a sub package as VT suits. >Purpose: > UT: for developers, more likely a white box, running on building >environment. > VT: for user and deployment, more likely a black box, running on > product or testing environment, all known issue should be covered. > >Supposed approach: > 1 modify building system to generate package: vdsm-VT.rpm.
I would prefer the package name 'vdsm-test.rpm' and this package should include unit tests and verification tests.
> 2 install as an option, after install, user type "vdsm-VT" would make >the test begin.
The test runner should be able to run the full suite of unit tests and verification tests (with an option to run only unit tests or only verification tests). This can be the same program that we use in the build environment except that it will set the PYTHONPATH differently to target the installed files.
good idea, checking on the details of vdsm-hack in tests.
>Planned details: > 1 Going to place cases in vdsm project in ./tests/VT. > 2 On installation will move some useful UT cases into VT.
If you follow my approach above, you would simply package the whole tests/ directory and no copying would be necessary.
> 3 use same framework UT used. > 4 two sub dir in test/VT: user_case_test;general_test.
What is the difference between these two types of tests?
user_cases would holds tests simulating the scenario user calling vdsm, and it could be examples for user. xmlrpc(vdscli) and rest-api cases would go here. I guess xmlrpc testcases already exist in engine side, and moving them here makes vdsm more independent and robust. general_test would holds test detecting target environment problem, such as authority settings, and thread pipe tests, etc. The names of the two directories are not good, need to be justified.
> It is just a scratch from my mind, so I'd like hear your opinions.
Thanks for the idea! Do you have a sample test for the verification test suite? Will it be your pipe deadlock test?
Yeah, plan to place the test in it.
As summary, I guess a better structure of test would be as following:
vdsm |--tests |test_runner.sh |--UT |--VT |--xmlrpc |--rest |--general
Sounds reasonable. But what's "general"? Isn't VT going to be using some kind of binding in any case?
planning to place some environment test case here, such as "thread and pipe test".
Why would you need a running Vdsm instance for such a test?
no need indeed, just want to find a place to holds test for generic issue. Maybe more check about the environment settings such as user vdsm/kvm authority check could also set here.(just an example, for now this test is not needed).
Well, when the need rise, we can add more directories. Luckily git does not have empty dirs ;-)