1) Why there are 2 different tests directories (tests and newtests)? Are there XML-RPC tests in "tests" directory which are not covered in XML-RPC tests in "newtests" directory? I'd like to merge those.
2) What's the benefit in numbering tests? It seems to me it only introduces overhead of renaming tests when one is deleted (if we want to avoid numbering gaps).
3) CLI import tests access a /vagrant directory which I obviously developers will not have in their local workstations. Is there a test server we could use to test that? Otherwise, it should be upstream?
4) CLI tests only check return code. Should they not test at least a part of the output (eg "TASK COMPLETE" string) ?
5) I see there are no CLI tests for object operations (eg cobbler distro add). I thought that might be on purpose because XML-RPC tests already cover the main logic of these operations, but then the same would apply to direct commands and testing them would not be needed. I assume these tests are missing because nobody implemented them yet?
Regards, Alan Evangelista
Hi,
Sometime ago.. someone started work to overhaul the unit/integration tests, i suppose this work never got finished. Frankly i'm not all that familiar with the current tests and to be honest my time atm is too limited to dive into that subject...
Too "answer" your questions: 1) Merging both test directories seems wise. 2) I don't think there's any real benefit; your reasoning makes sense 3) I suppose this test server is hosted by James. I agree that it should be available to all developers or the test should be changed. 4) Return-codes are an issue with the CLI, so yes output parsing should probably be added 5) Yes
So, If you have an interest in improving Cobblers quality and are willing to step up and improve the test framework i'm all for that. I'm willing to merge any improvements!
On Tue, Mar 25, 2014 at 3:40 PM, Alan Evangelista <alanoe@linux.vnet.ibm.com
wrote:
- Why there are 2 different tests directories (tests and newtests)? Are
there XML-RPC tests in "tests" directory which are not covered in XML-RPC tests in "newtests" directory? I'd like to merge those.
- What's the benefit in numbering tests? It seems to me it only
introduces overhead of renaming tests when one is deleted (if we want to avoid numbering gaps).
- CLI import tests access a /vagrant directory which I obviously
developers will not have in their local workstations. Is there a test server we could use to test that? Otherwise, it should be upstream?
- CLI tests only check return code. Should they not test at least a part
of the output (eg "TASK COMPLETE" string) ?
- I see there are no CLI tests for object operations (eg cobbler distro
add). I thought that might be on purpose because XML-RPC tests already cover the main logic of these operations, but then the same would apply to direct commands and testing them would not be needed. I assume these tests are missing because nobody implemented them yet?
Regards, Alan Evangelista
cobbler-devel mailing list cobbler-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/cobbler-devel
On 04/09/2014 10:18 AM, Jörgen Maas wrote:
Hi,
Sometime ago.. someone started work to overhaul the unit/integration tests, i suppose this work never got finished. Frankly i'm not all that familiar with the current tests and to be honest my time atm is too limited to dive into that subject...
Too "answer" your questions:
- Merging both test directories seems wise.
- I don't think there's any real benefit; your reasoning makes sense
- I suppose this test server is hosted by James. I agree that it
should be available to all developers or the test should be changed. 4) Return-codes are an issue with the CLI, so yes output parsing should probably be added 5) Yes
So, If you have an interest in improving Cobblers quality and are willing to step up and improve the test framework i'm all for that. I'm willing to merge any improvements!
Thanks for the feedback. I have already updated CLI / XML-RPC tests and merged both tests directories. I will rebase that on top of master and submit the patches upstream as soon as I can (probably next few weeks).
I do not know James. He accesses this mailing list often or should I contact him directly to talk about the test server?
Regards, Alan Evangelista
cobbler-devel@lists.fedorahosted.org