Hi all,
As part of implementing https://bugzilla.redhat.com/show_bug.cgi?id=846185 and https://bugzilla.redhat.com/show_bug.cgi?id=1121462, I was thinking that the best user experience would be if we ask nothing of the user. They need to just click on a system's "Update Inventory" button or run the "bkr update-inventory <fqdn>" command to update the inventory data for the system.
So, the key step is to figure out which distro to use. RHEL6/CentOS6 would cover most of the hardware, but there may be exceptions for older/newer hardware archs. So, here is what I am thinking:
1. Start with RHEL6/CentOS6, and check if the system is compatible with it. If yes, go to step 4, else go to step 2.
2. Is the system compatible with RHEL7/CentOS7? If yes, go to step 4, else go to step 3.
3. Is the system compatible with RHEL5/CentOS5? If yes, go to step 4, else error out.
4. Submit a job with the chosen distro and /distribution/install and /distribution/inventory tasks. (The job will use the <hostRequires force = .. /> element so that all non-removed systems can be inventoried as long as the job owner has the sufficient rights on the system)
One immediate drawback of this is using hard coded distro names above.
Another idea I can think of is have a system level ksmeta variable defined such as "inventory_distro=MyOSMajor" and use that. But that is asking of system owners/beaker admins to have to do that explicitly. It would also vastly simplify the implementation.
Thoughts?
Best, Amit.
Excerpts from Amit Saha's message of 2015-05-04 14:56 +10:00:
So, the key step is to figure out which distro to use. RHEL6/CentOS6 would cover most of the hardware, but there may be exceptions for older/newer hardware archs. So, here is what I am thinking:
- Start with RHEL6/CentOS6, and check if the system is compatible
with it. If yes, go to step 4, else go to step 2.
- Is the system compatible with RHEL7/CentOS7? If yes, go to step 4,
else go to step 3.
- Is the system compatible with RHEL5/CentOS5? If yes, go to step 4,
else error out.
- Submit a job with the chosen distro and /distribution/install and
/distribution/inventory tasks. (The job will use the <hostRequires force = .. /> element so that all non-removed systems can be inventoried as long as the job owner has the sufficient rights on the system)
Yes, this sounds good. Beaker will need to pick the family based on what is not excluded and also what is available in the lab. RHEL should be preferred over CentOS. So I guess the priority list is:
RedHatEnterpriseLinux6 CentOS6 RedHatEnterpriseLinux7 CentOS7 RedHatEnterpriseLinux5 CentOS5
Can we also add Fedora (any release I suppose) to the bottom of the list? lshw will be there, albeit without our patches, but what about beaker-system-scan? Can we add it to our Fedora harness repos? I am just thinking of e.g. beaker.fedoraproject.org which only has Fedora trees.
One immediate drawback of this is using hard coded distro names above.
Another idea I can think of is have a system level ksmeta variable defined such as "inventory_distro=MyOSMajor" and use that. But that is asking of system owners/beaker admins to have to do that explicitly. It would also vastly simplify the implementation.
Allowing the admin/system owner to specify might be handy but I don't think we can use ksmeta for it. All the current ksmeta computation assumes you already have a distro tree to start with, whereas here we would be using the ksmeta to actually pick a distro tree. I suspect the implementation would actually be very messy.
I think having some hardcoded rules in Beaker is fine, if someone wants to use a different distro they can always construct a job explicitly (using bkr machine-test or workflow-simple or any other way).
On 05/04/2015 03:27 PM, Dan Callaghan wrote:
I think having some hardcoded rules in Beaker is fine, if someone wants to use a different distro they can always construct a job explicitly (using bkr machine-test or workflow-simple or any other way).
Right, and when it comes to "What works best for inventorying systems?", the Beaker team are actually the best positioned to come up with an answer.
One thing that did occur to me is that it's likely desirable to use the latest *released* version of the relevant family, rather than the latest version available (since the latter might be a testing tree).
Cheers, Nick.
Excerpts from Nick Coghlan's message of 2015-05-04 15:33 +10:00:
One thing that did occur to me is that it's likely desirable to use the latest *released* version of the relevant family, rather than the latest version available (since the latter might be a testing tree).
Yes that's true, the filter criterion should be
<distro_arch value="$thearch"/> <distro_family value="$selectedfamily"/> <distro_tag value="RELEASED"/>
which reminds me that the logic also needs to pick a preferred arch when the system supports multiple arches (should prefer 64-bit over 32-bit/31-bit, and for PPC prefer big-endian).
And it also needs to handle systems which don't support *any* RHEL/CentOS/Fedora but only a RHEL-alike under a different name. For example the internal-only aarch64 product.
Also note that in those cases, there will probably be no RELEASED trees. Also note that RELEASED is done by hand by admins nowadays so a fresh Beaker installation may not have any trees with that tag. Therefore if there are no trees in the family tagged RELEASED, it should just use the latest available tree.
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Monday, May 4, 2015 3:39:28 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-04 15:33 +10:00:
One thing that did occur to me is that it's likely desirable to use the latest *released* version of the relevant family, rather than the latest version available (since the latter might be a testing tree).
Yes that's true, the filter criterion should be
<distro_arch value="$thearch"/> <distro_family value="$selectedfamily"/> <distro_tag value="RELEASED"/>
which reminds me that the logic also needs to pick a preferred arch when the system supports multiple arches (should prefer 64-bit over 32-bit/31-bit, and for PPC prefer big-endian).
And it also needs to handle systems which don't support *any* RHEL/CentOS/Fedora but only a RHEL-alike under a different name. For example the internal-only aarch64 product.
Also note that in those cases, there will probably be no RELEASED trees. Also note that RELEASED is done by hand by admins nowadays so a fresh Beaker installation may not have any trees with that tag. Therefore if there are no trees in the family tagged RELEASED, it should just use the latest available tree.
Thanks for the suggestions and comments. I have some code which now implements the above:
http://fpaste.org/218852/30883888/
As far as my tests tell me, i think this does what we want as far as distro selection is concerned.
The exception is for distros which are RHEL like but has a different name. I will think about it.
Excerpts from Amit Saha's message of 2015-05-06 13:48 +10:00:
Thanks for the suggestions and comments. I have some code which now implements the above:
http://fpaste.org/218852/30883888/
As far as my tests tell me, i think this does what we want as far as distro selection is concerned.
Looks pretty good, although it looks like the method will return either a Distro or a DistroTree in different code paths.
The exception is for distros which are RHEL like but has a different name. I will think about it.
I think if none of the pre-defined list of OS majors matches then it should just pick *anything* and we just hope it will work. That should cover the RHEL-alikes.
On 05/06/2015 02:34 PM, Dan Callaghan wrote:
Excerpts from Amit Saha's message of 2015-05-06 13:48 +10:00:
The exception is for distros which are RHEL like but has a different name. I will think about it.
I think if none of the pre-defined list of OS majors matches then it should just pick *anything* and we just hope it will work. That should cover the RHEL-alikes.
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Cheers, Nick.
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Good idea.
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Good idea.
Agree, thought about it earlier as well.
----- Original Message -----
From: "Amit Saha" asaha@redhat.com To: "Beaker development" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:52:09 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Good idea.
Agree, thought about it earlier as well.
Okay, uploaded a preliminary patch for the API endpoint: http://gerrit.beaker-project.org/#/c/4201/
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Would be even better to store this into the db as people can change it via Web UI?
Excerpts from Matt Jia's message of 2015-05-07 08:38 +10:00:
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Would be even better to store this into the db as people can change it via Web UI?
Storing this in the db is overkill, the only reason we need a "preferred list of distro families" at all is because we still haven't finished the migration to lshw. Once that is done, the choice of distro becomes basically irrelevant because lshw should be giving us the same info everywhere. This is only a stop-gap while we have smolt on RHEL5/6 and lshw elsewhere.
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Storing this in the db is overkill, the only reason we need a "preferred list of distro families" at all is because we still haven't finished the migration to lshw. Once that is done, the choice of distro becomes basically irrelevant because lshw should be giving us the same info everywhere. This is only a stop-gap while we have smolt on RHEL5/6 and lshw elsewhere.
Fair enough, I did not see the context.
On 05/07/2015 12:51 AM, Dan Callaghan wrote:
Excerpts from Matt Jia's message of 2015-05-07 08:38 +10:00:
----- Original Message -----
From: "Dan Callaghan" dcallagh@redhat.com To: "beaker-devel" beaker-devel@lists.fedorahosted.org Sent: Wednesday, May 6, 2015 3:48:16 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
Excerpts from Nick Coghlan's message of 2015-05-06 15:40 +10:00:
Could we pull the list of preferred inventory distros from a configuration file rather than hardcoding it? That should be easier to update when Fedora versions cycle, and also provides a way for instance administrators to replace the default list with a different one.
Would be even better to store this into the db as people can change it via Web UI?
Storing this in the db is overkill, the only reason we need a "preferred list of distro families" at all is because we still haven't finished the migration to lshw. Once that is done, the choice of distro becomes basically irrelevant because lshw should be giving us the same info everywhere. This is only a stop-gap while we have smolt on RHEL5/6 and lshw elsewhere.
What about tagging the distros in beaker and use them for inventory based on these tags? Something like inventory_prio1, inventory_prio2,...
That's both WebUI and CLI friendly and does not require any config changes.
Cheers, Jaroslav.
Excerpts from Jaroslav Kortus's message of 2015-05-13 01:41 +10:00:
What about tagging the distros in beaker and use them for inventory based on these tags? Something like inventory_prio1, inventory_prio2,...
That's both WebUI and CLI friendly and does not require any config changes.
Using distro tags is quite a good idea, the downside is that it would require continual maintenance. For example every RHEL x.y GA we would have to remember to tag the new version. Similarly every Fedora GA. And then there are the other in-development products like the aarch64 distro, which has never had a GA and usually the best thing we can do is use the latest compose available.
Since this is really just a temporary measure I think an admin configurable list with hardcoded default is an okay approach.
On Mon, 2015-05-04 at 15:27 +1000, Dan Callaghan wrote:
Yes, this sounds good. Beaker will need to pick the family based on what is not excluded and also what is available in the lab. RHEL should be preferred over CentOS. So I guess the priority list is:
RedHatEnterpriseLinux6 CentOS6 RedHatEnterpriseLinux7 CentOS7 RedHatEnterpriseLinux5 CentOS5
Can we also add Fedora (any release I suppose) to the bottom of the list? lshw will be there, albeit without our patches, but what about beaker-system-scan? Can we add it to our Fedora harness repos? I am just thinking of e.g. beaker.fedoraproject.org which only has Fedora trees.
Hi all,
I would like just remind that the POWERPC machines support the compatibility modes, so a POWER(x) machine can always run in POWER(x-1) mode. This happens on POWER8 machines when RHEL6 is installed there. Since RHEL6 is not compatible with POWER8, the machine is switched into POWER7-compatible mode when you run RHEL6 there [1].
So with the mentioned priority list, POWER8 machines will be classified as POWER7 (as they usualy are). It doesn't matter for machine filtering presets, since they are fine with the revision number, which is (should be, at least I hope it still is) propagated into "model" field in the DB. But it is confusing for humans when viewing the systems in the WebUI. So in this case, prefering RHEL7 over RHEL6 would solve the problem.
Thanks. Michael
[1] http://www-01.ibm.com/support/knowledgecenter/linuxonibm/liacd/liacd-distro-...
----- Original Message -----
From: "Michael Petlan" mpetlan@redhat.com To: "Beaker development" beaker-devel@lists.fedorahosted.org Sent: Tuesday, June 2, 2015 6:59:52 PM Subject: Re: [Beaker-devel] Simplify inventorying systems in Beaker
On Mon, 2015-05-04 at 15:27 +1000, Dan Callaghan wrote:
Yes, this sounds good. Beaker will need to pick the family based on what is not excluded and also what is available in the lab. RHEL should be preferred over CentOS. So I guess the priority list is:
RedHatEnterpriseLinux6 CentOS6 RedHatEnterpriseLinux7 CentOS7 RedHatEnterpriseLinux5 CentOS5
Can we also add Fedora (any release I suppose) to the bottom of the list? lshw will be there, albeit without our patches, but what about beaker-system-scan? Can we add it to our Fedora harness repos? I am just thinking of e.g. beaker.fedoraproject.org which only has Fedora trees.
Hi all,
I would like just remind that the POWERPC machines support the compatibility modes, so a POWER(x) machine can always run in POWER(x-1) mode. This happens on POWER8 machines when RHEL6 is installed there. Since RHEL6 is not compatible with POWER8, the machine is switched into POWER7-compatible mode when you run RHEL6 there [1].
So with the mentioned priority list, POWER8 machines will be classified as POWER7 (as they usualy are). It doesn't matter for machine filtering presets, since they are fine with the revision number, which is (should be, at least I hope it still is) propagated into "model" field in the DB. But it is confusing for humans when viewing the systems in the WebUI. So in this case, prefering RHEL7 over RHEL6 would solve the problem.
Thank you for the note, Michael. I have added it to the bugzilla so that we can add it to our release note.
beaker-devel@lists.fedorahosted.org