resource type ignoring

Jay Shaughnessy jshaughn at redhat.com
Mon Mar 11 14:02:08 UTC 2013


On 3/8/2013 5:06 PM, Jay Shaughnessy wrote:
>
> On 3/8/2013 11:49 AM, John Mazzitelli wrote:
>> [this is more for charles - but I want to put this on the devel list 
>> to have a transparent and public thread]
>>
>> I would like to start a thread dealing with BZ 535289 (aka RHQ-1) - 
>> this was a use-case being discussed:
>>
>>> https://bugzilla.redhat.com/show_bug.cgi?id=rhq-1 specifically:
>>>
>>> Suppose I have machines with multiple agents, each potentially
>>> having ~40k child service resources of a particular type in a given 
>>> agent
>>> inventory.
>>>
>>> Suppose I do not care about all of those resources, but I do care 
>>> about some of them.
>>> How can we support monitoring certain resources while not all 40k.
>> This is where RHQ-1 comes in - the ability to "disable" resource 
>> types (or, said another way, ignore a bunch of resources of a given 
>> type).
>>
>> There are three things here.
>>
>> 1) Be able to disable all new resources for a specific resource type. 
>> So, if I go to a (as yet non-existent) Administration page, I click 
>> some buttons to say "disable type 'foo'", then anytime the agent 
>> sends a inventory report with resources of that 'foo' type, we ignore 
>> them.  They never get into the discovery queue and never get a row in 
>> RHQ_RESOURCE table. If they are already in the discovery queue, you 
>> already have the option to "IGNORE" them, so that's already a feature 
>> and has been for a while. But what if their parent is in the queue? 
>> (the parent is not an ignored type). If we commit the parent, the 
>> agent will send up the child services of type "foo" and we simply 
>> ignore them - we don't commit those ignored resources and we throw 
>> them away.
>>
>> This is the "all of them" approach - we ignore all resources of a 
>> given type.
> This to me sounds very useful.  In fact I could even imaging pushing 
> this all the way out to the plugin descriptor such that a type 
> *starts* out as ignored.  This allows the plugin dev to make a really 
> fine-grained plugin but initially not overwhelm the user, just setting 
> various "core" types as initially enabled.  This would also perhaps 
> help in iterative plugin development.
>>
>> 2) Be able to ignore a select number of resources already in 
>> inventory. We could do this from the inventory pages - I select some 
>> resources, click a (as yet non-existent) "Ignore" button, and they 
>> get ignored - hidden from view - and the agent will somehow need to 
>> disable or ignore them as well. This sounds just like uninventory - 
>> but the problem is, the agent will simply rediscover them during the 
>> next discovery run. We need to somehow tell the agent "uninventory 
>> these resources and don't discover them again". How do we indicate 
>> this inside the agent? I don't know. We probably need to keep the 
>> resource in the agent's inventory with a status of IGNORED. But if we 
>> have 40,000 resources, and we want to IGNORE all of them (or at least 
>> most of them), the agent's memory utilization could be large for 
>> holding resources we don't care about. I don't know a way around this 
>> - we may have to just suck it up and use the memory.
> I agree with pretty much everything you say here. It's like an 
> uninventory that adds to a blacklist.  My only question is around how 
> to unignore or basically take resources off the blacklist. How would 
> you be able to even see what you are ignoring?
>>
>> 3) Be able to not ignore a small amount of resources, but ignore all 
>> the rest. This is very similar to #2 above, but this goes to the fact 
>> that I might have an agent discover 40,000 resources, but I only care 
>> about, say 100 of them. How do we do this?
>>
>> This #3 is the most difficult one. How do we ask the user to select 
>> 39,900 resources and press the ignore button? Not realistic. Not only 
>> that, but in the case where those 40k are child services, today we 
>> never get a chance to accept or reject them from the discovery queue 
>> - once the parent is committed, only then does the server even get to 
>> know what's underneath it (the agent will only discovery child 
>> services after the parent is committed).
>>
>> I think this may involve a big change - in the discovery queue, we 
>> may have to support a new option when pressing the Import button - 
>> somehow we need to support an option "do not committed child 
>> services".  This would then commit the server but then somehow show 
>> the child services in the discovery queue, at which time you can pick 
>> which ones you want to commit to inventory (thus, you only pick 100, 
>> and you keep the other 39,900 in the queue in NEW state - I suppose 
>> you could opt to then IGNORE them? Many problems abound with this. 
>> The UI tree in the queue view doesn't support N layers deep - only 
>> the top platform and the immediate child servers are shown. Also, how 
>> do we show 40,000 (or something that large) rows in the tree?  Do we 
>> hit performance problems in the UI?
>>
>> This is a huge change - from the discovery queue UI down to the 
>> server then to the agent, changes would need to be made to support this.
> This is pretty tricky.  One mechanism that may work here and in #2 is 
> to just make it pattern based in some way.  Some list of regex 
> patterns in an include/exclude manner. Not sure of any details but 
> you'd need to be able to do comparisons against info provided by 
> initial discovery.
>
> Another option is to look at "keep" as opposed to, or complementing 
> "ignore".

Configuration filters could be very powerful.  For example, a plugin 
config that is used to perform filtering based on the actual plugin 
config discovered for a resource.  For example name "==Foo".  The search 
bar syntax may be useful for string operators here.

The nice thing is that this filtering could potentially be performed 
agent-side.  Minimally, it could be applied at inventory merge.

Group definitions, and groups, could possibly be used to act like 
filters. The big downside here is that it's probably too "lazy". The 
resources would already have to be in inventory to be ignored, as group 
defs are applied via db queries on resources in inventory. We'd likely 
want to have Ignored resource be kept agent-side, or minimally be 
filtered at merge-time.

  The nice thing about a filter approach is it lends itself to being 
able to see what is being ignored.


>>
>> So, #1 and #2 should be doable (though difficult to fully implement). 
>> #3 is very difficult and would require some design changes and some 
>> deep thinking as to how this would effect all tiers of RHQ. At this 
>> point in time, I'm thinking we just don't attempt #3.
> I think #1 has a lot of value.  As soon as you start cherry-picking 
> resources it gets trickier for sure.
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel at lists.fedorahosted.org
>> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel



More information about the rhq-devel mailing list