Another strange stacktrace in agent log

Jay Shaughnessy jshaughn at redhat.com
Mon Feb 10 16:34:53 UTC 2014


OK, looks like no one is going for my soft fix :)    The proposed 
datatype change is fine, too.  Performing discovery on the concurrently 
modified children list should dwindle to nothing as needed, I think.  
I'm not totally sure it won't end up throwing some sort of exception if 
it ends up doing work under a "dead" parent, but I guess we'll see if 
that falls out later.  We want discovery to complete without exception 
even when modifications are taking place.

A couple of notes/questions on the current branch changes:
- I'm not sure I quite understand the changes to Resource, with the 
customChildResourcesCollection flag. is this really necessary?
- I think you want to update InventoryManager.compactResource() with:

         if (resource.getChildResources().isEmpty()) {
             resource.setChildResources(Collections.EMPTY_SET);
         } else {
             resource.setChildResources(new 
CopyOnWriteArraySet<Resource>(resource.getChildResources());
         }

   This will replace the server-side type with the thread-safe type as 
resources get processed by inventory sync.

- While you're in there, please kill the unnecessary comments at the top 
of cloneResourceWithoutChildren().


On 2/10/2014 8:40 AM, Thomas Segismont wrote:
> Le 08/02/2014 03:40, Elias Ross a écrit :
>>
>> On Fri, Feb 7, 2014 at 10:33 AM, Jay Shaughnessy <jshaughn at redhat.com
>> <mailto:jshaughn at redhat.com>> wrote:
>>
>>
>>     That datatype could be useful for us in places.  I don't think we
>>     use it anywhere currently.  As for this issue, though, I actually
>>     think we should stop doing discovery on that parent node if its
>>     child Set is being modified (likely uninventory).  As such, I just
>>     pushed a change to catch the ConcurrentModificationExceptio__n and
>>     log a warning (__dbd05bd84ff577c98fbd723bf557b0__face03736b).
>>
>>
>> I've seen infinite loops (100% CPU) resulting in two threads modifying
>> the same HashMap (which is or was the underlying implementation for
>> HashSet.) This may or may not result in an exception thrown, depending
>> on thread state. Although this is fairly unlikely, it is possible. What
>> I saw was two threads modifying the same bucket. This was some years ago
>> in a program I wrote.
>>
>> See also this:
>> http://mailinator.blogspot.com/2009/06/beautiful-race-condition.html
>>
>> So just the remote chance two threads can modify the same non-threadsafe
>> data structure has me worried and catching an exception seems like a
>> very casual fix.
>>
>>
>>
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel at lists.fedorahosted.org
>> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel
>>
>
> I had this topic near the top of my todo list but found not time last 
> week to investigate, so sorry for the late feedback.
>
> I have pushed a commit in a topic branch (bug/CME_In_PC):
> https://git.fedorahosted.org/cgit/rhq/rhq.git/commit/?h=bug/CME_In_PC&id=38505e7b5c1efbdbf31b12e31cd83dcdc202ad5e 
>
>
> It reverts Jay's commit and makes use of a CopyOnWriteArraySet for the 
> childResources field in Resource objects (only on the agent side).
>
> The upside is that:
> * it is thread-safe again
> * it is less memory hungry than the default-initialized 
> ConcurrentHashMap we used to have
>
> The downside:
> * it is less conservative from a memory allocation perspective as all 
> Resource objects (in the agent view of the inventory) will hold an 
> instance of this collection.
>
> I think we should run the perf tests again and see how it impacts the 
> agent footprint. I'm quite confident but actual figures are a better 
> indicator :)
>
> Thomas
>
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/rhq-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.fedorahosted.org/pipermail/rhq-devel/attachments/20140210/16cb2c09/attachment.html>


More information about the rhq-devel mailing list