From the looks of this, I think we may need to add some code in InventoryFile.loadInventory() that checks the childResources field of every Resource in the hierarchy loaded from inventory.dat and if it is not a ConcurrentHashSet, converts it to one.

Mazz, what do you think?

-------- Original Message --------
Subject: [RHQ and Jopr] - Agent doesn't send reports to server
Date: Wed, 09 May 2012 05:51:38 -0400
From: konrad biel <do-not-reply@jboss.com>
To: Ian Springer <ian.springer@redhat.com>


JBoss Community

Agent doesn't send reports to server

created by konrad biel in RHQ and Jopr - View the full discussion

Agent was running for a long time without problem. Then suddenly all resources discovered by agent went down.

I've tried to restart agent, RHQ server, Jboss, install agent one more time, uninventory resources but reports are not send to server.

Strange is that I have values in Platform Utilization but I can't connect to Jboss or Apache. Remaining agents are running without any problems.

 

RHQ 4.3

OS: Solaris 10

 


2012-05-09 11:11:39,132 WARN  [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Failed to synchronize local inventory with Server inventory for Resource [10129] and its descendants: null
2012-05-09 11:11:39,132 WARN  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Exception caught while running server discovery
java.lang.RuntimeException: java.util.ConcurrentModificationException
    at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:1072)
    at org.rhq.core.pc.inventory.InventoryManager.handleReport(InventoryManager.java:1006)
    at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.call(AutoDiscoveryExecutor.java:121)
    at org.rhq.core.pc.inventory.AutoDiscoveryExecutor.run(AutoDiscoveryExecutor.java:92)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.util.ConcurrentModificationException
    at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
    at java.util.HashMap$KeyIterator.next(HashMap.java:828)
    at org.rhq.core.pc.inventory.InventoryManager.mergeResource(InventoryManager.java:2708)
    at org.rhq.core.pc.inventory.InventoryManager.mergeUnknownResources(InventoryManager.java:2610)
    at org.rhq.core.pc.inventory.InventoryManager.synchInventory(InventoryManager.java:1041)
    ... 12 more
2012-05-09 11:11:39,139 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Found 1 servers.
2012-05-09 11:11:39,139 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Running runtime discovery scan rooted at [platform]
2012-05-09 11:12:00,811 INFO  [ResourceDiscoveryComponent.invoker.daemon-2] (org.rhq.plugins.apache.ApacheServerDiscoveryComponent)- SNMPConf directive not found. Skipping SNMP configuration.
2012-05-09 11:12:01,357 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery
java.lang.NullPointerException
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:239)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResourceRecursive(RuntimeDiscoveryExecutor.java:158)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.runtimeDiscover(RuntimeDiscoveryExecutor.java:140)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.call(RuntimeDiscoveryExecutor.java:99)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.run(RuntimeDiscoveryExecutor.java:88)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
2012-05-09 11:12:01,368 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery
java.lang.NullPointerException
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:239)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResource(RuntimeDiscoveryExecutor.java:277)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.discoverForResourceRecursive(RuntimeDiscoveryExecutor.java:158)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.runtimeDiscover(RuntimeDiscoveryExecutor.java:140)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.call(RuntimeDiscoveryExecutor.java:99)
    at org.rhq.core.pc.inventory.RuntimeDiscoveryExecutor.run(RuntimeDiscoveryExecutor.java:88)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
2012-05-09 11:12:01,378 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error in runtime discovery


or:


2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished server sync for drift definitions. 0 added and 0 deleted in 30 ms
2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting drift content sync...
2012-05-09 11:27:12,224 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished drift content sync in 0 ms
2012-05-09 11:27:12,292 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting server sync for drift definitions...
2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished server sync for drift definitions. 0 added and 0 deleted in 31 ms
2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Starting drift content sync...
2012-05-09 11:27:12,323 INFO  [InventoryManager.discovery-1] (core.pc.drift.sync.DriftSyncManager)- Finished drift content sync in 0 ms
2012-05-09 11:27:12,688 INFO  [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Found 1 servers.
2012-05-09 11:33:31,558 WARN  [Content.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Cannot get a resource container for an invalid resource ID=0
2012-05-09 11:33:31,559 WARN  [Content.discovery-1] (rhq.core.pc.content.ContentDiscoveryRunner)- Exception received from component while attempting content retrieval
org.rhq.core.clientapi.agent.PluginContainerException: Resource component container could not be retrieved for resource: 0
    at org.rhq.core.pc.util.ComponentUtil.getComponent(ComponentUtil.java:102)
    at org.rhq.core.pc.content.ContentManager.performContentDiscovery(ContentManager.java:418)
    at org.rhq.core.pc.content.ContentDiscoveryRunner.call(ContentDiscoveryRunner.java:123)
    at org.rhq.core.pc.content.ContentDiscoveryRunner.run(ContentDiscoveryRunner.java:90)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)



    at java.lang.Thread.run(Thread.java:619)

2012-05-09 11:33:31,560 WARN  [Content.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Cannot get a resource container for an invalid resource ID=0

 

 

I don't have any idea what I can check?

Reply to this message by going to Community

Start a new discussion in RHQ and Jopr at Community