Change in vdsm[master]: sp: Fix stopping domain monitors

danken at redhat.com danken at redhat.com
Thu Dec 5 16:24:33 UTC 2013


Dan Kenigsberg has posted comments on this change.

Change subject: sp: Fix stopping domain monitors
......................................................................


Patch Set 1:

(1 comment)

....................................................
Commit Message
Line 22: jobs, where there is no storage space left, while storage has plenty of space.
Line 23: The root cause of the error was anonymous thread running at unexpected times
Line 24: and stopping silently all domain monitors.  Adding some logging revealed that
Line 25: this thread was started from StoragePool.__del__ method. This thread was
Line 26: running 14 minutes after the original pool was disconnected, stopping domain
Your patch seems fine to me, but I am not sure we completely understand the root cause.

It may well be that __del__ was called 14 minutes after the sp object as lost context, but it's unlikely. It is more likely that it has happened much earlier, but the created thread was blocking on a long-disconnected storage.

I suspect that we failed to stop monitoring when we should have, and that sp.__del__ was called while self.domainMonitor.poolMonitoredDomains still had a couple of monitored domains. This suspicion is based on cases when we've seen a destroyStoragePool being received without a prior disconnectStoragePool, and after a disconnectStorageServer.

Could this be the case here?
Line 27: monitors used by the current pool object. It seems that a task was holding a
Line 28: reference to the old pool, and when the task was finished, the old pool was
Line 29: finally destroyed.
Line 30: 


-- 
To view, visit http://gerrit.ovirt.org/22058
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iad80984ab44dd4a4e36b92330eb9320cf68f1580
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>
Gerrit-Reviewer: Allon Mureinik <amureini at redhat.com>
Gerrit-Reviewer: Ayal Baron <abaron at redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken at redhat.com>
Gerrit-Reviewer: Eduardo <ewarszaw at redhat.com>
Gerrit-Reviewer: Federico Simoncelli <fsimonce at redhat.com>
Gerrit-Reviewer: Liron Ar <laravot at redhat.com>
Gerrit-Reviewer: Nir Soffer <nsoffer at redhat.com>
Gerrit-Reviewer: Saggi Mizrahi <smizrahi at redhat.com>
Gerrit-Reviewer: Sergey Gotliv <sgotliv at redhat.com>
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: Yes


More information about the vdsm-patches mailing list