Additional data purging needed; e.g. RHQ_CONFIG_UPDATE

Elias Ross genman at noderunner.net
Mon Sep 29 20:22:09 UTC 2014


On Mon, Sep 29, 2014 at 9:26 AM, Thomas Segismont <tsegismo at redhat.com> wrote:
> Hi,
>
> I agree, we need more sub-tasks in the purge job.
>
> In the past, when adding such tasks (operations history purge), we made them
> disabled by default with a configurable cut-off date. To me, it still sounds
> like the way to go.

Thanks for your response. I would argue that deletes should happen out
of the box, but I'm very happy now that RHQ is automating a lot of
this.

>
>
> Le 23/09/2014 20:11, Elias Ross a écrit :
>>
>> Bug 1131726 - Data from RHQ_CONFIG_UPDATE never purged
>> https://bugzilla.redhat.com/show_bug.cgi?id=1131726
>>
>
> OK
>
>> I saw data from 2012 in here. Not deleting data means more often
>> running into issues like this: Bug 1131724 - Full table scan of
>> RHQ_CONFIG_UPDATE in delete for RHQ_CONFIG
>>
>
> Can you elaborate on the symptoms? A full scan of a table is not a problem
> per se. What do you see? Timeouts?

Something like this:

20:27:37,279 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper]
(RHQScheduler_Worker-2) SQL Error: 2049, SQLState: 42000
20:27:37,279 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper]
(RHQScheduler_Worker-2) ORA-02049: timeout: distributed transaction
waiting for lock
20:27:37,279 ERROR
[org.rhq.enterprise.server.resource.ResourceManagerBean]
(RHQScheduler_Worker-2) Bulk named query delete error for
'ResourceConfigurationUpdate.deleteByResources2' for [732478]:
org.hibernate.exception.SQLGrammarException: could not execute
statement

Sorry I didn't have it in the Bugzilla.

>
> I believe this is the method responsible for the delete statement:
> ResourceManagerBean#clearResourceConfigErrorByType

Okay, for some reason that statement wasn't a prepared one so I didn't
see it. :-(

What you can do is change that to a select, then delete specific rows
with the entity manager.

> Can you elaborate on the symptoms for this one as well? In the BZ
> description you say "there is a possibility of deadlock". Did Oracle fail
> any execution of these requests with ORA-00060?

I'm guessing so, since that's what it said on the Oracle side, hence
Oracle generated a deadlock log file. I copied the relevant text from
the file that's generated on deadlock. Unfortunately, I didn't add in
the Java stack trace and that server is long gone, so I can't confirm
this.

>
> I'm not sure yet. Maybe a new "Advanced database purge settings" section
> would make sense.

Since I wasn't going to wait, I created a server plugin to do this
work in the meantime.

It doesn't use your framework, mostly because I wanted it to work on
RHQ 4.9, but also dealing with deleting all associations is a big
hassle as well.

I have 800,000+ rows in RHQ_CONFIG and RHQ_CONFIG_PROPERTY has
87,000,000+ rows, and this is just my lab system. My suspicion is data
is not being deleted here there either.

As RHQ_RESOURCE is 150K rows, and none of the other tables come close
to that many rows, there's quite a lot of orphaned configurations I
suspect.

https://bugzilla.redhat.com/show_bug.cgi?id=1147669


More information about the rhq-devel mailing list