On 23.06.20 19:19, Mark Reynolds wrote:
On 6/23/20 12:22 PM, David Boreham wrote:
>
> On 6/23/2020 10:07 AM, Mark Reynolds wrote:
>>
>> In 389 what we are seeing is that our backend txn plugins are doing
>> unindexed searches, but I would not call it a bug.
>
> The unindexed search is fine per se (although probably not a great
> idea if you want the op the plugin hooked to complete quickly).
>
> What's not fine is that all the DB reads under that search should be
> done in the same transaction with strong isolation.
First I'm not that intimately familiar with this issue, Thierry and
Ludwig did most of that investigation. But this happens during a
modify operation that triggers some BE txn plugins that do searches
and updates under the same parent transaction. So under these
conditions is when it just starts consuming a ton of db locks.
If a transactional
operation (eg modify) triggers a search by a plugin
it already holds a coupke of page locks as write locks. If the search
would try to access this pages without using the txn of the parent it
would have to wait - and the whole operation would self deadlock. So all
db accesses inside a txn need to use this txn directly or as a paent txn.
>
> Unindexed searches by themselves do not cause this issue, it's when we
> are updating the database under the same txn. So the mod takes a lock
> on a db page, then we call the be postop plugins, which in turn starts
> doing these expensive searches and updates - that is when the db lock
> issue pops up. I seem to recall from previous similar cases that this
> "mod update" involved a very large static group, and the RI or
> memberOf plugin doing its work. Maybe Thierry recalls some of the past
> cases?
>
>>
>>> It's really a configuration/indexing issue. But yes, there are long
>>> running operations/txns in regards to many plugins doing a lot of
>>> things while the database is being updated in the same nested
>>> operation. Now when these internal searches are properly indexed
>>> the db lock issue completely goes away.
>>
>> If missing an index were to result in poor performance, agreed --
>> it's a configuration issue. The server process exiting seems quite an
>> extreme consequence.
> It's not exactly crashing, but the db can get corrupted and it needs
> to be reinitialized. That sounds like a libdb bug to me :-) Running
> out of db locks should not corrupt the database.
>>
>>
>> Wondering if this is the result of an old fix for a deadlock problem
>> (bringing the internal op under the main transaction to cure the
>> deadlock)?
> Maybe :-) Haven't looked at that code in quite a few years...
>>
>> How is a regular (non-internal) unindexed search run? Surely that
>> doesn't burn through one lock per page touched?
>
> No it doesn't. See my comment above, standalone unindexed searches do
> not trigger this issue.
>
> Mark
>
>>
>> _______________________________________________
>> 389-users mailing list -- 389-users(a)lists.fedoraproject.org
>> To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org
>> Fedora Code of Conduct:
>>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>>
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproje...
>