Generally the advice is:
* autotune everything
* handtune everything.
IMO, autotuning is better (but I did write it, so I'm biased), and don't touch the
"split" because we've seen lots of communication issues and challenges
trying to educate about how the query processing works in the server core.
I can write a longer piece of how all the query server layers fit together and what each
tuning does, but I won't be able to for a few weeks (I'm traveling).
On 17 Jul 2019, at 02:30, Paul Whitney
<paul.whitney(a)chesapeake-it.com> wrote:
Hi Mark,
What I was referring to is:
"nsslapd-cache-autosize-split
The value sets the percentage of RAM that is used for the database cache. The remaining
percentage is used for the entry cache.
More than 512 MB RAM database cache do not improve the performance. Therefore, the
database cache is limited to 512 MB." -
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
Then further down is a table that tops off at 512MB for dbcache in the same document.
But even further down it contradicts by suggesting the results of the dbmon.sh script can
provide insight into how much more the dbcache can be to improve performance with the 2.2%
free example.
All these tuning features are great, but gets confusing when trying to mix and match the
best combination of settings.
I did go over your recommendations and have settled for now on increasing the dbcache to
5GB. Our consumers are configured with 64GB and the masters 128GB. While the percentage
of free space in the dbcache is not optimal (more than 12%), we are achieving a hit ratio
of 99%.
It's confusing because we never explain how queries and cache's affect each other
in a coherent document ... :) Which is tottaly our fault.
summary version - dbcache will cache the raw pages for bdb (IE indexes, serialised entry
pages etc).
entry cahce is deserialised entries in a seperate hashmap internal to the server.
So entries go through:
BDB -> dbcache -> entrycache -> entries in results etc.
You generally want to cache more in the entrycache because the deserialisation between
dbcachce -> entrycache is really expensive.
If your entrycache is sized properly, the dbcacche ends up caching index pages instead.
This goes through:
BDB -> dbcache -> IDL -> Query processing
There is a "slight" cost from dbccache -> IDL, but not enough to justify a
cacche.
So your dbccache becomes an index cache effectively, and as a result, generally is
"smaller" because IDLs are very very small - you can probably size what a
dbcache should be by listing the content in
var/lib/dirsrv/slapd-instance/db/userRoot/*.db, excluding "id2entry". Then add
maybe 50% or so to get some breathing room.
You can size your entrycache by looking at the size of the id2entry file, and again, add a
bit extra fro breathing room and struct overheads.
Hope that helps - perhaps a future autotuning will do this same process.
Paul M. Whitney, RHCSA, CISSP
Chesapeake IT Consulting, Inc.
2680 Tobacco Rd
Chesapeake Beach, MD 20732
Work: 443-492-2872
Cell: 410.493.9448
Email: paul.whitney(a)chesapeake-it.com
CONFIDENTIALITY NOTICE
The information contained in this facsimile or electronic message is confidential
information intended for the use of the individual or entity named above. If the reader of
this message is not the intended recipient, or an employee or agent responsible for
delivering this facsimile message to the intended recipient, you are hereby notified that
any dissemination, or copying of this communication is strictly prohibited. If this
message contains non-public personal information about any consumer or customer of the
sender or intended recipient, you are further prohibited under penalty of law from using
or disclosing the information to any third party by provisions of the federal
Gramm-Leach-Bliley Act. If you have received this facsimile or electronic message in
error, please immediately notify us by telephone and return or destroy the original
message to assure that it is not read, copied, or distributed by others.
From: Mark Reynolds <mreynolds(a)redhat.com>
Sent: Tuesday, July 16, 2019 9:30 AM
To: General discussion list for the 389 Directory server project.; Paul Whitney
Subject: Re: [389-users] Recommended SLAPD cache sizes
Hi Paul,
On 7/16/19 9:16 AM, Paul Whitney wrote:
> Is there some formula or recommendation on determining what would be the optimal
cache settings for a directory server (389-ds of course) with following sizes? I looked at
the DS 10 Admin Guide online and am getting conflicting information. But the manual shows
a table and suggests that the max size for the cn=config dbcache is 512MB.
There is no limit on the cache size value besides what you are limited to by a unsigned
long integer. Do you have a link to where it says 512 MB is a maximum?
> But I tried with 5GB and it appears to work fine with it, although over time start
to see negative values (prob not enough cache) and roevicts while running the dbmon.sh
script.
>
> __db files = ~6GB
> groupRoot = 1.5GB
> userRoot = 20GB
>
> What values would yield best performance for:
>
> cn=config dbcache? currently set to 5GB but are still seeing lots of roevicts. As
much
>
> userRoot entry cache? currently set to 21GB (hit ratio so far at 95%)
>
> groupRoot entry cache? currently set to 2GB (hit ration so far at 96%)
>
> Virtual machine configured with 8CPU, 128GB RAM. Load is in the low 20's with
occasional spikes to high 20's.
If you have 128 GB available to you then I would continue to increase the cache sizes
until you get to 99% hit ratio for the two entry caches, and the db cache. Also make sure
the DN and NDN caches also have a 99% cache hit ratio:
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
https://www.port389.org/docs/389ds/design/normalized-dn-cache.html
I would also look into running the DB on a RAM disk since you have the resources to do
so:
https://access.redhat.com/documentation/en-us/red_hat_directory_server/10...
HTH,
Mark
>
> Paul M. Whitney, RHCSA, CISSP
> Chesapeake IT Consulting, Inc.
> 2680 Tobacco Rd
> Chesapeake Beach, MD 20732
>
> Work: 443-492-2872
> Cell: 410.493.9448
> Email: paul.whitney(a)chesapeake-it.com
> CONFIDENTIALITY NOTICE
> The information contained in this facsimile or electronic message is confidential
information intended for the use of the individual or entity named above. If the reader of
this message is not the intended recipient, or an employee or agent responsible for
delivering this facsimile message to the intended recipient, you are hereby notified that
any dissemination, or copying of this communication is strictly prohibited. If this
message contains non-public personal information about any consumer or customer of the
sender or intended recipient, you are further prohibited under penalty of law from using
or disclosing the information to any third party by provisions of the federal
Gramm-Leach-Bliley Act. If you have received this facsimile or electronic message in
error, please immediately notify us by telephone and return or destroy the original
message to assure that it is not read, copied, or distributed by others.
>
>
>
> _______________________________________________
> 389-users mailing list --
> 389-users(a)lists.fedoraproject.org
>
> To unsubscribe send an email to
> 389-users-leave(a)lists.fedoraproject.org
>
> Fedora Code of Conduct:
>
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>
> List Guidelines:
>
https://fedoraproject.org/wiki/Mailing_list_guidelines
>
> List Archives:
>
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproje...
--
389 Directory Server Development Team
_______________________________________________
389-users mailing list -- 389-users(a)lists.fedoraproject.org
To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproje...
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs