Hello Everyone,
We are merging our student directory (about 200,000 entries) into our existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from basic tuning are here any big pitfalls we should look out for?
We're still on CentOS 7 for the time being: [root@prdds21 morgan]# rpm -qa|grep 389 389-admin-1.1.46-4.el7.x86_64 389-console-1.1.19-6.el7.noarch 389-dsgw-1.1.11-5.el7.x86_64 389-admin-console-1.1.12-1.el7.noarch 389-ds-1.2.2-6.el7.noarch 389-ds-base-libs-1.3.10.2-13.el7_9.x86_64 389-ds-base-1.3.10.2-13.el7_9.x86_64 389-adminutil-1.1.22-2.el7.x86_64 389-admin-console-doc-1.1.12-1.el7.noarch 389-ds-console-doc-1.2.16-1.el7.noarch 389-ds-console-1.2.16-1.el7.noarch [root@prdds21 morgan]#
thank you,
-morgan
No, unless you have some unusually large attributes (storing high-resolution profile pictures, something like that), and/or unusually high write traffic (constantly changing users' status, something like that), you should be fine on modern hardware.
On Wed, May 18, 2022, at 8:48 AM, Morgan Jones wrote:
Hello Everyone,
We are merging our student directory (about 200,000 entries) into our existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from basic tuning are here any big pitfalls we should look out for?
On 19 May 2022, at 00:48, Morgan Jones morgan@morganjones.org wrote:
Hello Everyone,
We are merging our student directory (about 200,000 entries) into our existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from basic tuning are here any big pitfalls we should look out for?
I think extra databases creates more administration overhead than benefit. The benefit to extra databases is "improved write performance" generally speaking. But the trade is subtree queries are more complex to eval for the server.
It's far easier for you the admin, and also support staff if you keep it as a single db. We have done huge amounts to improve parallel reads in recent years, so you should see large gains when you change from 1.3 to 1.4 or 2.0 :)
We're still on CentOS 7 for the time being: [root@prdds21 morgan]# rpm -qa|grep 389 389-admin-1.1.46-4.el7.x86_64 389-console-1.1.19-6.el7.noarch 389-dsgw-1.1.11-5.el7.x86_64 389-admin-console-1.1.12-1.el7.noarch 389-ds-1.2.2-6.el7.noarch 389-ds-base-libs-1.3.10.2-13.el7_9.x86_64 389-ds-base-1.3.10.2-13.el7_9.x86_64 389-adminutil-1.1.22-2.el7.x86_64 389-admin-console-doc-1.1.12-1.el7.noarch 389-ds-console-doc-1.2.16-1.el7.noarch 389-ds-console-1.2.16-1.el7.noarch [root@prdds21 morgan]#
thank you,
-morgan
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia
On 5/19/22 1:51 AM, William Brown wrote:
On 19 May 2022, at 00:48, Morgan Jones morgan@morganjones.org wrote:
Hello Everyone,
We are merging our student directory (about 200,000 entries) into our existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from basic tuning are here any big pitfalls we should look out for?
I think extra databases creates more administration overhead than benefit. The benefit to extra databases is "improved write performance" generally speaking. But the trade is subtree queries are more complex to eval for the server.
It's far easier for you the admin, and also support staff if you keep it as a single db. We have done huge amounts to improve parallel reads in recent years, so you should see large gains when you change from 1.3 to 1.4 or 2.0 :)
I fully agree with William points. Just a comment if you decide to merge the students entries into the employee directory. Either you will have to ADD 200K students entries into your existing directory and it will take several hours. Either you will use import (e.g. merge employee/students ldifs and reimport), that will require a reinit of the topology. The second option would be the fastest as import + reinit of a db with 225K entries should be fast.
We're still on CentOS 7 for the time being: [root@prdds21 morgan]# rpm -qa|grep 389 389-admin-1.1.46-4.el7.x86_64 389-console-1.1.19-6.el7.noarch 389-dsgw-1.1.11-5.el7.x86_64 389-admin-console-1.1.12-1.el7.noarch 389-ds-1.2.2-6.el7.noarch 389-ds-base-libs-1.3.10.2-13.el7_9.x86_64 389-ds-base-1.3.10.2-13.el7_9.x86_64 389-adminutil-1.1.22-2.el7.x86_64 389-admin-console-doc-1.1.12-1.el7.noarch 389-ds-console-doc-1.2.16-1.el7.noarch 389-ds-console-1.2.16-1.el7.noarch [root@prdds21 morgan]#
thank you,
-morgan
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Thanks for the info William and Thierry, this all makes sense.
We'll start testing in the coming weeks and see how it goes.
-morgan
On May 19, 2022, at 03:25, Thierry Bordaz tbordaz@redhat.com wrote:
On 5/19/22 1:51 AM, William Brown wrote:
On 19 May 2022, at 00:48, Morgan Jones morgan@morganjones.org wrote:
Hello Everyone,
We are merging our student directory (about 200,000 entries) into our existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from basic tuning are here any big pitfalls we should look out for?
I think extra databases creates more administration overhead than benefit. The benefit to extra databases is "improved write performance" generally speaking. But the trade is subtree queries are more complex to eval for the server.
It's far easier for you the admin, and also support staff if you keep it as a single db. We have done huge amounts to improve parallel reads in recent years, so you should see large gains when you change from 1.3 to 1.4 or 2.0 :)
I fully agree with William points. Just a comment if you decide to merge the students entries into the employee directory. Either you will have to ADD 200K students entries into your existing directory and it will take several hours. Either you will use import (e.g. merge employee/students ldifs and reimport), that will require a reinit of the topology. The second option would be the fastest as import + reinit of a db with 225K entries should be fast.
We're still on CentOS 7 for the time being: [root@prdds21 morgan]# rpm -qa|grep 389 389-admin-1.1.46-4.el7.x86_64 389-console-1.1.19-6.el7.noarch 389-dsgw-1.1.11-5.el7.x86_64 389-admin-console-1.1.12-1.el7.noarch 389-ds-1.2.2-6.el7.noarch 389-ds-base-libs-1.3.10.2-13.el7_9.x86_64 389-ds-base-1.3.10.2-13.el7_9.x86_64 389-adminutil-1.1.22-2.el7.x86_64 389-admin-console-doc-1.1.12-1.el7.noarch 389-ds-console-doc-1.2.16-1.el7.noarch 389-ds-console-1.2.16-1.el7.noarch [root@prdds21 morgan]#
thank you,
-morgan
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Hi Morgan,
In terms of scalability there should not be much issue with 225000 entries It is still a relatively small db and scalability is not a criteria to use several backends. That said it may still be handy for administrative reasons because it allows reinitializing the student db without impacting the employees db. About the pitfalls I can think about: - Make sure that you do not have large static groups static groups of 25K member are painful and slow down things (but usually still usable in practice) but static groups of 100K members just get unmanageable - Should be careful about the number of simultaneous connections (If you have peak of connection, you may need to add consumer and a load balancer. But usually it is not a problem because LDAP connections should be short lived) Then the usual ds389 tuning (mainly about cache size) should be enough ...
Regards Pierre
On Thu, May 19, 2022 at 1:57 AM William Brown william.brown@suse.com wrote:
On 19 May 2022, at 00:48, Morgan Jones morgan@morganjones.org wrote:
Hello Everyone,
We are merging our student directory (about 200,000 entries) into our
existing employee directory (about 25,000 entries).
They're a pair of multi-master replicas on virtual hardware that can
easily be expanded if needed though hardware performance hasn't been an issue.
Does this justify creating separate database for students? Aside from
basic tuning are here any big pitfalls we should look out for?
I think extra databases creates more administration overhead than benefit. The benefit to extra databases is "improved write performance" generally speaking. But the trade is subtree queries are more complex to eval for the server.
It's far easier for you the admin, and also support staff if you keep it as a single db. We have done huge amounts to improve parallel reads in recent years, so you should see large gains when you change from 1.3 to 1.4 or 2.0 :)
We're still on CentOS 7 for the time being: [root@prdds21 morgan]# rpm -qa|grep 389 389-admin-1.1.46-4.el7.x86_64 389-console-1.1.19-6.el7.noarch 389-dsgw-1.1.11-5.el7.x86_64 389-admin-console-1.1.12-1.el7.noarch 389-ds-1.2.2-6.el7.noarch 389-ds-base-libs-1.3.10.2-13.el7_9.x86_64 389-ds-base-1.3.10.2-13.el7_9.x86_64 389-adminutil-1.1.22-2.el7.x86_64 389-admin-console-doc-1.1.12-1.el7.noarch 389-ds-console-doc-1.2.16-1.el7.noarch 389-ds-console-1.2.16-1.el7.noarch [root@prdds21 morgan]#
thank you,
-morgan
389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject....
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
-- Sincerely,
William Brown
Senior Software Engineer, Identity and Access Management SUSE Labs, Australia _______________________________________________ 389-users mailing list -- 389-users@lists.fedoraproject.org To unsubscribe send an email to 389-users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-users@lists.fedoraproject.... Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
389-users@lists.fedoraproject.org
-
David Boreham -
Morgan Jones -
Pierre Rogier -
Thierry Bordaz -
William Brown