Roadmap for rust as a requirement in the project
by William Brown
Hi all,
Background: https://pagure.io/389-ds-base/issue/51140
Ludwig and Mark both raised some good points here. First is that features today (EntryUUID, LDAPSsoToken) are both locked behind another language. Rightly so, they shouldn't be hidden here forever. As well, Mark wants to ask what it would take to enable by default.
** Why did I use the fedse.c to load a plugin entry?
Because we don't (fully) support startup migrations yet. Mark has privately reminded me that he has reviewed https://pagure.io/389-ds-base/pull-request/49579 and that I should revive it. This v4 plugin adds a stateful assertion of entries allowing better migrations beyond what fedse.c's bootstrap can achieve. IE we can create an entry if it does not exist, and modify it partially if an admin has changed it (ie we can tweak defaults but without affecing say pluginEnabled). There was an issue viktor raised about pbkdf2 where it could not be disabled, and this would resolve it. Most likely, I will break the on startup migrations out from the v4 plugin patch set, and make it it's own change, and move some of the content from fedse.c and friends into this.
** TODO list to get rust enabled by default.
I think there is still a bit of work to do to enable by default, but I think we are pretty close. Roughly ordered, this is the list of things that has to happen to let us enable rust by default.
- everyone -> test that you can build and run tests with --enable-rust --disable-asan in your development environment so that we can work out and issues that may exist for us as developers.
- william -> fix the intentional name leak in the rust slapi plugin interface to use lazy_static. Today this triggers LSAN which breaks basic test suites.
- william -> clean up libsds linking and some of the related elements
- william -> revive and make on upgrade configs better (see above), potentially break it out from the v4 plugin patch.
- william + mark -> check that our respective major distros/releases can build with --enable-rust in release rpms
- william + viktor -> check that we can pass with --enable-rust and --enable-asan, especially in CI
- anyone interested -> update wiki contributing guide to include rust as a requirement
- (optional) anyone interested (but not william) -> add a section to the wiki on using rust in development
- team -> agree we are happy, and make configure.ac have --enable-rust by default
- team -> after a few weeks / months, remove the ifdefs, duplicate C versions of Rust features, and enable/disable features from configure.ac
Does this seem reasonable to everyone? I really want to make sure we do this right as a team, and we are all happy with it (I don't want a repeat of nunc-stans ;) ). If we agree on this, I will open some tickets up, add the relevant blocks/ordering and assign out the work.
Thank you all!
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs
3 years, 1 month
Odd behaviour in import
by William Brown
Hey there,
I'm seeing some odd behaviour in an import test. I'm seeing that a large number of entries won't import unless the directory is restarted before the import task is performed.
The error appears to be:
[25/Aug/2020:14:14:58.973490600 +1000] - WARN - import_foreman - import userRoot: Skipping entry "cn=group0,ou=Groups,dc=example,dc=com" which has no parent, ending at line 154 of file "/opt/dirsrv/var/lib/dirsrv/slapd-standalone1/ldif/4f8afb8d-ec97-4246-94a2-ec343c0eacb4.ldif"
...
[25/Aug/2020:14:14:59.307477400 +1000] - INFO - bdb_import_main - import userRoot: Import complete. Processed 14 entries (10 were skipped) in 1 seconds. (14.00 entries/sec)
This is where a newly created backend *with* example entries, then has it's entire content overwriten during an import. Anything that is underneath the ou=* entries is not imported, but the ou= and dc=are fine.
I'm wondering if this is something related to the fact we are replacing the ou= entries with different ids/nsunique ids. IE
id 3
rdn: ou=groups
objectClass: top
objectClass: organizationalunit
ou: groups
aci: (targetattr="cn || member || gidNumber || nsUniqueId || description || ob
jectClass")(targetfilter="(objectClass=groupOfNames)")(version 3.0; acl "Enab
le anyone group read"; allow (read, search, compare)(userdn="ldap:///anyone")
;)
aci: (targetattr="member")(targetfilter="(objectClass=groupOfNames)")(version
3.0; acl "Enable group_modify to alter members"; allow (write)(groupdn="ldap:
///cn=group_modify,ou=permissions,dc=example,dc=com");)
aci: (targetattr="cn || member || gidNumber || description || objectClass")(ta
rgetfilter="(objectClass=groupOfNames)")(version 3.0; acl "Enable group_admin
to manage groups"; allow (write, add, delete)(groupdn="ldap:///cn=group_admi
n,ou=permissions,dc=example,dc=com");)
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20200827015033Z
modifyTimestamp: 20200827015033Z
nsUniqueId: b0fce42b-e80711ea-8141c872-2df18128
parentid: 1
entryid: 3
numSubordinates: 1
Becomes:
id 4
rdn: ou=Groups
createTimestamp: 20200224023755Z
creatorsName: cn=Manager,dc=example,dc=com
entryUUID: 67cc2212-eafa-1039-8830-152569770969
modifiersName: cn=Manager,dc=example,dc=com
modifyTimestamp: 20200224023755Z
objectClass: organizationalUnit
objectClass: top
ou: Groups
nsUniqueId: 87b64988-e68911ea-a943c898-6d74ab17
parentid: 1
entryid: 4
Given that these id's are changing I'm wondering if this is somehow breaking our import ordering? Any ideas on where I should start to investigate this?
Thanks!
—
Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server
SUSE Labs
3 years, 3 months