Subject: Use thread aware library for complex regex searches
Link: http://nhosoi.fedorapeople.org/0001-Use-thread-aware-library-for-complex-reg...
Unfortunately, the fedoraproject.org is closed now. I'm going to create the page http://directory.fedoraproject.org/wiki/Thread_Aware_Regex which contains the following notes as soon as the wiki is reopened. ------------------------------------------------------------------------ Directory Server used to use the regular expression pattern matching and replacing library which was not thread safe. Thus, the operation should have been protected by the mutex lock. Regular expression could be used by the search filters (syntax plugin), acl, schema file load, and SASL Mapping. If one of them took a long time, the rest should have been blocked. Replacing the library with the thread aware library solves the problem and improves the throughput.
[http://www.pcre.org/ PCRE - Perl Compatible Regular Expressions] library is installed on RHELs/Fedoras, by default. We need just a subset of the APIs, thus we provide simplified slapi APIs wrapping the PCRE APIs.
NAME slapi_re_comp -- compiles a regular expression pattern. A thin wrapper of pcre_compile. SYNOPSIS Slapi_Regex *slapi_re_comp( char *pat, char **error ); PARAMS pat: Pattern to be compiled. error: The error string is set if the compile fails. RETURN VALUE a pointer to the regex handler which stores the compiled pattern. NULL if the compile fails. WARNING The regex handler should be released by slapi_re_free().
NAME slapi_re_exec -- matches a compiled regular expression pattern against a given string. A thin wrapper of pcre_exec. SYNOPSIS int slapi_re_exec( Slapi_Regex *re_handle, char *subject, time_t time_up ); PARAMS re_handle: The regex handler returned from slapi_re_comp. subject: A string to be checked against the compiled pattern. time_up: If the current time is larger than the value, this function returns immediately. (-1) means no time limit. RETURN VALUE 0 if the string did not match. 1 if the string matched. other values if any error occurred.
NAME slapi_re_subs -- substitutes '&' or '#' in the param src with the matched string. SYNOPSIS int slapi_re_subs( Slapi_Regex *re_handle, char *subject, char *src, char **dst, unsigned long dstlen ); PARAMS re_handle: The regex handler returned from slapi_re_comp. subject: A string checked against the compiled pattern. src: A given string which could contain the substitution symbols. dst: A pointer pointing to the memory which stores the output string. dstlen: Size of the memory dst. RETURN VALUE 1 if the substitution was successful. 0 if the substitution failed.
NAME slapi_re_free -- releases the regex handler which was returned from slapi_re_comp. SYNOPSIS void slapi_re_free(Slapi_Regex *re_handle); PARAMS re_handle: The regex handler to be released. RETURN VALUE none ------------------------------------------------------------------------
Thanks, --noriko
Noriko Hosoi wrote:
Subject: Use thread aware library for complex regex searches
Link: http://nhosoi.fedorapeople.org/0001-Use-thread-aware-library-for-complex-reg...
Unfortunately, the fedoraproject.org is closed now. I'm going to create the page http://directory.fedoraproject.org/wiki/Thread_Aware_Regex which contains the following notes as soon as the wiki is reopened.
Directory Server used to use the regular expression pattern matching and replacing library which was not thread safe. Thus, the operation should have been protected by the mutex lock. Regular expression could be used by the search filters (syntax plugin), acl, schema file load, and SASL Mapping. If one of them took a long time, the rest should have been blocked. Replacing the library with the thread aware library solves the problem and improves the throughput.
[http://www.pcre.org/ PCRE - Perl Compatible Regular Expressions] library is installed on RHELs/Fedoras, by default. We need just a subset of the APIs, thus we provide simplified slapi APIs wrapping the PCRE APIs.
NAME slapi_re_comp -- compiles a regular expression pattern. A thin wrapper of pcre_compile. SYNOPSIS Slapi_Regex *slapi_re_comp( char *pat, char **error ); PARAMS pat: Pattern to be compiled. error: The error string is set if the compile fails. RETURN VALUE a pointer to the regex handler which stores the compiled pattern. NULL if the compile fails. WARNING The regex handler should be released by slapi_re_free().
NAME slapi_re_exec -- matches a compiled regular expression pattern against a given string. A thin wrapper of pcre_exec. SYNOPSIS int slapi_re_exec( Slapi_Regex *re_handle, char *subject, time_t time_up ); PARAMS re_handle: The regex handler returned from slapi_re_comp. subject: A string to be checked against the compiled pattern. time_up: If the current time is larger than the value, this function returns immediately. (-1) means no time limit. RETURN VALUE 0 if the string did not match. 1 if the string matched. other values if any error occurred.
NAME slapi_re_subs -- substitutes '&' or '#' in the param src with the matched string. SYNOPSIS int slapi_re_subs( Slapi_Regex *re_handle, char *subject, char *src, char **dst, unsigned long dstlen ); PARAMS re_handle: The regex handler returned from slapi_re_comp. subject: A string checked against the compiled pattern. src: A given string which could contain the substitution symbols. dst: A pointer pointing to the memory which stores the output string. dstlen: Size of the memory dst. RETURN VALUE 1 if the substitution was successful. 0 if the substitution failed.
NAME slapi_re_free -- releases the regex handler which was returned from slapi_re_comp. SYNOPSIS void slapi_re_free(Slapi_Regex *re_handle); PARAMS re_handle: The regex handler to be released. RETURN VALUE none
Looks good. I notice you have some unrelated fixes in this patch too - those look ok - be sure to document those. You can edit your commit message by using git commit --amend to amend the last commit you made.
Thanks,
--noriko
-- Fedora-directory-devel mailing list Fedora-directory-devel@redhat.com https://www.redhat.com/mailman/listinfo/fedora-directory-devel
Rich Megginson wrote:
[...]
Looks good. I notice you have some unrelated fixes in this patch too
- those look ok - be sure to document those. You can edit your commit
message by using git commit --amend to amend the last commit you made.
Thank you, Rich, for reviewing my changes. I updated the commit message as follows and pushed the patch to master.
Use thread aware library for complex regex searches
For more details, see the design doc at http://directory.fedoraproject.org/wiki/Thread_Aware_Regex
Additional 2 unrelated changes are being made:
- dbgen.pl.in: secretary and manager are having a dn format value
"cn=...". 2) slapi_counter_sunos_sparcv9.S: adding "#define _ASM 1" to force to set an assembler code macro _ASM.
Thanks, --noriko
389-devel@lists.fedoraproject.org