Hi Endi:
We haven’t done any customizations. I’ve owned this FreeIPA deployment since way back in
the CentOS 6.X days, when all the CA stuff lived in a second LDAP instance. Since then
there have been numerous upgrades, cleanup of old data, some minor tweaks to things like
nsslapd-idletimeout etc. but nothing major.
We have a pretty simple use case, about 800 hosts, 3 nodes, about 30 active users, we
mainly use FreeIPA to enforce hosts access, via HBAC and sudo rules, user public ssh
keys.
I don’t know if this is related but back in February of this year we upgraded from
4.8.4-7 to 4.8.7-14 and started to see a slow and steady apparent memory leak from
ns-slapd:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
<
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Did you guys want me to create the Bugzilla ticket? I’m not sure if I can as a third
party. Happy to though.
Jim Richard
System Administrator III
jrichard(a)placeiq.com | (646) 338-8905 |
www.placeiq.com
On Apr 26, 2021, at 9:53 AM, Endi Dewata <edewata(a)redhat.com>
wrote:
Hi,
I have not seen this error before. Was there any customization done to the system? Could
you open a Bugzilla ticket against pki-core component on RHEL 8 and provide the steps to
reproduce? Also, please include the LDAP entry
cn=268174648,ou=certificateRepository,ou=ca,o=ipaca. Thanks.
--
Endi S. Dewata
On Mon, Apr 26, 2021, 7:40 AM Rob Crittenden <rcritten(a)redhat.com
<mailto:rcritten@redhat.com>> wrote:
Jim Richard via FreeIPA-users wrote:
> From /var/log/pki/pki-tomcat/ca/debug.2021-04-26.log
>
>
> 2021-04-26 04:13:42 [ajp-nio-127.0.0.1-8009-exec-4] INFO: DBVirtualList: dn:
cn=268174647,ou=certificateRepository,ou=ca,o=ipaca
> 2021-04-26 04:13:42 [ajp-nio-127.0.0.1-8009-exec-4] INFO: DBVirtualList: dn:
cn=268174648,ou=certificateRepository,ou=ca,o=ipaca
>
> 2021-04-26 04:13:42 [ajp-nio-127.0.0.1-8009-exec-4] SEVERE: Operation Error -
netscape.ldap.LDAPException cannot be cast to netscape.ldap.LDAPEntry
> java.lang.ClassCastException: netscape.ldap.LDAPException cannot be cast to
netscape.ldap.LDAPEntry
> at
com.netscape.cmscore.dbs.DBVirtualList.getEntries(DBVirtualList.java:477)
> at com.netscape.cmscore.dbs.DBVirtualList.getPage(DBVirtualList.java:610)
> at com.netscape.cmscore.dbs.DBVirtualList.getPage(DBVirtualList.java:602)
> at
com.netscape.cmscore.dbs.DBVirtualList.getElementAt(DBVirtualList.java:754)
> at
com.netscape.cmscore.dbs.CertRecordList.getCertRecord(CertRecordList.java:110)
> at
org.dogtagpki.server.ca.rest.CertService.searchCerts(CertService.java:473)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)
> at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406)
> at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213)
> at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228)
> at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
> at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:225)
> at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
> at java.security.AccessController.doPrivileged(Native Method)
> at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
> at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
> at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
> at java.security.AccessController.doPrivileged(Native Method)
> at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
> at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
> at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
> at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
> at
com.netscape.cms.tomcat.ExternalAuthenticationValve.invoke(ExternalAuthenticationValve.java:82)
> at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
> at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
> at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
> at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
> at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
> at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:428)
> at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
> at
org.apache.tomcat.util.net
<
http://org.apache.tomcat.util.net/>.NioEndpoint$SocketProcessor.doRun(...
> at
org.apache.tomcat.util.net
<
http://org.apache.tomcat.util.net/>.SocketProcessorBase.run(SocketProc...
> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748)
>
> 2021-04-26 04:13:42 [ajp-nio-127.0.0.1-8009-exec-4] SEVERE: Unable to search for
certificates: java.lang.ClassCastException: netscape.ldap.LDAPException cannot be cast to
netscape.ldap.LDAPEntry
> java.lang.RuntimeException: java.lang.ClassCastException:
netscape.ldap.LDAPException cannot be cast to netscape.ldap.LDAPEntry
> at
com.netscape.cmscore.dbs.DBVirtualList.getEntries(DBVirtualList.java:523)
> at com.netscape.cmscore.dbs.DBVirtualList.getPage(DBVirtualList.java:610)
> at com.netscape.cmscore.dbs.DBVirtualList.getPage(DBVirtualList.java:602)
> at
com.netscape.cmscore.dbs.DBVirtualList.getElementAt(DBVirtualList.java:754)
> at
com.netscape.cmscore.dbs.CertRecordList.getCertRecord(CertRecordList.java:110)
> at
org.dogtagpki.server.ca.rest.CertService.searchCerts(CertService.java:473)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:140)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249)
> at
org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236)
> at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:406)
> at
org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:213)
> at
org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:228)
> at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56)
> at
org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:170)
> at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:225)
> at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
> at java.security.AccessController.doPrivileged(Native Method)
> at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
> at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:282)
> at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:279)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
> at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:314)
> at
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:253)
> at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:191)
> at
org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:47)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:149)
> at
org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:145)
> at java.security.AccessController.doPrivileged(Native Method)
> at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:144)
> at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
> at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
> at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
> at
com.netscape.cms.tomcat.ExternalAuthenticationValve.invoke(ExternalAuthenticationValve.java:82)
> at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
> at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
> at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:678)
> at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
> at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
> at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:428)
> at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
> at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
> at
org.apache.tomcat.util.net
<
http://org.apache.tomcat.util.net/>.NioEndpoint$SocketProcessor.doRun(...
> at
org.apache.tomcat.util.net
<
http://org.apache.tomcat.util.net/>.SocketProcessorBase.run(SocketProc...
> at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
> at java.lang.Thread.run(Thread.java:748)
>
> Caused by: java.lang.ClassCastException: netscape.ldap.LDAPException cannot be cast
to netscape.ldap.LDAPEntry
> at
com.netscape.cmscore.dbs.DBVirtualList.getEntries(DBVirtualList.java:477)
> ... 70 more
>
>
>
>
>
> Current versions are:
>
> CentOS 8:
>
> ipa-client.x86_64 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
> ipa-client-common.noarch 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
> ipa-common.noarch 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
> ipa-healthcheck.noarch 0.4-6.module_el8.3.0+482+9e103aab
@AppStream
> ipa-healthcheck-core.noarch 0.4-6.module_el8.3.0+482+9e103aab
@AppStream
> ipa-selinux.noarch 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
> ipa-server.x86_64 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
> ipa-server-common.noarch 4.8.7-14.module_el8.3.0+698+d6d67052
@appstream
>
> 389-ds-base.x86_64 1.4.3.8-6.module_el8.3.0+604+ab7bf9cc
@AppStream
> 389-ds-base-libs.x86_64 1.4.3.8-6.module_el8.3.0+604+ab7bf9cc
@AppStream
>
> Linux sso-111 4.18.0-240.15.1.el8_3.x86_64 #1 SMP Mon Mar 1 17:16:16 UTC 2021 x86_64
x86_64 x86_64 GNU/Linux
cc'ing one of the CA developers.
rob