Here's ChatGPT feedback regarding your server logs...
Analyzing the provided log messages from a RHEL 9 Linux server, we can
identify several events. Here's a breakdown of each event, including its
probable cause and some examples:
1. ipa-dnskeysyncd LDAP Operations:
- Messages from `ipa-dnskeysyncd` show LDAP operations related to DNSSEC
key synchronization.
- Example: `ipa-dnskeysyncd[75834]: ipaserver.dnssec.bindmgr: INFO
attrs: <ldap.cidict.cidict object at 0x7f283b53f7f0>`.
- Probable Cause: Routine operations of DNSSEC synchronization with IPA
server.
2. DNS Notify Messages:
- Messages from `named` indicate it received DNS notify messages for
specific zones.
- Example: `named[1463]: client @0x7fd61803ae68 10.12.112.3#49016:
received notify for zone '0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa'`.
- Probable Cause: Notification of changes in DNS zones, likely part of
routine DNS zone updates.
3. dnssec-keyfromlabel Configuration Warnings:
- Warnings from `dnssec-keyfromlabel` about missing configuration
parameters.
- Example: `dnssec-keyfromlabel[75984]: Configuration.cpp(96): Missing
log.level in configuration. Using default value: INFO`.
- Probable Cause: Default configuration being used due to missing
specific configuration settings.
4. Systemd Service Messages:
- `systemd` messages about `ipa-ods-exporter.service`.
- Example: `systemd[1]: ipa-ods-exporter.service: Consumed 4.047s CPU
time`.
- Probable Cause: Normal systemd logging of service status.
5. Kernel Segmentation Fault (Core Dump):
- Critical: A segmentation fault in `dnssec-keyfroml`.
- Example: `kernel: dnssec-keyfroml[75991]: segfault at 18 ip
00007f7fba586284 sp 00007ffff3e834a8 error 4 in
libsofthsm2.so[7f7fba4ef000+9b000]`.
- Probable Cause: A crash in the `dnssec-keyfromlabel` process, possibly
due to an issue with `libsofthsm2`.
6. Systemd Core Dump Processing:
- Systemd handling of the core dump created by the crash.
- Example: `systemd-coredump[75993]: Process 75991 (dnssec-keyfroml) of
user 984 dumped core`.
- Indicates that a core dump was generated for further analysis.
7. ipa-dnskeysyncd Python Traceback:
- A Python traceback from `ipa-dnskeysyncd`, indicating an error in the
script.
- Example: `ipa-dnskeysyncd[75834]:
ipapython.ipautil.CalledProcessError: Command [...] returned non-zero exit
status -11`.
- Probable Cause: The error seems related to the `dnssec-keyfromlabel`
command failure.
8. Named TCP Receive Error:
- Named server shutting down a TCP connection due to an error.
- Example: `named[1463]: dispatch 0x7fd618009200: shutting down due to
TCP receive error: connection reset`.
- Probable Cause: Network-related issue or remote client reset the
connection.
### Steps to Troubleshoot Core Dump
To investigate the core dump caused by `dnssec-keyfroml`:
1. Locate the Core Dump:
- Core dumps are typically stored in `/var/lib/systemd/coredump/` or
`/var/crash/` on RHEL systems.
- Use `coredumpctl list` to list all core dumps and find the one related
to `dnssec-keyfroml`.
2. Analyze the Core Dump:
- Use `coredumpctl info <PID>` or `gdb` to analyze the core dump.
Replace `<PID>` with the process ID of `dnssec-keyfroml`.
- This will provide stack traces and other details about the state of
the program at the time of the crash.
3. Check Dependencies:
- Ensure that all dependencies, especially `libsofthsm2`, are correctly
installed and not corrupted.
4. Review Configuration and Logs:
- Check the configuration files for `dnssec-keyfromlabel` and related
services.
- Review system logs for any anomalies or errors leading up to the crash.
5. Test in a Controlled Environment:
- If possible, replicate the issue in a test environment to better
understand the conditions leading to the creation of the core dump file.
On Mon, Mar 4, 2024 at 10:54 PM Harry G Coin via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> wrote:
FYI. El9, latest stable.
Mar 04 22:46:05
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at
0x7f283b53f7f0>
Mar 04 22:46:05
registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68
10.12.112.3#49016: received notify for zone
'0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa'
Mar 04 22:46:05
registry1.1.quietfountain.com named[1463]: client @0x7fd618191538
fc00:1002:c7::3#41560: received notify for zone
'0.0.0.0.0.0.a.0.0.8.9.2.4.0.6.2.ip6.arpa'
Mar 04 22:46:05
registry1.1.quietfountain.com dnssec-keyfromlabel[75984]:
Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO
Mar 04 22:46:05
registry1.1.quietfountain.com dnssec-keyfromlabel[75984]:
Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value:
ALL
Mar 04 22:46:05
registry1.1.quietfountain.com dnssec-keyfromlabel[75984]:
Configuration.cpp(124): Missing slots.removable in configuration. Using default value:
false
Mar 04 22:46:05
registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service:
Deactivated successfully.
Mar 04 22:46:05
registry1.1.quietfountain.com systemd[1]: ipa-ods-exporter.service:
Consumed 4.047s CPU time.
Mar 04 22:46:05
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
ipaserver.dnssec.bindmgr: INFO attrs: <ldap.cidict.cidict object at
0x7f283b53faf0>
Mar 04 22:46:06
registry1.1.quietfountain.com dnssec-keyfromlabel[75991]:
Configuration.cpp(96): Missing log.level in configuration. Using default value: INFO
Mar 04 22:46:06
registry1.1.quietfountain.com dnssec-keyfromlabel[75991]:
Configuration.cpp(96): Missing slots.mechanisms in configuration. Using default value:
ALL
Mar 04 22:46:06
registry1.1.quietfountain.com dnssec-keyfromlabel[75991]:
Configuration.cpp(124): Missing slots.removable in configuration. Using default value:
false
Mar 04 22:46:06
registry1.1.quietfountain.com named[1463]: client @0x7fd61803ae68
10.12.112.3#49016: received notify for zone
'0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa'
Mar 04 22:46:06
registry1.1.quietfountain.com named[1463]: client @0x7fd618191538
fc00:1002:c7::3#41560: received notify for zone
'0.0.0.0.0.0.f.0.0.8.9.2.4.0.6.2.ip6.arpa'
Mar 04 22:46:06
registry1.1.quietfountain.com kernel: dnssec-keyfroml[75991]: segfault at
18 ip 00007f7fba586284 sp 00007ffff3e834a8 error 4 in libsofthsm2.so[7f7fba4ef000+9b000]
likely on CPU 0 (core 0, socket 0)
Mar 04 22:46:06
registry1.1.quietfountain.com kernel: Code: 75 f4 48 8b 53 30 49 89 4c 24
20 49 89 44 24 10 49 89 54 24 28 48 83 c4 08 4c 89 e0 5b 41 5c c3 66 0f 1f 44 00 00 f3 0f
1e fa <48> 8b 47 18 48 85 c0 74 44 4c 8d 47 10 4c 89 c7 eb 12 66 2e 0f 1f
Mar 04 22:46:06
registry1.1.quietfountain.com systemd[1]: Started Process Core Dump (PID
75992/UID 0).
Mar 04 22:46:06
registry1.1.quietfountain.com systemd-coredump[75993]: Resource limits
disable core dumping for process 75991 (dnssec-keyfroml).
Mar 04 22:46:06
registry1.1.quietfountain.com systemd-coredump[75993]: [🡕] Process 75991
(dnssec-keyfroml) of user 984 dumped core.
Mar 04 22:46:06
registry1.1.quietfountain.com systemd[1]:
systemd-coredump(a)40-75992-0.service: Deactivated successfully.
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: Traceback (most
recent call last):
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/libexec/ipa/ipa-dnskeysyncd", line 130, in <module>
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: while
ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib64/python3.9/site-packages/ldap/syncrepl.py", line 464, in
syncrepl_poll
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
self.syncrepl_refreshdone()
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib/python3.9/site-packages/ipaserver/dnssec/keysyncer.py", line 128, in
syncrepl_refreshdone
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
self.bindmgr.sync(self.dnssec_zones)
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 232, in
sync
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
self.sync_zone(zone)
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 205, in
sync_zone
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
self.install_key(zone, uuid, attrs, tempdir)
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib/python3.9/site-packages/ipaserver/dnssec/bindmgr.py", line 146, in
install_key
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: result =
ipautil.run(cmd, capture_output=True)
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: File
"/usr/lib/python3.9/site-packages/ipapython/ipautil.py", line 599, in run
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]: raise
CalledProcessError(
Mar 04 22:46:07
registry1.1.quietfountain.com ipa-dnskeysyncd[75834]:
ipapython.ipautil.CalledProcessError: CalledProcessError(Command
['/usr/sbin/dnssec-keyfromlabel', '-E', 'pkcs11', '-K',
'/var/named/dyndb-ldap/ipa/master/_domainkey.rockstable.systems/XXXXXX',
'-a', b'RSASHA256', '-l',
b'pkcs11:object=YYYYY;pin-source=/var/lib/ipa/dnssec/softhsm_pin', '-P',
b'20231101161909', '-A', 'none', '-I', 'none',
'-D', 'none', '-E', 'pkcs11',
'_domainkey.rockstable.systems.'] returned non-zero exit status -11: 'Found
uninitialized token\nFound uninitialized token\nKey not found.\nPKCS11_load_public_key
returned NULL\nFound uninitialized token\nKey not found.\nPKCS11_get_private_key returned
NULL\ndnssec-keyfromlabel: warning: ENGINE_load_private_key failed (not
found)\ndnssec-keyfromlabel: fatal: failed to get key
_domainkey.rockstable.systems/RSASHA256: not found\n')
Mar 04 22:46:06
registry1.1.quietfountain.com named[1463]: dispatch 0x7fd618009200:
shutting down due to TCP receive error: 2001:13c7:7002:3000::11#53: connection reset
--
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue