During an upgrade from 4.5.0-21.el7.centos.1.2 to 4.5.0-21.el7.centos.2.2 on a CentOS 7.4 machine, FreeIPA's DNS server briefly returned NXDOMAIN for records which existed in FreeIPA. These invalid responses were returned for a very short amount of time, but caused long-running issues with Java clients which tend to cache DNS responses. Upgraded packages included: 389-ds-base, 389-ds-base-libs, 389-ds-base-snmp, ipa-client, ipa-client-common, ipa-python-compat, ipa-server, ipa-server-common, ipa-server-dns, ipa-server-trust-ad, python2-ipa-server, and a dozen sss-related packages.
I reproduced this in a FreeIPA test environment by running `while true; do dig some.dns.entry.managed.by.freeipa @ip.address.of.freeipa | tee -a a-log-file; done` from one server, and running `yum update` on the FreeIPA machine. The invalid NXDOMAIN responses were returned some time after the `yum update` logged 'Cleanup' for the RPMs, and seemed to be during the 'Verifying' phase.
These NXDOMAIN responses claimed that an upstream nameserver ( a.root-servers.net) was the authority for my zone:
a-log-file-; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.37.rc1.el6_7.7 <<>> some.dns.entry.managed.by.freeipa @172.16.0.77 a-log-file-;; global options: +cmd a-log-file-;; Got answer: a-log-file:;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 2889 a-log-file-;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0 a-log-file- a-log-file-;; QUESTION SECTION: a-log-file-;some.dns.entry.managed.by.freeipa. IN A a-log-file- a-log-file-;; AUTHORITY SECTION: a-log-file-. 60 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2017102400 1800 900 604800 86400 a-log-file- a-log-file-;; Query time: 227 msec a-log-file-;; SERVER: 172.16.0.77#53(172.16.0.77) a-log-file-;; WHEN: Tue Oct 24 18:30:28 2017 a-log-file-;; MSG SIZE rcvd: 130
Usually when querying an invalid DNS entry, the dig output still claims that my FreeIPA server is authoritative for the zone: $ dig doesntexist.zone.managed.by.freeipa @172.16.0.77
; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.37.rc1.el6_7.7 <<>> doesntexist.zone.managed.by.freeipa @172.16.0.77 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 59953 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0
;; QUESTION SECTION: ;doesntexist.zone.managed.by.freeipa. IN A
;; AUTHORITY SECTION: zone.managed.by.freeipa. 30 IN SOA idm01.freeipa. hostmaster.zone.managed.by.freeipa. 1508869828 30 900 1209600 30
;; Query time: 0 msec ;; SERVER: 172.16.0.77#53(172.16.0.77) ;; WHEN: Tue Oct 24 19:27:12 2017 ;; MSG SIZE rcvd: 113
Is it possible that during a yum update, the FreeIPA DNS server temporarily forgets what zones it's authoritative for (or forgets all DNS records) and just delegates to the upstream DNS server for half a second or so? Or is something else going on here?
I'm open to suggestions.
Thanks, Nicholas.