On ma, 14 joulu 2020, Alexander Bokovoy via FreeIPA-devel wrote:
>On ti, 01 joulu 2020, Alexander Bokovoy via FreeIPA-devel wrote:
>>On ti, 01 joulu 2020, Florence Blanc-Renaud via FreeIPA-devel wrote:
>>>On 11/27/20 12:12 PM, Alexander Bokovoy via FreeIPA-devel wrote:
>>>>On ke, 18 marras 2020, Alexander Bokovoy via FreeIPA-devel wrote:
>>>>>On ma, 16 marras 2020, Alexander Bokovoy via FreeIPA-devel wrote:
>>>>>>On pe, 13 marras 2020, Alexander Bokovoy via FreeIPA-devel wrote:
>>>>>>>On ke, 11 marras 2020, Stanislav Levin via FreeIPA-devel
wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>11.11.2020 14:11, Alexander Bokovoy via FreeIPA-devel
пишет:
>>>>>>>>>On ke, 11 marras 2020, Stanislav Levin wrote:
>>>>>>>>>>>
>>>>>>>>>>>On top of that we have a worrying behavior of
>>>>>>>>>>>the Azure CI with regards
>>>>>>>>>>>to DNSSEC that waits for investigation.
>>>>>>>>>>please, where to see the failure?
>>>>>>>>>
>>>>>>>>>You can look, for example, at
>>>>>>>>>https://github.com/freeipa/freeipa/pull/5248
>>>>>>>>It is like
https://pagure.io/freeipa/issue/8538
>>>>>>>>
>>>>>>>>At least, 389-ds logging may be raised to 8192 from
>>>>>>>>the current one (0)
>>>>>>>>for debugging.
>>>>>>>>
>>>>>>>We already have debugging enabled in Azure CI builds. I
>>>>>>>uploaded logs to
>>>>>>>the issue 8538.
>>>>>>>
>>>>>>>To me this looks like 389-ds issue 4363 is not really fixed
yet.
>>>>>>
>>>>>>I ran few experiments with Rawhide and git master over weekend.
Here is
>>>>>>my status before 4.9.0-rc1 release preparation:
>>>>>>
>>>>>>- Master branch seems to be no worse than 4.8.0 in terms of
running on
>>>>>> Fedora 32 in Azure Pipelines CI and PR CI.
>>>>>>
>>>>>>- Rawhide has fixes for certmonger and 389-ds-base but I was
unable to
>>>>>> get them fully tested due to upgrade of glibc that made
impossible to
>>>>>> use Azure Pipelines with Rawhide anymore on kernels less than
v5.8.
>>>>>>
>>>>>>glibc changed implementation of faccessat() to use faccessat2() if
this
>>>>>>syscall is available at the compile time -- requires kernel v5.8
or
>>>>>>later. As a result, systemd cannot start anymore in unprivileged
>>>>>>container on Azure Pipelines CI even with host Ubuntu 20.04 which
uses
>>>>>>v5.4. The exact solution is unclear yet because it is a general
issue
>>>>>>with libseccomp not knowing about newer syscalls and not being
able to
>>>>>>filter out unknown syscalls in a way that would trigger a fallback
to
>>>>>>faccessat() in glibc.
>>>>>>
>>>>>>This is a generic issue -- other projects saw a similar fallout
when
>>>>>>coreutils and other projects started to use statx() syscall. For
>>>>>>example,
https://bugzilla.redhat.com/show_bug.cgi?id=1784228
outlines
>>>>>>this for libuv which is used by Node.js.
>>>>>>
>>>>>>libseccomp only added support for faccessat2() in version 2.5:
>>>>>>https://github.com/seccomp/libseccomp/commit/5696c896409c1feb37eb502df33cf36efb2e8e01,
>>>>>>
>>>>>>this version is available in Debian Sid already, so one option
would be
>>>>>>to try to update the host image at runtime to use newer
libseccomp2
>>>>>>package from Sid (it is easily installable on top of Focal
repositories,
>>>>>>I checked), then restart docker and reuse our unprivileged
containers.
>>>>>
>>>>>An update to the FreeIPA 4.9.0 release candidate releases.
>>>>>
>>>>>We merged most of fixes regarding Rawhide runs to git master and I
>>>>>branched ipa-4-9 for a new release.
>>>>>
>>>>>FreeIPA 4.9.0 release candidate 1 is out now and is built in Rawhide.
>>>>>There is a bug in client-only build which should now be addressed
with
>>>>>PR:
https://github.com/freeipa/freeipa/pull/5276
>>>>>
>>>>>Armando did set up PR CI to track ipa-4-9 branch. I did the same for
>>>>>Azure Pipelines. There is also a label 'ipa-4-9' for proposing
pull
>>>>>requests for the backports.
>>>>
>>>>Another update.
>>>>
>>>>I am planning for FreeIPA 4.9.0 release candidate 2 for December 1st.
>>>>
>>>>Rawhide state:
>>>>
>>>> - bind-dyndb-ldap 11.6-1.fc34 should be in a working state against BIND
>>>> 9.11 now. Installing IPA master with integrated DNS works just fine.
>>>>
>>>> - python3-dns 2.1.0-0.1.rc1.fc34 is broken and does not allow to
>>>> install IPA replica. This should be fixed with python3-dns
>>>> 2.1.0-0.2.rc1.fc34:
>>>>https://bodhi.fedoraproject.org/updates/FEDORA-2020-622a2dccdc
>>>> With this fix installing IPA replica works fine.
>>>>
>>>> - Spec file for FreeIPA needs updates based on our recent discussions
>>>> with Thomas for RHEL 8.4 packaging. I'll handle this in
>>>>
https://github.com/freeipa/freeipa/pull/5279
>>>>
>>>>Pull requests I expect to land before 4.9.0rc2 release:
>>>> 5294 Allow Apache to answer to ipa-ca requests without
>>>>ipa-4-9
https://github.com/freeipa/freeipa/pull/5294
>>>>{'failure': 1, 'success': 1, 'pending': 28}
>>>> 5292 Always define the path DNSSEC_OPENSSL_CONF ipa-4-9
>>>>https://github.com/freeipa/freeipa/pull/5292 {'success': 1,
>>>>'pending': 3}
>>>5292 has been merged in master and backported to ipa-4-9.
>>>
>>>> 5290 Improve PKI subsystem detection ipa-4-6 ipa-4-8
>>>>ipa-4-9
https://github.com/freeipa/freeipa/pull/5290
>>>>{'success': 1, 'failure': 1, 'pending': 24}
>>>5290 needs discussions with pki team, we can skip this fix for the next rc.
>>>
>>>> 5279 freeipa.spec.in: unify spec files across upstream
>>>>WIP ipa-4-9
https://github.com/freeipa/freeipa/pull/5279
>>>>{'success': 1, 'pending': 24}
>>>> 5199 Change KRA profiles in certmonger tracking so they
>>>>ipa-4-6 ipa-4-8 ipa-4-9
>>>>https://github.com/freeipa/freeipa/pull/5199 {'success': 1,
>>>>'pending': 27, 'failure': 1, 'error': 1}
>>>5199 has been merged on the master branch and a backport to
>>>ipa-4-9 is in progress.
>>
>>
>>Thanks, Flo. So I need to complete 5279 to proceed with RC2.
>
>Small update to catch up with the last two weeks.
>
>RC2 was released on December 4th:
>https://www.freeipa.org/page/Releases/4.9.0rc2
>
>RC3 was releasde on December 10th:
>https://www.freeipa.org/page/Releases/4.9.0rc3
>
>We are considerably close to the final 4.9.0 release now. The only
>remaining weirdness to figure out is why a combination of NetworkManager
>and systemd-resolved on the IPA client in Rawhide and F33 breaks client
>enrollment in OpenQA with
https://bodhi.fedoraproject.org/updates/FEDORA-2020-7851982ff6
>
>Arguably, this is NetworkManager issue -- it is not reproducible without
>this failing update.
>
>Thierry and Flo are looking into topology issues with a recent 389-ds
>which includes monotonic clock fixes. These changes seem to encounter
>other issues in 389-ds related to entryUUID introduction. We need to
>talk to 389-ds team to see what are the plans for the release on the
>directory server side and what we should be targetting as the stable
>version.
>
>Final bit to investigate is a set of SELinux AVCs as seen in RHEL 8.4
>development composes. They all look like this one:
>
>type=AVC msg=audit(1607701574.844:1426): avc: denied { search } for pid=31965
comm="dogtag-ipa-rene" name="opencryptoki" dev="tmpfs"
ino=72868 scontext=system_u:system_r:certmonger_t:s0
tcontext=system_u:object_r:pkcs_slotd_lock_t:s0 tclass=dir permissive=0
>
>which most likely needs a rule to be added in the general SELinux
>policy. We need to explore it a bit more and see if it is reproducible
>with F33/Rawhide.
>
>If we aren't able to get through it in next couple days, I'll do FreeIPA
>4.9.0 release without this fix and we'll set to process it in 4.9.1.
>There are few more SELinux-related issues along trust to AD path but
>with upcoming holidays there hardly be any time to look into them
>together with SELinux maintainers.
>
>Tentatively, FreeIPA 4.9.0 release would be set to December 15-16th.
One more issue I forgot about:
https://bugzilla.redhat.com/show_bug.cgi?id=1902811
Something in the deployment process changes ownership of
/var/named/dyndb-ldap from root:named to root:root. This causes startup
of named to fail because bind-dyndb-ldap driver cannot operate on
/var/named/dyndb-ldap/* content.
https://openqa.fedoraproject.org/tests/739654#step/role_deploy_domain_con...
shows it for latest Rawhide compose:
...
Dec 13 19:31:59 localhost named[33269]: generating session key for dynamic DNS
Dec 13 19:31:59 localhost named[33269]: sizing zone task pool based on 6 zones
Dec 13 19:31:59 localhost named[33269]: none:106: 'max-cache-size 90%' - setting
to 1759MB (out of 1954MB)
Dec 13 19:32:00 localhost named[33269]: set up managed keys zone for view _default, file
'/var/named/dynamic/managed-keys.bind'
Dec 13 19:32:00 localhost named[33269]: loading DynDB instance 'ipa' driver
'/usr/lib64/bind/ldap.so'
Dec 13 19:32:00 localhost named[33269]: bind-dyndb-ldap version 11.6 compiled at 00:00:00
Nov 23 2020, compiler 10.2.1 20201112 (Red Hat 10.2.1-8)
Dec 13 19:32:00 localhost named[33269]: unable to open directory 'dyndb-ldap',
working directory is '/var/named': permission denied
Dec 13 19:32:00 localhost named[33269]: LDAP config validation failed for database
'ipa': permission denied
Dec 13 19:32:00 localhost named[33269]: dynamic database 'ipa' configuration
failed: permission denied
Dec 13 19:32:00 localhost named[33269]: loading configuration: permission denied
Dec 13 19:32:00 localhost named[33269]: exiting (due to fatal error)
Dec 13 19:32:00 localhost systemd[1]: named.service: Control process exited, code=exited,
status=1/FAILURE
Dec 13 19:32:00 localhost systemd[1]: named.service: Failed with result
'exit-code'.
Dec 13 19:32:00 localhost systemd[1]: Failed to start Berkeley Internet Name Domain
(DNS).
If we look at bind-dyndb-ldap package, it has intended permissions:
[root@m2 ~]# rpm -qlv bind-dyndb-ldap|grep var
drwxrwx--- 2 root named 0 Nov 23 16:12 /var/named/dyndb-ldap
Interesting enough, if I'd re-run ipa-server-install after fixing the
ownership back to root:named, everything succeeds. So there is something
happening during installation that triggers a change. Since named user
account is created during named installation with
[root@m2 ~]# rpm -q --scripts bind|grep var/named
/usr/sbin/useradd -u 25 -r -N -M -g named -s /sbin/nologin -d /var/named -c Named named
>/dev/null 2>&1 || :;
the home directory /var/named should not be created or otherwise
modified during installation.
Yet, we have no explicit handling of /var/named/dyndb-ldap in the
installer code. There are places where we create
/var/named/dyndb-ldap/ipa but we use Python's os.mkdir() there which
should not modify permissions of the parent folder.
Good news: I figured it out and fixed package dependencies in
bind-dyndb-ldap. In my tests it succeeds now. The details can be seen at
With that, we've got few more issues reported that make it warrant to
wait until the corresponding PRs are merged:
1. kadmin.local's getprincs command doesn't list all principals:
This is incomplete implementation of a feature asked by krb5 QE in
RHEL/Fedora. We added a feature but didn't check that it actually didn't
return the whole list.
2. configuredService conversion to enabledService on upgrade:
This is a fallout of hidden replicas introduction in FreeIPA 4.8.0. If
you have deployments that started before FreeIPA 4.8.0 and you did
upgrade to FreeIPA 4.8.7 or recent, your server will get its services
marked as configuredService and not enabledService. This will cause
the services to not start on 'ipactl restart' as they will not match
anymore a filter we have (enabledService). Conversion is missing on
upgrade.
3. ipa-client-install: unilaterally set dns_lookup_kdc to True
This is something we wanted to do for long time. A lookup of KDCs
through DNS simplifies trust to Active Directory setup as administrators
don't need to add AD-specific realm configuration anymore. It is worth
to add this to 4.9.0 release as it is a defaults change for new clients.
4. Fixes to Debian/Ubuntu service names:
--
/ Alexander Bokovoy
Sr. Principal Software Engineer
Security / Identity Management Engineering
Red Hat Limited, Finland