On Fri, 2018-05-25 at 20:31 +0000, Alexander Bokovoy wrote:
To be totally honest, I did not get this message either, Alex - my
understanding was that once we finally got all the intended packages
landing and the automated tests worked, you actually thought FreeIPA
was in acceptable shape fore real use. If it was known that it was not,
we absolutely ought to have communicated this *far* more widely than on
a niche mailing list: FreeIPA is supposed to be a key feature of Fedora
Server which is itself a key edition of Fedora. This should have been
up-front in the release notes and the release announcement, or frankly,
should've caused us to rethink the release plans.
I did tell that several times
but the only real answer I've got: "these issues are not blocking criteria for
Fedora Server". At some point you choose your own fights: fixing software or fixing
release criteria. For Fedora 29 I'd like us to extend Fedora Server blocking criteria,
now that majority of porting has been completed.
For us a push with Python3 migration (we have to migrate all Python base, not a selected
module here or there), NSS to OpenSSL migration, mod_nss to mod_ssl migration, NSS default
database format migration, Apache ignorance of its ecosystem (changes in ABI in mod_proxy
in minor versions), modularity inconsistence through the course of year 2017, have killed
a lot of the productive time.
Obviously there was some sort of significant communication fail if
enough people missed the message that this got totally whiffed on, so
we should absolutely figure out what we can do better there.
Perhaps this also suggests our existing release criteria and test cases
for FreeIPA are insufficient: if it can pass our existing tests and
thus appear to meet our existing criteria, yet be in your judgment "not
ready for production", that seems fundamentally wrong. How do you we
think we could address that? Can you give some kind of summary of the
issues here, which we can use to think about how to extend the test
cases and criteria?
The issues were listed in the email referenced by Jonathan
already.
- Replication failures should have been a blocker alone (they are for FreeIPA team) but
Fedora Server criteria does not include them.
- Broken NSS sqldb defaults caused us several months working on fixes. The latest one,
https://bodhi.fedoraproject.org/updates/FEDORA-2018-8cf042000b, was only pushed after
Fedora 28 release.
https://bugzilla.redhat.com/show_bug.cgi?id=1568271 was found in late
April, after we did fight all the previous issues. We started with NSS sqldb adaptation in
October 2017.
- Only on Thursday this week we've finally tracked down a nasty python-ldap bug that
crashed FreeIPA framework on every time --all option was used on a host or service entry
with additional access controls defined. This is not part of Fedora Server criteria but
kills FreeIPA use with delegated permissions to retrieve Kerberos credentials.
- We had to do a lot of Python 3 porting work for other projects. Time is not unlimited,
especially when it comes to releases and blocking criteria.
- Dogtag had to work on Tomcat 8.5 adaptation where existing API it dependent on was
removed.
So on May 15th we released
https://www.freeipa.org/page/Releases/4.6.90.pre2 which is now
in Fedora 28 stable updates. We consider it as one of closer candidates to being stable.
Between 4.6.90.pre1 and pre2 are two months of hard work across several sizeable projects
(freeipa, sssd, 389-ds, MIT Kerberos, dogtag, nss, authselect, gssproxy, to name a few).
We have a testing setup at FreeIPA upstream that allows us to test complex topologies.
Only recently we were able to move to Fedora 28 testing there as we had issues with our
components. There we test also what OpenQA is unable to test so far. I think 4.6.90.pre2
is in much better shape than what Fedora 28 had released. However, if we were to get it as
a blocking release, Fedora 28 would have been delayed by at least a month.
As I said, we had no choice: a push of NSS sqldb defaults change forced us to work on both
nss-related code and openssl migration at the same time. It made impossible to keep
FreeIPA from Fedora 27 and do our work in a separate module. This was known since autumn
2017 and was a well voiced situation.
--
/ Alexander Bokovoy