Once the portables will be built in f37, the repacked binary in f37
and in f38 (39...)will be identical.
Rpms have some additional value - split to subpakcages, few symlinks
due to system integration, system tzdata, system crypto binding and so
on.
On Thu, 29 Jun 2023 at 22:20, Tom Stellard <tstellar(a)redhat.com> wrote:
On 6/29/23 13:07, Jiri Vanek wrote:
> nn. You were right. There are going to be two separated packages.
> Portable, built once in oldest live, and "normal" which is going to
> repack them for all and shipp them.
> My apologise for typo in last second change:
>
https://fedoraproject.org/w/index.php?title=Changes%2FBuildJdkOncePackEve...
>
> narrowed.
>
Ok, that makes sense now.
My next question is what is the difference between the java-xy-openjdk-portable package
that is built on f37 and the repacked java-xy-openjdk package that is shipped on f38.
Are the bits inside the package exactly the same?
-Tom
> On Thu, 29 Jun 2023 at 21:16, Tom Stellard <tstellar(a)redhat.com> wrote:
>>
>> On 6/29/23 11:06, Jiri Vanek wrote:
>>> Nope, xy stands for 1.8.0, 11, 17 and latest. It is enumerated several
>>> time in the proposal. Still the
>>>
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoret...
>>> adjusted
>>>
>>
>> OK, I see. I thought there were going to be two different packages.
java-xy-openjdk-portable
>> and java-xy-openjdk. Where java-xy-openjdk is the one that gets shipped in
Fedora and
>> java-xy-openjdk-portable lives only in the side-tags.
>>
>> -Tom
>>
>>> Tahnx!
>>>
>>> On Thu, 29 Jun 2023 at 19:14, Tom Stellard <tstellar(a)redhat.com>
wrote:
>>>>
>>>> On 6/29/23 09:52, Jiri Vanek wrote:
>>>>> Hi Tom!
>>>>>
>>>>> Thanx a lot of for input. Although I did my bes with the tagging,
it
>>>>> will be learning on the go.
>>>>> I had adapted
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#theoret...
>>>>> as you suggested. It is great improvement.
>>>>>
>>>>
>>>> Thanks, this looks better.
>>>>
>>>> For step 5. should the first mention of java-xy-openjdk-portable
actually
>>>> be java-xy-openjdk ? Same question for step 7.
>>>>
>>>> -Tom
>>>>
>>>>> Especially the config, I was not aware about. That woudl indeed help
a lot.
>>>>> The usage of pernament tag is someging I have to learn, but is
>>>>> moreover necessary for proper srpm rebuil.
>>>>>
>>>>> TYVM!
>>>>> J.
>>>>>
>>>>> On Wed, 28 Jun 2023 at 21:31, Tom Stellard
<tstellar(a)redhat.com> wrote:
>>>>>>
>>>>>> On 6/26/23 09:21, Aoife Moloney wrote:
>>>>>>>
https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere#includi...
>>>>>>>
>>>>>>> This document represents a proposed Change. As part of the
Changes
>>>>>>> process, proposals are publicly announced in order to
receive
>>>>>>> community feedback. This proposal will only be implemented
if approved
>>>>>>> by the Fedora Engineering Steering Committee.
>>>>>>>
>>>>>>>
>>>>>>> == Summary ==
>>>>>>>
>>>>>>> This is the last step in
>>>>>>>
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
>>>>>>> effort. JDKs in fedora are already static, and we repack
portable
>>>>>>> tarballs into RPMs. Currently, the portable tarball is built
for each
>>>>>>> Fedora and EPEL version. Goal here is to build each JDK
>>>>>>> (8,11,17,21,latest (20)) only once, in oldest live Fedora
repack in
>>>>>>> all live Fedoras. If jdk is buitl in epel, it will be built
in oldest
>>>>>>> possible epel and repacked in newer live epels.
>>>>>>>
>>>>>>>
>>>>>>> == Owner ==
>>>>>>> * Name: [[User:jvanek| Jiri Vanek]]
>>>>>>>
>>>>>>> * Email: jvanek(a)redhat.com
>>>>>>>
>>>>>>>
>>>>>>> == Detailed Description ==
>>>>>>>
>>>>>>> As described in
>>>>>>>
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs ;
>>>>>>> during last year, packaging of JDKs had changed
dramatically. As
>>>>>>> described in the same wiki page and in individual sub
changes and
>>>>>>> devel threads, the primary reason for this is to lower
maintenance and
>>>>>>> still keep Fedora Java friendly.
>>>>>>>
>>>>>>> * In the first system wide change, we have changed the JDKs
to build
>>>>>>> properly as standalone, portable JDK - the way JDK is
supposed to be
>>>>>>> built. I repeat, we spent ten years by patching JDK to
become properly
>>>>>>> dynamic against system libs, and all patches went upstream,
but this
>>>>>>> has become a fight which can not be won.
>>>>>>>
>>>>>>> * As a second step we introduced portable RPMs, which do not
have any
>>>>>>> system integration, only build JDK and pack the final
tarball in RPM
>>>>>>> for Fedora use.
>>>>>>>
>>>>>>> * In third step - without any noise, just verified with
fesco -
>>>>>>>
https://pagure.io/fesco/issue/2907 - we stopped building JDK
in fully
>>>>>>> integrated RPMs. Instead of this, normal RPMs BUildRequire
portable
>>>>>>> RPMs and just unpack it, and repack it.
>>>>>>>
>>>>>>> Now last step is ahead - to build portable LTS JDKs 8,11,17
and 21 in
>>>>>>> oldest live Fedora, and repack everywhere.
java-latest-openjdk, which
>>>>>>> contains latests STS JDK - currently 20, soon briefly 21 and
a bit
>>>>>>> after 22... If we would built java-latest-openjdk in oldest
live EPEL
>>>>>>> - epel8 now, we have verified, that such repacked JDKs works
fine,
>>>>>>> however repack from epel seem to not be acceptable, thus
>>>>>>> ajva-latest-openjdk will be built twice - one in oldest live
fedora,
>>>>>>> and once in oldest live epel. Build forme oldest possible
epel will be
>>>>>>> repacked to that one or newer epels, and build from oldest
live fedroa
>>>>>>> to all fedoras.
>>>>>>>
>>>>>>> === theoretical tagging solution ===
>>>>>>>
>>>>>>> 1. request side tags for all releases
>>>>>>> 2. build the actual Java in the side tag for the
oldest thing
>>>>>>
>>>>>> Could you use the real package name here. I think that will
make it easier
>>>>>> to understand. You can still put 'actual Java' in
parens or something.
>>>>>>
>>>>>>> 3. tag the result ot (2) to all side tags from (1)
>>>>>>> 4. waitrepo them
>>>>>>> 5. build the repacked java packages in all the side
tags from (1)
>>>>>>
>>>>>> Same thing here, can you use the real package name.
>>>>>>
>>>>>>> 6. untag the result of (2) from all the side tags from
(1)
>>>>>>> 7. ship bodhi updates from side tags OR retag the
builds to candidate tags
>>>>>>> (and delete the side tags)
>>>>>>>
>>>>>>> The build from (2) will be eventually garbage collected. To
prevent that, it
>>>>>>> might be re-tagged regularly. This is where releng might be
able to help by
>>>>>>> creating a long lived tag to tag this into for preserving.
>>>>>>>
>>>>>>> Yes, we could make a 'fN-openjdk' tag and mark it
protected... that part
>>>>>>> would be easy enough.
>>>>>>>
>>>>>>
>>>>>> I think this process could be improved if the side-tags in 1.
are permanent side-tags,
>>>>>> and step 6 is skipped. That would make it easier to rebuild the
packages from
>>>>>> step 5 if needed.
>>>>>>
>>>>>> Also, can you put a config file in the dist-git repos to tell
fedpkg which target
>>>>>> to build against? I thought I remembered seeing that feature in
the past. If so,
>>>>>> then you could configure the dist-gits for the repacked javas to
automatically build
>>>>>> from those side-tags, which I think would be a lot easier for
package maintainers and
>>>>>> may help make automated rebuilds possible.
>>>>>>
>>>>>> -Tom
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> ==== including portable srpms in release (improving of step
6) ====
>>>>>>>
>>>>>>> To include portable rpms in all live Fedoras is currently
not
>>>>>>> possible. Best solution would be simply make and bodhi
update of one
>>>>>>> portable rpm to all live fedoras. Bodhi is currenlty not
capable to do
>>>>>>> so, issue was raised:
>>>>>>>
https://github.com/fedora-infra/bodhi/issues/5387
investigating
>>>>>>> possibility to deliver single build as update to several
releases.
>>>>>>>
>>>>>>> "..It's not possible ATM, it would require a heavy
rewrite of the
>>>>>>> code, starting from the database structure (every build is
now related
>>>>>>> to a single release)..." Maybe on long run..."
>>>>>>>
>>>>>>> On long run, if bodhi will allow this, that will be way to
go.
>>>>>>> On short run, there are following options:
>>>>>>> a) ask releng to tag the portables directly
>>>>>>> - this needs manual approach of rare humans, thus no
go unless
>>>>>>> strictly enforced by unpredicted conditions
>>>>>>> - this walks around whole testing repos. For
portables tarballs, as
>>>>>>> nothing should depend on them, and are tested indirectly
after repack,
>>>>>>> this should be technically ok, but is heavily discouraged
in
>>>>>>> principle.
>>>>>>> b) build portable for all OSes, but do not ship them
(don't do bodhi update)
>>>>>>> - this would probably work for all frontiers, only
the real
>>>>>>> repacked JDK will be different
>>>>>>> - pros is, that we will be sure that portables builds
on live fedoras
>>>>>>> - cons is, that the portable JDK will not be
available by dnf install anyway
>>>>>>> c) build portable for all OSes, including bodhi update
>>>>>>> - pros is, that we will be sure that portables builds
on live fedoras
>>>>>>> - another pros is that the portable JDK will be
available by dnf
>>>>>>> install anyway
>>>>>>> - there may be clash during the build which will
cause to repack
>>>>>>> wrong (newer, non certified) portables
>>>>>>> d) include SRPM_REBUILD.readme in srpm and generated
>>>>>>> PORTABLES_INSTALL.readme in RPMs, which will ideally at
least contain:
>>>>>>> - instruction why you need portables
>>>>>>> - instruction how to find the portables
>>>>>>> - from SRPM_REBUILD.readme pointing to
PORTABLES_INSTALL.readme
>>>>>>> - generated link to the koji, allowing to download
the SRPM
>>>>>>> - generated link to the koji, allowing to download
the binaries
>>>>>>> - generated instruction how to dnf install used
portables
>>>>>>>
>>>>>>> I would currently vote for d). If there will be complains
about broken
>>>>>>> SRPM rebuild, or need to install portables without hacking,
then
>>>>>>> fall-back a, b or c via Change Proposal.
>>>>>>> Once Bodhi allows single build to be tagged to several
release, I will
>>>>>>> move to that.
>>>>>>>
>>>>>>> == Feedback ==
>>>>>>>
>>>>>>>
>>>>>>> == Benefit to Fedora ==
>>>>>>>
>>>>>>> Java maintainers will finally have some free time... No
kidding -
>>>>>>> maintenance and *certification* of so much supported JDKs on
so much
>>>>>>> Fedora versions is brutal. By building once, and repack, we
will
>>>>>>> regain cycles to continue support Fedora with all LTS and
one STS JDK.
>>>>>>>
>>>>>>> If we fail to build once and repack everywhere, Java
maintainers will
>>>>>>> most likely need to lower the number of JDKs in fedora to
system one
>>>>>>> only.
>>>>>>>
>>>>>>> == Scope ==
>>>>>>> * Proposal owners: Technically all JDKs (except 8, where
some more
>>>>>>> tuning is needed, and EPEL for java-latest) are prepared, as
they have
>>>>>>> a portable version, and RPMs just repack it. Except tuning
up the JDK8
>>>>>>> and EPEL for latest, scope owners are done.
>>>>>>>
>>>>>>> * Other developers: There will be needed significant support
from RCM
>>>>>>> and maybe senior Fedora leadership to help to finish the
build in
>>>>>>> oldest and enable to repack everywhere
>>>>>>>
>>>>>>> * '''Release engineering:
[
https://pagure.io/releng/issue/11438
>>>>>>> #11438]''' There will be needed significant
support from RCM, where
>>>>>>> I'm actually unsure what they will have to do to enable
this. The mas
>>>>>>> rebuild will not be needed.
>>>>>>>
>>>>>>> * Policies and guidelines: AFAIK none (not needed for this
Change)
>>>>>>> * Trademark approval: N/A (not needed for this Change)
>>>>>>>
>>>>>>> * Alignment with Community Initiatives: All supported JDKs
will remain
>>>>>>> in Fedora in highest possible quality with full QA and
certification,
>>>>>>> and its packagers will not lose their minds. Note that QA
will still
>>>>>>> run on all live Fedoras, not only on the builder one.
>>>>>>>
>>>>>>>
>>>>>>> == Upgrade/compatibility impact ==
>>>>>>>
>>>>>>> The change should be completely transparent to any user.
>>>>>>>
>>>>>>>
>>>>>>> == How To Test ==
>>>>>>>
>>>>>>> `sudo dnf update/install "java*"` will install
expected set of working packages.
>>>>>>>
>>>>>>> SRPM rebuild of both portables (which were built once) and
of any rpms
>>>>>>> (from this freshly rebuild portbales) have to remain
possible
>>>>>>>
>>>>>>>
>>>>>>> == User Experience ==
>>>>>>> The change should be absolutely transparent to any user.
>>>>>>>
>>>>>>>
>>>>>>> == Dependencies ==
>>>>>>> To finish this we will need heavy support from RCM, and
maybe others.
>>>>>>> Although there are precedents with such pacakge, they all
bites. From
>>>>>>> SW point of view, the dependece chain is `normal RPMs build
requires
>>>>>>> portable RPMs` and thats all.
>>>>>>>
>>>>>>>
>>>>>>> == Contingency Plan ==
>>>>>>> * Contingency mechanism: Even if It should be straight
forward to
>>>>>>> revert back to building per OS, it '''may be
impossible for current
>>>>>>> maintainers to save time''' for it. If this
change is approved, we
>>>>>>> will be building '''4-5'''
(jdk8,11,17,sts and 21) builds for all
>>>>>>> fedoras. If this change is not finished in time, we may
'''need to
>>>>>>> orphan some of the JDKs'''. In better case, we
will be able to keep
>>>>>>> living '''one LTS as system JDK, and
java-latest-openjdk''' as future
>>>>>>> system JDK. That is 2*(3-5) builds (rawhide, (forked,),
latest live,
>>>>>>> oldest live (oldest not yet dropped)). '''In
worst case''', we may be
>>>>>>> able to maintain only java-latest-openjdk. On long run
changing it to
>>>>>>> '''rolling system JDK,''' which are
the expected 3-5 builds.
>>>>>>> * Contingency deadline: N/A
>>>>>>> * Blocks release? No. The change can be introduced even on
the fly to
>>>>>>> live distributions.
>>>>>>>
>>>>>>> == Documentation ==
>>>>>>>
>>>>>>> N/A (not a System Wide Change)
>>>>>>>
>>>>>>> == Release Notes ==
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> devel mailing list -- devel(a)lists.fedoraproject.org
>>>>>> To unsubscribe send an email to
devel-leave(a)lists.fedoraproject.org
>>>>>> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>>>>>> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>>> List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>>>>>> Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
>>>>>
>>>>
>>>
>>
>