https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.
== Owner == * Name: [[User:Otaylor|Owen Taylor]] * Email: otaylor@redhat.com
== Detailed Description == There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app. Currently, the rebuild process is done using modules - packages are either rebuilt in a shared flatpak-common module, or in a module specific to the application. This change replaces this with a separate build target that all rebuilds are done in.
=== Rebuilding RPMs === For each Fedora release, we'll have two additional targets
f39-flatpak-runtime (inherits f39), tags: f39-flatpak-runtime-build, f39-flatpak-runtime f39-app (inherits f39-flatpak-runtime), tags: f39-app-build, f39-app
The f39-flatpak-runtime target has a limited use - it's used to build packages that go into the Flatpak runtimes (standard and KDE). This would include flatpak-runtime-config, flatpak-rpm-macros, but also rebuilds of standard packages if needed to prune dependencies: currently this is done for gstreamer-plugins-good.
The f39-app target is the main target that is used for prefix=/app rebuilds. flatpak-rpm-macros is part of the build group for this target, resulting in builds that target /app. As currently, spec files can be conditionalized with '%if 0%{?flatpak}'
Builds in f39-flatpak-runtime and f39-app have the dist tags .f39runtime and .f39app respectively. This is necessary so that we can automatically rebuild a package without having to bump the release in the specfile. (Koji requires everything to have a unique NVR.) It also allows humans and tools to easily distinguish these specialized builds.
Any Fedora packager can rebuild a package into f39-app using 'fepdkg build --target=f39-app'. There will also be a new convenience subcommands of fedpkg 'fedpkg build-flatpak-rpms', 'fedpkg build-flatpak-rpms-local' that can be used to rebuild all missing bundled dependencies of a Flatpak, in Koji or locally.
Once a package exists in f39-app or f38-app, then [[https://gitlab.com/redhat/centos-stream/ci-cd/distrosync/distrobuildsync distrobuildsync]] will be used to do a build into f39-app each time a standard build completes.
=== Container Versioning ===
Currently, the version is based on the module version:
firefox:stable:3820230427145713:b1edb643 => firefox-stable-3820230427145713.b1edb643
Where the module version is created from platform versions and git commit timestamps for the module repository. We could reproduce something very much like this based on the runtime branch and git commit timestamp, but since application Flatpaks have an identifiable "Main RPM" (unlike the general case of modules and containers), a better scheme is:
firefox-112.0.2-1.fc38 => firefox-flatpak-112.0.2-1 name: main RPM name + -flatpak version: main RPM version release: automatically incremented based on existing Koji builds
Because upgrades are done on Flatpak ID (org.mozilla.Firefox), there is no compatibility impact of switching Flatpaks from 'firefox' to 'firefox-flatpak'.
There is considerable implementation complexity within OSBS to implement this, because the N/V/R need to be written into generated Dockerfile as labels ''before'' building it, but it should be manageable. If not, we'll need to switch to a different scheme. (Running dnf with <code>--best</code> when building the container will help make this reliable and would be a general improvement.)
=== Updates ===
The current plan is that builds of RPMs will be tagged directly into f39-app and containers will be composed from the contents of f39-app with no involvement of Bodhi. Bodhi is only involved in releasing the container.
We can add CI checks on container updates to compare the rebuilt packages in the container to the latest released versions for Fedora, though it's not always easy to distinguish:
* Newer because of a RPM change that hasn't passed testing yet * vs. Newer because a spec file was needed to get the package to rebuild with prefix=/app
Since Fedora does not embargo builds, there's no potential information leak if a change to dist-git is released via container before it is released by RPM.
=== Disadvantages and issues=== In moving to this system, we lose the ability to take advantage of the capabilities of modularity - to use a different stream branch of a library for an application than the one shipped in F39, say. There are essentially no examples of this being used in all the Flatpaks created for Fedora over the last 5 years, probably because modularity has gained little traction within Fedora more broadly.
Some Flatpak modules strip down bundled packages by setting macros in the buildopts/rpms/macros section of the modulemd file. For example, flatpak-common has:
buildopts: rpms: macros: | %_without_mingw 1 %_without_tkinter 1
This is no longer possible, so these changes need to be fed back into spec files with '%if 0%{?flatpak}' style conditionalization.
Additionally a couple of packages (evolution-data-services and tracker-miners) are set up so they can be built with an application-specific D-Bus prefix. Evolution has:
buildopts: rpms: macros: | %_eds_dbus_services_prefix org.gnome.Evolution
This will need to be redone so that evolution-data-services doesn't need recompilation and the prefixing can be done by changing configuration files at container build time.
== Feedback ==
== Benefit to Fedora == The experience for Fedora developers who want to create Flatpaks of their applications will be improved: * They no longer need to understand modularity and modulemd files - instead of a modulemd file *and* a container.yaml, they simply create a container.yaml. * Rebuilds are automatically shared between Flatpaks, without having to get them added to the flatpak-common module. * Because the local build experience is no longer using the poorly maintained local-build mode of MBS, it can be more efficient and easier to understand.
This should result in more Fedora Flatpaks and more up-to-date Fedora Flatpaks.
We also greatly reduce the usage of Module Build Service within Fedora. This is important because upstream maintenance is currently reduced, and it's not clear that we'll be able to sustain a deployment indefinitely.
== Scope ==
* Proposal owners: ** Update flatpak-module-tools to be able to work without modularity ** Create changes to fedpkg to add 'rebuild-flatpak-packages' (implementation likely within flatpak-module-tools) ** Update atomic-reactor to be able to build Flatpaks without modularity
* Other developers: ** Flatpaks will need to be migrated to the new system by Flatpak packagers. While we could make the new system opt-in for F39, it's probably easiest to just require all Flatpaks to be moved over. In many cases, this should consist of just a few minor changes to container.yaml. ** Changes to fedpkg and atomic-reactor need to be reviewed, and for atomic-reactor deployed. '''Note''' Fedora is on OSBS 1, which is not a current release. Backporting changes to OSBS 1 seems more realistic than upgrading Fedora to OSBS 2 at this point.
* Release engineering: [https://pagure.io/releng/issue/11415 #Releng issue number] ** Release engineering will need to manually create the tags/targets for F39 and update the tag maintenance scripts to do this for future releases.
* Policies and guidelines: https://docs.fedoraproject.org/en-US/flatpak/ will need to be updated once we have an initial implementation. No impact to packaging guidelines is expected.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives: No
== Upgrade/compatibility impact == The Flatpaks built will look exactly the same as before from a user perspective.
== How To Test ==
Testing here will be done by selecting a few Flatpaks and trying to rebuild them using the new system, first locally and then on infrastructure.
The final validation will be when we mass rebuild Flatpaks for F39.
== User Experience == This should be largely invisible to users. Hopefully, by simplifying the developer experience around creating Flatpaks, more Fedora Flatpaks will be available and they will be kept more up-to-date.
== Dependencies == This is unrelated to other Fedora changes or to changes within packages.
== Contingency Plan ==
* Contingency mechanism: The current infrastructure for module-based builds will be maintained, so that if we can't get this done for F39, we'll build Flatpaks as modules for F39. * Contingency deadline: By 2023-08-01 we should have runtimes built and a number of applications or we'll plan on using modules for F39 Flatpaks. * Blocks release? No
== Documentation == At the moment, this change proposal is standalone documentation of the change. As noted above, https://docs.fedoraproject.org/en-US/flatpak/ will need to be updated once we have an initial implementation. If you are interested in getting involved in planning and implementing this change, please contact the [https://fedoraproject.org/wiki/SIGs/Flatpak Flatpak SIG] matrix room.
== Release Notes ==
On Wed, 2023-05-10 at 11:54 +0100, Aoife Moloney wrote:
Additionally a couple of packages (evolution-data-services and tracker-miners) are set up so they can be built with an application-specific D-Bus prefix. Evolution has:
buildopts: rpms: macros: | %_eds_dbus_services_prefix org.gnome.Evolution
This will need to be redone so that evolution-data-services doesn't need recompilation and the prefixing can be done by changing configuration files at container build time.
Hi, I cannot speak for the tracker-miners, but in case of the Evolution, it runs in its own sandbox, separated from the host system, with bundled evolution-data-server (eds) services, thus when the user installs the Flatpak version, he/she gets the expected Evolution and eds versions, independent of what the host system has installed. Advantage: the user gets all fixes, not only client-side (Evolution) fixes. It's important, from my point of view.
In many cases, this should consist of just a few minor changes to container.yaml.
Do you mean the `evolution.yaml` is gone with this change and the dependencies are taken directly from the .spec file? The eds dependency is a problem here, as you noted, not talking that not everything from the .spec file requires a rebuild for the flatpak (most dependencies are part of the runtime).
Bye, Milan
On Wed, May 10, 2023 at 8:06 AM Milan Crha mcrha@redhat.com wrote:
On Wed, 2023-05-10 at 11:54 +0100, Aoife Moloney wrote:
Additionally a couple of packages (evolution-data-services and tracker-miners) are set up so they can be built with an application-specific D-Bus prefix. Evolution has:
buildopts: rpms: macros: | %_eds_dbus_services_prefix org.gnome.Evolution
This will need to be redone so that evolution-data-services doesn't need recompilation and the prefixing can be done by changing configuration files at container build time.
Hi,
I cannot speak for the tracker-miners, but in case of the Evolution, it runs in its own sandbox, separated from the host system, with bundled evolution-data-server (eds) services, thus when the user installs the Flatpak version, he/she gets the expected Evolution and eds versions, independent of what the host system has installed. Advantage: the user gets all fixes, not only client-side (Evolution) fixes. It's important, from my point of view.
We'll still have an evolution-data-server that runs inside the sandbox and has prefixed names, it will just have to be done a different way. My best current idea is that for the (single) Flatpak build, the evolution-data-services would have
-DDBUS_SERVICES_PREFIX=com.example.FlatpakApp
Then we'd change things (preferably upstream) so instead of building DBUS_SERVICES_PREFIX into the source code,
com.example.FlatpakApp.org.gnome.evolution.dataserver.AddressBook10.service
would have:
Exec=/app/libexec/evolution-addressbook-factory --dbus-services-prefix=com.example.FlatpakApp
Then container.yaml would have something like:
dbus-service-substitute=com.example.FlatpakApp:org.gnome.evolution
And that would do that substitution in the names and contents of any service files when building the container. Does that sound workable? Are there better ways we could do it?
In many cases, this should consist of just a few minor changes to
container.yaml.
Do you mean the `evolution.yaml` is gone with this change and the dependencies are taken directly from the .spec file? The eds dependency is a problem here, as you noted, not talking that not everything from the .spec file requires a rebuild for the flatpak (most dependencies are part of the runtime).
Yeah, evolution.yaml is gone with the change. Any dependency that is not part of the runtime will have to have been rebuilt in the f39-app target, or the container build will fail. Then we'll have some convenience at the fedpkg level to make that not too painful. ('fedpkg flatpak-build' will check that things seem ok before starting the build, and there will be 'fedpkg flatpak-build-rpms --all-missing' to rebuild the missing dependencies.)
- Owen
On Wed, 2023-05-10 at 09:30 -0400, Owen Taylor wrote:
Does that sound workable? Are there better ways we could do it?
Hi, if I recall correctly, using the custom D-Bus prefix is there to match application's D-Bus prefix defined for the flatpak, thus: a) the services run independently from the system, inside the sandbox; b) they can be started without any additional effort on the Flatpak configuration side (because the services share the app's D-Bus prefix).
I briefly grepped the evolution-data-server sources and it seems that most of the places in the .c files can be changed in runtime, but there are places where the things can break, like the `evolution-data- server.pc` file or the D-Bus .service files, which both reference the D-Bus name from the compile time and those .service files are also named by the service (otherwise there's a runtime warning in the journal about the file not matching the service name). Those might not be show stoppers, I guess.
By the way, when people install Evolution flatpak, they have preinstalled evolution-ews inside it. How will they install it after this change?
Bye, Milan
On Mon, May 15, 2023 at 1:51 AM Milan Crha mcrha@redhat.com wrote:
On Wed, 2023-05-10 at 09:30 -0400, Owen Taylor wrote:
Does that sound workable? Are there better ways we could do it?
Hi,
if I recall correctly, using the custom D-Bus prefix is there to match application's D-Bus prefix defined for the flatpak, thus: a) the services run independently from the system, inside the sandbox; b) they can be started without any additional effort on the Flatpak configuration side (because the services share the app's D-Bus prefix).
I briefly grepped the evolution-data-server sources and it seems that most of the places in the .c files can be changed in runtime, but there are places where the things can break, like the `evolution-data- server.pc` file or the D-Bus .service files, which both reference the D-Bus name from the compile time and those .service files are also named by the service (otherwise there's a runtime warning in the journal about the file not matching the service name). Those might not be show stoppers, I guess.
The idea is that we'd substitute the name and contents of the service files at container creation time. The .pc file is tricker, since it is build time. Those variables aren't used much. I found two examples: - libfolks uses it when setting up a private bus for test cases - shouldn't matter - You use it in to fix bus names in flatpak-evolution-wrapper.sh - those could just be hardcoded since the bus name will be always the same for the Evolution Flatpak
By the way, when people install Evolution flatpak, they have
preinstalled evolution-ews inside it. How will they install it after this change?
The idea is that the container.yaml can list any number of packages to be installed in the Flatpak. The first one is the "main" one, but that only means that the name/version of the Flatpak come from it by default. Adding evolution-ews as an additional package will work fine.
- Owen
Hi,
On Wed, 2023-05-17 at 12:23 -0400, Owen Taylor wrote:
You use it in to fix bus names in flatpak-evolution-wrapper.sh - those could just be hardcoded since the bus name will be always the same for the Evolution Flatpak
It will work unless the version of the D-Bus service is bumped for whatever reason. That's the only reason why the variables are part of the .pc file, to have correct D-Bus service name provided, not hard- coded.
In any case, I understand these are not obstacles, not the big one, if the runtime change for the D-Bus service names will be possible. Feel free to file an upstream bug against the evolution-data-server, thus it's tracked (and not forgotten).
Thanks and bye, Milan
On Wed, May 10, 2023 at 11:54:39AM +0100, Aoife Moloney wrote:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
For each Fedora release, we'll have two additional targets
f39-flatpak-runtime (inherits f39), tags: f39-flatpak-runtime-build, f39-flatpak-runtime f39-app (inherits f39-flatpak-runtime), tags: f39-app-build, f39-app
The f39-flatpak-runtime target has a limited use - it's used to build packages that go into the Flatpak runtimes (standard and KDE). This would include flatpak-runtime-config, flatpak-rpm-macros, but also rebuilds of standard packages if needed to prune dependencies: currently this is done for gstreamer-plugins-good.
The f39-app target is the main target that is used for prefix=/app rebuilds. flatpak-rpm-macros is part of the build group for this target, resulting in builds that target /app. As currently, spec files can be conditionalized with '%if 0%{?flatpak}'
Builds in f39-flatpak-runtime and f39-app have the dist tags .f39runtime and .f39app respectively. This is necessary so that we can automatically rebuild a package without having to bump the release in the specfile. (Koji requires everything to have a unique NVR.) It also allows humans and tools to easily distinguish these specialized builds.
I love how simple this is. Mostly just using koji as it's supposed to be used.
Zbyszek
On Wed, May 10, 2023 at 9:48 AM Florian Weimer fweimer@redhat.com wrote:
- Aoife Moloney:
There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app.
Is this relocation still required?
Yes - the /app vs. /usr split is pretty fundamental to the operation of Flatpak, and this Change doesn't propose any changes to how Flatpak works, just how Flatpaks are built in Fedora.
- Owen
Makes a lot of sense and improves the information value of the version greatly.
I'm just wondering whether the name suffix is the right place to put "flatpak" as if it were a subpackage - can we use the dist tag instead, say `fi39` for "fedora immutable based on fedora 39" or something like that? Or `fp39` for `fedora flatpak ...`?
I'm just wondering whether the name suffix is the right place to put
"flatpak" as if it were a subpackage - can we use the dist tag instead, say `fi39` for "fedora immutable based on fedora 39" or something like that? Or `fp39` for `fedora flatpak ...`?
The main difference is what happens when you type firefox into koji - do you see
firefox-112.0.2-1.fc38 [the RPM] firefox-112.0.2-1.flatpak [the Flatpak]
Or do you just see the RPM build and you have to type firefox-flatpak to see the container build? Which one is clearer? Which one is most useful? I don't honestly have a big opinion here. I'll bring this and some other naming issues up in the Fedora Flatpaks SIG next Monday.
- Owen
On Thu, May 11, 2023 at 7:44 AM Michael J Gruber mjg@fedoraproject.org wrote:
Makes a lot of sense and improves the information value of the version greatly.
I'm just wondering whether the name suffix is the right place to put "flatpak" as if it were a subpackage - can we use the dist tag instead, say `fi39` for "fedora immutable based on fedora 39" or something like that? Or `fp39` for `fedora flatpak ...`? _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
* Owen Taylor:
On Wed, May 10, 2023 at 9:48 AM Florian Weimer fweimer@redhat.com wrote:
- Aoife Moloney:
There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app.
Is this relocation still required?
Yes - the /app vs. /usr split is pretty fundamental to the operation of Flatpak, and this Change doesn't propose any changes to how Flatpak works, just how Flatpaks are built in Fedora.
Do you have a pointer where I can read up on the current approach?
I'm concerned that this is just like Software Collections with a different path, which I believe where previously banned in Fedora (and are generally a huge hassle on the releng side, I think).
Thanks, Florian
On Fri, May 12, 2023 at 1:34 PM Florian Weimer fweimer@redhat.com wrote:
- Owen Taylor:
On Wed, May 10, 2023 at 9:48 AM Florian Weimer fweimer@redhat.com
wrote:
- Aoife Moloney:
There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app.
Is this relocation still required?
Yes - the /app vs. /usr split is pretty fundamental to the operation of Flatpak, and this Change doesn't propose any changes to how Flatpak works, just how Flatpaks are built in Fedora.
Do you have a pointer where I can read up on the current approach?
Some references that I found quickly:
https://docs.fedoraproject.org/en-US/flatpak/concepts/ https://docs.flatpak.org/en/latest/basic-concepts.html
But neither goes into a lot of detail to motivation. I'm certainly willing to describe things in more detail or look around further if you still have questions.
I'm concerned that this is just like Software Collections with a different path, which I believe where previously banned in Fedora (and are generally a huge hassle on the releng side, I think).
The big distinction here is that there is no idea that these RPMs will be installed directly on a Fedora system. They are used only when building Flatpak containers. So I don't think there is much similarity to SCL's other than "built with a different prefix".
- Owen
* Owen Taylor:
On Fri, May 12, 2023 at 1:34 PM Florian Weimer fweimer@redhat.com wrote:
- Owen Taylor:
On Wed, May 10, 2023 at 9:48 AM Florian Weimer fweimer@redhat.com wrote:
- Aoife Moloney:
There are two types of Flatpak containers - runtimes - which contain unmodified Fedora packages, and applications - which contain Fedora packages rebuilt to relocate them from /usr to /app.
Is this relocation still required?
Yes - the /app vs. /usr split is pretty fundamental to the operation of Flatpak, and this Change doesn't propose any changes to how Flatpak works, just how Flatpaks are built in Fedora.
Do you have a pointer where I can read up on the current approach?
Some references that I found quickly:
https://docs.fedoraproject.org/en-US/flatpak/concepts/ https://docs.flatpak.org/en/latest/basic-concepts.html
But neither goes into a lot of detail to motivation. I'm certainly willing to describe things in more detail or look around further if you still have questions.
As far as I can see, it doesn't explain why we need rebuilds.
Toolbx seems to be able achieve something similar without rebuilds.
Thanks, Florian
On Thu, May 25, 2023 at 4:02 AM Florian Weimer fweimer@redhat.com wrote:
- Owen Taylor:
On Fri, May 12, 2023 at 1:34 PM Florian Weimer fweimer@redhat.com
wrote:
Do you have a pointer where I can read up on the current approach?
Some references that I found quickly:
https://docs.fedoraproject.org/en-US/flatpak/concepts/ https://docs.flatpak.org/en/latest/basic-concepts.html
But neither goes into a lot of detail to motivation. I'm certainly willing to describe things in more detail or look around further if you still have questions.
As far as I can see, it doesn't explain why we need rebuilds.
Toolbx seems to be able achieve something similar without rebuilds.
In the docker/oci ecosystem, layers are explicitly dependent on the exact underlying layers. If I modify a base image to upgrade one library, I have to rebuild images that depend on it. Similarly, once you've created a Toolbx container based on a verison of the image, you can't replace the image without recreating the container and losing local modifications.
If Flatpak was based on docker/oci, then every time a single library in the runtime was modified, all 2000+ apps on Flathub would need to be rebuilt and the upgraded versions downloaded by every single user. In the ideal case, the upgraded application layers delta-compress well, but we're still talking about a huge amount of wasted resources. It works better for server-side containers because of datacenter-level networking and resources, and because server-side container images are often simpler and are based on simpler base images.
So, we need to preserve the ability to separately upgrade the application and the runtime. This is currently achieved in Flatpak by the "low-tech" way that the runtime is in /usr and the application in /app and we mount those together into final namespace. If we want to achieve the same thing with everything in /usr, then we have a series of technical challenges to overcome. Among others:
* We need a way to merge filesystems (fuse-overlayfs, composefs, huge quantities of bind mounts, hardlink trees...) * We need to rebuild files that can contain data from both parts (/etc/ld.so.conf, /usr/lib/fontconfig/cache, etc.) * We have to handle the case where the application layer needs to bundle a newer/different version of a library than the runtme
These aren't impossible to solve, but to come up with solutions, implement them in the Flatpak codebase, validate them with real applications, update all the build infrastructure is a large (year or years scale) project. Beyond that, you have to convince upstream Flatpak that this is worthwhile - to add a whole second mode of operation for Flatpak when the current mode of putting application code in /app largely works fine in the "normal" case where the application is rebuilt from source code.
So, to recap, Flatpak separates out /usr and /app because that allows the runtime and application to be separately upgraded. Modifying Flatpak to solve that problem in a different way would be a technically difficult and long process that, even if we had the resources to do it, would not solve immediate problems with our dependency on modularity.
- Owen
V Thu, May 25, 2023 at 06:04:51AM -0400, Owen Taylor napsal(a):
- We need to rebuild files that can contain data from both parts
(/etc/ld.so.conf, /usr/lib/fontconfig/cache, etc.)
How does splitting files from /usr to /app helps with this item? Or is it that application flatpaks are not supposed to deliver their own libraries and fonts and if they do, they have to vendor them in a specific way (e.g. storing a location of the application libraries into application executables with RPATH)?
-- Petr
On Thu, May 25, 2023 at 6:57 AM Petr Pisar ppisar@redhat.com wrote:
V Thu, May 25, 2023 at 06:04:51AM -0400, Owen Taylor napsal(a):
- We need to rebuild files that can contain data from both parts
(/etc/ld.so.conf, /usr/lib/fontconfig/cache, etc.)
How does splitting files from /usr to /app helps with this item? Or is it that application flatpaks are not supposed to deliver their own libraries and fonts and if they do, they have to vendor them in a specific way (e.g. storing a location of the application libraries into application executables with RPATH)?
Generally, if the runtime includes a library (like fontconfig) where it's useful for an application to drop data into /app, the runtime is expected to set the necessary configuration files, environment variables, or whatever so that they are found:
For example, fontconfig is handled by:
https://src.fedoraproject.org/rpms/flatpak-runtime-config/blob/f38/f/50-flat...
which is installed in all Fedora runtime images. Setting LD_LIBRARY_PATH is handled by the Flatpak core.
- Owen
On Fri, May 12, 2023 at 07:33:56PM +0200, Florian Weimer wrote:
I'm concerned that this is just like Software Collections with a different path, which I believe where previously banned in Fedora (and are generally a huge hassle on the releng side, I think).
The main sticking point with Software Collections was the complixity it introduced into spec files -- lots of special macros that had to be used just-so.
(And of course there were a lot of disagreements about naming things.)
Hi,
Aoife Moloney amoloney@redhat.com writes:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
snip
There is considerable implementation complexity within OSBS to implement this, because the N/V/R need to be written into generated Dockerfile as labels ''before'' building it, but it should be manageable. If not, we'll need to switch to a different scheme. (Running dnf with <code>--best</code> when building the container will help make this reliable and would be a general improvement.)
Just out of interest, has this complicated part been already implemented or is it still on the todo list? Would using a different container build system reduce the complexity here?
Cheers,
Dan
On Thu, May 11, 2023 at 2:31 PM Dan Čermák dan.cermak@cgc-instruments.com wrote:
Hi,
Aoife Moloney amoloney@redhat.com writes:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
snip
There is considerable implementation complexity within OSBS to implement
this,
because the N/V/R need to be written into generated Dockerfile as labels ''before'' building it, but it should be manageable. If not, we'll need to switch to a different
scheme.
(Running dnf with <code>--best</code> when building the container will help make this reliable and would be a general improvement.)
Just out of interest, has this complicated part been already implemented or is it still on the todo list? Would using a different container build system reduce the complexity here?
I have a fairly strong sense of what needs to be done, and have written similar code before, but haven't actually written the patches for the atomic-reactor component of OSBS.
Honestly, the "complexity" here is dwarfed both by the complexity of maintaining OSBS in Fedora *and* by the complexity of switching to a different container build system. I don't think it factors in to whether Fedora switches and on what timescale.
- Owen
Il 10/05/23 12:54, Aoife Moloney ha scritto:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
I've never tried to make a flatpak because I was scared by the need of firstly build modules and I'm really happy to see this change moving on.
However, there's something I can't understand: AFAIK, a flatpak is platform independent, so a flatpak built on F39 can be installed on any Fedora version or even on other Linux distributions... right? So, why having all those "Fedora Containers" releases in Bodhi which follow Fedora branches? Isn't just one Fedora Containers release enough? What happens if one builds the same flatpak on multiple Fedora Containers releases?
Mattia
On Fri, May 12, 2023, 1:03 PM Mattia Verga via devel < devel@lists.fedoraproject.org> wrote:
Il 10/05/23 12:54, Aoife Moloney ha scritto:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
I've never tried to make a flatpak because I was scared by the need of firstly build modules and I'm really happy to see this change moving on.
However, there's something I can't understand: AFAIK, a flatpak is platform independent, so a flatpak built on F39 can be installed on any Fedora version or even on other Linux distributions... right? So, why having all those "Fedora Containers" releases in Bodhi which follow Fedora branches? Isn't just one Fedora Containers release enough? What happens if one builds the same flatpak on multiple Fedora Containers releases? infrastructure/new_issue https://pagure.io/fedora-infrastructure/new_issue
The "F38 Flatpaks" release in Bodhi represents Flatpaks built with the F38 package set against the F8 runtime. But, yes, as you say we handle Flatpaks as a single stream. Once we release Firefox into "F39 Flatpaks", everybody on all releases gets that and we never do an update in "F38 Flatpaks" again.
If updates *do* get pushed on multiple releases, last pushed wins. Might be useful if we found that we pushed something to early or broken - but isn't normal.
But what if we had a single release instead?
On a technical level, we rely on separate releases because Bodhi is using that to know what koji tag to pull from. So to merge them, we'd probably need a single dest tag in Koji as well. But would it be more convenient and less confusing for packagers and users? Would there be any performance or UI problems from having a release in Bodhi that extends indefinitely?
In any case, a change to how we handle Flatpak releases in Koji could be done separately from this change proposal :-)
- Owen
Il 13/05/23 03:04, Owen Taylor ha scritto:
On Fri, May 12, 2023, 1:03 PM Mattia Verga via devel devel@lists.fedoraproject.org wrote:
Il 10/05/23 12:54, Aoife Moloney ha scritto:
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
I've never tried to make a flatpak because I was scared by the need of firstly build modules and I'm really happy to see this change moving on.
However, there's something I can't understand: AFAIK, a flatpak is platform independent, so a flatpak built on F39 can be installed on any Fedora version or even on other Linux distributions... right? So, why having all those "Fedora Containers" releases in Bodhi which follow Fedora branches? Isn't just one Fedora Containers release enough? What happens if one builds the same flatpak on multiple Fedora Containers releases? [infrastructure/new_issue](https://pagure.io/fedora-infrastructure/new_issue)
The "F38 Flatpaks" release in Bodhi represents Flatpaks built with the F38 package set against the F8 runtime. But, yes, as you say we handle Flatpaks as a single stream. Once we release Firefox into "F39 Flatpaks", everybody on all releases gets that and we never do an update in "F38 Flatpaks" again.
If updates *do* get pushed on multiple releases, last pushed wins. Might be useful if we found that we pushed something to early or broken - but isn't normal.
I was wondering if that was due to have the ability of building the flatpak against a specific runtime. I mean, as I understand how a flatpak works, a flatpak created from a koji build for f39 will be built against a specific runtime (say, KDE available in f39 stable tags), but when we build the same koji build for f40 we will be using a different runtime which may not be compatible... so still having a Fedora Containers 39 release will make the maintainer able to continue using the f39 runtime... I know I'm quite confused, aren't I?
If I search for fedora-toolbox updates in Bodhi, I now see there are multiple updates released alongside each others at the same time.
But what if we had a single release instead?
On a technical level, we rely on separate releases because Bodhi is using that to know what koji tag to pull from. So to merge them, we'd probably need a single dest tag in Koji as well. But would it be more convenient and less confusing for packagers and users? Would there be any performance or UI problems from having a release in Bodhi that extends indefinitely?
Would be simpler to have a single Koji dest tag which doesn't change alongside Fedora branches, so that packagers only have to remember that one? (provided that we don't want the ability to build multiple updates as discussed above)
From a Bodhi POV, I don't think there would be any problem in extending a release indefinitely... we currently have Fedora ELN running since 2020 and with something like 80K updates and nothing as blown up (yet...).
In any case, a change to how we handle Flatpak releases in Koji could be done separately from this change proposal :-)
- Owen
On Sat, May 13, 2023, 4:24 AM Mattia Verga via devel < devel@lists.fedoraproject.org> wrote:
Il 13/05/23 03:04, Owen Tay rt I look
On Fri, May 12, 2023, 1:03 PM Mattia Verga via devel < devel@lists.fedoraproject.org> wrote:
Il 10/05/23 12:54, Aoife Moloney ha scritto: M kk kk kk kk o kk kk kk I I'll k> https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
I've never tried to make a flatpak because I was scared by the need of firstly build modules and I'm really happy to see this change moving on.
However, there's something I can't understand: AFAIK, a flatpak is platform independent, so a flatpak built on F39 can be installed on any Fedora version or even on other Linux distributions... right? So, why having all those "Fedora Containers" releases in Bodhi which follow Fedora branches? Isn't just one Fedora Containers release enough? What happens if one builds the same flatpak on multiple Fedora Containers releases? infrastructure/new_issue https://pagure.io/fedora-infrastructure/new_issue
The "F38 Flatpaks" release in Bodhi represents Flatpaks built with the F38 package set against the F8 runtime. But, yes, as you say we handle Flatpaks as a single stream. Once we release Firefox into "F39 Flatpaks", everybody on all releases gets that and we never do an update in "F38 Flatpaks" again.
If updates *do* get pushed on multiple releases, last pushed wins. Might be useful if we found that we pushed something to early or broken - but isn't normal.
I was wondering if that was due to have the ability of building the flatpak against a specific runtime. I mean, as I understand how a flatpak works, a flatpak created from a koji build for f39 will be built against a specific runtime (say, KDE available in f39 stable tags), but when we build the same koji build for f40 we will be using a different runtime which may not be compatible... so still having a Fedora Containers 39 release will make the maintainer able to continue using the f39 runtime... I know I'm quite confused, aren't I?
If I search for fedora-toolbox updates in Bodhi, I now see there are multiple updates released alongside each others at the same time
I think some of the confusion comes from the fact that fedora-toolbox is not a Flatpak at all and we have different bodhi releases and policies for Flatpaks and other containers. We naturally do want parallel versions of the toolbox container - you should be able to create a F37 Toolbox or a F38 Toolbox or ...
But what if we had a single release instead?
On a technical level, we rely on separate releases because Bodhi is using that to know what koji tag to pull from. So to merge them, we'd probably need a single dest tag in Koji as well. But would it be more convenient and less confusing for packagers and users? Would there be any performance or UI problems from having a release in Bodhi that extends indefinitely?
Would be simpler to have a single Koji dest tag which doesn't change alongside Fedora branches, so that packagers only have to remember that one? (provided that we don't want the ability to build multiple updates as discussed above)
fedpkg automatically determined the target based on the runtime specified in container.yaml. We'd need separate targets in any case because the build tag where the packages installed in the container come from needs to be different based on the runtime version / fedora release. But the different targets could potentially share a single dest tag.
One thing that would need to change in this model would be the name of the runtime container in koji/bodhi. We'd probably need to have it be flatpak-runtime-f38 or something like that, or it would be confusing to be doing parallel updates into a single dest tag and bodhi release.
(The runtime needs security updates as long as we have any applications still using it.)
From a Bodhi POV, I don't think there would be any problem in extending a release indefinitely... we currently have Fedora ELN running since 2020 and with something like 80K updates and nothing as blown up (yet...)
Well, that answers that question :-)
I've created: https://gitlab.com/fedora/sigs/flatpak/fedora-flatpaks/-/issues/15 to track this idea, and will bring it up in the next Fedora Flatpaks SIG meeting.
Owen
On Fri, May 12, 2023 at 09:04:40PM -0400, Owen Taylor wrote:
So, why having all those "Fedora Containers" releases in Bodhi which follow Fedora branches? Isn't just one Fedora Containers release enough?
[...]
The "F38 Flatpaks" release in Bodhi represents Flatpaks built with the F38 package set against the F8 runtime. But, yes, as you say we handle Flatpaks as a single stream. Once we release Firefox into "F39 Flatpaks", everybody on all releases gets that and we never do an update in "F38 Flatpaks" again.
If updates *do* get pushed on multiple releases, last pushed wins. Might be useful if we found that we pushed something to early or broken - but isn't normal.
But what if we had a single release instead?
A few years ago, a design team contributor was working on an animated video extolling our painless upgrade process. But, in the midst of that, they updated to a new Fedora release, and discovered that Inkscape had dropped an obscure export format which they happened to rely on.
I'd love it if we could produce both "fast" and "slow" streams. Ideally that's orthogonal to Fedora runtime release, but practically speaking that's what we have now. Maybe in the future we could build a "slow" stream as part of EPEL on a UBI or CentOS Stream runtime?
V Wed, May 10, 2023 at 11:54:39AM +0100, Aoife Moloney napsal(a):
https://fedoraproject.org/wiki/Changes/FlatpaksWithoutModules
[...]
=== Rebuilding RPMs === For each Fedora release, we'll have two additional targets
f39-flatpak-runtime (inherits f39), tags: f39-flatpak-runtime-build, f39-flatpak-runtime f39-app (inherits f39-flatpak-runtime), tags: f39-app-build, f39-app
The f39-flatpak-runtime target has a limited use - it's used to build packages that go into the Flatpak runtimes (standard and KDE). This would include flatpak-runtime-config, flatpak-rpm-macros, but also rebuilds of standard packages if needed to prune dependencies: currently this is done for gstreamer-plugins-good.
The f39-app target is the main target that is used for prefix=/app rebuilds. flatpak-rpm-macros is part of the build group for this target, resulting in builds that target /app. As currently, spec files can be conditionalized with '%if 0%{?flatpak}'
[...]
Any Fedora packager can rebuild a package into f39-app using 'fepdkg build --target=f39-app'.
[...]
Once a package exists in f39-app or f38-app, then [[https://gitlab.com/redhat/centos-stream/ci-cd/distrosync/distrobuildsync distrobuildsync]] will be used to do a build into f39-app each time a standard build completes.
Do I understand correctly that flapaks containing the same package actually reuses the same package build? E.g. if a flatpak for Firefox requires a nss package and a flatpak for Thunderbird requires the same nss package, then nss is built into f39-app only once and both Firefox and Thunderbird flapaks are created from the same nss package build?
Because if there were a need for rebuilding the same package for different flapaks differently, using a single Koji target would not not work. A Koji tag only presents the latest build of a package.
-- Petr
On Wed, May 17, 2023 at 4:42 AM Petr Pisar ppisar@redhat.com wrote:
[...]
Any Fedora packager can rebuild a package into f39-app using 'fepdkg build --target=f39-app'.
[...]
Once a package exists in f39-app or f38-app, then [[
https://gitlab.com/redhat/centos-stream/ci-cd/distrosync/distrobuildsync
distrobuildsync]] will be used to do a build into f39-app each time a standard build completes.
Do I understand correctly that flapaks containing the same package actually reuses the same package build? E.g. if a flatpak for Firefox requires a nss package and a flatpak for Thunderbird requires the same nss package, then nss is built into f39-app only once and both Firefox and Thunderbird flapaks are created from the same nss package build?
Yes, that's the idea. While we had the theoretical ability to build libraries differently when bundling into different Flatpaks, in the vast majority of cases, we did not use that capability.
Because if there were a need for rebuilding the same package for different flapaks differently, using a single Koji target would not not work. A Koji tag only presents the latest build of a package.
Yep. The side tag mechanism could possibly be used if a situation arose where we *did* need different builds, but I don't see that need.
- Owen
This proposal has now been submitted to FESCo for voting https://pagure.io/fesco/issue/3005 _______________________________________________ devel-announce mailing list -- devel-announce@lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapro... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue