In the medium-term future, we might want to provide some application software to users as Flatpak bundles in addition to traditional RPM form. This should be generally transparent to users but provide (at least) two advantages. First, these applications can be sandboxed, and only able to access resources they're specifically allowed to, increasing security. And second, users may have easier options for mixing and matching application versions that match their personal needs. And as a project, as Flatpak gains traction as a cross-platform solution, providing a whole bunch of Fedora packages in this way even to users of other distros helps increase our impact and usefulness to the world.
The Flatpak design splits the runtime from the application in the filesystem — apps are built expecting the runtime at /usr as normal, but are installed into a tree at /app. I'm not proposing that we move to having /app, but the idea of having a separate tree is pretty fundamental. As it stands right now, if we want to build existing RPMs as Flatpaks, we have to either do some very sketchy magic with relocation, or else hack on the specfiles.
So, the crazy idea: what if, for applications which would be useful as a Flatpak, we allow /opt/fedora/apps/[appname-version] as an allowed prefix? Then, the exact same RPMs could be used by someone who doesn't care about Flatpaks _and_ to construct the Flatpak, with no really-invasive changes. (Perhaps by making a link inside the Flatpak, or simply by asking for an extension to Flatpak to support the different application root.)
From the FHS 3.0:
/opt is reserved for the installation of add-on application software packages.
... but it doesn't actually define what "add-on application software packages" are. Although we integrate it all together in Fedora, arguably a lot of our software could qualify.
From our current Packaging Guidelines:
/opt and its related directories (/etc/opt and /var/opt) is reserved for the use of vendors in the FHS. We have reserved the fedora name with LANANA for our use. If a package installs files into /opt it may only use directories in the /opt/fedora hierarchy. Fedora attempts to organize this directory by allocating a subdirectory of our /opt/fedora directory for specific subsystems. If you think you need to use /opt/fedora please file an FPC ticket to decide whether it's a valid use of /opt and what subdirectory should be allocated for your use.
.... so, it's not completely outside the realm of precedent. I know it's very different from our existing "blend everything into /usr" approach, but I think it might be worth doing for the increased flexibility. What do you all think?
On 10/19/2016 01:50 PM, Matthew Miller wrote:
/opt is reserved for the installation of add-on application software packages.... but it doesn't actually define what "add-on application software packages" are.
They are referring to any arbitrary non-OS-vendor provided add-on package/suite (esp. CSS packages) etc.
For them, /opt/<vendor> is the playground they can play any crazy packaging games they want to.
Have a look into google's chrome packaging, if you want an example, which pretty nicely respects the FHS.
Although we integrate it all together in Fedora, arguably a lot of our software could qualify.
Over the years, FPC repeatedly has discussed this.
IIRC, we basically had ruled out this idea for Fedora packages, because their should not be any technical need to resort to /opt for opensource packages, which are part of the distro.
Weather putting third-party packages (rpmfusion, etc.) into /opt/<third-party> instead of /usr (I.e. merging with the distro) is a matter of taste.
Weather putting flatpaks into /opt is a clever idea, I don't want to comment on, because I consider flatpaks to be an idea which is best to be abandoned, ASAP.
Ralf
"MM" == Matthew Miller mattdm@fedoraproject.org writes:
MM> So, the crazy idea: what if, for applications which would be useful MM> as a Flatpak, we allow /opt/fedora/apps/[appname-version] as an MM> allowed prefix? Then, the exact same RPMs could be used by someone MM> who doesn't care about Flatpaks _and_ to construct the Flatpak, with MM> no really-invasive changes. (Perhaps by making a link inside the MM> Flatpak, or simply by asking for an extension to Flatpak to support MM> the different application root.)
I'm not entirely sure I understand all of what's being proposed here.
Certainly /opt/fedora/flatpack or something (but not "apps", which is too generic for a flatpack-specific location) would be a great place to place flatpaks; certainly better than /flatpack or whatever which was proposed previously.
But.. if you're suggesting that the regularly-packaged libeoffice be moved out of /usr and into /opt/fedora/apps/libreoffice (or libreoffice-5.1.6.1 or libreoffice-5.1.6.1-2 or some other variation incorporating more of the version-release pair) then... that's a big deal.
Some will say that it really goes against the whole Unix spirit and the organizational setup which we've had for 30+ years. I would agree that it's a break with history, but hipster-coders raised on /Libraries and /Applications certainly won't care, and if it carries measurable benefits (beyond just making something internal to flatpaks easier) then I think there's certainly an argument to be made for it. (The possibility of having multiple versions is, for me, enticing.)
Really my concern is how you'd deal with the more basic issues:
* How does typing "libreoffice" work? Do you also drop symlinks into /usr/bin? * How do things like desktop configuration, icons and whatnot get installed where desktops will look for them? * Can these "apps" provide libraries or content used by other "apps" or even just regular programs on the system? How do those other things actually find the content? * Do you allow multiple versions? How does that change the above three answers? How does ordering work in that case? How do you decide which version is the "main" one, which gets /usr/bin/libreoffice, etc.?
I really hope that the answer for the first three is to just add each of these directories to various search paths. That gets hilarious pretty quickly.
- J<
On 10/19/2016 03:52 PM, Jason L Tibbitts III wrote:
But.. if you're suggesting that the regularly-packaged libeoffice be moved out of /usr and into /opt/fedora/apps/libreoffice (or libreoffice-5.1.6.1 or libreoffice-5.1.6.1-2 or some other variation incorporating more of the version-release pair) then... that's a big deal.
Some will say that it really goes against the whole Unix spirit
In particular, this would go against the OpenSource spirit and thus contradict Fedora's objectives.
Ralf
On 19 October 2016 at 12:50, Matthew Miller mattdm@fedoraproject.org wrote:
apps are built expecting the runtime at /usr as normal, but are installed into a tree at /app.
Alex isn't on this list (cc'd) but this is what he said to me on IRC:
--- Its unclear exactly how this would work for a non-trivial application. If you need any dependencies, then those will either be in /usr, which is fine for a regular rpm, but then what is the point of the whole exercise. However, if you want to put it in a container with a limited /usr (such as with flatpak or atomic workstation) then you need all the additional dependencies built in that same prefix too. Or alternatively, each in its own prefix plus a rats nest of environment variables so they all find each other.
Additionally, having an app in a separate prefix means it doesn't integrate well with the system. For instance, gnome-shell will not find the icon and desktop files in /opt/app/org.gnome.gedit/share/. In order to handle this flatpak has the "export" system that manages a separate combined directory of such files. You'd need something similar for this to work outside flatpak.
And for flatpak, using /app rather than /opt is actually an important decision, because we often need to export the real host filesystem to an app, and if the app is in /opt then exposing the host /opt to the application becomes very tricky.
Basically, I think this approach isn't really useful for flatpak, nor do I see it as useful for regular rpms. ---
Richard
On Wed, Oct 19, 2016 at 08:52:26AM -0500, Jason L Tibbitts III wrote:
But.. if you're suggesting that the regularly-packaged libeoffice be moved out of /usr and into /opt/fedora/apps/libreoffice (or libreoffice-5.1.6.1 or libreoffice-5.1.6.1-2 or some other variation incorporating more of the version-release pair) then... that's a big deal.
Yeah, I'm suggesting considering the big deal. That way, we basically get Flatpaks "for free", with no need for forked specfiles or crazy specfile macros (or behind-the-scenes slight-of-hand).
Some will say that it really goes against the whole Unix spirit and the organizational setup which we've had for 30+ years. I would agree that it's a break with history, but hipster-coders raised on /Libraries and /Applications certainly won't care, and if it carries measurable benefits (beyond just making something internal to flatpaks easier) then I think there's certainly an argument to be made for it. (The possibility of having multiple versions is, for me, enticing.)
It definitely is a change to the organizational setup Fedora has had. I'm not so sure it goes against the Unix spirit in general -- I'm certainly used to having all sorts of stuff in /opt from back in my sysadmin days.
Really my concern is how you'd deal with the more basic issues:
- How does typing "libreoffice" work? Do you also drop symlinks into /usr/bin?
FHS has a provision for /opt/<provider>/bin -- we could put that /opt/fedora/bin in the default $PATH and use that.
- How do things like desktop configuration, icons and whatnot get installed where desktops will look for them?
Something similar, I guess. :)
- Can these "apps" provide libraries or content used by other "apps" or even just regular programs on the system? How do those other things actually find the content?
I would suggest no, at least for libraries. Maybe the Modularity initiative people have a different answer here, though.
- Do you allow multiple versions? How does that change the above three answers? How does ordering work in that case? How do you decide which version is the "main" one, which gets /usr/bin/libreoffice, etc.?
I think allowing multiple versions would be a nice side-effect, but we *could* say that if you need that, use the Flatpak versions. Maybe the alternatives system could be used to moderate which version owns the shared bin link.
I really hope that the answer for the first three is to just add each of these directories to various search paths. That gets hilarious pretty quickly.
Yeah, I think that would get out of control. :)
On Wed, Oct 19, 2016 at 04:08:35PM +0200, Ralf Corsepius wrote:
Some will say that it really goes against the whole Unix spirit In particular, this would go against the OpenSource spirit and thus contradict Fedora's objectives.
I don't understand the connection you are making, there. Can you elaborate?
On Wed, Oct 19, 2016 at 03:20:34PM +0100, Richard Hughes wrote:
And for flatpak, using /app rather than /opt is actually an important decision, because we often need to export the real host filesystem to an app, and if the app is in /opt then exposing the host /opt to the application becomes very tricky.
Can you give an example of an app which would want to access the host /opt in that way?
I don't see packaging regular RPMs with /app as a prefix as a likely option in Fedora, although I guess we *could* discuss that. I *do*, see a high value in having our packages share a common build configuration. For many programs, PREFIX at build time leads to many runtime assumptions, and we can't just shuffle prebuilt binaries things to assemble a container with a different structure.
I'm not set in concrete on this idea, but do you have any alternate suggestions for how this could work?
I would like to avoid
1. Forking specfiles and doing everything twice 2. Adding complicated macros to specfiles for different prefixes (history has shown this to not work well) 3. Counting on a shuffle-the-built-bits tool of some sort (this seems fragile and I'm not sure we have anyone committed to doing it)
On 10/19/2016 04:31 PM, Matthew Miller wrote:
On Wed, Oct 19, 2016 at 04:08:35PM +0200, Ralf Corsepius wrote:
Some will say that it really goes against the whole Unix spirit In particular, this would go against the OpenSource spirit and thus contradict Fedora's objectives.
I don't understand the connection you are making, there. Can you elaborate?
To build everything from source yourself and not to package other parties binaries, is one of the foundations of an open source distro, such as Fedora.
What your are trying to achieve is to abandon this and to allow foreign binaries in Fedora.
IMHO, this is betrayal of Fedora's fundamental principles.
Ralf
On 19 October 2016 at 11:17, Ralf Corsepius rc040203@freenet.de wrote:
On 10/19/2016 04:31 PM, Matthew Miller wrote:
On Wed, Oct 19, 2016 at 04:08:35PM +0200, Ralf Corsepius wrote:
Some will say that it really goes against the whole Unix spirit In particular, this would go against the OpenSource spirit and thus contradict Fedora's objectives.
I don't understand the connection you are making, there. Can you elaborate?
To build everything from source yourself and not to package other parties binaries, is one of the foundations of an open source distro, such as Fedora.
What your are trying to achieve is to abandon this and to allow foreign binaries in Fedora.
What? I think there is a serious misunderstanding here.
IMHO, this is betrayal of Fedora's fundamental principles.
Ralf
packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org
On Wed, Oct 19, 2016 at 05:17:32PM +0200, Ralf Corsepius wrote:
Some will say that it really goes against the whole Unix spirit In particular, this would go against the OpenSource spirit and thus contradict Fedora's objectives.
I don't understand the connection you are making, there. Can you elaborate?
To build everything from source yourself and not to package other parties binaries, is one of the foundations of an open source distro, such as Fedora.
Okay.
What your are trying to achieve is to abandon this and to allow foreign binaries in Fedora.
There are at least two things to untangle from that statement.
First, "foreign binaries" are already allowed to run on Fedora. I certainly don't think we should break that.
But that's not my interest *here*. Here, I am talking about how we build and ship *Fedora-built* binaries. My goal is to expand that, not, somehow, the opposite.
On Wed, Oct 19, 2016 at 12:01:41PM -0400, Matthew Miller wrote:
To build everything from source yourself and not to package other parties binaries, is one of the foundations of an open source distro, such as Fedora.
Okay.
In thinking about this part a bit more: I don't think that in itself is a foundation, or should be. That's an implementation. The foundational principle is that users should be confident that the binaries they receive are built from a particular open codebase with a known, replicable recipe. We do that in Fedora with our rigorous central build system. In Debian, they're working on addressing the same problem by making reproducable builds (so it doesn't matter so much who happens to have made the build you have or where it was done). I don't think the Debian approach is fundamentally counter to open source or free software principles.
Again, though, this drifts from the conversation at hand, because I'm explicitly talking about Fedora-built packages. And it certainly seems completely irrelevant to PREFIX=/usr vs PREFIX=/app vs PREFIX=/opt/fedora/apps/<appname>.
On 10/19/2016 4:50 AM, Matthew Miller wrote:
As it stands right now, if we want to build existing RPMs as Flatpaks, we have to either do some very sketchy magic with relocation, or else hack on the specfiles.
Pretty much 100% against the proposal, but this caught my attention after having done some recent work with RPM relocations.
I suspect *whatever* we do is going to require hacking on spec files somewhat, if only to add various support lines. But given that RPM relocation (if it works right and the application is prepared for it) was designed specifically to support this type of thing, wouldn't that be a good place to start for this?
Ideally, you could end up with something like SCL support macros on top of the regular spec -- allow normal, default builds (and don't diverge too much from other RPM uses), but put the hooks there for a build system and 'rpm' to do something special.
-jc
On Wednesday, October 19, 2016 7:50:23 AM EDT Matthew Miller wrote:
In the medium-term future, we might want to provide some application software to users as Flatpak bundles in addition to traditional RPM form. This should be generally transparent to users but provide (at least) two advantages. First, these applications can be sandboxed, and only able to access resources they're specifically allowed to, increasing security. And second, users may have easier options for mixing and matching application versions that match their personal needs. And as a project, as Flatpak gains traction as a cross-platform solution, providing a whole bunch of Fedora packages in this way even to users of other distros helps increase our impact and usefulness to the world.
The Flatpak design splits the runtime from the application in the filesystem — apps are built expecting the runtime at /usr as normal, but are installed into a tree at /app. I'm not proposing that we move to having /app, but the idea of having a separate tree is pretty fundamental. As it stands right now, if we want to build existing RPMs as Flatpaks, we have to either do some very sketchy magic with relocation, or else hack on the specfiles.
So, the crazy idea: what if, for applications which would be useful as a Flatpak, we allow /opt/fedora/apps/[appname-version] as an allowed prefix? Then, the exact same RPMs could be used by someone who doesn't care about Flatpaks _and_ to construct the Flatpak, with no really-invasive changes. (Perhaps by making a link inside the Flatpak, or simply by asking for an extension to Flatpak to support the different application root.)
From the FHS 3.0:
/opt is reserved for the installation of add-on application software packages.... but it doesn't actually define what "add-on application software packages" are. Although we integrate it all together in Fedora, arguably a lot of our software could qualify.
From our current Packaging Guidelines:
/opt and its related directories (/etc/opt and /var/opt) is reserved for the use of vendors in the FHS. We have reserved the fedora name with LANANA for our use. If a package installs files into /opt it may only use directories in the /opt/fedora hierarchy. Fedora attempts to organize this directory by allocating a subdirectory of our /opt/fedora directory for specific subsystems. If you think you need to use /opt/fedora please file an FPC ticket to decide whether it's a valid use of /opt and what subdirectory should be allocated for your use..... so, it's not completely outside the realm of precedent. I know it's very different from our existing "blend everything into /usr" approach, but I think it might be worth doing for the increased flexibility. What do you all think?
One of the major issues I see with this approach is the use of outdated libraries and other outdated packages. I know that Flatpak is important to a large number of people, but that isn't enough to suggest creating a prefix for each individual application, unless applications packaged in this way were separate from other packages, at which point we'd essentially have a distro on a distro.
The solution to the above, as people who are for Flatpak would say, is to just keep the packages that make up your Flatpak as up to date as possible. If that were the case, we would have duplicate packages in each prefix increasing the size of a Fedora installation on all systems not using btrfs or a similar filesystem with deduplication or a similar technology.
The solution to packaging RPMs as Flatpaks should be obvious. Simply changing the prefix to a temporary directory and using the resulting file tree would be the way to go.
On 19 October 2016 at 12:50, Matthew Miller mattdm@fedoraproject.org wrote:
First, these applications can be sandboxed, and only able to access resources they're specifically allowed to, increasing security.
With sandboxing using SELinux you don't need to change anything in existing packages. Even better .. one of the reasons to develop SELinux was sandboxing. Instead fiddling around generic packages start working on new package holding SELinux definitions holding exact services in sandboxes.
kloczek
- Do you allow multiple versions? How does that change the above three answers? How does ordering work in that case? How do you decide which version is the "main" one, which gets /usr/bin/libreoffice, etc.?
I think allowing multiple versions would be a nice side-effect, but we *could* say that if you need that, use the Flatpak versions. Maybe the alternatives system could be used to moderate which version owns the shared bin link.
Don't forget that there is already a process for doing this sort of thing (multiple versions) using environment modules https://fedoraproject.org/wiki/PackagingDrafts/EnvironmentModules
($dayjob does this wth various compilers / libraries)
packaging@lists.fedoraproject.org
-
Andrew Elwell -
Japheth Cleaver -
Jason L Tibbitts III -
John M. Harris, Jr. -
Matthew Miller -
Ralf Corsepius -
Richard Hughes -
Stephen John Smoogen -
Tomasz Kłoczko