This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Some specifics:
- it could feed from COPRs -- add your packages by checking a box - packages would only go in if they meet minimal automatic gating -- can't conflict with other packages - possibly some sort of automatic flagging of crazy %pre or %post package scripts? - if they don't meet the gating, package owner would get an email explaining the problem - Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate? - degree to which packages would be allowed here forever vs. encouraged to improve so they can eventually be in the main repo is an open question - repo would be off by default, but easily enabled in yum or in Gnome Software - packages would be signed, possibly by a different key from the main Fedora one. - signing could be automatic rather than manual - ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
Matthew Miller (mattdm@fedoraproject.org) said:
This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Some specifics:
- it could feed from COPRs -- add your packages by checking a box
- packages would only go in if they meet minimal automatic gating -- can't conflict with other packages
- possibly some sort of automatic flagging of crazy %pre or %post package scripts?
- if they don't meet the gating, package owner would get an email explaining the problem
- Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate?
- degree to which packages would be allowed here forever vs. encouraged to improve so they can eventually be in the main repo is an open question
- repo would be off by default, but easily enabled in yum or in Gnome Software
- packages would be signed, possibly by a different key from the main Fedora one.
- signing could be automatic rather than manual
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
I'm not necessarily against this, but how does this differ from the 'Fedora Commons' part of the original proposal? What would we be accomplishing here above and beyond a set of curated COPRs?
Bill
On Sun, Feb 09, 2014 at 09:03:14AM +0100, Bill Nottingham wrote:
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
I'm not necessarily against this, but how does this differ from the 'Fedora Commons' part of the original proposal? What would we be accomplishing here above and beyond a set of curated COPRs?
The original idea was that the Commons would be a bubble of packages under the current strict packaging guidelines; this is a suggestion for a different bubble for packages that don't yet conform.
The difference from curated COPRs is:
- dodges the "signing in coprs is hard" problem - packages would be expected to work together -- it's an integration point - could still have some rules (I didn't mention this, but "don't update base packages here" would be one) - potentially easier discoverability for each package - depending on the focus between "ugly" vs. "incubation" or "staging" as a name, the implication would be that these are packages which are _striving_ to be part of the commons ("Fedora Collection"?) but aren't ready yet, whereas some other things might live in COPRs forever and have no intention of ever playing nice.
Matthew Miller (mattdm@fedoraproject.org) said:
The original idea was that the Commons would be a bubble of packages under the current strict packaging guidelines; this is a suggestion for a different bubble for packages that don't yet conform.
The difference from curated COPRs is:
- dodges the "signing in coprs is hard" problem
- packages would be expected to work together -- it's an integration point
- could still have some rules (I didn't mention this, but "don't update base packages here" would be one)
- potentially easier discoverability for each package
- depending on the focus between "ugly" vs. "incubation" or "staging" as a name, the implication would be that these are packages which are _striving_ to be part of the commons ("Fedora Collection"?) but aren't ready yet, whereas some other things might live in COPRs forever and have no intention of ever playing nice.
So, from a user perspective, what does this buy us? If (for example) the only reason it's not in the main repo is that it has bundled libs, what are we accomplishing by making the user have an extra checkbox?
IOW, why not just restrict the super strict guidelines to the core product repos of Fedora, and have this repo be for things outside of that, which can include things that follow the guidelines, and things like chef that don't.
Bill
On Sun, Feb 09, 2014 at 06:56:41PM +0100, Bill Nottingham wrote:
So, from a user perspective, what does this buy us? If (for example) the only reason it's not in the main repo is that it has bundled libs, what are we accomplishing by making the user have an extra checkbox?
If it's *just* bundled libs (or depending on your perspective, "just" bundled libs with scare quotes), not very much. In that case, I think it would just be a decision based on whether a new repo structure or allowing bundled libs mixed in would be less work overall.
But I'm thinking that there are probably a whole host of other things. So, it makes it at least a little more clear from a user perspective that this might be not as pretty. I'm not necessarily opposed to letting the idea be tried for a couple of releases as optional and if it's not destroying the world, enabling it by default. Possibly software installers could present visual cues based on repo policy level.
IOW, why not just restrict the super strict guidelines to the core product repos of Fedora, and have this repo be for things outside of that, which can include things that follow the guidelines, and things like chef that don't.
I think there are different levels here. If we keep the core reasonably small, we can more reasonably say things like "must be ported to systemd timer units" for that subset.
Also, I hear from a lot of Fedora developers who strongly believe that even if we allow messier software, improving the packaging quality is still important. The idea of calling this "staging" or "incubating" implies that packages at this level can be "in" easily but are expected to be polished over time, with the aim of graduating to the fully-FPG-compliant level.
(At the DevConf panel discussion, someone (Stephen, I think) made the suggestion that packages that aren't aiming for this could comfortably live in COPRs forever. Or, we *could* have both fedora-incubator and fedora-ugly, but I don't know how much value that gives.)
On Mon, Feb 10, 2014 at 03:49:51AM -0500, Matthew Miller wrote:
On Sun, Feb 09, 2014 at 06:56:41PM +0100, Bill Nottingham wrote:
IOW, why not just restrict the super strict guidelines to the core product repos of Fedora, and have this repo be for things outside of that, which can include things that follow the guidelines, and things like chef that don't.
I think there are different levels here. If we keep the core reasonably small, we can more reasonably say things like "must be ported to systemd timer units" for that subset.
Also, I hear from a lot of Fedora developers who strongly believe that even if we allow messier software, improving the packaging quality is still important. The idea of calling this "staging" or "incubating" implies that packages at this level can be "in" easily but are expected to be polished over time, with the aim of graduating to the fully-FPG-compliant level.
(At the DevConf panel discussion, someone (Stephen, I think) made the suggestion that packages that aren't aiming for this could comfortably live in COPRs forever. Or, we *could* have both fedora-incubator and fedora-ugly, but I don't know how much value that gives.)
I think we need to support the use-cases of both Incubator (packages here are destined for Commons once the packages are polished up) and Ugly (these packages are usful for Fedora End Users to have access to but the upstream and downstream packaging is such that we'd never want to push them to people features as is). We might also want to satisfy a third use case that's "This package is too ugly for Commons and no one wants to make it prettier but Product X desperately wants to ship it" but I think that could be its own topic.
I would like to avoid having two repositories for these two use cases if one repository would fit the bill. Additional repositories are not without cost for releng and infrastructure and packagers so I think it would be good to think about whether we can meet all needs within one repository before thinking about making two.
I think the top things that people want to shortcut in the packaging guidelines are about how your package fits in with the other packages in the distribution.
* Bundled libraries are essentially a disagreement between an application package and a library package on what the upstream library's code should do (One of hte things the FPC asks about bundling libraries is whether the Fedora package could/would carry a patch to the library to support the needed additions). * Conflicts of all sorts: filesystem, package naming, multiple versions that are not parallel installable * alternate systems that do similar things: systemd, SysVinit, minit, etc; rsyslog, syslog-ng, systemd journal * Different compilations of the same thing: lua, libev, and liblzma all create -source packages so that dependencies can compile the libraries in particular ways without actually bundling. * Combinations of the above two: compiling with clang vs gcc, compiling against dietlibc vs glibc,
There's also fesco policy that's largely about the relation between packages that people want to circumvent: * New, incompatible library versions for some software while other software needs the old versions. * Legal addons to the kernel * Things that indirectly point to non-free software (which we'd probably allow into the main repositories if the Board voted to allow it)
I think that many of these things get left out in the cold if we're specifying that the new repo doesn't support conflicts between the new repo and the Fedora Commons repo, the Core (once we figure out how to distribute that), or each other. On the other hand, the inter-package conflicts are one of the major drawbacks that you see with the idea of people running copr repos so...
Perhaps we need a repository where we don't have rpm Conflicts and we don't have filesystem Conflicts but we have some specified method of handling parallel versions. SCLs could fit here (and we do need a repository for non-fedora SCLs to be available for fedora users... so that could kill two birds with one stone) or other, similar technologies that are essentially bundling of requirements at the distribution level instead of bundling at the upstream level.
We would definitely want people to know that we weren't promising security, timely updates, etc... (indeed, we might want to tell people to expect insecure code, packages that would languish, etc...) as there would be a much higher duplication of code in this repo and likely more packages which included versions that were no longer accepted upstream.
-Toshio
On Tue, Feb 11, 2014 at 02:23:37PM -0800, Toshio Kuratomi wrote:
I think we need to support the use-cases of both Incubator (packages here are destined for Commons once the packages are polished up) and Ugly (these
^^^^^^^
Someone really needs to come up with a word we can use for this. "Collection"? "Curated"?
On Tue, Feb 11, 2014 at 02:23:37PM -0800, Toshio Kuratomi wrote:
I think we need to support the use-cases of both Incubator (packages here are destined for Commons once the packages are polished up) and Ugly (these
^^^^^^^
Someone really needs to come up with a word we can use for this. "Collection"? "Curated"?
Collection sounds alright to me. If software collections will live in the Commons (in Ring 2) maybe it makes more sense too.
Jens
On Tue, 2014-02-11 at 14:23 -0800, Toshio Kuratomi wrote:
I think we need to support the use-cases of both Incubator (packages here are destined for Commons once the packages are polished up) and Ugly (these packages are usful for Fedora End Users to have access to but the upstream and downstream packaging is such that we'd never want to push them to people features as is).
+1
I would like to avoid having two repositories for these two use cases if one repository would fit the bill.
+1. I also think the name of the repository should reflect that. I would suggest something catchy, like Fedora Easy or Fedora Playground (Honza Horak's suggestion), or something that would go with the Rings theme, like Fedora Mantle (implying the main Fedora repo is Core).
I think that many of these things get left out in the cold if we're specifying that the new repo doesn't support conflicts between the new repo and the Fedora Commons repo, the Core (once we figure out how to distribute that), or each other.
I think conflicts are an important thing to figure out before we continue laying plans for this new repo.
I wouldn't throw all conflicts in the same bin. For example, I think it's ok to bundle libraries (though it would be cool if we could somehow detect it automatically and know which libraries are bundled), but I would not be in favour of allowing package naming conflicts.
Regarding multiple parallelly installable versions of the same packages, are SCLs going to give us that? Also, SCLs could be useful when packages have file system conflicts. On the other hand, would this be abusing SCLs?
We would definitely want people to know that we weren't promising security, timely updates, etc... (indeed, we might want to tell people to expect insecure code, packages that would languish, etc...) as there would be a much higher duplication of code in this repo and likely more packages which included versions that were no longer accepted upstream.
+1.
Tadej
On 02/13/2014 11:05 AM, Tadej Janež wrote:
On Tue, 2014-02-11 at 14:23 -0800, Toshio Kuratomi wrote:
I think we need to support the use-cases of both Incubator (packages here are destined for Commons once the packages are polished up) and Ugly (these packages are usful for Fedora End Users to have access to but the upstream and downstream packaging is such that we'd never want to push them to people features as is).
+1
I would like to avoid having two repositories for these two use cases if one repository would fit the bill.
+1. I also think the name of the repository should reflect that. I would suggest something catchy, like Fedora Easy or Fedora Playground (Honza Horak's suggestion), or something that would go with the Rings theme, like Fedora Mantle (implying the main Fedora repo is Core).
I think that many of these things get left out in the cold if we're specifying that the new repo doesn't support conflicts between the new repo and the Fedora Commons repo, the Core (once we figure out how to distribute that), or each other.
I think conflicts are an important thing to figure out before we continue laying plans for this new repo.
I wouldn't throw all conflicts in the same bin. For example, I think it's ok to bundle libraries (though it would be cool if we could somehow detect it automatically and know which libraries are bundled), but I would not be in favour of allowing package naming conflicts.
Regarding multiple parallelly installable versions of the same packages, are SCLs going to give us that?
Yes.
Also, SCLs could be useful when packages have file system conflicts. On the other hand, would this be abusing SCLs?
It depends how hard would be to package it. I wouldn't push for SCL in every case.
We would definitely want people to know that we weren't promising security, timely updates, etc... (indeed, we might want to tell people to expect insecure code, packages that would languish, etc...) as there would be a much higher duplication of code in this repo and likely more packages which included versions that were no longer accepted upstream.
+1.
Tadej
env-and-stacks mailing list env-and-stacks@lists.fedoraproject.org https://lists.fedoraproject.org/mailman/listinfo/env-and-stacks
Yesterday I sent email to devel list about content and use-cases of the repo. No replies yet. Server WG will probably package Roles into SCL, but have them in main repo. Some incubating Roles might go in Playground repository.
Marcela
Matthew Miller (mattdm@fedoraproject.org) said:
- depending on the focus between "ugly" vs. "incubation" or "staging" as a name, the implication would be that these are packages which are _striving_ to be part of the commons ("Fedora Collection"?) but aren't ready yet, whereas some other things might live in COPRs forever and have no intention of ever playing nice.
So, from a user perspective, what does this buy us? If (for example) the only reason it's not in the main repo is that it has bundled libs, what are we accomplishing by making the user have an extra checkbox?
Well there are many packages (either with submitted reviews or not) that take a very long time to get into Fedora because of the review process. Good examples might be javascript (*cough* jquery cough;), but personally there are also loads of Haskell libraries I would like to put into Fedora but am not able to also due to the limited review manpower (Fedora Haskell lags behind Debian and Ubuntu because of this).
IOW, why not just restrict the super strict guidelines to the core product repos of Fedora, and have this repo be for things outside of that, which can include things that follow the guidelines, and things like chef that don't.
I am not following the implication/difference, you mean to build out from Core (koji and current infrastructure) rather than inwards from the outside (copr etc)? That could also be a reasonable approach - particularly for Commons. There are certainly good arguments for re-using current infrastructure where possible while keeping the new repo accessible.
Certainly I have questions about the new repo: will packages be in git? tracked by bugzilla? Can packages be pulled from the repo: either by the maintainer or because of breakage say? How to distinguish packages?
Is "Ugly" also a part of Ring 2 btw?
BTW Honza Horak suggested the name Playground in the E&S Meeting, which I rather like.
Jens
Matthew Miller mattdm@fedoraproject.org writes:
This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Some specifics:
- it could feed from COPRs -- add your packages by checking a box
- packages would only go in if they meet minimal automatic gating -- can't conflict with other packages
How about updates for packages in given Fedora release? I'd say no because that easily breaks integration/other parts. That's what copr is for.
- possibly some sort of automatic flagging of crazy %pre or %post package scripts?
- if they don't meet the gating, package owner would get an email explaining the problem
- Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate?
Basic licensing should apply, but most likely I'd ignore bundling stuff because that's often going to be main reason for being in this repo.
- degree to which packages would be allowed here forever vs. encouraged to improve so they can eventually be in the main repo is an open question
I'd say people would be expected to work on integration but there's no guarantee.
I mentioned this during the talk, but basically I think adopting "Incubator" name and generic process from Apache project and Eclipse[2]. It's well know terminology in OSS world IMO.
- repo would be off by default, but easily enabled in yum or in Gnome Software
Sounds good
- packages would be signed, possibly by a different key from the main Fedora one.
Sensible as well I'd say
- signing could be automatic rather than manual
Not sure about this one...probably not.
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
They'll have to match the releases. It should basically follow the same workflow/lifecycle as normal packages
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
As said up top I'd prefer Fedora Incubator naming but otherwise +1!
[1] http://incubator.apache.org/ [2] http://wiki.eclipse.org/Development_Resources/Process_Guidelines/What_is_Inc...
-- Stanislav Ochotnicky sochotnicky@redhat.com Software Engineer - Developer Experience
PGP: 7B087241 Red Hat Inc. http://cz.redhat.com
On Sun, Feb 09, 2014 at 01:18:25PM +0100, Stanislav Ochotnicky wrote:
- it could feed from COPRs -- add your packages by checking a box
- packages would only go in if they meet minimal automatic gating -- can't conflict with other packages
How about updates for packages in given Fedora release? I'd say no because that easily breaks integration/other parts. That's what copr is for.
Yes, that makes sense to me too. In fact, I thought that that was in my original mail, but I see that it is not. I am a very tired person right now. :)
- Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate?
Basic licensing should apply, but most likely I'd ignore bundling stuff because that's often going to be main reason for being in this repo.
Yep. Exact rules to be worked out. (Possibly known bundling needs to be registered somewhere, but again wanting to strike a balance where this doesn't require a lot of paperwork.)
I'd say people would be expected to work on integration but there's no guarantee. I mentioned this during the talk, but basically I think adopting "Incubator" name and generic process from Apache project and Eclipse[2]. It's well know terminology in OSS world IMO.
And in line with what you said about spins that aspire to be products, too. :)
[some stuff snipped]
- signing could be automatic rather than manual
Not sure about this one...probably not.
We don't have the infrastructure to do this right now -- it certainly couldn't be signed with the main Fedora key in the way packages are signed now. But other than protection of the private key, the current system does not really give us strong advantages over automatic signing. (As I understand it, there's no careful human verification of package provenance anyway.) So maybe something could be developed.
On 02/09/2014 01:18 PM, Stanislav Ochotnicky wrote:
Matthew Miller mattdm@fedoraproject.org writes:
This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Some specifics:
- it could feed from COPRs -- add your packages by checking a box
- packages would only go in if they meet minimal automatic gating -- can't conflict with other packages
How about updates for packages in given Fedora release? I'd say no because that easily breaks integration/other parts. That's what copr is for.
- possibly some sort of automatic flagging of crazy %pre or %post package scripts?
- if they don't meet the gating, package owner would get an email explaining the problem
- Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate?
Basic licensing should apply, but most likely I'd ignore bundling stuff because that's often going to be main reason for being in this repo.
- degree to which packages would be allowed here forever vs. encouraged to improve so they can eventually be in the main repo is an open question
I'd say people would be expected to work on integration but there's no guarantee.
I mentioned this during the talk, but basically I think adopting "Incubator" name and generic process from Apache project and Eclipse[2]. It's well know terminology in OSS world IMO.
- repo would be off by default, but easily enabled in yum or in Gnome Software
Sounds good
- packages would be signed, possibly by a different key from the main Fedora one.
Sensible as well I'd say
- signing could be automatic rather than manual
Not sure about this one...probably not.
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
They'll have to match the releases. It should basically follow the same workflow/lifecycle as normal packages
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
As said up top I'd prefer Fedora Incubator naming but otherwise +1!
[1] http://incubator.apache.org/ [2] http://wiki.eclipse.org/Development_Resources/Process_Guidelines/What_is_Inc...
-- Stanislav Ochotnicky sochotnicky@redhat.com Software Engineer - Developer Experience
PGP: 7B087241 Red Hat Inc. http://cz.redhat.com
env-and-stacks mailing list env-and-stacks@lists.fedoraproject.org https://lists.fedoraproject.org/mailman/listinfo/env-and-stacks
I prefer name incubator. So we have topic for tomorrow meeting, yay!
Marcela
Date: Sun, 9 Feb 2014 01:25:01 -0500 From: mattdm@fedoraproject.org To: env-and-stacks@lists.fedoraproject.org Subject: half baked idea for further baking: "fedora-ugly" repo
This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Some specifics:
- it could feed from COPRs -- add your packages by checking a box
- packages would only go in if they meet minimal automatic gating --
can't conflict with other packages
- possibly some sort of automatic flagging of crazy %pre or %post package
scripts?
- if they don't meet the gating, package owner would get an email
explaining the problem
- Fedora packaging guidelines don't apply fully, but maybe some subset
is appropriate?
- degree to which packages would be allowed here forever vs. encouraged
to improve so they can eventually be in the main repo is an open question
- repo would be off by default, but easily enabled in yum or in Gnome
Software
- packages would be signed, possibly by a different key from the main
Fedora one.
- signing could be automatic rather than manual
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to
be figured out!
What do people think of this idea overall? Anyone interested in polishing this up into a Real Thing?
-- Matthew Miller -- Fedora Project -- mattdm@fedoraproject.org _______________________________________________ env-and-stacks mailing list env-and-stacks@lists.fedoraproject.org https://lists.fedoraproject.org/mailman/listinfo/env-and-stacks
Putting on my QA hat here, I think that we need to ensure that packages run through full depcheck rather than just get checked for conflicts. In fact, they should run through the full automation process once we that up and running.
Also, I do not believe that we should ever have a package in our distro that does not meet licensing. If Fedora even hints at shipping non-open source software, then you're going to drive a bunch of people away.
John.
On Sun, Feb 09, 2014 at 11:13:03AM -0500, John Dulaney wrote:
Putting on my QA hat here, I think that we need to ensure that packages run through full depcheck rather than just get checked for conflicts.
Sounds reasonable.
In fact, they should run through the full automation process once we that up and running.
Also sounds good, although depending on what those tests are some failures could be warnings rather than blockers.
Also, I do not believe that we should ever have a package in our distro that does not meet licensing. If Fedora even hints at shipping non-open source software, then you're going to drive a bunch of people away.
Agreed -- this would be quite a few steps beyond a third-party repo. License compliance is required for COPRs, and this is a step more towards the center of Fedora than that.
Hi!
On Sun, 2014-02-09 at 01:25 -0500, Matthew Miller wrote:
COPRs are cool, but very much the uncharted wilderness. The Fedora distribution proper is a stockade fort with high walls -- it's very nice iside (with some rough parts of town... I can take this analogy all the way...) but the barrier to get in is giantic and topped with spikes.
And it's not just that there's a lack of guidelines. Different COPRs repos don't need to work together, and it's imposible to tell which will or to have any epectation that two which work together today will work together tomorrow. And, packages aren't signed at all.
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
Generally, a big +1 from me for the idea.
I really don't like the name "Fedora ugly". While it may make sense to us, it would quite possible send the wrong message to those who we target with this repo. I'm also not so in favour of naming it "Fedora Staging" or "Fedora Incubator", since it would send the message that packages would eventually come out of this repo. Well, they might be just fine sitting in the repo forever.
Of the top of my head, I would say we need to send the right message to two audiences: 1) Upstream application authors/Wanna be Fedora packagers: we need to tell them that we want them to "play" with us, create packages for Fedora and that it is not so difficult and strict as it is now (i.e. they don't need to follow the current Packaging guidelines). 2) Fedora users: we want to tell them: "Hey, we have this cool new repo, which provides so much more packages than just Fedora's main repo, but you should know that: - these packages might have security issues which won't be fixed as quickly as they are right now in Fedora's main repo, - these packages might not have gone through the same QA scrutiny as packages in Fedora's main repo, - these packages might disappear or become unmaintained quickly, if their author is not interested in them anymore - etc." Basically, I advocate to set the right expectations for users and tell them it's a *best effort* thing.
So, in-line with that, I'm suggesting that we name it: - "Fedora Easy(TM)", since it will be easier for upstream application authors to get their apps in Fedora (the same for Wanna be Fedora packagers) and we need to tell the users to "ease up" their expectations for packages in this repo, - or "Fedora Mantle", since it would mean upstream application authors can stay in the "mantle" zone and not "dig" to the "core" if they don't want to, and the users will know that the packages would not be the "core" focus of Fedora (like Fedora's main repo).
- it could feed from COPRs -- add your packages by checking a box
+1. I think it would be a very natural workflow to first make your own (scratch) packages, improve them, build them and then, if you see interest/demand, propose them to inclusion in the "ugly" repo.
- packages would only go in if they meet minimal automatic gating -- can't conflict with other packages
I think we should define a small core policy (e.g. non-conflicting with packages in Fedora's main repo, no over-riding of packages in Fedora's main repo, licenses compatible with Fedora) and have an automated way to check and enforce it. Having a manual review process would unnecessarily slow the process of populating this repository.
- possibly some sort of automatic flagging of crazy %pre or %post package scripts?
Yes, definitely, since these are going to be executed as root, right? We could have automated testing that would warn if the package's %pre or %post scripts tamper with other parts of the system. Another idea would be to only allow "macroized" %pre and %post scripts and the macros would be provided and controlled by main Fedora?
- if they don't meet the gating, package owner would get an email explaining the problem
+1 (or some other notification).
- Fedora packaging guidelines don't apply fully, but maybe some subset is appropriate?
Yes, it would make sense to not reinvent the wheel. But to reach a consensus on the subset might be challenging. I'm in favour of the subset being as small as possible.
- repo would be off by default, but easily enabled in yum or in Gnome Software
+1. We could set users' expectations in the spirit of what I wrote in the beginning by presenting this in Gnome Software (or as a comment in yum/dnf's repo file).
- packages would be signed, possibly by a different key from the main Fedora one.
+1
- signing could be automatic rather than manual
+1
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
I agree that, in principle, the repo should follow the current lifecycle of Fedora's main repo. We don't know yet how the products might change that in the future, so this will need to be figured out.
Tadej
On Tue, Feb 11, 2014 at 03:04:12PM +0100, Tadej Janež wrote:
Generally, a big +1 from me for the idea.
[still tired from travel. longer reply later. deleting all of message except the one point I want to respond to now...]
I think we should define a small core policy (e.g. non-conflicting with packages in Fedora's main repo, no over-riding of packages in Fedora's main repo, licenses compatible with Fedora) and have an automated way to check and enforce it. Having a manual review process would unnecessarily slow the process of populating this repository.
I'm afraid that this becomes an easy route for malware into the distribution if we don't have a human check. Obviously that can't be perfect either but it raises the bar significantly.
On 02/11/2014 07:35 PM, Matthew Miller wrote:
On Tue, Feb 11, 2014 at 03:04:12PM +0100, Tadej Janež wrote:
Generally, a big +1 from me for the idea.
[still tired from travel. longer reply later. deleting all of message except the one point I want to respond to now...]
I think we should define a small core policy (e.g. non-conflicting with packages in Fedora's main repo, no over-riding of packages in Fedora's main repo, licenses compatible with Fedora) and have an automated way to check and enforce it. Having a manual review process would unnecessarily slow the process of populating this repository.
I'm afraid that this becomes an easy route for malware into the distribution if we don't have a human check. Obviously that can't be perfect either but it raises the bar significantly.
I'm not sure that significantly is correct. It's not such a big problem now to package some sane code, go through formal review and then change the code to malware. As far as there is some quick "stop" button, I'd be ok with automatic reviews.
Honza
On Tue, Feb 11, 2014 at 07:53:14PM +0100, Honza Horak wrote:
I'm not sure that significantly is correct. It's not such a big problem now to package some sane code, go through formal review and then change the code to malware. As far as there is some quick "stop" button, I'd be ok with automatic reviews.
Sure. The current model is based on the idea of sponsorship, and the idea that the high bar of getting a package in creates a system of contributor trust. We should probably look at how true that is, too.
On 02/11/2014 07:58 PM, Matthew Miller wrote:
On Tue, Feb 11, 2014 at 07:53:14PM +0100, Honza Horak wrote:
I'm not sure that significantly is correct. It's not such a big problem now to package some sane code, go through formal review and then change the code to malware. As far as there is some quick "stop" button, I'd be ok with automatic reviews.
Sure. The current model is based on the idea of sponsorship, and the idea that the high bar of getting a package in creates a system of contributor trust. We should probably look at how true that is, too.
If the packager has devel cla, I would be fine with adding such collections into the repository. He should have at least some knowledge about proper packaging. I wouldn't start normal review process for such packages, because we would have even more packages with "high bar" for inclusion into Fedora repositories.
Marcela
On 02/09/2014 07:25 AM, Matthew Miller wrote:
This idea has come up several times at DevConf, and I thought I'd throw it out here so it can maybe get further development and discussion.
Just subscribed, so coming little bit later to discussion :)
So, the proposal: a new repository in the Fedora Project which I am tentatively calling "Fedora Ugly". It could be "Fedora Staging", but I think that promises a bit much (some things may remain here for a really, really long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
What would be content? All Copr packages. I could not imagine who would wanted all of those weird packages.
What I would make sense to me is mimic "experimental"[1] from Debian. This is branch with packages, which may soon land in "unstable" (which is more or less equivalent of rawhide without broken deps).
https://wiki.debian.org/DebianExperimental
It would be opt-in and by checking some box, you would make promise that this is something which you plan to deliver in Fedora soon. And I would even restrict it to packages which are already in Fedora or Package review has been already submitted.
Or you can create Outer Fedora. Which would match 3rd ring from Fedora.next, where would be layered application which are too big to get in Fedora in reasonable time. And again opt-in. And by subscribing in, you would promise to build into that repo only stable packages (and build nightly into different project in Copr).
- degree to which packages would be allowed here forever vs. encouraged to improve so they can eventually be in the main repo is an open question
I would be ok with such soft policy as is in https://www.debian.org/doc/manuals/developers-reference/resources#s4.6.4 (see end of section 4.6.4.3)
- repo would be off by default, but easily enabled in yum or in Gnome Software
Note that this would be possible soon with each individual Copr repository anyway by: dnf copr enable msuchy/myproject https://git.fedorahosted.org/cgit/copr.git/tree/dnf-plugin/copr_hook.py (currently under development)
- packages would be signed, possibly by a different key from the main Fedora one.
- signing could be automatic rather than manual
This is not trivial! It would mean either rebuilds in Koji. Or setup another instance of Sigul, and if you do that you can set it up for Corp too. But both are quite tricky.
- ugly-testing or ugly-updates repo, and rolling release vs versioning? to be figured out!
+1 to rolling - against last branched version in case of "Outer Fedora" - against rawhide in case of "experimental"
On Thu, Feb 13, 2014 at 05:25:57PM +0100, Miroslav Suchý wrote:
long time). This repo would provide an integration space where packages from diverse COPRs repos could come together, and also be more discoverable by other Fedora developers and users (just add one repo).
What would be content? All Copr packages. I could not imagine who would wanted all of those weird packages.
I'm not sure if this is a question. No, it wouldn't be all Copr packages, because as you say, that's probably not what people want.
I would be ok with such soft policy as is in https://www.debian.org/doc/manuals/developers-reference/resources#s4.6.4 (see end of section 4.6.4.3)
The use case is different from that -- I think "software which is likely to damage your system" should actually _stay out_ of this.
- repo would be off by default, but easily enabled in yum or in Gnome Software
Note that this would be possible soon with each individual Copr repository anyway by: dnf copr enable msuchy/myproject https://git.fedorahosted.org/cgit/copr.git/tree/dnf-plugin/copr_hook.py (currently under development)
Also not the same, because here you have to know the Copr to begin with.
- packages would be signed, possibly by a different key from the main Fedora one.
- signing could be automatic rather than manual
This is not trivial! It would mean either rebuilds in Koji. Or setup another instance of Sigul, and if you do that you can set it up for Corp too. But both are quite tricky.
It's not trivial, but it doesn't require either of those things. I think a signing server with less security than Sigul would be adequate, or else a Sigul instance with more relaxed policies.
On 02/13/2014 08:09 PM, Matthew Miller wrote:
It's not trivial, but it doesn't require either of those things. I think a signing server with less security than Sigul would be adequate, or else a Sigul instance with more relaxed policies.
-1 Rather no signing at all than less secure signing.
Mirek
On Thu, Feb 13, 2014 at 08:23:51PM +0100, Miroslav Suchy wrote:
It's not trivial, but it doesn't require either of those things. I think a signing server with less security than Sigul would be adequate, or else a Sigul instance with more relaxed policies.
-1 Rather no signing at all than less secure signing.
Let me elaborate, then. The current signing setup requires human intervention, but we are using it in a way that provides no actual benefit from a human being in the chain, and in fact introduces the possibility of human error. So by less secure, I don't necessarily mean "less actual security", but "less potential security, which is wasted anyway".
env-and-stacks@lists.fedoraproject.org