On Fri, Jul 31, 2015 at 10:27 AM, Gerald Henriksen <ghenriks(a)gmail.com> wrote:
So the discussion about Spark has revealed:
1) the Big Data community doesn't trust the Fedora packages
From my perspective, I see a lot of things happening in the "Big
Data
community" which indicates not so much distrust for Fedora packages
(or RHEL, for that matter), but more a complete lack of understanding
about the benefits of packaging for these projects.
2) it seems a consensus that the realities of the Big Data ecosystem
don't work very well with making packages for Fedora
I don't think that's necessarily true. I think a lot of the community
is following the "Hadoop Way", where developers are just trying to
mimic the way Hadoop does things, but Hadoop seems to be pretty poorly
packaged and has had a lot of problems over time, from complete lack
of dependency convergence, lack of packaging standards, improper use
of the Linux filesystem hierarchy, confusing use of shell scripts and
environment variables set all over the place, etc.
A lot of this is where downstream packagers can bring good things back
to the upstream communities. So, I don't think it's that the ecosystem
doesn't work well with Fedora packaging... I think it's more that they
haven't had sufficient feedback from those communities to push them to
make their software more easily integrated. (Part of this is a Java
thing more than a big data thing... let's face it... Java packaging
hasn't really had a lot of good support until quite recently, with
xmvn and related tools.)
3) result of 2, move towards newer (Docker) style of making Big Data
available on Fedora, perhaps with a side benefit of it then also being
possible to have it available on RHEL.
Docker still has some limitations, and is not necessarily suitable to
everybody's situation. I'd still prefer Fedora RPMs over Docker-based
packaging. I think the community could benefit from both.
Should we then also have a discussion, or just do it, of removing the
existing packages from Fedora 23?
I'd be against that. It's one thing if there's no maintainer, but it's
another to decide to oust a whole class of projects.
The packages that are there are now in some cases at least several
versions out of date, which isn't doing either Fedora or the Big Data
SIG any favours.
Big Data packages aren't the only packages to suffer from this.
There's many projects that are out of date. I'd rather Fedora have a
relatively stable version than no version at all. Maybe instead we
should try working with the upstream communities more to ease
integration, so it's easier to update?
Additionally, if we do create Docker images they offer the possibility
of creating bug/error opportunities if a user somehow brings an rpm
package into the Docker image.
To use the now favourite term, do we "reboot" the Big Data SIG and
start fresh?
Thoughts?
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii