Re: Bringing Hadoop to Fedora
by Tim St. Clair
Hi Arun -
It's been a week so I figured I would follow up on the status regarding the patch-set listed below.
Cheers,
Tim
----- Original Message -----
> From: "Arun C Murthy" <acm(a)hortonworks.com>
> To: yarn-dev(a)hadoop.apache.org
> Cc: common-dev(a)hadoop.apache.org, mapreduce-dev(a)hadoop.apache.org, hdfs-dev(a)hadoop.apache.org, dev(a)bigtop.apache.org
> Sent: Tuesday, June 11, 2013 12:54:11 PM
> Subject: Re: Bringing Hadoop to Fedora
>
> Tim,
>
> This is great! I'll take a look at some of these patches, welcome!
>
> Arun
>
> On Jun 11, 2013, at 8:54 AM, Tim St Clair wrote:
>
> > Greetings Hadoop Development Community,
> >
> > As some of you are aware, we've been looking to bring the upstream 2.X
> > series into the Fedora channels. We believe that there are several
> > benefits that this can bring to the community:
> >
> > - Updated & streamlined rpm packaging
> > - Deeper level of system integration
> > - System managed dependencies (security + defect tracking)
> > - Proving ground for OpenJDK7
> > - Greater exposure for the community (both adoption and defect tracking)
> > ...
> >
> > During our evaluation (https://fedoraproject.org/wiki/Features/Hadoop),
> > we've come across some build issues as they relate to Fedora packaging
> > guidelines (http://fedoraproject.org/wiki/Packaging:Java). Most of these
> > issues (listed below) are very minor dependency changes, and we were
> > wondering if there are folks who would be willing to review.
> >
> > fedora-patch-math: https://issues.apache.org/jira/browse/HADOOP-9594
> > fedora-patch-collections: https://issues.apache.org/jira/browse/HADOOP-9610
> > fedora-patch-cglib: https://issues.apache.org/jira/browse/HADOOP-9611
> > fedora-patch-jersey: https://issues.apache.org/jira/browse/HADOOP-9613
> > fedora-patch-jets3t: https://issues.apache.org/jira/browse/HADOOP-9623
> > (uber deprecated)
> >
> > There are other patches we have in the queue but they still need to be
> > flushed out, and in an effort to follow the KISS principles we figure this
> > would be a good start.
> >
> > Cheers,
> > Tim
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
10 years, 9 months
Fwd: Bringing Hadoop to Fedora
by Tim St. Clair
Folks,
As a point of note, all tracking details have been updated on the wiki:
https://fedoraproject.org/wiki/Features/Hadoop#Upstream_Patch_Tracking
I plan on following up with upstream this week re: the message below.
Cheers,
Tim
----- Forwarded Message -----
> From: "Arun C Murthy" <acm(a)hortonworks.com>
> To: yarn-dev(a)hadoop.apache.org
> Cc: common-dev(a)hadoop.apache.org, mapreduce-dev(a)hadoop.apache.org, hdfs-dev(a)hadoop.apache.org, dev(a)bigtop.apache.org
> Sent: Tuesday, June 11, 2013 12:54:11 PM
> Subject: Re: Bringing Hadoop to Fedora
>
> Tim,
>
> This is great! I'll take a look at some of these patches, welcome!
>
> Arun
>
> On Jun 11, 2013, at 8:54 AM, Tim St Clair wrote:
>
> > Greetings Hadoop Development Community,
> >
> > As some of you are aware, we've been looking to bring the upstream 2.X
> > series into the Fedora channels. We believe that there are several
> > benefits that this can bring to the community:
> >
> > - Updated & streamlined rpm packaging
> > - Deeper level of system integration
> > - System managed dependencies (security + defect tracking)
> > - Proving ground for OpenJDK7
> > - Greater exposure for the community (both adoption and defect tracking)
> > ...
> >
> > During our evaluation (https://fedoraproject.org/wiki/Features/Hadoop),
> > we've come across some build issues as they relate to Fedora packaging
> > guidelines (http://fedoraproject.org/wiki/Packaging:Java). Most of these
> > issues (listed below) are very minor dependency changes, and we were
> > wondering if there are folks who would be willing to review.
> >
> > fedora-patch-math: https://issues.apache.org/jira/browse/HADOOP-9594
> > fedora-patch-collections: https://issues.apache.org/jira/browse/HADOOP-9610
> > fedora-patch-cglib: https://issues.apache.org/jira/browse/HADOOP-9611
> > fedora-patch-jersey: https://issues.apache.org/jira/browse/HADOOP-9613
> > fedora-patch-jets3t: https://issues.apache.org/jira/browse/HADOOP-9623
> > (uber deprecated)
> >
> > There are other patches we have in the queue but they still need to be
> > flushed out, and in an effort to follow the KISS principles we figure this
> > would be a good start.
> >
> > Cheers,
> > Tim
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
10 years, 9 months
Re: Package Review Requests for Hadoop dependencies
by gil
Il 12/06/2013 19:04, Björn Esser ha scritto:
> Am Mittwoch, den 12.06.2013, 12:54 -0400 schrieb Tim St Clair:
>> Many thanks Björn!
>>
>> It looks like we are green on dependencies!
>> "for now" ;-)
>>
>> Cheers,
>> Tim
> Hi Tim!
>
> You're welcome! If there are any further reviews or pkgs needed during
> intergration of hadoop, just feel free to ask me for "fixing" this.
>
> Cheers,
> Björn
thanks very thanks
gil
10 years, 9 months
Re: Package Review Requests for Hadoop dependencies
by Tim St. Clair
Many thanks Björn!
It looks like we are green on dependencies!
"for now" ;-)
Cheers,
Tim
----- Original Message -----
> From: "Björn Esser" <bjoern.esser(a)gmail.com>
> To: "gil" <puntogil(a)libero.it>
> Cc: devel(a)lists.fedoraproject.org
> Sent: Tuesday, June 11, 2013 1:35:50 PM
> Subject: Re: Package Review Requests for Hadoop dependencies
>
> Am Dienstag, den 11.06.2013, 20:16 +0200 schrieb gil:
> > hi
> > yes , many thanks Björn
> > have some trouble with boost with bookkeeper
> >
> > data.cpp:249:3: error: 'lock_guard' is not a member of 'boost'
> > boost::lock_guard<boost::mutex> lock(mutex);
> > ^
> > data.cpp:249:33: error: expected primary-expression before '>' token
> > boost::lock_guard<boost::mutex> lock(mutex);
> > ^
> > data.cpp:249:45: error: 'lock' was not declared in this scope
> > boost::lock_guard<boost::mutex> lock(mutex);
> >
> > channel.cpp:705:43: error: 'shared_dynamic_cast' is not a member of 'boost'
> > boost::shared_dynamic_cast<AsioSSLDuplexChannel>(shared_from_this()),
> > ^
> > channel.cpp:705:90: error: expected primary-expression before '>' token
> > boost::shared_dynamic_cast<AsioSSLDuplexChannel>(shared_from_this()),
> > ^
> > channel.cpp: In member function 'void
> > Hedwig::AsioSSLDuplexChannel::sslShutdown()':
> > channel.cpp:752:42: error: 'shared_dynamic_cast' is not a member of 'boost'
> > boost::shared_dynamic_cast<AsioSSLDuplexChannel>(shared_from_this()),
> > ^
> > channel.cpp:752:89: error: expected primary-expression before '>' token
> > boost::shared_dynamic_cast<AsioSSLDuplexChannel>
> > (shared_from_this()),
> >
> > boost problems
> > any ideas?
> > thanks
>
> Which boost version resp. which fedora-release? I think it would be the
> best to join #fedora-devel irc and and have our discussion there,
> wouldn't it?
>
> --
> devel mailing list
> devel(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/devel
10 years, 9 months
Re: Packaging Hadoop in the Fedora community
by Tim St. Clair
Bruno -
Feel free to join the dev-lists and IRC channels(fedora-devel, fedora-bigdata), as there are numerous items we are still hashing out here (https://fedoraproject.org/wiki/Features/Hadoop), and we would be happy to coordinate. For example, getting parity across the tests is a stubborn foe. Once Hadoop we've tackled the main package we can eval and coordinate other ecosystem packages.
Cheers,
Tim
----- Original Message -----
> From: "Bruno Mahé" <bruno(a)bmahe.net>
> To: "Matthew Farrellee" <matt(a)redhat.com>
> Cc: puntogil(a)libero.it, tstclair(a)redhat.com
> Sent: Wednesday, June 12, 2013 1:56:57 AM
> Subject: Re: Packaging Hadoop in the Fedora community
>
> +Tim
>
> Hi Matthew and Tim,
>
> I noticed Tim's email on the Apache Bigtop/Hadoop mailing list
> and was wondering how we could all collaborate and work together to
> improve the state of Apache Hadoop (and its ecosystem) on
> Fedora/CentOS/RHEL.
>
> Thanks,
> Bruno
>
> On 04/26/2013 10:38 AM, Bruno Mahé wrote:
> > On 04/25/2013 06:49 AM, Matthew Farrellee wrote:
> >> On 04/25/2013 01:55 AM, Bruno Mahé wrote:
> >>> On 04/24/2013 05:40 AM, Matthew Farrellee wrote:
> >>>> Hello Big Data SIG!
> >>>>
> >>>> A few of us in the Fedora community have been working to package Apache
> >>>> Hadoop for Fedora. The feature page is,
> >>>>
> >>>> https://fedoraproject.org/wiki/Features/Hadoop
> >>>>
> >>>> You'll find what we plan to package, our progress working through all
> >>>> the dependencies, a process for how we're attacking dependencies and a
> >>>> pre-Fedora repo of the packages in progress.
> >>>>
> >>>> It's a big effort, but will form a foundation for packaging more Hadoop
> >>>> ecosystem packages for Fedora.
> >>>>
> >>>> We're at a point where we need a better place to engage the community
> >>>> and the Big Data SIG is a natural choice. I hope you will welcome us to
> >>>> the list and IRC, and maybe lend a hand!
> >>>> _______________________________________________
> >>>> bigdata mailing list
> >>>> bigdata(a)lists.fedoraproject.org
> >>>> https://admin.fedoraproject.org/mailman/listinfo/bigdata
> >>>
> >>>
> >>> Hi,
> >>>
> >>> I am not CC-ing the Bigdata SIG as to not spam the list.
> >>>
> >>> One of the project I work on intersects closely with this SIG. This
> >>> project is Apache Bigtop (http://bigtop.apache.org/).
> >>> The goal of Apache Bigtop is three folds:
> >>> 1/ Provide top notch packages for Apache Hadoop related projects
> >>> 2/ Provide a point of integration and testing for all these projects
> >>> 3/ Provide means to reliably deploy a complete stack.
> >>>
> >>> So there is quite a bit of overlapping between Apache Bigtop, this SIG
> >>> and your efforts.
> >>>
> >>> Would you see any way we could collaborate?
> >>>
> >>>
> >>> Please, see this previous message for an explanation about Apache Bigtop
> >>> with regards to this SIG and Fedora:
> >>> http://lists.fedoraproject.org/pipermail/bigdata/2013-March/000012.html
> >>>
> >>>
> >>> Thanks,
> >>> Bruno
> >>
> >> Definitely. I hold an assumption that Bigtop is a good place to -
> >>
> >> 0) drive consistency across the Hadoop ecosystem
> >> 1) provide deliverables that are integrated into various OS platforms
> >>
> >> For (0), our packaging effort is finding old and inconsistent (different
> >> versions of the same) dependencies across projects.
> >>
> >> For (1), at a minimum I assume that the packaging work done for Fedora
> >> can be contributed to Bigtop. This would provide integration with Fedora
> >> at a level that does not currently exist.
> >>
> >> Beyond that. I'm interested in the testing framework that Bigtop
> >> provides. Immediately, for the Fedora packaging effort, we need a way to
> >> make sure the packages we provide aren't breaking core functionality. I
> >> don't think TestDFSIO and some GridMix2 will be sufficient.
> >>
> >> Best,
> >>
> >>
> >> matt
> >>
> >
> > Hi,
> >
> > For (0), we are already working on this. We also spend a lot of efforts
> > helping verifying that release candidates do not break their downstream
> > projects. So the more people helping on that front, the merrier!
> >
> > For (1), we would definitely gladly accepts all contributions! And we
> > would look at the best way to integrate your work.
> > The sticky point may be related to Fedora specific macros. Our spec file
> > works across a bunch of RPM distributions, so Fedora specific macros
> > would have to be inside some ifdef or encapsulated or any other solution
> > (even using Apache Bigtop as a common effort and forking the specs to
> > add the Fedora specific macros). But in any case that would be
> > implementation details and we can look at it whenever you feel to
> > contribute your work.
> >
> > Apache Bigtop has some smoke tests, package tests (verify they can be
> > installed, removed, upgraded, permissions are rights...) as well as
> > integration tests. So I believe that could be leveraged by you whether
> > or not you decide to put in common the efforts to work on packaging or
> > not. If you have any issue using the tests, feel free to ping me or to
> > send an email to the mailing list.
> >
> >
> >
> > Thanks,
> > Bruno
>
>
10 years, 9 months
Re: Package Review Requests for Hadoop dependencies
by Tim St. Clair
Many thanks Björn!
----- Original Message -----
> From: "Björn Esser" <bjoern.esser(a)gmail.com>
> To: "Robert Rati" <rrati(a)redhat.com>
> Cc: devel(a)lists.fedoraproject.org, bigdata(a)lists.fedoraproject.org
> Sent: Tuesday, June 11, 2013 10:23:12 AM
> Subject: Re: Package Review Requests for Hadoop dependencies
>
> Hi Rob!
>
> You're welcome.
>
> Just approved ZooKeeper; BookKeeper is last to be reviewed (and
> currently FTBFS) on the list. So hadoop should reach it's target F20 in
> time :)
>
> Cheers,
> Björn
>
> Am Dienstag, den 11.06.2013, 10:21 -0400 schrieb Robert Rati:
> > Hi Bjorn,
> >
> > Thanks so much for picking these up and reviewing them!
> >
> > Rob
> >
> > On 06/11/2013 03:04 AM, Björn Esser wrote:
> > > Am Montag, den 10.06.2013, 15:42 -0400 schrieb Robert Rati:
> > >> I'm working with the Big Data SIG to package hadoop for Fedora. We're
> > >> targeting Fedora 20, but right now there are a number of packages that
> > >> hadoop will depend upon that are awaiting review. All dependent
> > >> packages have review BZs logged.
> > >>
> > >> Packages needing review for inclusion into Fedora:
> > >> bookkeeper (BZ948589)
> > >> groovy (BZ858127)
> > >> jets3t (BZ847109)
> > >> jspc-compiler (BZ960720)
> > >> maven-native (BZ864084)
> > >> zookeeper (BZ823122)
> > >>
> > >> One of those packages also has a dependency which also needs to be
> > >> reviewed:
> > >> jtoaster (zookeeper, BZ957337)
> > >>
> > >> If we are to get hadoop into Fedora 20 we need some movement on the
> > >> review requests for these packages. Would anyone, or a group of people,
> > >> be willing to review these packages and help us get them into Fedora?
> > >>
> > >> Rob
> > >
> > > Hello Rob!
> > >
> > > I'll take them one by one during this week. Would be fine if another
> > > volunteer will join-in, though.
> > >
> > > Cheers,
> > > Björn
> > >
>
>
> --
> devel mailing list
> devel(a)lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/devel
10 years, 9 months
Re: fas name
by Tim St. Clair
You can add me.
FAS Name: Tstclair
Cheers,
Tim
----- Original Message -----
> From: "gil" <puntogil(a)libero.it>
> To: "Robert Rati" <rrati(a)redhat.com>, "Tim St Clair" <tstclair(a)redhat.com>
> Sent: Tuesday, June 11, 2013 5:18:26 AM
> Subject: fas name
>
> hi
> can you give me your fas name for append this one to the fiels owner for
> package:
> bookkeeper
> zookeeper
> and for all those who feel appropriate
> thanks
> regards
> gil
>
>
10 years, 9 months
Re: Package Review Requests for Hadoop dependencies
by Robert Rati
Hi Bjorn,
Thanks so much for picking these up and reviewing them!
Rob
On 06/11/2013 03:04 AM, Björn Esser wrote:
> Am Montag, den 10.06.2013, 15:42 -0400 schrieb Robert Rati:
>> I'm working with the Big Data SIG to package hadoop for Fedora. We're
>> targeting Fedora 20, but right now there are a number of packages that
>> hadoop will depend upon that are awaiting review. All dependent
>> packages have review BZs logged.
>>
>> Packages needing review for inclusion into Fedora:
>> bookkeeper (BZ948589)
>> groovy (BZ858127)
>> jets3t (BZ847109)
>> jspc-compiler (BZ960720)
>> maven-native (BZ864084)
>> zookeeper (BZ823122)
>>
>> One of those packages also has a dependency which also needs to be reviewed:
>> jtoaster (zookeeper, BZ957337)
>>
>> If we are to get hadoop into Fedora 20 we need some movement on the
>> review requests for these packages. Would anyone, or a group of people,
>> be willing to review these packages and help us get them into Fedora?
>>
>> Rob
>
> Hello Rob!
>
> I'll take them one by one during this week. Would be fine if another
> volunteer will join-in, though.
>
> Cheers,
> Björn
>
10 years, 9 months