FYI
-------- Original Message --------
Subject: F21 Self Contained Change: Apache Spark
Date: Fri, 21 Mar 2014 13:28 +0100
From: Jaroslav Reznik <jreznik(a)redhat.com>
Reply-To: devel(a)lists.fedoraproject.org
Organization: Red Hat, Inc.
To: devel-announce(a)lists.fedoraproject.org
= Proposed Self Contained Change: Apache Spark =
https://fedoraproject.org/wiki/Changes/ApacheSpark
Change owner(s): William Benton <willb(a)redhat.com>
Apache Spark is a fast and general engine for large-scale data processing.
This change brings Spark to Fedora, allowing easy deployment and
development
of Spark applications on Fedora.
== Detailed Description ==
Apache Spark is a fast and general engine for large-scale data
processing. It
supports developing custom analytic processing applications over large data
sets or streaming data. Because it has the capability to cache intermediate
results in cluster memory and schedule DAGs of computations, Spark programs
can run up to 100x faster than equivalent Hadoop MapReduce jobs. Spark
applications are easy to develop, parallel, fast, and resilient to failure,
and they can operate on data from in-memory collections, local files, a
Hadoop-
compatible filesystem, or from a variety of streaming sources. Spark also
includes libraries for distributed machine learning and graph algorithms.
== Scope ==
* Proposal owners: Currently our Spark package has been accepted into
Fedora
[1]. It features nearly all of the functionality available from the
upstream
release. (The missing features -- specifically, Python bindings, the Spark
REPL, Kryo-based serialization, primitives for approximate cardinalities of
very large sets, and Mesos integration -- were missing from the initial
packages due to unavailable dependencies and bundling issues; we're
working to
close the gap with upstream as quickly as possible.) This work depended
upon
Fedora 21's improved support for the Scala ecosystem [2].
* Other developers: N/A (not a System Wide Change)
* Release engineering: N/A (not a System Wide Change)
* Policies and guidelines: N/A (not a System Wide Change)
[1]
http://pkgs.fedoraproject.org/cgit/spark.git
[2]
https://fedoraproject.org/wiki/Changes/ImprovedScalaEcosystem
_______________________________________________
devel-announce mailing list
devel-announce(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel-announce