Re: multi-CPU optimization inside a distribution
by Dan Horák
On Thu, 30 Jun 2016 15:59:38 -0400
Solomon Peachy <pizza(a)shaftnet.org> wrote:
> On Thu, Jun 30, 2016 at 08:04:57PM +0100, Richard W.M. Jones wrote:
> > I will just snipe here that the situation would be better if POWER
> > > 4 hardware was generally available to buy.
>
> The upcoming Raptor Talos POWER8 board [1] looks promising, but at
> $3700 a pop just for the motherboard and an entry-level [1] CPU, it's
> out of reach of many otherwise-interested developers. Assuming it
> goes into production at all..
unfortunately it does not in foreseeable future (based on official
update on their IRC channel). It was my hope to get newer workstation
class HW and be able to lift the ppc64 (BE) support to Power7 and up
(or so). With the OpenPOWER ecosystem there is still chance someone
else will try it.
Dan
7 years, 10 months
Re: multi-CPU optimization inside a distribution
by Josh Boyer
On Thu, Jun 30, 2016 at 12:01 PM, Jan Kratochvil
<jan.kratochvil(a)redhat.com> wrote:
> On Thu, 30 Jun 2016 16:13:59 +0200, Josh Boyer wrote:
>> On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote:
>> > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer
>> >>=Power8 hardware does use ppc64le which is already Power8 optimized.
>> > So isn't this problem (somehow temporarily) solved for now?
>>
>> POWER8 can run in either endian mode. So no, the problem is not solved.
>
> End-users do not even know what is endianity. And all application developers
> need to write software BE/LE-independently (such as using ntohl()/htonl()
> etc.) so they also do not care whether the host is BE or LE. Only <1% of
> developers for high performance core or very low-end libraries (like glibc or
> glib) need to care about BE/LE.
>
> So why to run Power8 in BE? Power is switching to LE AFAIK which makes it
> also better compatible with buggy software ignoring BE/LE difference which is
> (as all the software) targeting primarily x86* (=LE).
It isn't switching. It's adding. I'd maybe agree that LE will become
more prevalent over time, particularly as it is used in cloud
workloads. However, BE mode isn't going away anytime soon.
>> Realistically, it isn't about either specifically. Each iteration of
>> POWER tends to require tuning specifically for that generation if you
>> want to get the most performance out of your software.
>
> Latest CPU features get utilized mostly only in inner loops of high
> performance code which should use IFUNC. So building the whole distro for the
> latest CPU brings only incompatibility with no real performance advantage.
>
>
> At least this is my expectation, the OP misses any real benchmark numbers
> whether the distro build for the latest CPU generation is worth it.
> IIRC for x86* it was not worth it.
You might not be wrong, but you're dismissing the market dynamics of
POWER that I mentioned. Those same dynamics lead to the distribution
components for ppc64 being built as 64-bit binaries, even though at
the time 32-bit binaries for 99% of packages were actually more than
sufficient.
Comparing performance numbers within POWER is fine, but applying the
"this is how we do it in x86" logic isn't really suitable. They are
different markets. I believe the markets will perhaps merge over
time, but they clearly haven't yet today.
josh
7 years, 10 months
Re: multi-CPU optimization inside a distribution
by Peter Robinson
On Thu, Jun 30, 2016 at 5:01 PM, Jan Kratochvil
<jan.kratochvil(a)redhat.com> wrote:
> On Thu, 30 Jun 2016 16:13:59 +0200, Josh Boyer wrote:
>> On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil <jan.kratochvil(a)redhat.com> wrote:
>> > But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer
>> >>=Power8 hardware does use ppc64le which is already Power8 optimized.
>> > So isn't this problem (somehow temporarily) solved for now?
>>
>> POWER8 can run in either endian mode. So no, the problem is not solved.
>
> End-users do not even know what is endianity. And all application developers
> need to write software BE/LE-independently (such as using ntohl()/htonl()
> etc.) so they also do not care whether the host is BE or LE. Only <1% of
> developers for high performance core or very low-end libraries (like glibc or
> glib) need to care about BE/LE.
>
> So why to run Power8 in BE? Power is switching to LE AFAIK which makes it
> also better compatible with buggy software ignoring BE/LE difference which is
> (as all the software) targeting primarily x86* (=LE).
Data conversion is expensive, slow and problematic so once it's in one
format it tends to stay that way. There's also legacy/proprietary
tools that run as big endian on the Power64 platform. For a new
deployment there's no reason not to do it as LE from the outset, and
migration from other LE platforms makes sense but if you're in a BE
world (AIX, Solaris, Z-series and friends) it makes sense to run in
BE.
There are some applications where the data comes in in BE format and
it's costly to deal with that in real time, especially in high
throughput scenarios.
>> Realistically, it isn't about either specifically. Each iteration of
>> POWER tends to require tuning specifically for that generation if you
>> want to get the most performance out of your software.
>
> Latest CPU features get utilized mostly only in inner loops of high
> performance code which should use IFUNC. So building the whole distro for the
> latest CPU brings only incompatibility with no real performance advantage.
We're looking at IFUNC as one option, Function multi-versioning in GCC
6 [1] is presumably also another option, although wouldn't be useful
for already released enterprise releases, although obviously
irrelevant for the Fedora conversation.
> At least this is my expectation, the OP misses any real benchmark numbers
> whether the distro build for the latest CPU generation is worth it.
> IIRC for x86* it was not worth it.
[1] https://lwn.net/Articles/691932/
7 years, 10 months
Re: multi-CPU optimization inside a distribution
by Peter Robinson
On Thu, Jun 30, 2016 at 2:53 PM, Jan Kratochvil
<jan.kratochvil(a)redhat.com> wrote:
> On Thu, 30 Jun 2016 12:12:37 +0200, Dan Horák wrote:
>> For ppc64 (the big endian POWER) the base is set by the toolchain
>> default which is Power4/ppc970. When Power7 came we were asked what are
>> the options to take the advantage of these CPUs, 3 generations newer
>> than the base. The solution was the introduction of ppc64p7 subarch
>
> But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer
>>=Power8 hardware does use ppc64le which is already Power8 optimized.
> So isn't this problem (somehow temporarily) solved for now?
>
> Or are really the Power7 (and not yet Power8) machines the real problem?
No your confusing endian with CPU features. We support both Big Endian
(ppc64) and Little Endian (ppc64le) options for the Power64
architecture. Both can run on Power8 hardware, in fact the builder VMs
run like with with both BE and LE VMs running on the same physical
Power8 host side by. There's shortly going to also be Power9 [1] with
options that need optimisation to make the most of new architectural
enhancements which will be primarily focused towards ppc64le (but will
also work on ppc64) but we don't want to exclude Power8 from ppc64le.
Peter
[1] http://www.nextplatform.com/2016/04/18/power9-will-bring-competition-data...
7 years, 10 months
Re: multi-CPU optimization inside a distribution
by Josh Boyer
On Thu, Jun 30, 2016 at 9:53 AM, Jan Kratochvil
<jan.kratochvil(a)redhat.com> wrote:
> On Thu, 30 Jun 2016 12:12:37 +0200, Dan Horák wrote:
>> For ppc64 (the big endian POWER) the base is set by the toolchain
>> default which is Power4/ppc970. When Power7 came we were asked what are
>> the options to take the advantage of these CPUs, 3 generations newer
>> than the base. The solution was the introduction of ppc64p7 subarch
>
> But now there is ppc64le for >=Power8. So <=Power7 can use ppc64 but newer
>>=Power8 hardware does use ppc64le which is already Power8 optimized.
> So isn't this problem (somehow temporarily) solved for now?
POWER8 can run in either endian mode. So no, the problem is not solved.
> Or are really the Power7 (and not yet Power8) machines the real problem?
Realistically, it isn't about either specifically. Each iteration of
POWER tends to require tuning specifically for that generation if you
want to get the most performance out of your software. That tuning is
incompatible with older generations of hardware (endian completely
aside). So as Dan alludes to, this is fairly cumbersome. To make it
widely useable, you default to the oldest generation you want to
support.
x86 theoretically has similar problems, but we tend to just not care
as much about tuning at the distro level. The applications can tune
however they'd like, but given that most datacenters are not
homogeneous among x86 machines, it often means people cannot tune for
one specific x86 generation because they have to run on a wider
variety.
Contrast that with POWER, where people using that tend to buy one or
two very large systems to consolidate their workload on. They can and
often do want the best performance they can eek out since they're
consolidating, which means they often want the distro components tuned
as well, particularly system libraries.
Going forward, I'd seriously look at dropping anything older than
POWER7 support. I know we all dearly love our Mac G5 machines
(ppc970), but those are getting very very old at this point. POWER5
was a non-event for Fedora, POWER6 was a weird generation even for
IBM. POWER7 and POWER8 are where most of the effort is, and that will
only continue as next generation POWER chips come out. Given that
Rackspace and others have actually started using the openPOWER stuff
with POWER8, it probably makes sense to focus on where the users are
headed and not where they were 10 years ago.
josh
7 years, 10 months
multi-CPU optimization inside a distribution
by Dan Horák
Hi all,
we were talking about this item for some time, so let's start a thread
for it to have the discussion and hopefully also a solution documented.
This is meant as discussion initiation based on the situation in
Fedora on POWER. I would like to ask the bigger experts than me to fill
the missing details and options.
Currently we set a minimal CPU level for an architecture (or use the
toolchain default) on the distribution level
(/usr/lib/rpm/redhat/rpmpc owned by redhat-rpm-config [1]). It allows
the distro to run on such CPU and any newer evolution of it (omitting
any kernel or hw issues), but it also means it doesn't generally take
advantage of features and improvements in the newer CPUs.
For ppc64 (the big endian POWER) the base is set by the toolchain
default which is Power4/ppc970. When Power7 came we were asked what are
the options to take the advantage of these CPUs, 3 generations newer
than the base. The solution was the introduction of ppc64p7 subarch into
the packaging and release engineering tools. But it showed more as a
hack than a solution, touching rpm, yum, koji, .... The list of packages
is manually maintained, requires manual updates to the buildsystem for
new releases, seems new packaging tools (dnf) don't understand it
correctly. Is there a way to make the subarch mechanism generic and
integrated with the other tools? So we could have ppc64p8 and ppc64p9
inside Fedora for POWER ...
Now I'm getting to an area where others are the experts :-) Glibc
allows to build and install multiple per-cpu optimized runtimes that
are selected based on the CPU. There is the IFUNC mechanism and
function multi-versioning in GCC (user-friendly IFUNC) [2] to allow
multiple implementations inside one library/binary.
Some packages do the CPU detection during runtime and select the
optimized functions themselves.
There is also an option to introduce a "tertiary architecture" and
rebuild everything for the new CPU, but keeping the rpm arch the same.
But it has its own costs too.
What do you think? Are there any recommendations for both the distro
and package maintainers and upstream developers? I suppose even primary
architectures are facing the same problem.
[1] http://pkgs.fedoraproject.org/cgit/rpms/redhat-rpm-config.git/tree/rpmrc
[2] https://lwn.net/Articles/691932/
Dan
7 years, 10 months
Fwd: Fedora 24 is here!
by Peter Robinson
For those that didn't notice the Secondary arches are included in the
Primary release announcement.
Congrats and kudos to all architecture teams for all the hard work to
get them out at the same time as primary!
Peter
---------- Forwarded message ----------
From: Matthew Miller <mattdm(a)fedoraproject.org>
Date: Tue, Jun 21, 2016 at 2:47 PM
Subject: Fedora 24 is here!
To: announce(a)lists.fedoraproject.org
Today the Fedora Project is pleased to announce the general release of
Fedora 24. Download it now from our Get Fedora site:
Workstation: https://getfedora.org/workstation/
Server: https://getfedora.org/server/
Cloud: https://getfedora.org/cloud/
Spins: https://spins.fedoraproject.org/
Labs: https://labs.fedoraproject.org/
ARM: https://arm.fedoraproject.org/
Another Step in the Fedora Journey
----------------------------------
The Fedora Project has embarked on a great journey... redefining what
an operating system should be for users and developers. Such innovation
does not come overnight, and Fedora 24 is one big step on the road to
the next generation of Linux distributions. But that does not mean that
Fedora 24 is some "interim" release; there are great new features for
Fedora users to deploy in their production environments right now!
Workstation
-----------
The Fedora 24 Workstation release features GNOME 3.20, with many
usability improvements such as easier input device and printer
settings, a better search interface, shortcut windows for keyboard
commands, and more convenient music controls.
Flatpak (formerly xdg-app) is another building-block feature, with
Software able to track installed Flatpaks and adding more features in
the future as the technology develops. The Software app has also grown
features to provide a full system upgrade directly from the desktop
from one Fedora release to the next, and the ability to provide
labeling as well as reviews of available software.
Fedora 24 continues our work on the X replacement, Wayland, a
next-generation graphics stack. Although this release will not default
to Wayland, it includes many improvements and is available as an option
for users to try out, and potentially will be the default stack in
Fedora 25.
Server
------
Fedora 24 Server edition is more streamlined and introduces more
modularity, which will become a major factor in future Fedora releases,
even as unnecessary packages were removed and the installer has a
smaller footprint.
FreeIPA 4.3 is a major feature for Fedora 24 Server.
FreeIPA is an integrated security information management solution. This
new version of FreeIPA features simplified replica installation and
improved replication technology management.
Cloud
-----
Fedora is on its way to being the best platform for containerized
applications, from base Fedora container images to a full-featured
platform as a service to run and manage them.
As we continue on this part of the journey, we are packaging OpenShift
Origin so it is easy to deploy. OpenShift Origin is a Platform as a
Service system based around Kubernetes, a production-grade container
orchestration project. OpenShift Origin is optimized for application
development and deployment. Origin makes it easy for developers to get
started building applications in containers and for operators to manage
them.
While not shipped in Fedora 24, per se, we have new infrastructure for
developing container images with applications layered on top of the
base Fedora Docker image. Fedora Developers will also see a layered
image build service, which provides tools for Fedora contributors to
start creating and shipping layered container images in Fedora 25 and
beyond.
Spins and Labs
--------------
Fedora Spins and Labs are alternative Fedora versions that offer
additional desktop environments, or other custom collections of
software, alongside the three editions that are the primary focus for
the project.
Our Spins make it easy for people to use other desktop environments.
Everyone has different needs and Spins are a great way for us to meet
them. The Fedora 24 spins release showcases KDE Plasma, Xfce, LXDE,
Mate-Compiz, and Cinnamon, all on the same Fedora base. Fedora Labs
offer collections of software for specific purposes: Games, Robotics,
and so on. They are pre-selected sets of software ideal for events or
audiences with specialized interests that need corresponding software.
Fedora 24 comes with a new lab focused on Astronomy, which contains a
set of tools for astronomers and astrophysicists.
ARM
---
ARM images are available, as usual, for several use cases. Fedora 24
ships desktop images, such as Spins and Workstation, but also provides a
Server image. A minimal Fedora image completes the wide set of install
options for your ARM board.
Alternate Fedora Architectures
------------------------------
For the first time, all of our secondary architectures, AArch64,
Power64 (ppc64, ppc64le) and s390x, are released simultaneously, rather
than trailing a little behind. This includes the Server edition for all
architectures and the Cloud and Docker base images for AArch64 and
Power64. Along with all the same enhancements as in the primary
architectures in Fedora 24, AArch64 and Power64 now have support for
golang, nodejs, mongodb; along with numerous other
architecture-specific enhancements. The architecture release notes
pages on the wiki have more specifics for each. Download these from:
https://dl.fedoraproject.org/pub/fedora-secondary/releases/24/
Atomic Host
-----------
Fedora Atomic Host releases on a two-week schedule, and each release is
built on the latest released Fedora version. This schedule means the
Atomic Host will now be currently built on Fedora 24. You can try one of
the newer features with recent Fedora Atomic Host builds today. Since
Fedora 23 was released, Atomic Host has added a developer mode that
gives a better developer experience overall. When running in developer
mode, the host will download and start Cockpit and a tmux session to
make it easier to work at the console and obtain necessary information
such as the root password or IP address.
Make Fedora Even Better
-----------------------
If you want to take an active hand in making Fedora even better, there
are many ways you can contribute. There are many roles that you can
participate. Visit http://whatcanidoforfedora.org/ for more
information! While this is a general release, there is always a chance
you may encounter bugs or missing features. To report issues
encountered during testing, contact the Fedora QA team via the mailing
list or in #fedora-qa on Freenode IRC. As testing progresses, common
issues are tracked on the Common F24 Bugs page:
https://fedoraproject.org/wiki/Common_F24_bugs
For tips on reporting a bug effectively, read "how to file a bug
report at https://fedoraproject.org/wiki/How_to_file_a_bug_report.
Flock 2016: Krakow, Poland
--------------------------
If you want to be a part of the journey that Fedora is taking, either
as a current or interested Fedora contributor, one way to engage with
our community is through Fedora premier events. The annual North
American/European conference for Fedora contributors is Flock, which
takes place August 2-5, 2016 in Krakow, Poland. Registration is now
open at https://register.flocktofedora.org. For more information about
our Latin American and Asia-Pacific Conferences, stay tuned for
announcements on the Fedora Community Blog:
https://communityblog.fedoraproject.org.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
--
announce mailing list
announce(a)lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/announce@lists.fedoraproject.org
7 years, 10 months