Hi Jaroslav,
For HPC workloads its seen that ondemand cpufreq governor
introduces some jitter.
It has been reported in several distros (eg Ubuntu, Redhat) [1].
Jitter is mainly
due to kernel worker threads which needs to get scheduled to
perform load
calculation and set frequencies.
A new cpufreq governor "schedutil" is being developed by Rafael J.
Wysocki which
seems to largely solve the problem. It hooks on to the scheduler
for load calculations
and sets frequency in interrupt contexts (hence no worker thread
which causes the problem).
But there is a long way to go before it stabilizes and reaches
distros.
A fix that could reduce jitter went into ubuntu recently (only for
ppc64) [1],
was by setting a ondemand governor's tuning parameter
"sampling_down_factor".
Sampling_down_factor is a multiplying factor for sampling rate
(rate at which cpu
load calculation happens) when the CPU is at its highest frequency
it reduces the
rate at which load calcuations are done.
By setting sampling_down_factor = 100, If cpu reaches max
frequency,
the sampling rate is reduced by 100 times which reduces jitter.
Typically, sampling_rate = 10 ms,
sampling_down_factor=100 ,
resulting sampling rate = 1000ms = 1 sec
The idea is that the effect of jitter is more pronounced when the
cpu load is closer to 100%,
as ondemand governor sets to max frequency only when there is
high cpu load. We could
reduce the sampling rate , when it effects the most.
Proposed fix through tuned :
- Adding a new balanced profile for hpc workloads called
balanced-hpc.
- We could set sampling_down_parameter to 100 for this profile.
[1]
https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1483586
[2]
https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
Following is a proposed patch that adds a balanced-hpc profile:
This profile derived from the balanced profile. The only changes
from balanced profile are the cpufreq governor and additional
setting of sampling_down_factor.
sampling_down_factor is set to 100.
diff --git a/profiles/balanced-hpc/scripts.sh
b/profiles/balanced-hpc/scripts.sh
new file mode 100755
index 0000000..b494e2d
--- /dev/null
+++ b/profiles/balanced-hpc/scripts.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+
+. /usr/lib/tuned/functions
+
+start() {
+ echo 100 >
/sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
+ return 0
+}
+
+stop() {
+ echo 1 >
/sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
+ return 0
+}
+
+process $@
diff --git a/profiles/balanced-hpc/tuned.conf
b/profiles/balanced-hpc/tuned.conf
new file mode 100644
index 0000000..777ef41
--- /dev/null
+++ b/profiles/balanced-hpc/tuned.conf
@@ -0,0 +1,21 @@
+#
+# tuned configuration
+#
+
+[cpu]
+governor=ondemand
+energy_perf_bias=normal
+
+[audio]
+timeout=10
+
+[video]
+radeon_powersave=auto
+
+[disk]
+# Comma separated list of devices, all devices if commented out.
+# devices=sda
+alpm=medium_power
+
+[script]
+script=scripts.sh
Regards
Akshay Adiga