On Wed, 2020-01-08 at 12:24 -0700, Chris Murphy wrote:
On Mon, Jan 6, 2020 at 11:09 AM Lennart Poettering
<mzerqung(a)0pointer.de> wrote:
> - facebook is working on making oomd something that just works for
> everyone, they are in the final rounds of canonicalizing the
> configuration so that it can just work for all workloads without
> tuning. The last bits for this to be deployable are currently being
> done on the kernel side ("iocost"), when that's in, they'll
submit
> oomd (or simplified parts of it) to systemd, so that it's just there
> and works. It's their expressive intention to make this something
> that also works for desktop stuff and requires no further
> tuning. they also will do the systemd work necessary. time frame:
> half a year, maybe one year, but no guarantees.
Looks like PSI based oom killing doesn't work without swap. Therefore
oomd can't be considered a universal solution. Quite a lot of
developers have workstations with quite a decent amount of RAM,
~64GiB, and do not use swap at all. Server baremetal are likewise
mixed, depending on workloads, and in cloud it's rare for swap to
exist.
https://github.com/facebookincubator/oomd/issues/80
We think earlyoom can be adjusted to work well for both the swap and
no swap use cases.
But so can oomd, after all, they are willing to implement a plugin that
uses the MaxAvailable heuristic. It just won't be available in the
short term.
In principle, I think what we are trying to achieve here is to keep the
system mostly responsive from a user perspective. This seems to imply
keeping pages in main memory that belong to "important" processes.
Should oomd not manage to do this well enough out of the box, then I
see two main methods we have to improve things:
* Aggressively kill when we think important pages might get evicted
- earlyoom does this based on MemAvailable
- oomd plugin could do the same if deemed the right thing
* Actively protect important processes[1]:
- set MemoryMin, MemoryLow on important units
- limit "normal" processes more e.g. MemoryHigh for applications
- in the long run: adjust the OOMScore/MemoryHigh dynamically based
on whether the user is interacting with an application at the time
earlyoom does the first and has the big advantage that it can be
shipped in F32. However, it is not clear to me that this aggressive
heuristic is actually better overall. And even if it is, we would
likely still move it into oomd in the long run.
Finally, for F32 we might already be able to improve things quite a lot
simply by setting a few configuration options in GNOME systemd units.
Benjamin
[1] I do not know how well this works, so it may be nice if people
experimented with it[2]. For GNOME you can easily add a systemd drop-in
for various services. e.g. to protect the shell (in a wayland session)
simply do:
$ systemctl edit --user gnome-shell-wayland.service
[Service]
MemoryMin=250M
MemoryLow=500M
Which I suspect should already help a lot in many scenarios.
[2] Unfortunately, I guess that such measurements may be skewed a lot
on systems that use swap due to unrelated lags. i.e. Jan Grulich's mail
from earlier today titled "Lagging system with latest kernels".