The following proposal comes out of the discussion at this weeks Server SIG
meeting[1]
Fedora Server will have:
* / (root) will be a minimum of 2 GiB and a maximum of 15 GiB
* SWAP will continue to be calculated automatically based on available RAM on
the system
* All unused space will be assigned to a volume group and available to be
assigned to new partitions or extend existing partitions.
* Anaconda will continue to handle the appropriate EFI and /boot settings
We also discussed during the meeting whether we should have a separate /var
partition by default, but the general sense was that we might be better served
by developing a mechanism to allow partitions to be split from existing mount
points, which would be more flexible going forward.
As we did not have quorum in the meeting by the point we got to this proposal,
I'm taking it to the list for discussion and votes.
For the record, the current behavior of the partitioning scheme is for / to be
given up to 50 GiB of space and then any remaining space after that is assigned
to a separate /home partition.
[1]
https://meetbot.fedoraproject.org/fedora-meeting-1/2016-03-15/serversig.201…
Hi,
Workstation working group continues to evaluate oom managers and seek
input from domain experts on the subject. I've come across this video
from the All Systems Go conference, on the larger subject of resource
control. Its server+cloud+container oriented. So I think it speaks
directly to your use cases.
Resource Control (2019) Dan Schatzberg
https://www.youtube.com/watch?v=30i7SamZxRU
I'm looking into setting up a discussion session with Dan, and do some
Q&A. I'll report back when I know more about that.
--
Chris Murphy
Hi server@ and cloud@ folks,
There is a system-wide change to enable earlyoom by default on Fedora
Workstation. It came up in today's Workstation working group meeting
that I should give you folks a heads up about opting into this change.
Proposal
https://fedoraproject.org/wiki/Changes/EnableEarlyoom
Devel@ discussion
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org…
The main issue on a workstation, heavy swap leading to an unresponsive
system, is perhaps not as immediately frustrating on a server. But
the consequences of indefinite hang or the kernel oom-killer
triggering, which is a SIGKILL, are perhaps worse.
On the plus side, earlyoom is easy to understand, and its first
attempt is a SIGTERM rather than SIGKILL. It uses oom_score, same as
kernel oom-killer, to determine the victim.
The SIGTERM is issued to the process with the highest oom_score only
if both memory and swap reach 10% free. And SIGKILL is issued to the
process with the highest oom_score once memory and swap reach 5% free.
Those percentages can be tweaked, but the KILL percentage is always
1/2 of the TERM percentage, so it's a bit rudimentary.
One small concern I have is, what if there's no swap? That's probably
uncommon for servers, but I'm not sure about cloud. But in this case,
SIGTERM happens at 10% of RAM, which leaves a lot of memory on the
table, and for a server with significant resources it's probably too
high. What about 4%? Maybe still too high? One option I'm thinking of
is a systemd conditional that would not run earlyoom on systems
without a swap device, which would leave these systems no worse off
than they are right now. [i.e. they eventually recover (?),
indefinitely hang (likely), or oom-killer finally kills something
(less likely).]
I've been testing earlyoom, nohang, and the kernel oom-killer for > 6
months now, and I think it would be completely sane for Server and
Cloud products to enable earlyoom by default for fc32, while
evaluating other solutions that can be more server oriented (e.g.
nohang, oomd, possibly others) for fc33/fc34. What is clear: this
isn't going to be solved by kernel folks, the kernel oom-killer only
cares about keeping the kernel alive, it doesn't care about user space
at all.
In the cases where this becomes a problem, either the kernel hangs
indefinitely or does SIGKILL for your database or whatever is eating
up resources. Whereas at least earlyoom's first attempt is a SIGTERM
so it has a chance of gracefully quitting.
There are some concerns, those are in the devel@ thread, and I expect
they'll be adequately addressed or the feature will not pass the FESCo
vote. But as a short term solution while evaluating more sophisticated
solutions, I think this is a good call so I thought I'd just mention
it, in case you folks want to be included in the change.
--
Chris Murphy