Sorry if this is the wrong place to post. I need to start somewhere.
Description of problem:
The Plymouth LUKS password prompt will wait FOREVER for me to type in a
password without powering down the monitor or blanking the screen. This
leaves me vulnerable to screen burn-in if 1) there was an unexpected
reboot like a crash, or 2) another OS (which shall remain nameless)
rebooted after update, but the dual-boot started in Linux instead of
the other OS.
Steps to Reproduce:
1. Wait at LUKS prompt until you need to throw away your monitor.
A bill for an expensive new monitor.
The monitor goes into powersaver mode after a reasonable time, like 20
minutes, or at least blanks.
GSL 2.6 has just been released, and it comes with an soname bump. All
dependent packages will need to be rebuilt in rawhide.
Fedora Project Contributor
This subject matches a Fedora Workstation Working Group issue of the
same name , and this post is intended to be an independent summary
of the findings so far, and call for additional testing and
discussion, in particular subject matter experts.
Problem and thesis statement:
Certain workloads, such as building webkitGTK from source, results in
heavy swap usage eventually leading to the system becoming totally
unresponsive. Look into switching from disk based swap, to swap on a
Summary of findings (restated, but basically the same as found at ):
Test system, Macbook Pro, Intel Core i7-2820QM (4/8 cores), 8GiB RAM,
Samsung SSD 840 EVO, Fedora Rawhide Workstation.
Test case, build WebKitGTK from source.
$ cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -GNinja
Case 1: 8GiB swap on SSD plain partition (not encrypted, not on LVM)
Case 2: 8GiB swap on /dev/zram0
In each case, that swap is exclusive, there are no other swap devices.
Within ~30 minutes in the first case, and ~10 minutes in the second
case, the GUI is completely unresponsive, mouse pointer has frozen and
doesn't recover after more than 30 minutes of waiting. By remote ssh,
the first case is semi-responsive, updates should be every 5 seconds
but are instead received every 2-5 minutes but it wasn't possible to
compel recovery by cancelling the build process after another 30
minutes. By remote ssh, the second case is totally unresponsive, no
updates for 30 minutes.
The system was manually forced power off at that point, in both cases.
oom killer never triggered.
NOTE: ninja, by default on this system, sets N concurrent jobs to
nrcpus + 2, which is 10 on this system. If I reboot with nr_cpus=4,
ninja sets N jobs to 6.
Case 3: 2GiB swap on /dev/zram0
In one test this resulted in system hang (no pointer movement) within
5 minutes of executing ninja, and within another 6 minutes oom killer
is invoked on a cc1plus process, which is fatal to the build process,
remaining build related processes quit on their own, and the system
But in two subsequent tests in this same configuration, oom killer
wasn't invoked, and the system meandered between responsive for ~1
minute, totally frozen for 5-6 minutes, in a cycle lasting beyond 1
hour without ever triggering oom killer.
Screenshot taken during one of the moments the remote ssh session updated
The state had not changed after 45 minutes following the above
screenshot so I forced power off on that system. But the point here is
this slightly different configuration has some non-determinism to it,
even though in the end it's a bad UX. The default, unprivileged build
command is effectively taking down the system all the same.
Case 4: 8GiB swap on SSD plain partition, `ninja -j 4`
This is the same setup as Case 1, except I manually set N jobs to 4.
Build succeeds, and except for a few mouse pointer stutters, the
system remains responsive, even Firefox with multiple tabs open, and
youtube video playing. Exactly the experience we'd like to see, albeit
not all CPU resources are used for the build, but clearly the limiting
factor is this particular package requires more than ~14GiB to build
successfully, and the system + shell + Firefox, just doesn't have
To what degree, and why, is this problem instigated by the build
application (ninja in this example) or its supporting configuration
files, including cmake? Or the kernel? Or the system configuration? Is
it a straightforward problem, or is this actually somewhat nuanced
with multiple components in suboptimal configuration coming together
as the cause? Is it expected that an unprivileged user can run a
command whose defaults eventually lead to a totally unrecoverable
system? From a security risk standpoint, the blame can't be entirely
on the user or the application configuration, but how should
application containment be enforced? Other than containerizing the
build programs, is there a practical way right now of enforcing CPU
and memory limits on unprivileged applications? Other alternatives? At
the very least it seems like getting to an oom killer sooner would
result in a better experience, fail the process before the GUI becomes
unresponsive and hangs out for 30+minutes (possibly many hours).
I agree that subpackages for py2 and py3 seems best, but I'll have to
see if mercurial can be parallel installed without too much effort.
I don't expect to have time to work on this for the next 2 weeks.
On Sun, Aug 18, 2019 at 9:16 PM Mads Kiilerich <mads(a)kiilerich.com> wrote:
> You are giving many good reasons why we need Mercurial packages for
> Python 2 and Python 3, side by side.
> That will make it possible for other packages to jump to Python 3. Your
> package will no longer be blocking, and it doesn't have to be your concern.
> It will also put the Mercurial package in the good position where it has
> done all the work of transitioning, and the python2 part will be
> entirely optional, only kept alive for benefit of other packages that
> still depend on it.
> I am working upstream with tortoisehg. While late, I don't expect any
> problems. The current blocker for Fedora packaging of tortoisehg is the
> lack of Python 3 Mercurial, and the biggest risk is that the Python 2
> Mercurial package goes away before I had time to transition the
> tortoisehg package to depend on Python 3 Mercurial.
> On 8/16/19 3:53 PM, Neal Becker wrote:
> > I think tortoisehg is not ported to python3
> > On Wed, Aug 14, 2019 at 1:39 PM Neal Becker <ndbecker2(a)gmail.com> wrote:
> >> I just tested hg-5.1.0 with the latest git version of hg-evolve on
> >> python3 and at least some basic things seem to be working. One
> >> problem:
> >> hg
> >> *** failed to import extension hggit: No module named 'compat'
> >> That's with the latest pip version of hg-git (0.8.12).
> >> hg-git is a supported fedora package, so I think we need this to work.
> >> On Tue, Aug 13, 2019 at 9:35 AM Petr Stodulka <pstodulk(a)redhat.com> wrote:
> >>> On 12. 08. 19 22:36, Miro Hrončok wrote:
> >>>> On 12. 08. 19 20:37, Petr Stodulka wrote:
> >>>>> Can you explain better what do you mean by that? I am little lost
> >>>>> here.
> >>>> Sure. The idea was:
> >>>> 1) When Fedora 31 is branched (scheduled for tomorrow ), push the switch to rawhide (Fedora 32)
> >>>> 2) See what happens, collect feedback.
> >>>> 3) Soon before F31 Beta Freeze (scheduled for 2019-08-29 ) decide whether this is worth pushing to F31 (probably not)
> >>>> 4) Soon before F32 mass Python 2 removal flag day (scheduled for mid November ), decide whether to revert and request and exception or not
> >>>>  https://fedoraproject.org/wiki/Releases/31/Schedule
> >>>>  https://fedoraproject.org/wiki/Changes/RetirePython2#Detailed_Description
> >>> Thanks Miro for explanation. That sounds asi the best idea now probably. From that point, I will not create new subpackages for Py2 / Py3 and I will just do the switch tomorrow.
> >>> @mercurial-maintainers: if anyone have something against or better solution, answer here guys, otherwise I will do the rebase.
> >>> Just FYI, I will be kinda offline between 2019-08-15 - 2019-08-25 because of PTO. So in case of troubles, I will not be able to do anything during that time. From that point, I could postpone the rebase if needed, but I hope that someone else will be able to help around instead of me in case of troubles.
> >>> --
> >>> Petr Stodulka
> >>> OS & Application Modernization
> >>> IRC nicks: pstodulk, skytak
> >>> Software Engineer
> >>> Red Hat Czech s.r.o.
> >> --
> >> Those who don't understand recursion are doomed to repeat it
Those who don't understand recursion are doomed to repeat it
I've tried to get a couple of epel8 branches created, and the tickets
have been closed (twice in one case after I re-opened it) with "The
branch in PDC already exists". (It worked for other packages.) I don't
know what PDC is, but the branch definitely isn't in git. An example is
<https://pagure.io/releng/fedora-scm-requests/issue/15061>. Can someone
tell me what to do about that?
Also, I don't get any notifications from pagure (as with various other
-- most? -- services), despite turning on anything relevant-looking in
the notifications app. I've asked several times about lack of
notifications with no luck or suggestions how to debug it. Can anyone
make any suggestions now?
I'm increasingly un-motivated to do maintenance particularly because of
continual infrastructure problems.
I've been following rawhide for the last few weeks to test some F31
things. Now that we've branched I've updated my .repo files to point to
the normal fedora repos and disabled the rawhide ones, but still
dnf update wants to install a whole load of .fc32 packages.
I've already manually updated to fedora-release-31-0.10 and cleared
my dnf cache, but that does not help.