cloud December 2019

cloud@lists.fedoraproject.org

5 participants
13 discussions

Python is missing in Fedora AWS AMIs
by Danishka Navin 29 Dec '22

29 Dec '22

Hi team, Is there any specific reason to drop python from Fedora AMIs in AWS? Regards, -- Danishka Navin http://danishkanavin.blogspot.com http://twitter.com/danishkanavin

6 5

[Fedocal] Reminder meeting : Fedora Cloud Workgroup
by dusty＠dustymabe.com 30 Dec '19

30 Dec '19

Dear all, You are kindly invited to the meeting: Fedora Cloud Workgroup on 2020-01-01 from 17:00:00 to 18:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Standing meeting for the Fedora Cloud Workgroup Source: https://apps.fedoraproject.org/calendar/meeting/1999/

1 0

[Fedocal] Reminder meeting : Fedora Cloud Workgroup
by dusty＠dustymabe.com 23 Dec '19

23 Dec '19

Dear all, You are kindly invited to the meeting: Fedora Cloud Workgroup on 2019-12-25 from 17:00:00 to 18:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Standing meeting for the Fedora Cloud Workgroup Source: https://apps.fedoraproject.org/calendar/meeting/1999/

1 0

Re: F32, enable fstrim.timer by default
by Chris Murphy 18 Dec '19

18 Dec '19

On Wed, Dec 18, 2019 at 5:32 AM Martin Kolman <mkolman(a)redhat.com> wrote: > This will also trim thin LVs on thin pools (if any), right ? > > So not just hardware, it can even make "software" storage layouts faster > & potentially even avoid pool exhaustion in some cases. :) Just a reminder, the underlying unit, fstrim.service, uses 'fstrim --fstab' so only fstab file systems are affected. The user would need to change the unit file to use --all instead of --fstab to affect all mounted file systems. I'll include that info in the change wiki. I imagine the best practice is to copy the original unit file, edit it, and use it as a drop in unit file in /etc ? Unfinished change, still in progress... https://fedoraproject.org/wiki/Changes/EnableFSTrimTimer -- Chris Murphy

1 0

Re: F32, enable fstrim.timer by default
by Chris Murphy 18 Dec '19

18 Dec '19

On Wed, Dec 18, 2019 at 11:10 AM Neal Gompa <ngompa13(a)gmail.com> wrote: > > On Wed, Dec 18, 2019 at 12:08 PM Chris Murphy <lists(a)colorremedies.com> wrote: > > > > On Wed, Dec 18, 2019 at 10:56 AM Chris Murphy <lists(a)colorremedies.com> wrote: > > > > > > One thing I see in the Ubuntu fstrim.service unit file that I'm not > > > seeing in the Fedora fstrim.service unit file, is a conditional for > > > containers (line 4). I'm not sure where to ask about that. Maybe > > > upstream systemd? > > > > Found it. I'm not sure if util-linux would typically be found in a > > container base image? Probably no point in calling fstrim in that > > case, but also doesn't hurt. > > > > $ sudo dnf provides /usr/lib/systemd/system/fstrim.timer > > util-linux-2.34-3.fc31.x86_64 : A collection of basic system utilities > > Repo : @System > > Matched from: > > Filename : /usr/lib/systemd/system/fstrim.timer > > > > Yeah, util-linux is pretty common in some types of containers. It > probably makes sense to send a PR to add that conditional. Appears to be in the upstream version already. -- Chris Murphy

1 0

Re: F32, enable fstrim.timer by default
by Chris Murphy 18 Dec '19

18 Dec '19

readd cloud@ list On Wed, Dec 18, 2019 at 5:32 AM Martin Kolman <mkolman(a)redhat.com> wrote: > > On Wed, 2019-12-18 at 13:11 +0100, Martin Pitt wrote: > > Hello Chris, > > > > Chris Murphy [2019-12-17 22:23 -0700]: > > > This desktop@ thread [1] about a slow device restored by enabling > > > fstrim.service, got me thinking about enabling fstrim.timer [2] by > > > default in Fedora Workstation. But I'm curious if it might be > > > desirable in other Fedora Editions, and making it a system-wide > > > change? > > > > This is a function/property of hardware, so it's IMO not desktop specific at > > all. Servers suffer just as well from hard disks becoming slower. > This will also trim thin LVs on thin pools (if any), right ? Correct. > So not just hardware, it can even make "software" storage layouts faster > & potentially even avoid pool exhaustion in some cases. :) Maybe. I'm not sure either way if someone would actually notice performance improvements, but it wouldn't make them worse. And yes, potentially avoid pool exhaustion, in particular in under provisioned cases. I forgot to mention: with qemu/kvm/libvirt VM's, the trim would not get passed down to the backing storage due to default settings. The discard mode "unmap" is supported with a SCSI disk using virtio SCSI controller; I see some curious works/doesn't work with "plain" virtio disk. But when it works, it does pass down to underlying thinp LV and raw files. If anyone is doing raw file backups, or otherwise paying for storage, it could save them some coins. And when it doesn't work, literally nothing happens: the file doesn't get holes punched out, but there's also no corruption (I test these things with a Btrfs scrub; it will consistently always complain if a single metadata or data checksum mismatches). Anyway, it's an optimization. Pretty well tested elsewhere at this point. And offhand not aware of any liabilities, but thought I'd ask about it before writing up a system wide change proposal. One thing I see in the Ubuntu fstrim.service unit file that I'm not seeing in the Fedora fstrim.service unit file, is a conditional for containers (line 4). I'm not sure where to ask about that. Maybe upstream systemd? $ cat /lib/systemd/system/fstrim.service [Unit] Description=Discard unused blocks on filesystems from /etc/fstab Documentation=man:fstrim(8) ConditionVirtualization=!container [Service] Type=oneshot ExecStart=/sbin/fstrim --fstab --verbose --quiet ProtectSystem=strict ProtectHome=yes PrivateDevices=no PrivateNetwork=yes PrivateUsers=no ProtectKernelTunables=yes ProtectKernelModules=yes ProtectControlGroups=yes MemoryDenyWriteExecute=yes SystemCallFilter=@default @file-system @basic-io @system-service chris@chris-Standard-PC-Q35-ICH9-2009:~$ -- Chris Murphy

1 1

F32, enable fstrim.timer by default
by Chris Murphy 18 Dec '19

18 Dec '19

Hi, This desktop@ thread [1] about a slow device restored by enabling fstrim.service, got me thinking about enabling fstrim.timer [2] by default in Fedora Workstation. But I'm curious if it might be desirable in other Fedora Editions, and making it a system-wide change? I've checked recent versions of openSUSE and Ubuntu, and they have it enabled. Therefore I estimate the likelihood of running into cons (below) is pretty remote. Most people won't notice anything. The pros: + when passed down to flash drives that support trim, it provides a hint to the drive firmware about erase blocks ready for erasure. Some devices will have improved wear leveling and performance as a result, but this is firmware specific. LVM [3] and dm-crypt [4] passdown appears to be enabled by default on Fedora. + with LVM thin provisioning, it will cause unused LV extents to be returned to the thin pool for use by other LVs, kind of a nifty work around for XFS not supporting fs shrink resize. The gotchas: + Few, but highly visible, reports of buggy SSDs that corrupt or lose data soon after trim being issued. By now, most have been blacklisted in the kernel, and/or have manufacturer firmware updates. We shouldn't run into this problem unless someone has older hardware that hasn't been update and for some reason also hasn't been blacklisted in the kernel. + Older SSD's have only non-queued trim support, which also can result in a brief hang while the command is processed. This is highly variable based on the device firmware, and the workload. But using weekly fstrim is preferred for these devices, instead of using the discard mount option in /etc/fstab. + Possible exposure of fs locality pattern may be a security risk for some workflows. [4] [5] [1] https://lists.fedoraproject.org/archives/list/desktop@lists.fedoraproject.o… [2] fstrim.timer, if enabled, runs fstrim.service weekly, specifically Monday at midnight local time; and if the system isn't available at that time, it runs during or very soon after the next startup. The command: ExecStart=/usr/sbin/fstrim --fstab --verbose --quiet fstab means only file systems in fstab are included; verbose reports the mount point and bytes potentially discarded and is recorded in the systemd journal; quiet suppresses errors which is typical for file systems and devices that don't support fstrim, e.g. the EFI System partition, which is FAT16/32; and USB flash "stick" drives, and hard drives. [3] /etc/lvm/lvm.conf, if I'm reading it correctly, file system discards are passed down: # This configuration option has an automatic default value. # thin_pool_discards = "passdown" Due to this Fedora 27 feature; trim is passed down by dm-crypt as well for LUKS volumes. Curiously because Fedora neither sets the discard mount option for any file system, nor enables fstrim.timer, this feature isn't being taken advantage of. [4] https://fedoraproject.org/wiki/Changes/EnableTrimOnDmCrypt [5] Trim on LUKS/dm-crypt note from upstream, section 5.19 https://gitlab.com/cryptsetup/cryptsetup/-/wikis/FrequentlyAskedQuestions#5… -- Chris Murphy

1 0

[Fedocal] Reminder meeting : Fedora Cloud Workgroup
by dusty＠dustymabe.com 16 Dec '19

16 Dec '19

Dear all, You are kindly invited to the meeting: Fedora Cloud Workgroup on 2019-12-18 from 17:00:00 to 18:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Standing meeting for the Fedora Cloud Workgroup Source: https://apps.fedoraproject.org/calendar/meeting/1999/

1 0

[Fedocal] Reminder meeting : Fedora Cloud Workgroup
by dusty＠dustymabe.com 09 Dec '19

09 Dec '19

Dear all, You are kindly invited to the meeting: Fedora Cloud Workgroup on 2019-12-11 from 17:00:00 to 18:00:00 UTC At fedora-meeting-1(a)irc.freenode.net The meeting will be about: Standing meeting for the Fedora Cloud Workgroup Source: https://apps.fedoraproject.org/calendar/meeting/1999/

1 0

Automatic Network Configuration in Cloud with NetworkManager (nm-cloud-setup)
by Thomas Haller 04 Dec '19

04 Dec '19

Hello, We (NetworkManager) are working on a tool that should help configuring NetworkManager in cloud. This is for example for configuring the secondary IP addresses on AWS/EC2 ([1], [2]) [1] https://aws.amazon.com/premiumsupport/knowledge-center/ec2-centos-rhel-seco… [2] https://aws.amazon.com/premiumsupport/knowledge-center/ec2-ubuntu-secondary… It's inspired by and serves a similar purpose as SuSE's cloud-netconfig ([3], [4]) and ec2-net-utils package on Amazon Linux ([5]). [3] https://www.suse.com/c/multi-nic-cloud-netconfig-ec2-azure/ [4] https://github.com/SUSE-Enceladus/cloud-netconfig [5] https://github.com/aws/ec2-net-utils The tool can be triggered via a systemd.timer and/or a NetworkManager dispatcher scripts. Then it fetches meta data from the Instance Metadata Service (thereby detecting whether it runs in a cloud environment). Finally it configures the networking using NetworkManager. Currently it only supports EC2 and IPv4. But if the tool shows to be useful, it could be extended to also support other cloud providers (Azure). There is a merge request on review ([6], [7]). There is a bit of a hurry to get this merged, because it should make the cut for the next NetworkManager release. So, I will merge it soon to master, albeit still in an experimental state. [6] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests… [7] https://bugzilla.redhat.com/show_bug.cgi?id=1642461 The plan is that cloud images could install and enable the tool, so you boot a (Fedora) image and networking would just work. Is there interrest in something like this? Any comments, questions? best, Thomas

2 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

cloud December 2019