lvm2-commits October 2018

lvm2-commits@lists.fedorahosted.org

4 participants
58 discussions

v2_03_01 annotated tag has been created
by Marian Csontos 31 Oct '18

31 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=1326762567c49c7305d83… Commit: 1326762567c49c7305d83ac59dff819d7de964fd Parent: 0000000000000000000000000000000000000000 Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: 2018-10-31 14:42 +0000 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: 2018-10-31 14:42 +0000 annotated tag: v2_03_01 has been created at 1326762567c49c7305d83ac59dff819d7de964fd (tag) tagging 913c28917e62577a2ef67152b2e5159237503dda (commit) replaces v2_03_00 Release 2.03.01 Important bugfix release. This addresses an issue introduced in 2.02.178 where concurrent write at the start of the logical volume and at end of VG metadata might result in overwriting the LV data. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJb2cIFAAoJELkRJDHlCQOfXfgQAMx+8KhR3aji3hAyzfxMqmaM EG7VO4DgoMHkXcGoiB+jiu9fFlJ0fDSExxZmMq/aIAIsJjnOwOwYMy6dCaPNq9nf ocMJtYK4plAs6ImizJoUgyPVlACNxgbxcm6vZIdiwmiXonfwYT26pfA5c7HZYxWt KMbF70vA0g3H8HvS8mcWXV6xmGpmyMl+MfTiMpciSIyJqYTKxXUuEQqBhaPkzPyW g+yWt16FyFseXj5y0poDn0oRqReMECVi9LiqgaLPvQTR9redXTzJVx88cwbFjfkI ATKtRMscAzolHieV7loNZTRrmGAcca1FQOMfj+aDHEpwpk/sqU2E9LRBTCZ2TelQ PS6q0oi1qBaz9+9xdgs9xrFH4t/Q0/TR2O2f/TEnJ/Ar+DNMwId7di+hrOWUERSo LH+qSfUa3XSYswYJFfwMEq4z9ZcMhaG9NSbNgSM9mI06XO2Y1GVnuM2dNe88keCQ au2gOJb79duS0QuaEY4PBrqxDyWTtB0H74UvpGRafSf0d9WRQm4HrfxDtK79+Aww 85ogfaWmAShv7W64rFkF/deXqkWDLu1QG1WSfGok7ClWcJXTA29bEQWjY5YAv/AL UEh116b+hfBJfGXGKczYWCaEP4MhDPPahZ2iaArZ5vIqHdG0DvHteDRphQZYpnKZ Z89xzH6R2YYSzYpW9hbn =wPqe -----END PGP SIGNATURE----- David Teigland (3): fix: cov: missed return value test tests: add new test for lvm on md devices metadata: prevent writing beyond metadata area Heinz Mauelshagen (2): raid: fix left behind SubLVs Revert "raid: fix left behind SubLVs" Marian Csontos (7): post-release build: Use PYTHON env. variable when provided config: Fix version for VDO build: make genrate config file spec: lvmdbusd requires python3 setuptools build: Fix CLDFLAGS default pre-release Zdenek Kabelac (36): cov: missed return value test cov: warn about failing sigaction cov: drop check for pointer cov: dmeventd plugin fix memleak cov: fix memleak on bcache io error path cov: fix leaking openned file descriptors cov: index off by one cov: dm stats missed terminating null cov: fix missing null allocation check cov: add at least ASSERT cov: dmstats check for failing malloc cov: add check for positive value cov: fix typo cov: fix failing filter initialization cov: fix error path cov: dm node message fix missing initilization cov: ensure vars are set cov: check dev_close_immediate cov: check dm_strncpy result cov: check closedir result cov: log failing unlink cov: make sure label scans valid lvinfo cov: add missing error path check for label_scan_open cov: ignore error of vsnprintf cov: pvscan ensure sigle_devs list is always initialized cov: drop uneeded header files cov: avoid selfrecursive inclusion of toolcontext.h revert "cov: dm stats missed terminating null" cov: shutdown warning tests: fix tested targeted tests: timing without /usr/bin/time tests: properly enforce v1 tests: fix conf generation with older shells tests: add raid target presence check tests: better support for /var/run tests: missed char in list

1 0

master - post-release
by Marian Csontos 31 Oct '18

31 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=420af27f088a3808816ba… Commit: 420af27f088a3808816ba3ab5b47dfa400fef308 Parent: 913c28917e62577a2ef67152b2e5159237503dda Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: Wed Oct 31 15:41:22 2018 +0100 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: Wed Oct 31 15:56:41 2018 +0100 post-release --- VERSION | 2 +- VERSION_DM | 2 +- WHATS_NEW | 3 +++ WHATS_NEW_DM | 3 +++ 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/VERSION b/VERSION index 3194d20..83eb504 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.03.01(2) (2018-10-31) +2.03.02(2)-git (2018-10-31) diff --git a/VERSION_DM b/VERSION_DM index d26e766..4866208 100644 --- a/VERSION_DM +++ b/VERSION_DM @@ -1 +1 @@ -1.02.153 (2018-10-31) +1.02.155-git (2018-10-31) diff --git a/WHATS_NEW b/WHATS_NEW index c4f690f..e152027 100644 --- a/WHATS_NEW +++ b/WHATS_NEW @@ -1,3 +1,6 @@ +Version 2.03.02 - +=================================== + Version 2.03.01 - 31st October 2018 =================================== diff --git a/WHATS_NEW_DM b/WHATS_NEW_DM index 193b7b0..7049a4c 100644 --- a/WHATS_NEW_DM +++ b/WHATS_NEW_DM @@ -1,3 +1,6 @@ +Version 1.02.155 - +==================================== + Version 1.02.153 - 31st October 2018 ====================================

1 0

master - pre-release
by Marian Csontos 31 Oct '18

31 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=913c28917e62577a2ef67… Commit: 913c28917e62577a2ef67152b2e5159237503dda Parent: 1e40e734e6da2ee4b20286a132eafe679d5695d1 Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: Wed Oct 31 15:39:50 2018 +0100 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: Wed Oct 31 15:39:50 2018 +0100 pre-release --- VERSION | 2 +- VERSION_DM | 2 +- WHATS_NEW | 2 +- WHATS_NEW_DM | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/VERSION b/VERSION index 749ef5f..3194d20 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.03.01(2)-git (2018-10-10) +2.03.01(2) (2018-10-31) diff --git a/VERSION_DM b/VERSION_DM index d9bb271..d26e766 100644 --- a/VERSION_DM +++ b/VERSION_DM @@ -1 +1 @@ -1.02.153-git (2018-10-10) +1.02.153 (2018-10-31) diff --git a/WHATS_NEW b/WHATS_NEW index a8a9445..c4f690f 100644 --- a/WHATS_NEW +++ b/WHATS_NEW @@ -1,4 +1,4 @@ -Version 2.03.01 - +Version 2.03.01 - 31st October 2018 =================================== Version 2.03.00 - 10th October 2018 diff --git a/WHATS_NEW_DM b/WHATS_NEW_DM index 52ce01b..193b7b0 100644 --- a/WHATS_NEW_DM +++ b/WHATS_NEW_DM @@ -1,4 +1,4 @@ -Version 1.02.153 - +Version 1.02.153 - 31st October 2018 ==================================== Version 1.02.151 - 10th October 2018

1 0

master - build: Fix CLDFLAGS default
by Marian Csontos 31 Oct '18

31 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=1e40e734e6da2ee4b2028… Commit: 1e40e734e6da2ee4b20286a132eafe679d5695d1 Parent: aecf542126640faa17c240afbb1ea61f11355c39 Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: Mon Oct 22 12:56:34 2018 +0200 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: Tue Oct 30 10:15:44 2018 +0100 build: Fix CLDFLAGS default --- configure | 7 +++++-- configure.ac | 7 +++++-- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/configure b/configure index 286733c..d8f7384 100755 --- a/configure +++ b/configure @@ -3067,7 +3067,7 @@ if test -z "$CFLAGS"; then : fi case "$host_os" in linux*) - CLDFLAGS="$CLDFLAGS -Wl,--version-script,.export.sym" + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"} -Wl,--version-script,.export.sym" # equivalent to -rdynamic ELDFLAGS="-Wl,--export-dynamic" # FIXME Generate list and use --dynamic-list=.dlopen.sym @@ -3087,7 +3087,7 @@ case "$host_os" in ;; darwin*) CFLAGS="$CFLAGS -no-cpp-precomp -fno-common" - CLDFLAGS="$CLDFLAGS" + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"}" ELDFLAGS= CLDWHOLEARCHIVE="-all_load" CLDNOWHOLEARCHIVE= @@ -3099,6 +3099,9 @@ case "$host_os" in FSADM=no BLKDEACTIVATE=no ;; + *) + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"}" + ;; esac ################################################################################ diff --git a/configure.ac b/configure.ac index c5d3207..70fd674 100644 --- a/configure.ac +++ b/configure.ac @@ -30,7 +30,7 @@ AC_CANONICAL_TARGET([]) AS_IF([test -z "$CFLAGS"], [COPTIMISE_FLAG="-O2"]) case "$host_os" in linux*) - CLDFLAGS="$CLDFLAGS -Wl,--version-script,.export.sym" + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"} -Wl,--version-script,.export.sym" # equivalent to -rdynamic ELDFLAGS="-Wl,--export-dynamic" # FIXME Generate list and use --dynamic-list=.dlopen.sym @@ -50,7 +50,7 @@ case "$host_os" in ;; darwin*) CFLAGS="$CFLAGS -no-cpp-precomp -fno-common" - CLDFLAGS="$CLDFLAGS" + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"}" ELDFLAGS= CLDWHOLEARCHIVE="-all_load" CLDNOWHOLEARCHIVE= @@ -62,6 +62,9 @@ case "$host_os" in FSADM=no BLKDEACTIVATE=no ;; + *) + CLDFLAGS="${CLDFLAGS:"$LDFLAGS"}" + ;; esac ################################################################################

1 0

v2_02_182 annotated tag has been created
by Marian Csontos 31 Oct '18

31 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=d64377b5bbd2d170f1d63… Commit: d64377b5bbd2d170f1d6365157e2245d90a63c1a Parent: 0000000000000000000000000000000000000000 Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: 2018-10-30 09:10 +0000 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: 2018-10-30 09:10 +0000 annotated tag: v2_02_182 has been created at d64377b5bbd2d170f1d6365157e2245d90a63c1a (tag) tagging b93aded0212a903f8a1e9e897dada1f34aa3de43 (commit) replaces v2_02_180 Release 2.02.182 Important bugfix release. This addresses an issue introduced in 2.02.178 where concurrent write at the start of the logical volume and at end of VG metadata might result in overwriting the LV data. Also MD RAID version 1.0 devices are correctly filtered out preventing them to be seen as duplicate PVs. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJb2CAQAAoJELkRJDHlCQOf9pMP/0QfI9y8xYxzrtvnMhErb/us FBdhgIxMEWkjcs48bY00DQLDWBgxTfngWeU2i01a64yNfo092JEa+UeOel/43KsV cxE9pYK9JyQNak8sXjNXZm2ldUYlkgenEDdw4CF3PEkEmyukdbUicwFE8pAr1bdt ntxG8qWD7mxekurO9ybCZ9Sjhhl6ZwCxuJxfGpYmSLQYmubL5JDGUGRnSo6Xjv9l CxhJ1yhwR0oSBdleH4OHA7zdu+9+12fDt0JQRTXYAGUo0oZXOtzcrboudbp3G5RB I2F4e1cmDShb7nommHF0gaWvzcw1gHuuSGdyVHn4EkEGk8XHVyKTbmadbJNru9pj pmHmX6e3+8gIP3h14QNq+KP4poHQBCCqMsdjbCv9ay1+85v4i1xPUkA20Cgc/KLA S1oWSOAKk6o44bETBwhYNmM03oMtnSdqYDo7rsIT9Qmtd6+/Aeda32Zj4RM9z+Eb NZx21xQ6V+N7d02GmT1Z1XN7BgYBuEzm7iQtmm5TvZBBqcqjC1g3TF8oltUW/ALA dt/H1V5VBZGqA4yu8vPMHwn4PSAW/gh9UFET5obfxWyyBRjFDkM4wBYe5LoovwW9 fG/Bty7u8+LIgyUb/GMUBPtoPOnXCyDHeZ3CBz4BmlASMHqxEi9aap4PBiwpY48I Jq8hEGpeAjv/vVo80xXT =Zp35 -----END PGP SIGNATURE----- Bryn M. Reeves (1): dmsetup: fix error propagation in _display_info_cols() David Teigland (12): lvconvert: restrict command matching for no option variant lvconvert: improve text about splitmirrors vgcreate: close exclusive fd after pvcreate mirrors: fix read_only_volume_list bcache: reduce MAX_IO to 256 lvmetad: improve scan for pvscan all lvmetad: fix pvs for many devices WHATS_NEW: recent fixes scan: use full md filter when md 1.0 devices are present scan: enable full md filter when md 1.0 devices are present tests: add new test for lvm on md devices metadata: prevent writing beyond metadata area Heinz Mauelshagen (8): lvconvert: reject conversions of LVs under snapshot lvconvert: reject conversions on raid1 split trackchanges SubLVs lvconvert: reject conversions on raid1 split trackchanges LVs lvconvert: fix regression preventing direct striped conversion lvconvert: fix conversion attempts to linear test: add striped -> raid0 test script lvconvert: avoid superfluous interim raid type lvconvert: fix interim segtype regression on raid6 conversions Marian Csontos (15): post-release test: Check flavour is used and exists Add BSD 2-Clause License WHATS_NEW build: make generate pre-release pre-release post-release Merge branch '2018-06-01-stable' of git://sourceware.org/git/lvm2 into 2018-06-01-stable spec: Add vdo plugin for dmeventd spec: Disable python bindings on newer versions Update WHATS_NEW spec: Fix python and applib interactions Update WHATS_NEW pre-release Peter Rajnoha (1): scripts: add After=rbdmap.service to {lvm2-activation-net,blk-availability}.service Zdenek Kabelac (8): dmeventd: base vdo plugin dmeventd: rebase to stable branch cache: drop metadata_format validation mirror: fix splitmirrors for mirror type tests: splitmirror for mirror type tests: check policy mq can be used with format2 dmeventd: lvm2 plugin uses envvar registry tests: check activation of many thin-pool

1 0

master - metadata: prevent writing beyond metadata area
by David Teigland 29 Oct '18

29 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=aecf542126640faa17c24… Commit: aecf542126640faa17c240afbb1ea61f11355c39 Parent: 8df2dd66ce53817b250f5dd6bd05fda3a38ac26e Author: David Teigland <teigland(a)redhat.com> AuthorDate: Mon Oct 29 16:53:17 2018 -0500 Committer: David Teigland <teigland(a)redhat.com> CommitterDate: Mon Oct 29 16:53:17 2018 -0500 metadata: prevent writing beyond metadata area lvm uses a bcache block size of 128K. A bcache block at the end of the metadata area will overlap the PEs from which LVs are allocated. How much depends on alignments. When lvm reads and writes one of these bcache blocks to update VG metadata, it can also be reading and writing PEs that belong to an LV. If these overlapping PEs are being written to by the LV user (e.g. filesystem) at the same time that lvm is modifying VG metadata in the overlapping bcache block, then the user's updates to the PEs can be lost. This patch is a quick hack to prevent lvm from writing past the end of the metadata area. --- lib/device/bcache.c | 79 +++++++++++++++++++++++++++++++++++++++- lib/device/bcache.h | 3 ++ lib/format_text/format-text.c | 10 +++++ lib/label/label.c | 35 +++++++++++++++++- lib/label/label.h | 2 + lib/metadata/mirror.c | 4 ++ 6 files changed, 130 insertions(+), 3 deletions(-) diff --git a/lib/device/bcache.c b/lib/device/bcache.c index 5e957fe..0183079 100644 --- a/lib/device/bcache.c +++ b/lib/device/bcache.c @@ -157,6 +157,10 @@ static void _async_destroy(struct io_engine *ioe) free(e); } +static int _last_byte_fd; +static uint64_t _last_byte_offset; +static int _last_byte_sector_size; + static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, sector_t sb, sector_t se, void *data, void *context) { @@ -164,12 +168,53 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, struct iocb *cb_array[1]; struct control_block *cb; struct async_engine *e = _to_async(ioe); + sector_t offset; + sector_t nbytes; + sector_t limit_nbytes; + sector_t extra_nbytes = 0; if (((uintptr_t) data) & e->page_mask) { log_warn("misaligned data buffer"); return false; } + offset = sb << SECTOR_SHIFT; + nbytes = (se - sb) << SECTOR_SHIFT; + + /* + * If bcache block goes past where lvm wants to write, then clamp it. + */ + if ((d == DIR_WRITE) && _last_byte_offset && (fd == _last_byte_fd)) { + if (offset > _last_byte_offset) { + log_error("Limit write at %llu len %llu beyond last byte %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)_last_byte_offset); + return false; + } + + if (offset + nbytes > _last_byte_offset) { + limit_nbytes = _last_byte_offset - offset; + if (limit_nbytes % _last_byte_sector_size) + extra_nbytes = _last_byte_sector_size - (limit_nbytes % _last_byte_sector_size); + + if (extra_nbytes) { + log_debug("Limit write at %llu len %llu to len %llu rounded to %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes, + (unsigned long long)(limit_nbytes + extra_nbytes)); + nbytes = limit_nbytes + extra_nbytes; + } else { + log_debug("Limit write at %llu len %llu to len %llu", + (unsigned long long)offset, + (unsigned long long)nbytes, + (unsigned long long)limit_nbytes); + nbytes = limit_nbytes; + } + } + } + cb = _cb_alloc(e->cbs, context); if (!cb) { log_warn("couldn't allocate control block"); @@ -180,10 +225,22 @@ static bool _async_issue(struct io_engine *ioe, enum dir d, int fd, cb->cb.aio_fildes = (int) fd; cb->cb.u.c.buf = data; - cb->cb.u.c.offset = sb << SECTOR_SHIFT; - cb->cb.u.c.nbytes = (se - sb) << SECTOR_SHIFT; + cb->cb.u.c.offset = offset; + cb->cb.u.c.nbytes = nbytes; cb->cb.aio_lio_opcode = (d == DIR_READ) ? IO_CMD_PREAD : IO_CMD_PWRITE; +#if 0 + if (d == DIR_READ) { + log_debug("io R off %llu bytes %llu", + (unsigned long long)cb->cb.u.c.offset, + (unsigned long long)cb->cb.u.c.nbytes); + } else { + log_debug("io W off %llu bytes %llu", + (unsigned long long)cb->cb.u.c.offset, + (unsigned long long)cb->cb.u.c.nbytes); + } +#endif + cb_array[0] = &cb->cb; do { r = io_submit(e->aio_context, 1, cb_array); @@ -1174,3 +1231,21 @@ bool bcache_invalidate_fd(struct bcache *cache, int fd) //---------------------------------------------------------------- +void bcache_set_last_byte(struct bcache *cache, int fd, uint64_t offset, int sector_size) +{ + _last_byte_fd = fd; + _last_byte_offset = offset; + _last_byte_sector_size = sector_size; + if (!sector_size) + _last_byte_sector_size = 512; +} + +void bcache_unset_last_byte(struct bcache *cache, int fd) +{ + if (_last_byte_fd == fd) { + _last_byte_fd = 0; + _last_byte_offset = 0; + _last_byte_sector_size = 0; + } +} + diff --git a/lib/device/bcache.h b/lib/device/bcache.h index 1d49af2..2edc54a 100644 --- a/lib/device/bcache.h +++ b/lib/device/bcache.h @@ -158,6 +158,9 @@ bool bcache_write_bytes(struct bcache *cache, int fd, uint64_t start, size_t len bool bcache_zero_bytes(struct bcache *cache, int fd, uint64_t start, size_t len); bool bcache_set_bytes(struct bcache *cache, int fd, uint64_t start, size_t len, uint8_t val); +void bcache_set_last_byte(struct bcache *cache, int fd, uint64_t offset, int sector_size); +void bcache_unset_last_byte(struct bcache *cache, int fd); + //---------------------------------------------------------------- #endif diff --git a/lib/format_text/format-text.c b/lib/format_text/format-text.c index b41f2b2..7fba881 100644 --- a/lib/format_text/format-text.c +++ b/lib/format_text/format-text.c @@ -387,10 +387,14 @@ static int _raw_write_mda_header(const struct format_type *fmt, MDA_HEADER_SIZE - sizeof(mdah->checksum_xl))); + dev_set_last_byte(dev, start_byte + MDA_HEADER_SIZE); + if (!dev_write_bytes(dev, start_byte, MDA_HEADER_SIZE, mdah)) { + dev_unset_last_byte(dev); log_error("Failed to write mda header to %s fd %d", dev_name(dev), dev->bcache_fd); return 0; } + dev_unset_last_byte(dev); return 1; } @@ -899,9 +903,12 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, (unsigned long long)(rlocn_new->size - new_wrap), (unsigned long long)new_wrap); + dev_set_last_byte(mdac->area.dev, mdac->area.start + mdah->size); + if (!dev_write_bytes(mdac->area.dev, mdac->area.start + rlocn_new->offset, (size_t) (rlocn_new->size - new_wrap), new_buf)) { log_error("Failed to write metadata to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); + dev_unset_last_byte(mdac->area.dev); goto out; } @@ -914,10 +921,13 @@ static int _vg_write_raw(struct format_instance *fid, struct volume_group *vg, if (!dev_write_bytes(mdac->area.dev, mdac->area.start + MDA_HEADER_SIZE, (size_t) new_wrap, new_buf + rlocn_new->size - new_wrap)) { log_error("Failed to write metadata wrap to %s fd %d", dev_name(mdac->area.dev), mdac->area.dev->bcache_fd); + dev_unset_last_byte(mdac->area.dev); goto out; } } + dev_unset_last_byte(mdac->area.dev); + rlocn_new->checksum = calc_crc(INITIAL_CRC, (uint8_t *)new_buf, (uint32_t)(rlocn_new->size - new_wrap)); diff --git a/lib/label/label.c b/lib/label/label.c index 22b6bc2..c92d2cf 100644 --- a/lib/label/label.c +++ b/lib/label/label.c @@ -174,6 +174,7 @@ int label_write(struct device *dev, struct label *label) { char buf[LABEL_SIZE] __attribute__((aligned(8))); struct label_header *lh = (struct label_header *) buf; + uint64_t offset; int r = 1; if (!label->labeller->ops->write) { @@ -208,11 +209,17 @@ int label_write(struct device *dev, struct label *label) return 0; } - if (!dev_write_bytes(dev, label->sector << SECTOR_SHIFT, LABEL_SIZE, buf)) { + offset = label->sector << SECTOR_SHIFT; + + dev_set_last_byte(dev, offset + LABEL_SIZE); + + if (!dev_write_bytes(dev, offset, LABEL_SIZE, buf)) { log_debug_devs("Failed to write label to %s", dev_name(dev)); r = 0; } + dev_unset_last_byte(dev); + return r; } @@ -1256,9 +1263,12 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) } } + dev_set_last_byte(dev, start + len); + if (!bcache_zero_bytes(scan_bcache, dev->bcache_fd, start, len)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } @@ -1266,9 +1276,11 @@ bool dev_write_zeros(struct device *dev, uint64_t start, size_t len) if (!bcache_flush(scan_bcache)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } + dev_unset_last_byte(dev); return true; } @@ -1302,9 +1314,12 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) } } + dev_set_last_byte(dev, start + len); + if (!bcache_set_bytes(scan_bcache, dev->bcache_fd, start, len, val)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } @@ -1312,9 +1327,27 @@ bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val) if (!bcache_flush(scan_bcache)) { log_error("Error writing device %s at %llu length %u.", dev_name(dev), (unsigned long long)start, (uint32_t)len); + dev_unset_last_byte(dev); label_scan_invalidate(dev); return false; } + + dev_unset_last_byte(dev); return true; } +void dev_set_last_byte(struct device *dev, uint64_t offset) +{ + unsigned int phys_block_size = 0; + unsigned int block_size = 0; + + dev_get_block_size(dev, &phys_block_size, &block_size); + + bcache_set_last_byte(scan_bcache, dev->bcache_fd, offset, phys_block_size); +} + +void dev_unset_last_byte(struct device *dev) +{ + bcache_unset_last_byte(scan_bcache, dev->bcache_fd); +} + diff --git a/lib/label/label.h b/lib/label/label.h index 447fc86..5939655 100644 --- a/lib/label/label.h +++ b/lib/label/label.h @@ -124,5 +124,7 @@ bool dev_read_bytes(struct device *dev, uint64_t start, size_t len, void *data); bool dev_write_bytes(struct device *dev, uint64_t start, size_t len, void *data); bool dev_write_zeros(struct device *dev, uint64_t start, size_t len); bool dev_set_bytes(struct device *dev, uint64_t start, size_t len, uint8_t val); +void dev_set_last_byte(struct device *dev, uint64_t offset); +void dev_unset_last_byte(struct device *dev); #endif diff --git a/lib/metadata/mirror.c b/lib/metadata/mirror.c index c86ff90..ba3f33d 100644 --- a/lib/metadata/mirror.c +++ b/lib/metadata/mirror.c @@ -302,10 +302,14 @@ static int _write_log_header(struct cmd_context *cmd, struct logical_volume *lv) return 0; } + dev_set_last_byte(dev, sizeof(log_header)); + if (!dev_write_bytes(dev, UINT64_C(0), sizeof(log_header), &log_header)) { + dev_unset_last_byte(dev); log_error("Failed to write log header to %s.", name); return 0; } + dev_unset_last_byte(dev); label_scan_invalidate(dev);

1 0

master - Revert "raid: fix left behind SubLVs"
by Heinz Mauelshagen 25 Oct '18

25 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=8df2dd66ce53817b250f5… Commit: 8df2dd66ce53817b250f5dd6bd05fda3a38ac26e Parent: 16ae968d24b4fe3264dc9b46063345ff2846957b Author: Heinz Mauelshagen <heinzm(a)redhat.com> AuthorDate: Thu Oct 25 14:30:32 2018 +0200 Committer: Heinz Mauelshagen <heinzm(a)redhat.com> CommitterDate: Thu Oct 25 14:35:56 2018 +0200 Revert "raid: fix left behind SubLVs" This reverts commit 16ae968d24b4fe3264dc9b46063345ff2846957b. We need to come up with a better fix, because we fall short wiping all known signatures when not using the wipe_lv API. --- lib/metadata/raid_manip.c | 140 +++++++++++++++++++++------------------------ 1 files changed, 66 insertions(+), 74 deletions(-) diff --git a/lib/metadata/raid_manip.c b/lib/metadata/raid_manip.c index 25960a3..3944dc4 100644 --- a/lib/metadata/raid_manip.c +++ b/lib/metadata/raid_manip.c @@ -689,90 +689,86 @@ static int _lv_update_and_reload_list(struct logical_volume *lv, int origin_only return r; } -/* - * HM Helper - * - * clear first @sectors of @lv - * - * Presuming we are holding an exclusive lock, we can clear the first - * @sectors of the (metadata) @lv directly on the respective PE(s) thus - * avoiding write+commit+activation of @lv altogether and hence superfluous - * latencies or left behind visible SubLVs on a command/system crash. +/* Makes on-disk metadata changes + * If LV is active: + * clear first block of device + * otherwise: + * activate, clear, deactivate * * Returns: 1 on success, 0 on failure - * - * HM FIXME: share with lv_manip.c! */ -static int _clear_lv(struct logical_volume *lv, uint32_t sectors) +static int _clear_lvs(struct dm_list *lv_list) { - struct lv_segment *seg; - struct physical_volume *pv; - uint64_t offset; - uint32_t cur_sectors; + struct lv_list *lvl; + struct volume_group *vg = NULL; + unsigned i = 0, sz = dm_list_size(lv_list); + char *was_active; + int r = 1; - if (test_mode()) + if (!sz) { + log_debug_metadata(INTERNAL_ERROR "Empty list of LVs given for clearing."); return 1; + } - if (!sectors) - return_0; + dm_list_iterate_items(lvl, lv_list) { + if (!lv_is_visible(lvl->lv)) { + log_error(INTERNAL_ERROR + "LVs must be set visible before clearing."); + return 0; + } + vg = lvl->lv->vg; + } + + if (test_mode()) + return 1; /* - * Rather than wiping lv->size, we can simply wipe the first 4KiB - * to remove the superblock of any previous RAID devices. It is much - * quicker than wiping a potentially larger metadata device completely. + * FIXME: only vg_[write|commit] if LVs are not already written + * as visible in the LVM metadata (which is never the case yet). */ - log_verbose("Clearing metadata area of %s.", display_lvname(lv)); - - dm_list_iterate_items(seg, &lv->segments) { - if (seg_type(seg, 0) != AREA_PV) - return_0; - if (seg->area_count != 1) - return_0; - if (!(pv = seg_pv(seg, 0))) - return_0; - if (!pv->pe_start) /* Be careful */ - return_0; - - offset = (pv->pe_start + seg_pe(seg, 0) * pv->pe_size) << 9; - cur_sectors = min(sectors, pv->pe_size); - sectors -= cur_sectors; - if (!dev_set(pv->dev, offset, cur_sectors << 9, DEV_IO_LOG, 0)) - return_0; - - if (!sectors) - break; - } - - return 1; -} + if (!vg || !vg_write(vg) || !vg_commit(vg)) + return_0; -/* - * HM Helper: - * - * wipe all LVs first sector on @lv_list avoiding metadata commit/activation. - * - * Returns 1 on success or 0 on failure - * - * HM FIXME: share with lv_manip.c! - */ -static int _clear_lvs(struct dm_list *lv_list) -{ - struct lv_list *lvl; + was_active = alloca(sz); - if (test_mode()) - return 1; + dm_list_iterate_items(lvl, lv_list) + if (!(was_active[i++] = lv_is_active(lvl->lv))) { + lvl->lv->status |= LV_TEMPORARY; + if (!activate_lv(vg->cmd, lvl->lv)) { + log_error("Failed to activate localy %s for clearing.", + display_lvname(lvl->lv)); + r = 0; + goto out; + } + lvl->lv->status &= ~LV_TEMPORARY; + } - if (!dm_list_size(lv_list)) { - log_debug_metadata(INTERNAL_ERROR "Empty list of LVs given for clearing."); - return 1; + dm_list_iterate_items(lvl, lv_list) { + log_verbose("Clearing metadata area %s.", display_lvname(lvl->lv)); + /* + * Rather than wiping lv->size, we can simply + * wipe the first sector to remove the superblock of any previous + * RAID devices. It is much quicker. + */ + if (!wipe_lv(lvl->lv, (struct wipe_params) { .do_zero = 1, .zero_sectors = 1 })) { + log_error("Failed to zero %s.", display_lvname(lvl->lv)); + r = 0; + goto out; + } } - - /* Walk list and clear first sector of each LV */ +out: + /* TODO: deactivation is only needed with clustered locking + * in normal case we should keep device active + */ + sz = 0; dm_list_iterate_items(lvl, lv_list) - if (!_clear_lv(lvl->lv, 1)) - return 0; + if ((i > sz) && !was_active[sz++] && + !deactivate_lv(vg->cmd, lvl->lv)) { + log_error("Failed to deactivate %s.", display_lvname(lvl->lv)); + r = 0; /* continue deactivating */ + } - return 1; + return r; } /* raid0* <-> raid10_near area reorder helper: swap 2 LV segment areas @a1 and @a2 */ @@ -5507,12 +5503,8 @@ static int _takeover_upconvert_wrapper(TAKEOVER_FN_ARGS) if (segtype_is_striped_target(initial_segtype) && !_convert_raid0_to_striped(lv, 0, &removal_lvs)) return_0; - if (!dm_list_empty(&removal_lvs)) { - if (!vg_write(lv->vg) || !vg_commit(lv->vg)) - return_0; - if (!_eliminate_extracted_lvs(lv->vg, &removal_lvs)) /* Updates vg */ - return_0; - } + if (!_eliminate_extracted_lvs(lv->vg, &removal_lvs)) /* Updates vg */ + return_0; return_0; }

1 0

master - raid: fix left behind SubLVs
by Heinz Mauelshagen 24 Oct '18

24 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=16ae968d24b4fe3264dc9… Commit: 16ae968d24b4fe3264dc9b46063345ff2846957b Parent: fc35a9169e9f5804e38e4a6a6a7bf3555c49b636 Author: Heinz Mauelshagen <heinzm(a)redhat.com> AuthorDate: Wed Oct 24 15:26:19 2018 +0200 Committer: Heinz Mauelshagen <heinzm(a)redhat.com> CommitterDate: Wed Oct 24 16:35:30 2018 +0200 raid: fix left behind SubLVs lvm metadata writes, commits and activations are performed for (newly) allocated RAID metadata SubLVs to wipe any preexisiting data thus avoid false raid superblock positives on RaidLV activation. This process can be interrupted by command or system crashs thus leaving stale SubLVs in the lvm metadata as a problem. Because we hold an exclusive lock in this metadata SubLV wiping process, we can address this problem by avoiding aforementioned commits/writes/activations altogether wiping the respective first sector of the first physical extent allocated to any metadata SubLV directly via the existing dev_set() API. Succeeds all LVM RAID tests. Related: rhbz1633167 --- lib/metadata/raid_manip.c | 140 ++++++++++++++++++++++++--------------------- 1 files changed, 74 insertions(+), 66 deletions(-) diff --git a/lib/metadata/raid_manip.c b/lib/metadata/raid_manip.c index 3944dc4..25960a3 100644 --- a/lib/metadata/raid_manip.c +++ b/lib/metadata/raid_manip.c @@ -689,86 +689,90 @@ static int _lv_update_and_reload_list(struct logical_volume *lv, int origin_only return r; } -/* Makes on-disk metadata changes - * If LV is active: - * clear first block of device - * otherwise: - * activate, clear, deactivate +/* + * HM Helper + * + * clear first @sectors of @lv + * + * Presuming we are holding an exclusive lock, we can clear the first + * @sectors of the (metadata) @lv directly on the respective PE(s) thus + * avoiding write+commit+activation of @lv altogether and hence superfluous + * latencies or left behind visible SubLVs on a command/system crash. * * Returns: 1 on success, 0 on failure + * + * HM FIXME: share with lv_manip.c! */ -static int _clear_lvs(struct dm_list *lv_list) +static int _clear_lv(struct logical_volume *lv, uint32_t sectors) { - struct lv_list *lvl; - struct volume_group *vg = NULL; - unsigned i = 0, sz = dm_list_size(lv_list); - char *was_active; - int r = 1; - - if (!sz) { - log_debug_metadata(INTERNAL_ERROR "Empty list of LVs given for clearing."); - return 1; - } - - dm_list_iterate_items(lvl, lv_list) { - if (!lv_is_visible(lvl->lv)) { - log_error(INTERNAL_ERROR - "LVs must be set visible before clearing."); - return 0; - } - vg = lvl->lv->vg; - } + struct lv_segment *seg; + struct physical_volume *pv; + uint64_t offset; + uint32_t cur_sectors; if (test_mode()) return 1; + if (!sectors) + return_0; + /* - * FIXME: only vg_[write|commit] if LVs are not already written - * as visible in the LVM metadata (which is never the case yet). + * Rather than wiping lv->size, we can simply wipe the first 4KiB + * to remove the superblock of any previous RAID devices. It is much + * quicker than wiping a potentially larger metadata device completely. */ - if (!vg || !vg_write(vg) || !vg_commit(vg)) - return_0; + log_verbose("Clearing metadata area of %s.", display_lvname(lv)); - was_active = alloca(sz); + dm_list_iterate_items(seg, &lv->segments) { + if (seg_type(seg, 0) != AREA_PV) + return_0; + if (seg->area_count != 1) + return_0; + if (!(pv = seg_pv(seg, 0))) + return_0; + if (!pv->pe_start) /* Be careful */ + return_0; - dm_list_iterate_items(lvl, lv_list) - if (!(was_active[i++] = lv_is_active(lvl->lv))) { - lvl->lv->status |= LV_TEMPORARY; - if (!activate_lv(vg->cmd, lvl->lv)) { - log_error("Failed to activate localy %s for clearing.", - display_lvname(lvl->lv)); - r = 0; - goto out; - } - lvl->lv->status &= ~LV_TEMPORARY; - } + offset = (pv->pe_start + seg_pe(seg, 0) * pv->pe_size) << 9; + cur_sectors = min(sectors, pv->pe_size); + sectors -= cur_sectors; + if (!dev_set(pv->dev, offset, cur_sectors << 9, DEV_IO_LOG, 0)) + return_0; - dm_list_iterate_items(lvl, lv_list) { - log_verbose("Clearing metadata area %s.", display_lvname(lvl->lv)); - /* - * Rather than wiping lv->size, we can simply - * wipe the first sector to remove the superblock of any previous - * RAID devices. It is much quicker. - */ - if (!wipe_lv(lvl->lv, (struct wipe_params) { .do_zero = 1, .zero_sectors = 1 })) { - log_error("Failed to zero %s.", display_lvname(lvl->lv)); - r = 0; - goto out; - } + if (!sectors) + break; } -out: - /* TODO: deactivation is only needed with clustered locking - * in normal case we should keep device active - */ - sz = 0; + + return 1; +} + +/* + * HM Helper: + * + * wipe all LVs first sector on @lv_list avoiding metadata commit/activation. + * + * Returns 1 on success or 0 on failure + * + * HM FIXME: share with lv_manip.c! + */ +static int _clear_lvs(struct dm_list *lv_list) +{ + struct lv_list *lvl; + + if (test_mode()) + return 1; + + if (!dm_list_size(lv_list)) { + log_debug_metadata(INTERNAL_ERROR "Empty list of LVs given for clearing."); + return 1; + } + + /* Walk list and clear first sector of each LV */ dm_list_iterate_items(lvl, lv_list) - if ((i > sz) && !was_active[sz++] && - !deactivate_lv(vg->cmd, lvl->lv)) { - log_error("Failed to deactivate %s.", display_lvname(lvl->lv)); - r = 0; /* continue deactivating */ - } + if (!_clear_lv(lvl->lv, 1)) + return 0; - return r; + return 1; } /* raid0* <-> raid10_near area reorder helper: swap 2 LV segment areas @a1 and @a2 */ @@ -5503,8 +5507,12 @@ static int _takeover_upconvert_wrapper(TAKEOVER_FN_ARGS) if (segtype_is_striped_target(initial_segtype) && !_convert_raid0_to_striped(lv, 0, &removal_lvs)) return_0; - if (!_eliminate_extracted_lvs(lv->vg, &removal_lvs)) /* Updates vg */ - return_0; + if (!dm_list_empty(&removal_lvs)) { + if (!vg_write(lv->vg) || !vg_commit(lv->vg)) + return_0; + if (!_eliminate_extracted_lvs(lv->vg, &removal_lvs)) /* Updates vg */ + return_0; + } return_0; }

1 0

master - spec: lvmdbusd requires python3 setuptools
by Marian Csontos 22 Oct '18

22 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=fc35a9169e9f5804e38e4… Commit: fc35a9169e9f5804e38e4a6a6a7bf3555c49b636 Parent: 7498f8383397a93db95655ca227257836cbcac82 Author: Marian Csontos <mcsontos(a)redhat.com> AuthorDate: Mon Oct 22 12:51:51 2018 +0200 Committer: Marian Csontos <mcsontos(a)redhat.com> CommitterDate: Mon Oct 22 12:51:51 2018 +0200 spec: lvmdbusd requires python3 setuptools --- spec/source.inc | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/spec/source.inc b/spec/source.inc index 4561e8a..8309c3f 100644 --- a/spec/source.inc +++ b/spec/source.inc @@ -82,7 +82,7 @@ %if %{enable_dbusd} %global buildreq_python3_devel python3-devel - %global buildreq_python_setuptools python-setuptools + %global buildreq_python3_setuptools python3-setuptools %global buildreq_python3_dbus python3-dbus %global buildreq_python3_pyudev python3-pyudev %endif @@ -168,7 +168,7 @@ BuildRequires: pkgconfig %maybe BuildRequires: %{?buildreq_lvmlockd_dlm} %maybe BuildRequires: %{?buildreq_lvmlockd_sanlock} %maybe BuildRequires: %{?buildreq_python3_devel} -%maybe BuildRequires: %{?buildreq_python_setuptools} +%maybe BuildRequires: %{?buildreq_python3_setuptools} %description LVM2 includes all of the support for handling read/write operations on

1 0

master - tests: add new test for lvm on md devices
by David Teigland 18 Oct '18

18 Oct '18

Gitweb: https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=7498f8383397a93db9565… Commit: 7498f8383397a93db95655ca227257836cbcac82 Parent: 9e1ee07d696fb0a1771f7dcd4490ed9ace0fa8d6 Author: David Teigland <teigland(a)redhat.com> AuthorDate: Thu Oct 18 13:06:42 2018 -0500 Committer: David Teigland <teigland(a)redhat.com> CommitterDate: Thu Oct 18 13:06:42 2018 -0500 tests: add new test for lvm on md devices --- test/shell/lvm-on-md.sh | 87 +++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 87 insertions(+), 0 deletions(-) diff --git a/test/shell/lvm-on-md.sh b/test/shell/lvm-on-md.sh new file mode 100644 index 0000000..ec8cc23 --- /dev/null +++ b/test/shell/lvm-on-md.sh @@ -0,0 +1,87 @@ +#!/usr/bin/env bash + +# Copyright (C) 2018 Red Hat, Inc. All rights reserved. +# +# This copyrighted material is made available to anyone wishing to use, +# modify, copy, or redistribute it subject to the terms and conditions +# of the GNU General Public License v.2. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software Foundation, +# Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + +SKIP_WITH_LVMPOLLD=1 + +. lib/inittest + +test -f /proc/mdstat && grep -q raid1 /proc/mdstat || \ + modprobe raid1 || skip + +aux lvmconf 'devices/md_component_detection = 1' +aux extend_filter_LVMTEST "a|/dev/md|" + +aux prepare_devs 2 + +# create 2 disk MD raid1 array +# by default using metadata format 1.0 with data at the end of device +aux prepare_md_dev 1 64 2 "$dev1" "$dev2" + +mddev=$(< MD_DEV) +pvdev=$(< MD_DEV_PV) + +vgcreate $vg "$mddev" + +lvs $vg + +lvcreate -n $lv1 -l 2 $vg +lvcreate -n $lv2 -l 2 -an $vg + +lvchange -ay $vg/$lv2 + +lvs $vg + +pvs -vvvv 2>&1|tee pvs.out + +vgchange -an $vg + +vgchange -ay -vvvv $vg 2>&1| tee vgchange.out + +lvs $vg +pvs + +vgchange -an $vg + +mdadm --stop "$mddev" + +# with md superblock 1.0 this pvs will report duplicates +# for the two md legs since the md device itself is not +# started +pvs 2>&1 |tee out +cat out +grep "prefers device" out + +pvs -vvvv 2>&1| tee pvs2.out + +# should not activate from the md legs +not vgchange -ay -vvvv $vg 2>&1|tee vgchange-fail.out + +# should not show an active lv +lvs $vg + +# start the md dev +mdadm --assemble "$mddev" "$dev1" "$dev2" + +# Now that the md dev is online, pvs can see it and +# ignore the two legs, so there's no duplicate warning + +pvs 2>&1 |tee out +cat out +not grep "prefers device" out + +vgchange -ay $vg 2>&1 |tee out +cat out +not grep "prefers device" out + +vgchange -an $vg + +vgremove -f $vg

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

lvm2-commits October 2018