From: Michael Hofmann <mhofmann(a)redhat.com>
gitlab-ci: use CI templates from production branch
The kernel_templates.yml file is pulled directly from
pipeline-definition in the parent pipeline. While CKI can test new
versions of the child pipelines via the cki-bot instrumentation, that is
not possible for anything related to the parent pipeline.
To at least make it possible to test-run changes before merging an MR in
pipeline-definition, pull the kernel_templates.yml file from the
production branch. This branch can be updated via a manual job from
pipeline-definition MRs and easily rolled back without going through a
full git revert cycle.
Bugzilla: INTERNAL
Upstream Status: RHEL-only
Signed-off-by: Michael Hofmann <mhofmann(a)redhat.com>
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index blahblah..blahblah 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -6,7 +6,7 @@
include:
- project: cki-project/pipeline-definition
- ref: main
+ ref: production
file: kernel_templates.yml
stages:
--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2265
From: Justin M. Forbes <jforbes(a)fedoraproject.org>
Turn off forced debug builds
For many years, rawhide kernels have forced users to run debug builds on
git snapshots by not building a non debug kernel as an option. While
this has served us well, in finding occasional bugs that are less likely
to surface otherwise, the performance of debug kernels has gotten
considerably worse over time. After evaluating the debug configs to see
if performance could be improved, it has become clear that we would have
to trade off too much to regain performance. We would be better served
by leaving more debug options enabled and offering a non debug kernel
for all users on all builds.
Signed-off-by: Justin M. Forbes <jforbes(a)fedoraproject.org>
diff --git a/redhat/scripts/genspec/genspec.sh b/redhat/scripts/genspec/genspec.sh
index blahblah..blahblah 100755
--- a/redhat/scripts/genspec/genspec.sh
+++ b/redhat/scripts/genspec/genspec.sh
@@ -4,14 +4,11 @@
UPSTREAM=$(git rev-parse -q --verify origin/"${UPSTREAM_BRANCH}" || \
git rev-parse -q --verify "${UPSTREAM_BRANCH}")
-if [ "$SNAPSHOT" = 0 ]; then
- # This is based off a tag on Linus's tree (e.g. v5.5 or v5.5-rc5).
- # Two kernels are built, one with debug configuration and one without.
- SPECDEBUG_BUILDS_ENABLED=1
-else
- # All kernels are built with debug configurations.
- SPECDEBUG_BUILDS_ENABLED=0
-fi
+# As debug kernels have gotten a bit slower over time, the forced debug
+# builds are going unused. We are no longer forcing debug builds only.
+# Keep the option around for one off scratch builds though.
+# Two kernels are built, one with debug configuration and one without.
+SPECDEBUG_BUILDS_ENABLED=1
if [ -n "$DISTLOCALVERSION" ]; then
SPECBUILDID=$(printf "%%define buildid %s" "$DISTLOCALVERSION")
--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2263
From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240
The -Werror flag should be enabled for RHEL. In ARK, however, it is a bit
tricky to enable -Werror as a build bug occasionally leaks through
upstream review and into Linus' tree, and the toolchain is rebased which
causes many new errors to be reported. As a consequence, -Werror cannot
be enabled in ARK but the code to enable it for RHEL can be added to the
top-level makefile.
Additional fixes: Alphabetize Makefile.variables (something I've wanted to
do for a while now) and remove a mention of Makefile.rhkpg.
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
---
redhat/configs/build_configs.sh | 5 +
redhat/self-test/data/centos-2585cf9dfaad.el7 | 1 +
redhat/self-test/data/centos-2585cf9dfaad.fc25 | 1 +
redhat/self-test/data/centos-78e36f3b0dae.el7 | 1 +
redhat/self-test/data/centos-78e36f3b0dae.fc25 | 1 +
redhat/self-test/data/centos-df0cc57e057f.el7 | 1 +
redhat/self-test/data/centos-df0cc57e057f.fc25 | 1 +
redhat/self-test/data/centos-fce15c45d3fb.el7 | 1 +
redhat/self-test/data/centos-fce15c45d3fb.fc25 | 1 +
redhat/self-test/data/fedora-2585cf9dfaad.el7 | 1 +
redhat/self-test/data/fedora-2585cf9dfaad.fc25 | 1 +
redhat/self-test/data/fedora-78e36f3b0dae.el7 | 1 +
redhat/self-test/data/fedora-78e36f3b0dae.fc25 | 1 +
redhat/self-test/data/fedora-df0cc57e057f.el7 | 1 +
redhat/self-test/data/fedora-df0cc57e057f.fc25 | 1 +
redhat/self-test/data/fedora-fce15c45d3fb.el7 | 1 +
redhat/self-test/data/fedora-fce15c45d3fb.fc25 | 1 +
redhat/self-test/data/rhel-2585cf9dfaad.el7 | 1 +
redhat/self-test/data/rhel-2585cf9dfaad.fc25 | 1 +
redhat/self-test/data/rhel-78e36f3b0dae.el7 | 1 +
redhat/self-test/data/rhel-78e36f3b0dae.fc25 | 1 +
redhat/self-test/data/rhel-df0cc57e057f.el7 | 1 +
redhat/self-test/data/rhel-df0cc57e057f.fc25 | 1 +
redhat/self-test/data/rhel-fce15c45d3fb.el7 | 1 +
redhat/self-test/data/rhel-fce15c45d3fb.fc25 | 1 +
redhat/Makefile.variables | 184 ++++++++++++------------
makefile | 7 +
27 files changed, 129 insertions(+), 91 deletions(-)
From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240
The -Werror flag should be enabled for RHEL. In ARK, however, it is a bit
tricky to enable -Werror as a build bug occasionally leaks through
upstream review and into Linus' tree, and the toolchain is rebased which
causes many new errors to be reported. As a consequence, -Werror cannot
be enabled in ARK but the code to enable it for RHEL can be added to the
top-level makefile.
Additional fixes: Alphabetize Makefile.variables (something I've wanted to
do for a while now) and remove a mention of Makefile.rhkpg.
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
---
redhat/configs/build_configs.sh | 5 +
redhat/Makefile.variables | 184 ++++++++++++++++++++-------------------
makefile | 7 +
3 files changed, 105 insertions(+), 91 deletions(-)
From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255
There are several groups of warnings and output that should be fixed in
the kernel's rpm build commands.
1) The output of pathfix.py is too noisy. Users do not need to see the
directory traversal messages, and only need to see files that were changed
(or not changed) by the script.
2) There is a vmlinux_decompressor error in !s390x builds. This code
must only be executed on s390x builds.
3) There are references to an old .tmp_versions directory that is no
longer used in the build. These references can be removed entirely from
the tree.
4) There are "File listed twice" rpm build errors that occur because the
rpm spec's kernel-modules-internal section %files section lists them
twice. Of course, they should only be listed once.
The four commits in this series fix the above four issues.
Test build: https://koji.fedoraproject.org/koji/taskinfo?taskID=96528106
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
---
redhat/scripts/mod/mod-denylist.sh | 11 -----------
redhat/kernel.spec.template | 27 +++++++++++----------------
2 files changed, 11 insertions(+), 27 deletions(-)
From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255
There are several groups of warnings and output that should be fixed in
the kernel's rpm build commands.
1) The output of pathfix.py is too noisy. Users do not need to see the
directory traversal messages, and only need to see files that were changed
(or not changed) by the script.
2) There is a vmlinux_decompressor error in !s390x builds. This code
must only be executed on s390x builds.
3) There are references to an old .tmp_versions directory that is no
longer used in the build. These references can be removed entirely from
the tree.
4) There are "File listed twice" rpm build errors that occur because the
rpm spec's kernel-modules-internal section %files section lists them
twice. Of course, they should only be listed once.
The four commits in this series fix the above four issues.
Test build: https://koji.fedoraproject.org/koji/taskinfo?taskID=96528106
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
---
redhat/scripts/mod/mod-denylist.sh | 11 -----------
redhat/kernel.spec.template | 27 ++++++++++-----------------
2 files changed, 10 insertions(+), 28 deletions(-)
From: Prarit Bhargava <prarit(a)redhat.com>
redhat/kernel.spec.template: Parallelize compression
This line in the kernel.spec file:
find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz;
implies that ${RPM_BUILD_NCPUS} xz instances are run. This is not true,
and this line actually applies ${RPM_BUILD_NCPUS} to a single instance of
xz. This means that the compression has been done one module at a time
(ie, in serial) rather than in parallel as is implied by the code.
Use xarg's -n option to assign 16 cpus per process and parallelize the
compression.
Suggested-by: "Herton R. Krzesinski" <herton(a)redhat.com>
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -2472,7 +2472,8 @@ find Documentation -type d | xargs chmod u+w
fi \
fi \
if [ "%{zipmodules}" -eq "1" ]; then \
- find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz; \
+ echo "Compressing kernel modules ..." \
+ find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -n 16 -P${RPM_BUILD_NCPUS} -r xz; \
fi \
%{nil}
--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2254
From: Prarit Bhargava <prarit(a)redhat.com>
redhat/kernel.spec.template: Parallelize compression
This line in the kernel.spec file:
find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz;
implies that ${RPM_BUILD_NCPUS} xz instances are run. This is not true,
and this line actually applies ${RPM_BUILD_NCPUS} to a single instance of
xz. This means that the compression has been done one module at a time
(ie, in serial) rather than in parallel as is implied by the code.
Use xarg's -n option to assign 16 cpus per process and parallelize the
compression.
Suggested-by: "Herton R. Krzesinski" <herton(a)redhat.com>
Signed-off-by: Prarit Bhargava <prarit(a)redhat.com>
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -2472,7 +2472,8 @@ find Documentation -type d | xargs chmod u+w
fi \
fi \
if [ "%{zipmodules}" -eq "1" ]; then \
- find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz; \
+ echo "Compressing kernel modules ..." \
+ find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -n 1 -P${RPM_BUILD_NCPUS} -r xz; \
fi \
%{nil}
--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2254