From: Vitaly Kuznetsov vkuznets@redhat.com

redhat: Use kspdx-tool output for the License: field

Use kspdx-tool (https://gitlab.com/vkuznets/kspdx-tool) to find all licenses in the source code and its output to the License: field of the specfile. Package the tool and its current output.

Signed-off-by: Vitaly Kuznetsov vkuznets@redhat.com

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template index blahblah..blahblah 100644 --- a/redhat/kernel.spec.template +++ b/redhat/kernel.spec.template @@ -636,7 +636,7 @@ Summary: The Linux kernel

Name: %{package_name} -License: GPLv2 and Redistributable, no modification permitted +License: ((GPL-2.0-only WITH Linux-syscall-note) OR BSD-2-Clause) AND ((GPL-2.0-only WITH Linux-syscall-note) OR BSD-3-Clause) AND ((GPL-2.0-only WITH Linux-syscall-note) OR CDDL-1.0) AND ((GPL-2.0-only WITH Linux-syscall-note) OR Linux-OpenIB) AND ((GPL-2.0-only WITH Linux-syscall-note) OR MIT) AND ((GPL-2.0-or-later WITH Linux-syscall-note) OR BSD-3-Clause) AND ((GPL-2.0-or-later WITH Linux-syscall-note) OR MIT) AND BSD-2-Clause AND BSD-3-Clause AND BSD-3-Clause-Clear AND GFDL-1.1-no-invariants-or-later AND GPL-1.0-or-later AND (GPL-1.0-or-later OR BSD-3-Clause) AND (GPL-1.0-or-later WITH Linux-syscall-note) AND GPL-2.0-only AND (GPL-2.0-only OR Apache-2.0) AND (GPL-2.0-only OR BSD-2-Clause) AND (GPL-2.0-only OR BSD-3-Clause) AND (GPL-2.0-only OR CDDL-1.0) AND (GPL-2.0-only OR GFDL-1.1-no-invariants-or-later) AND (GPL-2.0-only OR GFDL-1.2-no-invariants-only) AND (GPL-2.0-only OR Linux-OpenIB) AND (GPL-2.0-only OR MIT) AND (GPL-2.0-only OR X11) AND (GPL-2.0-only WITH Linux-syscall-note) AND GPL-2.0-or-later AND (GPL-2.0-or-later OR BSD-2-Clause) AND (GPL-2.0-or-later OR BSD-3-Clause) AND (GPL-2.0-or-later OR CC-BY-4.0) AND (GPL-2.0-or-later OR MIT) AND (GPL-2.0-or-later WITH GCC-exception-2.0) AND (GPL-2.0-or-later WITH Linux-syscall-note) AND ISC AND LGPL-2.0-or-later AND (LGPL-2.0-or-later OR BSD-2-Clause) AND (LGPL-2.0-or-later WITH Linux-syscall-note) AND LGPL-2.1-only AND (LGPL-2.1-only OR BSD-2-Clause) AND (LGPL-2.1-only WITH Linux-syscall-note) AND LGPL-2.1-or-later AND (LGPL-2.1-or-later WITH Linux-syscall-note) AND (Linux-OpenIB OR GPL-2.0-only) AND (Linux-OpenIB OR GPL-2.0-only OR BSD-2-Clause) AND Linux-man-pages-copyleft AND MIT AND (MIT OR Apache-2.0) AND (MIT OR GPL-2.0-only) AND (MIT OR GPL-2.0-or-later) AND (MIT OR LGPL-2.1-only) AND (MPL-1.1 OR GPL-2.0-only) AND (X11 OR GPL-2.0-only) AND (X11 OR GPL-2.0-or-later) AND Zlib AND (copyleft-next-0.3.1 OR GPL-2.0-or-later) AND (Redistributable, no modification permitted) URL: https://www.kernel.org/ Version: %{specrpmversion} Release: %{pkg_release} @@ -1069,7 +1069,6 @@ Epoch: %{gemini} %endif Summary: Performance monitoring for the Linux kernel Requires: bzip2 -License: GPLv2 %description -n perf This package contains the perf tool, which enables performance monitoring of the Linux kernel. @@ -1119,7 +1118,6 @@ This package provides debug information for the perf python bindings. %if %{with_tools} %package -n %{package_name}-tools Summary: Assortment of tools for the Linux kernel -License: GPLv2 %ifarch %{cpupowerarchs} Provides: cpupowerutils = 1:009-0.6.p1 Obsoletes: cpupowerutils < 1:009-0.6.p1 @@ -1137,14 +1135,12 @@ and the supporting documentation.

%package -n %{package_name}-tools-libs Summary: Libraries for the kernels-tools -License: GPLv2 %description -n %{package_name}-tools-libs This package contains the libraries built from the tools/ directory from the kernel source.

%package -n %{package_name}-tools-libs-devel Summary: Assortment of tools for the Linux kernel -License: GPLv2 Requires: %{package_name}-tools = %{version}-%{release} %ifarch %{cpupowerarchs} Provides: cpupowerutils-devel = 1:009-0.6.p1 @@ -1183,7 +1179,6 @@ and root causes of unexpected results.

%package -n rv Summary: RV: Runtime Verification -License: GPLv2 %description -n rv Runtime Verification (RV) is a lightweight (yet rigorous) method that complements classical exhaustive verification techniques (such as model @@ -1201,7 +1196,6 @@ analysing the logical and timing behavior of Linux.

%package -n bpftool Summary: Inspection and simple manipulation of eBPF programs and maps -License: GPLv2 Version: %{bpftoolversion} %description -n bpftool This package contains the bpftool, which allows inspection and simple @@ -1229,7 +1223,6 @@ This package provides debug information for the bpftool package.

%package selftests-internal Summary: Kernel samples and selftests -License: GPLv2 Requires: binutils, bpftool, iproute-tc, nmap-ncat, python3, fuse-libs %description selftests-internal Kernel sample programs and selftests. diff --git a/redhat/scripts/kspdx-tool/kspdx.py b/redhat/scripts/kspdx-tool/kspdx.py new file mode 100755 index blahblah..blahblah 100755 --- /dev/null +++ b/redhat/scripts/kspdx-tool/kspdx.py @@ -0,0 +1,226 @@ +#! /usr/bin/python3 + +# SPDX-License-Identifier: LGPL-2.1-or-later + +import argparse +import os +import re +import subprocess +import sys + +# Files to not search for SPDX patterns +ignored_files = [ + # license-rules.rst describe how to write SPDX-License-Identifier tags, skip it + "/process/license-rules.rst", +] + +# Generators, take the first SPDX identifier only to avoid +# parsing the code which adds 'SPDX-License-Identifier' to some +# other code. +generators = [ + "/scripts/atomic/gen-atomic-fallback.sh", + "/scripts/atomic/gen-atomic-instrumented.sh", + "/scripts/atomic/gen-atomic-long.sh", + "/tools/bpf/bpftool/gen.c", + "/tools/net/ynl/lib/nlspec.py", + "/tools/net/ynl/ynl-gen-c.py", + "/tools/testing/selftests/bpf/generate_udp_fragments.py", +] + +def run_command(cmdargs, sysexit=False, canfail=False, input=None): + res = subprocess.run(cmdargs, check=False, capture_output=True, text=True, input=input) + if res.returncode != 0 and not canfail: + print("%s returned %d, stdout: %s stderr: %s" % (res.args, res.returncode, res.stdout, res.stderr), file=sys.stderr) + if sysexit: + sys.exit(1) + else: + raise Exception("%s command failed" % cmdargs[0]) + return res + +def get_file_source(path, commit = None): + if not commit: + try: + with open(path, 'rb') as fp: + return fp.read().decode('utf-8', errors='ignore') + except Exception as e: + print("Failed to read file %s: %s" % (path, e), file=sys.stderr) + return None + else: + try: + res = run_command(['git', 'show', "%s:%s" % (commit, path)]) + return res.stdout + except Exception as e: + print("Failed to show file %s from commit %s: %s" % (path, commit, e), file=sys.stderr) + return None + +# Valid chars in SPDX tag: a-Z,0-9,-,+,_, ,\t,(,),. +spdx_pattern = re.compile(r"(?:--|*|#|//|..)\s*SPDX-License-Identifier:\s+([a-zA-Z0-9-_.\t ()+]+)") + +def get_spdx_string(fpath, commit, default, first_only=False): + content = get_file_source(fpath, commit) + if content is None: + print("Failed to get content of %s" % fpath, file=sys.stderr) + sys.exit(1) + + r = spdx_pattern.findall(content) + + if first_only: + r = r[:1] + elif len(set(r)) > 1: + print("WARNING: %s lists more than one different license, please check!" % fpath, file=sys.stderr) + + changed = True + while changed: + changed = False + for i in range(len(r)): + s = r[i] + # Remove extra spaces + s = " ".join(s.split()) + + # Remove trailing '--' (SVG) + s = re.sub("--$", "", s) + + # Drop unneded highest level parentheses + s = re.sub("^((.*))$", "\g<1>", s) + + # Drop unneeded inner parentheses when there are no spaces + s = re.sub("(([^ ]+))", "\g<1>", s) + + # (A OR B) OR C equals A OR B OR C + s = re.sub("((.*) OR (.*)) OR", "\g<1> OR \g<2> OR", s) + # A OR (B OR C) equals A OR B OR C + s = re.sub("OR ((.*) OR (.*))", "OR \g<1> OR \g<2>", s) + + # Assuming there's just one level of ORs, sort the licenses in reverse alphabetical order + # sort only when no parentheses + if s.find(' OR ') != -1 and s.find('(') == -1: + s = ' OR '.join([e.strip() for e in sorted(s.split(' OR '), reverse=True)]) + + # Split A and B into two items but make sure parenthes are balanced + and_pos = 0 + while True: + and_pos = s.find(' AND ', and_pos+1) + if and_pos > 0: + l1 = s[:and_pos] + l2 = s[and_pos+5:] + if l1.count('(') == l1.count(')') and l2.count('(') == l2.count(')'): + r.append(l2) + s = l1 + break + else: + break + + if s != r[i]: + r[i] = s + changed = True + if r == []: + r = [default] + + return r + +def convert_deprecated(license): + # Deprecated ids, see https://spdx.org/licenses/ + # GPL-1.0 equals GPL-1.0-only + license = re.sub("GPL-1.0($| )", "GPL-1.0-only\g<1>", license) + # GPL-1.0+ equals GPL-1.0-or-later + license = re.sub("GPL-1.0+($| )", "GPL-1.0-or-later\g<1>", license) + + # GPL-2.0 equals GPL-2.0-only + license = re.sub("GPL-2.0($| )", "GPL-2.0-only\g<1>", license) + # GPL-2.0+ equals GPL-2.0-or-later + license = re.sub("GPL-2.0+($| )", "GPL-2.0-or-later\g<1>", license) + + # LGPL-2.0 equals LGPL-2.0-only + license = re.sub("LGPL-2.0($| )", "LGPL-2.0-only\g<1>", license) + # LGPL-2.0+ equals LGPL-2.0-or-later + license = re.sub("LGPL-2.0+($| )", "LGPL-2.0-or-later\g<1>", license) + + # LGPL-2.1 equals LGPL-2.1-only + license = re.sub("LGPL-2.1($| )", "LGPL-2.1-only\g<1>", license) + # LGPL-2.1+ equals LGPL-2.1-or-latery + license = re.sub("LGPL-2.1+($| )", "LGPL-2.1-or-later\g<1>", license) + + # Use standard uppercase 'OR' + license = re.sub(" or ", " OR ", license) + return license + +def unique_licenses(licenses): + res = [] + for license in licenses: + license = convert_deprecated(license) + already_present = False + for existing in res: + if license.upper() == existing.upper(): + already_present = True + if already_present: + continue + res.append(license) + return sorted(res) + +def license_andlist(unique): + s = "" + for i in range(len(unique)): + # Parenthes are needed for everything but a singe item + if unique[i].find(' ') != -1 and len(unique) > 1: + s += '(' + unique[i] + ')' + else: + s += unique[i] + if i != len(unique) - 1: + s += ' AND ' + return s + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description='Report SPDX-License-Identifier tag for a kernel source file/directory', + formatter_class=argparse.ArgumentDefaultsHelpFormatter) + parser.add_argument('path', help='Path in the source tree') + parser.add_argument('-c', '--commit', help='Inspect given commit/HEAD instead of the current state') + parser.add_argument('-d', '--default', help='Default license', default="GPL-2.0-only") + parser.add_argument('-i', '--itemized', help='Print license[s] per file', action="store_true") + parser.add_argument('-j', '--joint', help='Print a single statement for all discovered licenses', action="store_true") + args = parser.parse_args() + + if os.path.isdir(args.path) and args.commit: + print("The specified path %s is a directory and --commit was given, this is unsupported." % args.path, file=sys.stderr) + + files = [] + if os.path.isdir(args.path): + w = os.walk(args.path) + for (dpath, dnames, fnames) in w: + # Skip .git objects + if '.git' in dpath.split('/'): + continue + files.extend([dpath.rstrip('/') + '/' + fname for fname in fnames]) + else: + files = [args.path] + + licenses = [] + for fpath in files: + ignore = False + for ignored in ignored_files: + if fpath.endswith(ignored): + ignore = True + continue + if ignore: + continue + + generator = False + for ignored in generators: + if fpath.endswith(ignored): + generator = True + continue + + file_licenses = get_spdx_string(fpath, args.commit, args.default, generator) + unique = unique_licenses(file_licenses) + if not args.itemized: + licenses.extend(unique) + else: + print("%s: %s" % (fpath, license_andlist(unique))) + + if not args.itemized: + if not args.joint: + for license in sorted(set(licenses)): + print(license) + else: + print(license_andlist(sorted(set(licenses)))) + + sys.exit(0)

-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2648

From: Vitaly Kuznetsov vkuznets@redhat.com

redhat: Add dist-check-licenses target

Introduce a dedicated make target to check source code licenses without the need to run all other selftests which may be failing for some other reason.

Signed-off-by: Vitaly Kuznetsov vkuznets@redhat.com

diff --git a/redhat/Makefile b/redhat/Makefile index blahblah..blahblah 100644 --- a/redhat/Makefile +++ b/redhat/Makefile @@ -877,6 +877,13 @@ dist-self-test: dist-self-test-data: ./self-test/data/create-data.sh

+dist-check-licenses: + @if test -x /usr/bin/bats; then \ + bats $(REDHAT)/self-test/1007-spdx-licenses.bats ; \ + else \ + echo "dist-check-licenses: The bats package is not installed" ; \ + fi + dist-help: @echo 'Cleaning targets:' @echo ' dist-clean - Clean redhat/configs/ and redhat/rpm/ directories.' @@ -1000,6 +1007,9 @@ dist-full-help: @echo ' dist-os-version - Displays the current Red Hat Enterprise Linux versioni' @echo ' target used by the current branch/tree.' @echo ' dist-self-test - Runs self-tests from the redhat/self-test directory' + @echo ' dist-check-licenses - Runs license selftest checking SPDX-License-Identifier:' + @echo ' tags in all source files against `License:` field in the' + @echo ' specfile template.' @echo ''

export

-- https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2648