Custom output build types (btype) for CG best practices
by Tomáš Hozza
Hello.
I'm from the Image Builder team at Red Hat. For those who don't know it,
our team works on tooling for building various OS images and integrates
with Koji as a CG (https://github.com/osbuild/koji-osbuild).
Our current integration works in the following way:
1. Koji builder plugin makes a REST API call to our service.
2. Our service builds the image, uploads the image file to Koji and then
imports it using CG API (
https://github.com/osbuild/osbuild-composer/blob/eb8b63f3ca166ecad13e10a2...
).
3. Koji builder plugin waits for the image build to finish and then
downloads logs and osbuild manifest (a kind of recipe allowing to rebuild
the image) and uploads them to the build task.
As a result, valuable files (such as logs and the osbuild manifest) are not
attached to the build in Koji, but to the build task. Files attached to the
task usually get garbage collected after some time (I guess depending on
the Koji instance configuration), so as a result we are losing these files
after some time.
I'm currently reworking the way osbuild-composer imports content to Koji,
so that we attach build logs and osbuild manifests to the Koji build as
well. When we import outputs to Koji, we need to specify the output build
type. For the image, we use "image", for logs we'll use "log", but the
osbuild manifest would probably benefit from its own build type (e.g.
"osbuild-manifest").
From what I checked, some content generators use their own build type (e.g.
when I run "koji --noauth call listBTypes" with Fedora Koji, there's a
"module" build type for MBS).
I would like to ask what is the common or usual way for a Content Generator
to mandate that a custom build type is available in Koji when using it? My
understanding is that a new build type can be added to the Koji instance by
an admin running the appropriate command. Is there some other way?
I'm trying to determine what's the best way to handle the situation that
our CG will start using a custom build type, while being already used by
multiple Koji instances. Is the only way to document this requirement and
work with responsible admins of Koji instances, which integrate with our
CG, to adjust the configuration before we start using the new build type?
Thanks in advance for any guidance.
Regards,
Tomas
--
Tomáš Hozza
Image Builder, Linux Engineering
PGP: 908AF740C0F898A1079968CDC5887AD51D9F3C2D
UTC+2 (CEST) / UTC+1 (CET)
Red Hat Czech http://cz.redhat.com
2 months, 2 weeks
Re: Inquiry Regarding CMake-Based Software Packaging for Koji
by Tomas Kopecek
On Mon, Sep 4, 2023 at 4:38 PM Joe (Jun-Yan) Chen <jychena(a)ambarella.com>
wrote:
> Dear Koji Development Team,
>
>
> I hope this email finds you well. I have a couple of questions regarding
> the packaging of CMake-based open-source software.
>
> Question 1: Can Koji's buildroot be deployed to access the Internet?
>
> For some open-source software projects, dependencies are downloaded during
> the CMake configuration phase. However, Koji's buildroot access towards
> internet(GitHub) is restricted. I attempted to pre-download these
> dependencies by cmake or set them as 'BuildRequires', but it appears that
> the intricate CMake system are inflexible and do not provide room for such
> customization.
>
> In the context of packaging such software for compilation on Koji, aside
> from modifying the CMake scripts, I would like to know if there are any
> other more straightforward methods that have been recommended based on the
> experience of previous packagers.
>
>
Nope, currently not. It is a part of koji's philosophy for reproducible
builds. Anything downloaded outside of curated git repos could change in
the future and rebuild of such thing will not be the same. It is also a
part of Fedora's packaging guidelines (
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_build_time_ne...
)
Normal solution is really to alter the package to not access the internet
during the build and package all dependencies.
> Question 2: Handling Hardcoded Paths in CMake
>
> When packaging certain open-source software using CMake, I noticed that
> during the CMake configuration phase, if I set
> "-DCMAKE_INSTALL_PREFIX=$RPM_BUILD_ROOT/opt/oss/", CMake tends to hardcode
> this path with $RPM_BUILD_ROOT into various places, such as the RPATH of
> binary libraries or CMake configuration files. Clearly, this is incorrect
> for the files that end up in /opt/oss/.
>
> As a workaround, I've experimented with running the command "find
> $RPM_BUILD_ROOT -type f -exec sed -i 's/${RPM_BUILD_ROOT}//g' {} \;" at the
> end of the %install section, which successfully passes the
> /usr/lib/rpm/check-buildroot check. However, I'm curious to know if this
> approach will introduce potential issues or concerns?
>
There are some predefined macros in Fedora which you can use to handle
this: https://docs.fedoraproject.org/en-US/packaging-guidelines/CMake/
> Your expertise and guidance on these matters would be highly valuable to
> me. Thank you for your time and assistance.
>
>
> Best Regards,
> Junyan CHEN
>
> ------------------------------
> This email and attachments contain Ambarella Proprietary and/or
> Confidential Information and is intended solely for the use of the
> individual(s) to whom it is addressed. Any unauthorized review, use,
> disclosure, distribute, copy, or print is prohibited. If you are not an
> intended recipient, please contact the sender by reply email and destroy
> all copies of the original message. Thank you.
> _______________________________________________
> koji-devel mailing list -- koji-devel(a)lists.fedorahosted.org
> To unsubscribe send an email to koji-devel-leave(a)lists.fedorahosted.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/koji-devel@lists.fedorahoste...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
--
Tomas Kopecek <tkopecek(a)redhat.com>
RHEL Build Development, RedHat
2 months, 3 weeks
Inquiry Regarding CMake-Based Software Packaging for Koji
by Joe (Jun-Yan) Chen
Dear Koji Development Team,
I hope this email finds you well. I have a couple of questions regarding the packaging of CMake-based open-source software.
Question 1: Can Koji's buildroot be deployed to access the Internet?
For some open-source software projects, dependencies are downloaded during the CMake configuration phase. However, Koji's buildroot access towards internet(GitHub) is restricted. I attempted to pre-download these dependencies by cmake or set them as 'BuildRequires', but it appears that the intricate CMake system are inflexible and do not provide room for such customization.
In the context of packaging such software for compilation on Koji, aside from modifying the CMake scripts, I would like to know if there are any other more straightforward methods that have been recommended based on the experience of previous packagers.
Question 2: Handling Hardcoded Paths in CMake
When packaging certain open-source software using CMake, I noticed that during the CMake configuration phase, if I set "-DCMAKE_INSTALL_PREFIX=$RPM_BUILD_ROOT/opt/oss/", CMake tends to hardcode this path with $RPM_BUILD_ROOT into various places, such as the RPATH of binary libraries or CMake configuration files. Clearly, this is incorrect for the files that end up in /opt/oss/.
As a workaround, I've experimented with running the command "find $RPM_BUILD_ROOT -type f -exec sed -i 's/${RPM_BUILD_ROOT}//g' {} \;" at the end of the %install section, which successfully passes the /usr/lib/rpm/check-buildroot check. However, I'm curious to know if this approach will introduce potential issues or concerns?
Your expertise and guidance on these matters would be highly valuable to me. Thank you for your time and assistance.
Best Regards,
Junyan CHEN
**********************************************************************
This email and attachments contain Ambarella Proprietary and/or Confidential Information and is intended solely for the use of the individual(s) to whom it is addressed. Any unauthorized review, use, disclosure, distribute, copy, or print is prohibited. If you are not an intended recipient, please contact the sender by reply email and destroy all copies of the original message. Thank you.
2 months, 3 weeks