Koji 2.0 planning

Pat Riehecky riehecky at fnal.gov
Tue Jun 2 13:37:30 UTC 2015


Hi Mike,

Exciting news! My initial thoughts are scattered below.

On 06/01/2015 10:43 PM, Mike McLean wrote:
> It's been eight years since koji 1.0. In that time Koji has grown a 
> lot, but always incrementally and with great care to avoid breaking 
> compatibility. Over the years, we've found numerous things that we 
> wanted to add or change, but that we dismissed as too big, too 
> complicated, or too invasive.
>
> Bumping the major release number gives use the freedom to shake things 
> up a bit. Koji 2.0 is about making major changes, otherwise it would 
> just be koji 1.13.
>
> Koji 2.0 will take some time. We will maintain Koji 1.x until 2.0 is 
> sufficiently stable. Some features (e.g. content generators) will also 
> appear in 1.x.
>
> The list below is probably not complete, it is ambitious, and it is 
> certainly open to discussion. If you are a Koji user, or otherwise 
> invested in Koji, then you are encouraged to join in. I expect there 
> will be a lot to say, so I've created a new mailing list devoted 
> specifically to Koji development.
>
> https://lists.fedorahosted.org/mailman/listinfo/koji-devel
>
> Also, apologies for the tense summary listing. I can certain expound 
> on any of these as necessary. I hope this suffices to start some 
> conversation.
>
>
> = High level goals =
>
> • better documentation
> • more community involvement
> • refactor/modernize code
> • more modular design
>   ∘ content generators
>   ∘ broader, better plugin framework
> • better support for different types of build processes
> • better support for for different types build output
> • make hard-wired restrictions more configurable
> • easier to deploy
> • better qa process
> • better release process
>
>
> = Highlights/Major changes =
>
> • python3 support
>   ∘ the bulk of the code will target python 2.6 + python-six
>   ∘ we'll create a basic client lib for older systems (e.g rhel5 
> clients/builders)
> • drop xmlrpc in favor a json based rpc
> • build namespaces
>   ∘ allow for use cases that require multiple builds of the same NVR 
> (or NVRA)

I'd also ask for the ability to migrate packages from one namespace into 
another - effectively replacing the one that exists there currently.

> • refactor task scheduling
> • extend content generator support
>   ∘ content generators will land in 1.x fairly soon, but in 2.0 they 
> will be more integral
>   ∘ refactor kojid to use content generator calls
>   ∘ (possibly) tighter integration in the db
> • unify handling of rpms with other build types in the db
>   ∘ e.g. unify rpminfo and archiveinfo tables
> • support different ways of building (possibly via content generators)
> • utilize jsonb fields in postgres
> • modular auth
>   ∘ make it easier to add new auth mechanisms
>   ∘ support openid auth
> • improve plugins
>   ∘ make the plugin framework cleaner and more pythonic
>   ∘ support plugins in cli and web ui
> • improve/update web ui
>   ∘ drop cheetah templates in favor of python-jinja2
>   ∘ more parity with cli
>   ∘ history browsing
>   ∘ landing page search or history
>   ∘ support plugins
> • change how tag pkglists (and blocking) work
> • refactor package ownership
> • refactor uploads
> • more flexible gc

Depending on what you mean by this, I've wondered if kojira could be 
replaced by a post action hook.

>
> = Yet more changes =
>
> • store all task requests by named args
>   ∘ (for ease of inspection)
> • get rid of tagBuild tasks

For the SL koji, I've a plugin that triggers off of tagBuild to run some 
tests.  While I'm not tied directly to the tagBuild itself, I am tied to 
"I finished building any package successfully against tag X".

> • drop odd event refererences in favor of timestamps
> • streamlined cli options
> • marker files for many things on disk
> • more history data
> • drop modpython support
> • policy code
>   ∘ more robust syntax
>     ‣ test negation
>     ‣ OR
>     ‣ parentheses
>     ‣ quoted strings
>   ∘ multiple result policies (non-terminal actions)
>   ∘ all-matches policy (needed for scheduler?)
>   ∘ break action (breaks out of nesting)
>   ∘ stop action (halts processing of an all-matches policy)
>

May I also request Database Schema migration?  Some other open source 
projects (django 1.8, wordpress) have frameworks for allowing the 
application to maintain its own schema.  I suspect this may simplify new 
installs and version upgrades.

-- 
Pat Riehecky
Scientific Linux developer

Fermi National Accelerator Laboratory
www.fnal.gov
www.scientificlinux.org



More information about the koji-devel mailing list