Koji 2.0 planning

Pat Riehecky riehecky at fnal.gov
Thu Jun 4 13:34:49 UTC 2015



On 06/01/2015 10:43 PM, Mike McLean wrote:
> It's been eight years since koji 1.0. In that time Koji has grown a 
> lot, but always incrementally and with great care to avoid breaking 
> compatibility. Over the years, we've found numerous things that we 
> wanted to add or change, but that we dismissed as too big, too 
> complicated, or too invasive.
>
> Bumping the major release number gives use the freedom to shake things 
> up a bit. Koji 2.0 is about making major changes, otherwise it would 
> just be koji 1.13.
>
> Koji 2.0 will take some time. We will maintain Koji 1.x until 2.0 is 
> sufficiently stable. Some features (e.g. content generators) will also 
> appear in 1.x.
>
> The list below is probably not complete, it is ambitious, and it is 
> certainly open to discussion. If you are a Koji user, or otherwise 
> invested in Koji, then you are encouraged to join in. I expect there 
> will be a lot to say, so I've created a new mailing list devoted 
> specifically to Koji development.
>
> https://lists.fedorahosted.org/mailman/listinfo/koji-devel
>
> Also, apologies for the tense summary listing. I can certain expound 
> on any of these as necessary. I hope this suffices to start some 
> conversation.
>
>
> = High level goals =
>
> • better documentation
> • more community involvement
> • refactor/modernize code
> • more modular design
>   ∘ content generators
>   ∘ broader, better plugin framework
> • better support for different types of build processes
> • better support for for different types build output
> • make hard-wired restrictions more configurable
> • easier to deploy
> • better qa process
> • better release process
>
>
> = Highlights/Major changes =
>
> • python3 support
>   ∘ the bulk of the code will target python 2.6 + python-six
>   ∘ we'll create a basic client lib for older systems (e.g rhel5 
> clients/builders)
> • drop xmlrpc in favor a json based rpc
> • build namespaces
>   ∘ allow for use cases that require multiple builds of the same NVR 
> (or NVRA)
> • refactor task scheduling
> • extend content generator support
>   ∘ content generators will land in 1.x fairly soon, but in 2.0 they 
> will be more integral
>   ∘ refactor kojid to use content generator calls
>   ∘ (possibly) tighter integration in the db
> • unify handling of rpms with other build types in the db
>   ∘ e.g. unify rpminfo and archiveinfo tables
> • support different ways of building (possibly via content generators)
> • utilize jsonb fields in postgres
> • modular auth
>   ∘ make it easier to add new auth mechanisms
>   ∘ support openid auth
> • improve plugins
>   ∘ make the plugin framework cleaner and more pythonic
>   ∘ support plugins in cli and web ui
> • improve/update web ui
>   ∘ drop cheetah templates in favor of python-jinja2
>   ∘ more parity with cli
>   ∘ history browsing
>   ∘ landing page search or history
>   ∘ support plugins
> • change how tag pkglists (and blocking) work
> • refactor package ownership
> • refactor uploads
> • more flexible gc
>
> = Yet more changes =
>
> • store all task requests by named args
>   ∘ (for ease of inspection)
> • get rid of tagBuild tasks
> • drop odd event refererences in favor of timestamps
> • streamlined cli options
> • marker files for many things on disk
> • more history data
> • drop modpython support
> • policy code
>   ∘ more robust syntax
>     ‣ test negation
>     ‣ OR
>     ‣ parentheses
>     ‣ quoted strings
>   ∘ multiple result policies (non-terminal actions)
>   ∘ all-matches policy (needed for scheduler?)
>   ∘ break action (breaks out of nesting)
>   ∘ stop action (halts processing of an all-matches policy)

I thought of a few more "extras" I'd like to propose:
  - kojid plugin sync
    kojid has a plugin architecture, it would be nice if the hub could 
say "Hey, you should have these plugins" and kojid could then fetch them 
over some kind of API

  - "fewer" settings within the koji-hub.conf (or any koji config)
    I'd love to see nearly any setting that can be set, being set over 
an API and stored in the database.  Mostly that makes it easier for me 
to backup and restore the database in a disaster.

-- 
Pat Riehecky
Scientific Linux developer

Fermi National Accelerator Laboratory
www.fnal.gov
www.scientificlinux.org



More information about the koji-devel mailing list