On Tue, 2006-03-14 at 22:24 -0500, Bill Nottingham wrote:
The build system should have a framework where simple post-build
checks can run. Simple design would be:
- tests are collected in a package of scripts (or in CVS)
- packagers can add extra ones for their packages
- tests have access to:
- the build logs
- the name of the user who submitted the build
- the owner of the package (if not the same)
- the previous version of the package
- the repodata for the repo being built for
- tests have a defined return status, such as
0 - everythings OK. Carry on, sir!
1 - some information here
7 - someone should review this and approve before pushing
11 - the package is bad, please throw away
These tests could then be:
- e-mailed to the builder
- e-mailed to the owner
- posted on the web
Doing this allows for simple, automated, QA; it can check that
things that are caught in the package review for initial import
are then caught later if they regress, and it can catch other
Examples of tests:
- test for files owned by the compile user
- test for files with executable stack
- look for specific compiler warnings in the build output, such as:
... will always overflow destination buffer ...
... warning: format argument XX unused before used argument XX ...
*** buffer overflow detected ***
... warning: ignoring return value of 'realloc' ...
- check for introduced multilib conflicts
- check the scriplets for:
- correct use of chkconfig
- correct use of useradd or equivalents
- excess complication (warning: package %post is 150 lines)
- using the repo data:
- check the package for broken deps
- check the repository for induced breakage
- using previous version of the same package:
- check for added/removed files
- check for added/removed/changed dependencies
- check for introduction of new setuid files
- check that symbols changed in libraries without changing soname
- heck, just run the package through rpmlint and mail it out
(or check it against a known list of exceptions for the package)
I'm sure people could come up with a lot more. By making it
a simple framework, it allows for easier contribution.
Sounds like a plan. We should do this.