[PATCH] [RHEL] Make new 0.1.17 release

Shawn Wells shawn at redhat.com
Tue May 13 23:26:59 UTC 2014


On 5/13/14, 8:52 AM, Jan Lieskovsky wrote:
> Hello Simon,
>
>    thanks for the suggestions.
>
> ----- Original Message -----
>> From: "Simon Lukasik" <isimluk at fedoraproject.org>
>> To: "SCAP Security Guide" <scap-security-guide at lists.fedorahosted.org>
>> Sent: Tuesday, May 13, 2014 10:37:22 AM
>> Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release
>>
>> On 05/12/2014 10:03 PM, Shawn Wells wrote:
>>> On 5/9/14, 2:04 PM, Jan Lieskovsky wrote:
>>>> ----- Original Message -----
>>>>> From: "Shawn Wells" <shawn at redhat.com>
>>>>> To: scap-security-guide at lists.fedorahosted.org
>>>>> Sent: Friday, May 9, 2014 6:03:20 PM
>>>>> Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release
>>>>>
>>>>>
>>>>> On 5/9/14, 10:24 AM, Jan Lieskovsky wrote:
>>>>>> ----- Original Message -----
>>>>>>>> From: "David Smith"<dsmith at secure-innovations.net>
>>>>>>>> To: "SCAP Security Guide"<scap-security-guide at lists.fedorahosted.org>
>>>>>>>> Sent: Friday, May 9, 2014 4:44:14 PM
>>>>>>>> Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Jan,
>>>>>>>>
>>>>>>>> Totally agreed on the need for the update; ack + thanks!
>>>>>> Thank you, David. Pushed to master. New tarball & EPEL build to come
>>>>>> shortly.
>>>>> Jan, would you be able to document (even informally) the various steps?
>>>>> While only extremely limited people have access to cut RPMs, it'd be
>>>>> good to document things.
>>>> Sure thing (should I forgot some item - doing them on per moment basis
>>>> when realized forgot to do some of them, will re-iterate later).
>>>>
>>>> Upstream process:
>>>> * update main scap-security-guide.spec file
>>>> (scap-security-guide/scap-security-guide.spec)
>>>>     with new version (value of "redhatssgversion" version variable).
>>>> Ensure
>>>>     the Release: field contains 1%{?dist} (1 as release version). Add
>>>> particular
>>>>     changelog entry (possibly verify for & fix whitespace noise)
>>>> * build & test the content (i.e. run 'make', 'make srpm', 'make rpm')
>>>> to verify
>>>>     it builds successfully. Also try to scan some systems with selected
>>>> profiles to see
>>>>     it the content works
>>>> * if it works, "make clean" in the git repository to start with clean
>>>> table.
>>>>     Make the source tarball via "make tarball". Upload the tarball to
>>>> repos.ssgproject.org.
>>>>
>>>> EPEL process:
>>>> * file-in new EPEL-6 bugzilla (Summary = "Upgrade scap-security-guide
>>>> package to scap-security-guide-X.Y.Z")
>>>>     Note: That bugzilla is required later when creating Bodhi update
>>>> request. See below.
>>>>     Note#2: It would be created automa{g,t}ically once the "latest
>>>> upstream source tarball checking Red Hat
>>>>             Bugzilla functionality" would realize there's is new source
>>>> tarball available. But since we wan't
>>>>             immediate upgrade, we create that bug manually.
>>>>
>>>> * take that bugzilla (state change NEW => ASSIGNED)
>>>> * clone the scap-security-guide git repository via fedpkg tool (as
>>>> documented in:
>>>>
>>>> https://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Import.2C_Commit.2Cand_Build_Your_Package
>>>>
>>>>
>>>>     section "Check out the module" and later ones). Split into coins
>>>> for our case it means:
>>>>
>>>>     $ fedpkg clone scap-security-guide
>>>>     $ cd scap-security-guide/
>>>> * ensure to change the git branch from master/ to origin/el6 via
>>>> 'switch-branch' fedpkg's option
>>>>     (this ensures the changes will be actually committed into EPEL-6
>>>> branch, and not into the master,
>>>>     IOW F-21 branch, which we don't want). To see the list of available
>>>> branches, issue the following:
>>>>
>>>>     $ fedpkg switch-branch -l
>>>>     Locals:
>>>>     * master
>>>>     Remotes:
>>>>       origin/el6
>>>>       origin/epel7
>>>>       origin/f18
>>>>       origin/f19
>>>>       origin/f20
>>>>       origin/master
>>>>
>>>>     To switch to el6 branch, issue:
>>>>
>>>>       $ fedpkg switch-branch el6
>>>>       Branch el6 set up to track remote branch el6 from origin.
>>>>
>>>>     command. Now it's possible actually to see the actual content of
>>>> EPEL-6 branch:
>>>>
>>>>       $ ls
>>>>       scap-security-guide.spec  sources
>>>>
>>>>       scap-security-guide.spec is the SPEC file used for build of
>>>> EPEL-6's rpm, sources
>>>>       text file contains md5sum of scap-security-guide tarball, which
>>>> will be build during
>>>>       SRPM / RPM build.
>>>>
>>>> * to refresh both of them (*.spec & content of source) at once, it's
>>>> possible to create source
>>>>     RPM package & import it into fedpkg.
>>>>
>>>>     Two important notes to mention here:
>>>>
>>>>     - The spec file needs to be the updated one => it's necessary to
>>>> update the actual epel-6 one
>>>>     with changes from upstream one or replace the epel-6 one with
>>>> upstream one (the latter is still
>>>>     possible because as of right now there aren't epel-6 downstream
>>>> specific patches that woulnd't
>>>>     be present in upstream already. But should there be such changes in
>>>> the future, the epel-6 spec
>>>>     should be updated to include changes from upstream spec but
>>>> simultaneously to keep epel-6 custom
>>>>     patches. IOW just replacing epel-6's spec with upstream's one
>>>> wouldn't work, but manual changes
>>>>     would be necessary).
>>>>
>>>>     - The new source tarball needs to be the last one uploaded to
>>>> repos.ssgproject.org (so md5sum
>>>>     would match during package build).
>>>>
>>>>     This means:
>>>>     * start with clean /rpmbuild directory structure
>>>>     * download latest tarball from repos.ssgproject.org into
>>>> /rpmbuild/SOURCES
>>>>     * place the modified epel-6 spec file into /rpmbuild/SPECS
>>>>     * build the source rpm (result will be in /rpmbuild/SRPMS)
>>>>
>>>> * return back to fedpkg & import the SRPM created in previous steps:
>>>>
>>>>     $ fedpkg import path_to_srpm
>>>>
>>>>     This will change content of 'sources' file (include new md5sum) &
>>>> update scap-security-guide.spec.
>>>>     $ git status
>>>>
>>>>     to see what will get committed.
>>>>
>>>>     $ git commit
>>>>
>>>>     to confirm the changes. The commit message should contain the string:
>>>>
>>>>     "Resolves: rh bz# id_of_the_epel_6_bug_we_created_before"
>>>>
>>>>     in point #1 of the EPEL process section.
>>>>
>>>>     Make scratch build to see the uploaded content (spec + tarball)
>>>> would actually build in the Koji
>>>>     build system via:
>>>>
>>>>     $ fedpkg scratch-build --srpm path_to_srpm_created_locally_before
>>>>
>>>>     Note: scratch-build to work with actually committed git repository
>>>> content, it requires the
>>>>     new content to be already "git push-ed" to the repository. But
>>>> since we want to verify if the
>>>>     content would build yet before pushing changes into the EPEL-6
>>>> repository, we need to provide
>>>>     the --srpm option pointing fedpkg to the local source RPM package
>>>> we have created one step before.
>>>>
>>>>     Once the scratch build passes (visible in Koji web interface, or
>>>> also on command line), we can push
>>>>     the changes to git repository via:
>>>>
>>>>     $ git push origin el6
>>>>
>>>>     After successful push, our / latest push should be visible at (in
>>>> el6 branch)
>>>>       http://pkgs.fedoraproject.org/cgit/scap-security-guide.git/
>>>>
>>>>     Now it's safe (scratch build succeeded & we pushed the changes to
>>>> the Fedora's git) to build real
>>>>     new package via:
>>>>
>>>>     $ fedpkg build
>>>>
>>>>     This again generates clickable link, at which it's possible to see
>>>> the progress / result of the build.
>>>>     Once the new package build in Koji finishes successfully, we flip
>>>> the previously created
>>>>     EPEL-6 bug to MODIFIED (ASSIGNED => MODIFIED) and mention the new
>>>> package name-version-release
>>>>     in the "Fixed in Version:" field of that bug.
>>>>
>>>> * Having new build available, it's necessary to schedule new Bodhi
>>>> update (something like
>>>>     advisory to be tied with new packages). I am using web UI:
>>>>       https://admin.fedoraproject.org/updates/new/
>>>>
>>>>     but there's command-line interface too (see [1] for further details).
>>>>
>>>>     Add New Update screen is shown (containing following fields / items):
>>>>
>>>>     Add New Update
>>>>     --------------
>>>>         Package: name-version-release of Koji build goes here (e.g.
>>>> scap-security-guide-0.1-16.el6)
>>>>         Type:    select one of - bugfix
>>>> /* intended for updates fixing bugs */
>>>>                                  enhancement
>>>> /* intended for updates adding new features */
>>>>                                  security
>>>> /* intended for updates fixing security flaws */
>>>>                                  newpackage
>>>> /* intended for updates introducing new RPM packages */
>>>>                  options
>>>>         Request: select 'testing' option of - testing
>>>> /* intended for updates that should reach -testing repository first
>>>> (before reaching -stable) */
>>>>                                               stable
>>>> /* intended for updates that should go into -stable directly - maybe
>>>> for critical security updates,
>>>>                                                                          other
>>>> package updates should be pushed to -testing repository first, then
>>>> only after some time
>>>>                                                                          into
>>>> -stable yum repository */
>>>>                                               none
>>>> /* not sure what's this one intended for */
>>>>
>>>>         Bugs:   provide previously created EPEL-6 RH BZ# id here
>>>> [v] Close bugs when update is stable
>>>>
>>>> /* Keep this checkbox checked in. It will automagically close the
>>>>                                                                          created
>>>> EPEL-6 bug, when the particular build is pushed to -stable yum
>>>> repository */
>>>>
>>>>         Notes: Describe the changes in this text field (i.e. which bugs
>>>> got fixed, which new functionality got introduced etc.)
>>>>                The content of this field appear in the advisory (sent
>>>> on fedora-package-announce mailing list), when the build is pushed to
>>>> -stable.
>>>>
>>>>        Suggest Reboot: []                   /* should the package
>>>> update require reboot after install */
>>>>        Enable karma automatism: [v]         /* if to use the karma
>>>> threshold the updates push system to use to decide if the build
>>>>                                                should be pushed to
>>>> -stable channel or not */
>>>>        Threshold for pushing to -stable: [3]      /* minimum threshold
>>>> / level of karma build needs to obtain from package testers to be
>>>>                                                      able to push it
>>>> into -stable channel */
>>>>        Threshold for unpushing [-3]:              /* lower bound for
>>>> negative karma, which is a sign for the push system to move the
>>>>                                                      package from the
>>>> -testing repository. IOW the build has received so much negative
>>>>                                                      karma /
>>>> experiences, it's not usable even for the -testing repository and
>>>> should be
>>>>                                                      rebuilt */
>>>>
>>>>       Once all the information is filed, click "Save Update".
>>>>       This will generate automated email about that build being pushed
>>>> to -testing. After some time at the same day (depending on TZ)
>>>>       the build is pushed to -testing repository.
>>>>
>>>> * The maintainer should check Bodhi pages for that update for positive
>>>> / negative karma / comments. If the build has reached positive karma >= 3
>>>>      it can be pushed to -stable (if it hasn't reached positive karma
>>>>> = 3 in 7 days, it will be pushed to -stable - 7 days is considered
>>>> sufficient
>>>>      period). If there are signs of negative karma, the build should be
>>>> either unpushed / deleted & new one made.
>>>>
>>>> * After 7 days the build can be pushed to -stable (under assumption it
>>>> didn't reach positive karma >= 3 sooner), meaning in next day or
>>>>     two it's reachable via yum subscribed to epel-6 repository directly.
>>>>
>>>> * And the whole process goes back to the first item.
>>>> Hopefully I have covered all the points. Should I realize there's
>>>> missing some yet, will clarify later.
>>> Whoa. This is great.
>> Yes. Thank You, Jan!
>>
>>> I've blocked off ~10 hours between now and Fri to catchup on SSG
>>> activities. I'll spend some of that converting this into DocBook and
>>> merging into our Developers Guide. It'll give us a solid starting point,
>>> and should make future edits easier. Will report back Fri on progress
>>> (+hopefully patches).
>> I believe that the el6 branch from Fedora repository should be dropped
>> once/when/if the scap-security-guide is released through standard Red
>> Hat Enterprise channel.
> Correct, is this process described somewhere? Or just after scap-security-guide
> RPM is released within standard Red Hat Enterprise Linux software updates
> channels it's enough to create FESCO ticket to drop the EPEL-6 branch?
>
>>
>> In the meantime, I have updated epel7 branch with the newest content
> Yeah, noticed new build. Thanks for that.
>
>> and
>> I have one observation to the ssg release process. Currently we have two
>> latest releases:
>>
>>       scap-security-guide-0.1.5.tar.gz
>>         - this contains latest fedora stuff
>>       scap-security-guide-0.1.17.tar.gz
>>         - this contains latest rhel stuff
>>
>> Note that Fedora and EPEL7 packages ship combined content from both
>> packages. The reasoning behind is that with scap-workbench on your
>> fedora box you may want to use RHEL6 content to assess servers in the
>> basement.
> Right, that's correct.
>
>> However, I believe it is not sound project governance to release
>> tarballs of the same name with the different content.
> Agree this is not fortunate solution, but it "somehow" evolved that
> way during the time (first there was Fedora rpm, then EPEL-6 rpm etc.)
> Before (while having the same name) scap-security-guide contained
> different content depending if we were on Red Hat Enterprise Linux or
> Fedora. But we started to ship Red Hat Enterprise Linux datastreams content
> also on Fedora (from reasons you outlined above), so now the originally
> disjoint is converging more and it would make sense to build one tarball
> containing everything (and to have just particular *.spec to decide which
> part of the content should be extracted and which filtered out).
>
>> I would love to
>> see either solution:
>>
>>    (1) Release everything in one tarball named scap-security-guide.
>> Timely releases would be plus. Or
> I like the first option better. Will have a look (if others won't have
> objections to it - if so, please speak up) which changes would be necessary
> to have just one scap-security-guide-N-V-R.tar.gz generated.

Having all RPM releases on the same version (EPEL-6, EPEL-7, Fedora) 
seems most ideal.

Content versioning (within the XCCDF itself) can be done on a 
per-sub-project basis if needed.

>>    (2) Rename Fedora tarball to scap-security-guide-extras
>>
>> If you could tag (`git tag` on particular commit) your releases in git
>> it would be also helpful.
> Yeah, another good recommendation. Was thinking recently about that when
> making 0.1.17 release, that it would be great if, e.g.
>
>    $ git describe
>
> in cloned repo would be able to provide version information. Now it does:
>
>    $ git describe
>    v0.5.0-InitialDraft-1070-gfc50879
>
> so there obviously was such effort in the past, but we hopefully stopped
> to perform the tagging?

IIRC, I added that tag during the first code submission to DISA FSO for 
the RHEL6 STIG. Eventually we started doing RPMs, versus sending raw 
code, so I stopped tagging. There's no reason why we shouldn't start 
that process up again.


More information about the scap-security-guide mailing list