Hi Docs gangstas,
I am steadily making progress toward a packaging solution. Thanks to Tommy jumping in with XSLT-based solutions, I learned a lot (although not nearly enough, of course) about how to solve other problems using the same building blocks. I also -- finally! -- figured out how to package documentation for KDE's khelpcenter, and believe me, it's not self-evident. Or rather, the end-state is comprehensible, but the procedure for getting there cleanly is not. In any case, that's solved too.
Also, the "fedora-doc-common" RPM is just about ready for rollout and will contain (hopefully) everything required for people not operating out of CVS to build docs. Mostly this just involves including our XSL snippets, entities, custom scripts, and so forth. This is not to say that by installing this RPM people will be able to just happily build away, but it puts that goal within reach.
Since the fedora-doc-* stuff will probably live in Fedora Extras, and because that is a rolling repository, it is much easier for us to work on this in phases. Phase One goals are for our currently available documents to be installable by an end user using yum, and that those documents should show up prominently in each of the locations a user might expect:
1. Launching "Desktop -> Help" for GNOME or KDE 2. Launching "Documentation -> [title]"
I don't know yet what Phase Two entails, but some goals might include:
1. Nice Python scripts for creating new XML document templates 2. " " " for building DRAFT-marked documents?
Some items I am still in the process of solving, by which I mean I should be able to finish them for Phase One:
- The rpm-info.dtd needs some lovin': - Packages should not get separate versions, too confusing - Docs should not get releases, also confusing - Figure out a way to condense generic person entries for use as authors, editors, translators, and/or packagers? Alternately, remove those not needed -- e.g. packagers only need full name and email, authors, editors and translators need "component" names, and people performing revisions only need initials. Some reference to dbpoolx.mod shows this shouldn't be too hard to do.
- Provide functionality to automatically sort "doc" or "rpm" revisions. Because RPM is particular about things like date formats in %changelog entries, we may need to "encourage" proper formatting using the DTD. We already have a *strong* hint there in the attributes Tommy provided, but I don't think there's anything built in to a DTD to check for ordering; I believe, however, that XSLT can sort. Anything we can do to bulletproof the process is probably good. If it proves too cumbersome to complete RSN, I'll push this off until Phase Two.
That's it from my neck of the woods. Hope everyone is enjoying the holidays. The holiday break and the patience of my ever-lovin' wife are the main reason I've been able to work on this stuff for the last few days. Best to all, and here's to a Happy New Year!
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
I am steadily making progress toward a packaging solution. Thanks to Tommy jumping in with XSLT-based solutions, I learned a lot (although not nearly enough, of course) about how to solve other problems using the same building blocks. I also -- finally! -- figured out how to package documentation for KDE's khelpcenter, and believe me, it's not self-evident. Or rather, the end-state is comprehensible, but the procedure for getting there cleanly is not. In any case, that's solved too.
I've seen that fast and furious activity ;-) Cheers that it is finally coming together. Even more, that you have drilled down for the crufty details for interfacing with those infuriating GUI's.
Also, the "fedora-doc-common" RPM is just about ready for rollout and will contain (hopefully) everything required for people not operating out of CVS to build docs. Mostly this just involves including our XSL snippets, entities, custom scripts, and so forth. This is not to say that by installing this RPM people will be able to just happily build away, but it puts that goal within reach.
Yes, necessary precursor step.
Phase One goals are for our currently available documents to be installable by an end user using yum, and that those documents should show up prominently in each of the locations a user might expect:
- Launching "Desktop -> Help" for GNOME or KDE
- Launching "Documentation -> [title]"
Agreed. Phase One should be getting our Docs and tools in place.
I don't know yet what Phase Two entails, but some goals might include:
- Nice Python scripts for creating new XML document templates
- " " " for building DRAFT-marked documents?
Reasonable. Why the python focus ;-? Not that I'm actually against it.
I would like to see a starter RPM that would explode into a new document skeleton and automatically connect to the docs-common RPM.
Some items I am still in the process of solving, by which I mean I should be able to finish them for Phase One:
- The rpm-info.dtd needs some lovin':
- Packages should not get separate versions, too confusing
- Docs should not get releases, also confusing
- Figure out a way to condense generic person entries for use as
authors, editors, translators, and/or packagers? Alternately, remove those not needed -- e.g. packagers only need full name and email, authors, editors and translators need "component" names, and people performing revisions only need initials. Some reference to dbpoolx.mod shows this shouldn't be too hard to do.
Use caution in trying to compress the person stuff. While the minimal information needed varies according to that person's current role, the information pool in "rpm-info.xml" is not large, not redundant, and not an issue. Consider if this is over-optimization.
Take a tip from the RDBMS crowd and collect all the person info into a central location (maybe just a <workers>..</workers> set) and then use ID linkage to pull the necessary stuff out of that.
Provide functionality to automatically sort "doc" or "rpm" revisions. Because RPM is particular about things like date formats in %changelog entries, we may need to "encourage" proper formatting using the DTD. We already have a *strong* hint there in the attributes Tommy provided, but I don't think there's anything built in to a DTD to check for ordering; I believe, however, that XSLT can sort. Anything we can do to bulletproof the process is probably good. If it proves too cumbersome to complete RSN, I'll push this off until Phase Two.
I considered this, but skipped it because of our current crazy numberings. Besides, with that strong hint, we can just shoot whomever screws it up. Updating the revision info isn't a frequent activity and just didn't seem worth the effort until everything else was working.
I wouldn't mind the XSLT doing a lint on the ordering, unless RPM will do that later.
That's it from my neck of the woods. Hope everyone is enjoying the holidays. The holiday break and the patience of my ever-lovin' wife are the main reason I've been able to work on this stuff for the last few days. Best to all, and here's to a Happy New Year!
Right back at 'cha.
Cheers
On Wed, 2005-12-28 at 19:27 -0600, Tommy Reynolds wrote:
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
Also, the "fedora-doc-common" RPM is just about ready for rollout and will contain (hopefully) everything required for people not operating out of CVS to build docs. Mostly this just involves including our XSL snippets, entities, custom scripts, and so forth. This is not to say that by installing this RPM people will be able to just happily build away, but it puts that goal within reach.
Yes, necessary precursor step.
And as I write this, that goal is very, VERY close.
[...snip...]
I don't know yet what Phase Two entails, but some goals might include:
- Nice Python scripts for creating new XML document templates
- " " " for building DRAFT-marked documents?
Reasonable. Why the python focus ;-? Not that I'm actually against it.
See any recent quasi-rant by Seth Vidal on f-devel-l regarding tools for the platform.
I would like to see a starter RPM that would explode into a new document skeleton and automatically connect to the docs-common RPM.
No new RPM needed, this will simply be a small script in the docs-common RPM.
Some items I am still in the process of solving, by which I mean I should be able to finish them for Phase One:
- The rpm-info.dtd needs some lovin':
- Packages should not get separate versions, too confusing
- Docs should not get releases, also confusing
- Figure out a way to condense generic person entries for use as
authors, editors, translators, and/or packagers? Alternately, remove those not needed -- e.g. packagers only need full name and email, authors, editors and translators need "component" names, and people performing revisions only need initials. Some reference to dbpoolx.mod shows this shouldn't be too hard to do.
Use caution in trying to compress the person stuff. While the minimal information needed varies according to that person's current role, the information pool in "rpm-info.xml" is not large, not redundant, and not an issue. Consider if this is over-optimization.
Take a tip from the RDBMS crowd and collect all the person info into a central location (maybe just a <workers>..</workers> set) and then use ID linkage to pull the necessary stuff out of that.
I like what you did with the worker stuff. I am going to change one small thing to make the option field "#IMPLIED" for othername as I believe is customary, but it's pretty much 6 of one, half-dozen of t'other.
Provide functionality to automatically sort "doc" or "rpm" revisions. Because RPM is particular about things like date formats in %changelog entries, we may need to "encourage" proper formatting using the DTD. We already have a *strong* hint there in the attributes Tommy provided, but I don't think there's anything built in to a DTD to check for ordering; I believe, however, that XSLT can sort. Anything we can do to bulletproof the process is probably good. If it proves too cumbersome to complete RSN, I'll push this off until Phase Two.
I considered this, but skipped it because of our current crazy numberings. Besides, with that strong hint, we can just shoot whomever screws it up. Updating the revision info isn't a frequent activity and just didn't seem worth the effort until everything else was working.
I had an interesting thought about this, which is that rather than having a "version" and "release" attribute, only one of which is applicable for each type of revision, why not use a single "revnumber" attribute? For docs, this would be a version change, and for new package rolls it would be a release change. The latest version for the doc and the RPM would always be the "revnumber" attribute of the topmost role="doc" revision, and the RPM's release would always be the "revnumber" attribute of the topmost role="rpm" revision.
I wouldn't mind the XSLT doing a lint on the ordering, unless RPM will do that later.
I thought the same thing, I'll see what I can do.
The only possible issue I see in the commit you made to rpm-info.dtd this evening is in requiring copyright holders to be workers. For example, Red Hat, Inc. and the Fedora Foundation won't fit that type of element, although one or both of them will certainly be copyright holders on several of our works. If we're not going to keep it as PCDATA, perhaps using subelements (worker|corp) might work. I think the KISS method, though, might dictate punting this one.
And in other news, RPM building now works for example-tutorial. To witness the awe-inspiring power, update your docs-common and example-tutorial modules. Then do the following:
1. cd docs-common 2. make rpm 3. su -c 'yum localinstall fedora-doc-common*.noarch.rpm' 4. cd ../example-tutorial # your path may be a little different 5. make rpm 6. su -c 'yum localinstall fedora-doc-example-tutorial*.noarch.rpm'
Note that the goodness comes through on both the GNOME and KDE menus, and if you are running Rawhide (FC5test1 or above), you'll also see our entries in the global "Help" menu. (Yes, the draft watermark is still there, but it will take me about 30 sec to get rid of that, but I'm too tired right now.)
There are still a few minor quirks to iron out, like how to make respectable %changelog entries that meet Fedora standards. And building from SRPM will work after I get some sleep. ;-) But these are pretty minor compared to the work already done.
2006 is looking pretty good, at least from where I'm standing!
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
- cd docs-common
- make rpm
This step failed for me:
+ for i in xmlformat xmldiff copy-figs bulkbuild tidy-bowl + install -m 755 /home/reynolds/src/f/fedora-docs/docs-common/rpmbuild/BUILD/fedora-doc-common-cvs20051230/bin/xmlformat /var/tmp/fedora-doc-common-0.1-cvs20051230-root-reynolds//usr/share/fedora/doc/docs-common/bin/ install: cannot stat `/home/reynolds/src/f/fedora-docs/docs-common/rpmbuild/BUILD/fedora-doc-common-cvs20051230/bin/xmlformat': No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.58603 (%install)
- su -c 'yum localinstall fedora-doc-common*.noarch.rpm'
- cd ../example-tutorial # your path may be a little different
- make rpm
- su -c 'yum localinstall fedora-doc-example-tutorial*.noarch.rpm'
Didn't try these, though.
Note that the goodness comes through on both the GNOME and KDE menus, and if you are running Rawhide (FC5test1 or above), you'll also see our entries in the global "Help" menu. (Yes, the draft watermark is still there, but it will take me about 30 sec to get rid of that, but I'm too tired right now.)
I'll add support for the Enlightenment E16, since that is really the window manager everybody should be using ;-)
There are still a few minor quirks to iron out, like how to make respectable %changelog entries that meet Fedora standards. And building from SRPM will work after I get some sleep. ;-) But these are pretty minor compared to the work already done. 2006 is looking pretty good, at least from where I'm standing!
Well, done, good and faithful coder! I'll recommend that quaid gives you a 15% raise!
P.S.: try to get this done before New Year's Eve, or just wait a few days...
Cheers
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
I had an interesting thought about this, which is that rather than having a "version" and "release" attribute, only one of which is applicable for each type of revision, why not use a single "revnumber" attribute? For docs, this would be a version change, and for new package rolls it would be a release change. The latest version for the doc and the RPM would always be the "revnumber" attribute of the topmost role="doc" revision, and the RPM's release would always be the "revnumber" attribute of the topmost role="rpm" revision.
Which ever is easier to parse.
Personally, I prefer the two-attribute approach because that seems more parallel to the actual update process: just change the version and release, similar to what would be done to manually construct the SPEC file. The 'role="rpm"' seems a bit artificial to me; perhaps it's just NIH syndrome.
The only possible issue I see in the commit you made to rpm-info.dtd this evening is in requiring copyright holders to be workers. For example, Red Hat, Inc. and the Fedora Foundation won't fit that type of element, although one or both of them will certainly be copyright holders on several of our works. If we're not going to keep it as PCDATA, perhaps using subelements (worker|corp) might work. I think the KISS method, though, might dictate punting this one.
Oops, I forgot about corporate ownership. OK, I'll revert it. /me updates CVS Done!
Happy New Year!
On Fri, 2005-12-30 at 02:22 -0500, Paul W. Frields wrote:
And in other news, RPM building now works for example-tutorial. To witness the awe-inspiring power, update your docs-common and example-tutorial modules. Then do the following:
- cd docs-common
- make rpm
- su -c 'yum localinstall fedora-doc-common*.noarch.rpm'
- cd ../example-tutorial # your path may be a little different
- make rpm
- su -c 'yum localinstall fedora-doc-example-tutorial*.noarch.rpm'
Note that the goodness comes through on both the GNOME and KDE menus, and if you are running Rawhide (FC5test1 or above), you'll also see our entries in the global "Help" menu. (Yes, the draft watermark is still there, but it will take me about 30 sec to get rid of that, but I'm too tired right now.)
Sorry to reply to myself, but that's fixed now. Though the list would like to know. ;-)
There are still a few minor quirks to iron out, like how to make respectable %changelog entries that meet Fedora standards. And building from SRPM will work after I get some sleep. ;-) But these are pretty minor compared to the work already done.
Building from SRPM now works too, and HTML stuff has the draft watermark if the SRPM is the source for the build.
Still remaining: %changelog lint stuff, and one added challenge: Need to draft watermark the XML stuff rendered in yelp by default. I think the DocBook in yelp may be rendered to HTML using the stuff in /usr/share/yelp, but if anyone could pitch in an explanation and advice, that would be great. Please don't be afraid to be pedantic, it helps not only me but the googlemind as well. (*grumble grumble* meinproc *grumble*)
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
Sorry to reply to myself, but that's fixed now. Though the list would like to know. ;-)
Confirmed: RPM building now works for docs-common and example-tutorial. Great work, Paul! I'll consider it as a late Christmas present ;-)
The "example-tutorial/Makefile" is ugly; once this stabilizes a bit, we can work on migrating it to either "docs-common/Makefile.common" or some separate shell scripts. That should be trivial, I think.
Major milestone, here. Take the rest of the year off...
Cheers!
On Fri, 2005-12-30 at 07:29 -0600, Tommy Reynolds wrote:
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
- cd docs-common
- make rpm
This step failed for me:
- for i in xmlformat xmldiff copy-figs bulkbuild tidy-bowl
- install -m 755 /home/reynolds/src/f/fedora-docs/docs-common/rpmbuild/BUILD/fedora-doc-common-cvs20051230/bin/xmlformat /var/tmp/fedora-doc-common-0.1-cvs20051230-root-reynolds//usr/share/fedora/doc/docs-common/bin/
install: cannot stat `/home/reynolds/src/f/fedora-docs/docs-common/rpmbuild/BUILD/fedora-doc-common-cvs20051230/bin/xmlformat': No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.58603 (%install)
WORKSFORME with a brand-new user. Does the 'xmlformat' binary appear in your ~/src/f/fedora-docs/docs-common/bin/ directory where it should? Part of the RPM specfile does a 'cvs up' in 'docs-common' -- are you doing a .cvsignore that prevents it from restoring this program, if it doesn't appear there already?
On Fri, 2005-12-30 at 07:39 -0600, Tommy Reynolds wrote:
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
I had an interesting thought about this, which is that rather than having a "version" and "release" attribute, only one of which is applicable for each type of revision, why not use a single "revnumber" attribute? For docs, this would be a version change, and for new package rolls it would be a release change. The latest version for the doc and the RPM would always be the "revnumber" attribute of the topmost role="doc" revision, and the RPM's release would always be the "revnumber" attribute of the topmost role="rpm" revision.
Which ever is easier to parse.
Personally, I prefer the two-attribute approach because that seems more parallel to the actual update process: just change the version and release, similar to what would be done to manually construct the SPEC file. The 'role="rpm"' seems a bit artificial to me; perhaps it's just NIH syndrome.
This seems counter-intuitive to me, since documents have a version, not a release -- any document change means a version change (or it should if the editor is doing his job). Meanwhile, RPMs have a release, and the version is tied to the document source inside. The RPM may change versions, but only in conjunction with a document source change. And documents may change versions several times without any release, as they are progressively edited -- especially now that we have the webtest host where drafts are rolled during the work-in-progress.
This might be just the occasion to move out of the attribute space and into the subelement space, thus:
<revision role="doc" date="2005-12-31"> <author worker="me"/> <revnumber dim="version">0.15</revnumber> <details>Style changes</details> </revision> <revision role="rpm" date="Sat Dec 31 2005"> <author worker="stickster"/> <revnumber dim="release">1</revnumber> <details>Update to 0.15</details> </revision>
But frankly, that doesn't get easier to parse for people, which I was under the impression was the whole point behind XML and DTDs. Plus it leaves wiggle room for valid but erroneous entries. We might be well-served by making the DTD do the work for us:
<docrevision date="2005-12-31" version="0.15"> <author worker="me"/> <details>Style changes</details> </docrevision> <pkgrevision date="Sat Dec 31 2005" release="1"> <author worker="me"/> <details>Update to 0.15</details> </pkgrevision>
This may be a case where it's good to sacrifice compactness on the altar of comprehensibility.
The only possible issue I see in the commit you made to rpm-info.dtd this evening is in requiring copyright holders to be workers. For example, Red Hat, Inc. and the Fedora Foundation won't fit that type of element, although one or both of them will certainly be copyright holders on several of our works. If we're not going to keep it as PCDATA, perhaps using subelements (worker|corp) might work. I think the KISS method, though, might dictate punting this one.
Oops, I forgot about corporate ownership. OK, I'll revert it. /me updates CVS Done!
Saw this, thanks. For some reason I'm getting your mail to the list really late, which is why my response is so delayed in real time.
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
<docrevision date="2005-12-31" version="0.15"> <author worker="me"/> <details>Style changes</details> </docrevision> <pkgrevision date="Sat Dec 31 2005" release="1"> <author worker="me"/> <details>Update to 0.15</details> </pkgrevision> This may be a case where it's good to sacrifice compactness on the altar of comprehensibility.
Since we simply disagree, let's just pick one, use it for a while, and then see which works better in the long run. Since we have multiple solutions, let evolution take its course. Pick one.
Saw this, thanks. For some reason I'm getting your mail to the list really late, which is why my response is so delayed in real time.
Yeah, I saw them arrive long after they were sent.
Thank BellSouth for blocking all 25/tcp and 25/udp access unless its to their very own SMTP server. They claim it is to help prevent relaying; they don't seem to believe there are other solutions. At that, they're better than the Charter setup that's my only alternative.
Cheers
Tommy Reynolds wrote:
Saw this, thanks. For some reason I'm getting your mail to the list really late, which is why my response is so delayed in real time.
Yeah, I saw them arrive long after they were sent.
Thank BellSouth for blocking all 25/tcp and 25/udp access unless its to their very own SMTP server. They claim it is to help prevent relaying; they don't seem to believe there are other solutions. At that, they're better than the Charter setup that's my only alternative.
Given the BellSouth/Charter/time proximity, I can only say the following: Happy New Year! Ein gutes neues Jahr! Feliz ano nuevo!
<my sincere apologies to the remaining languages of Terra>
Sincerely, Chris
On Sat, 2005-12-31 at 10:09 -0600, Tommy Reynolds wrote:
Thank BellSouth for blocking all 25/tcp and 25/udp access unless its to their very own SMTP server. They claim it is to help prevent relaying; they don't seem to believe there are other solutions. At that, they're better than the Charter setup that's my only alternative.
Blocking access to port 25 isn't such a bad plan in the grand scheme of things. What's the actual problem -- just that their servers are very slow to pass mail through? You could use an external mail server listening on a port other than 25, surely?
Uttered David Woodhouse dwmw2@infradead.org, spake thus:
Blocking access to port 25 isn't such a bad plan in the grand scheme of things. What's the actual problem -- just that their servers are very slow to pass mail through? You could use an external mail server listening on a port other than 25, surely?
Agreed: filtering port 25 is an acceptable part of an over-all security scheme. Unfortunately, BellSouth appear to never have heard of authentication, certificates, and the like.
Tunneling could work if there were someone on the other end. Wanna volunteer?
Cheers
On Sun, 2006-01-01 at 09:29 -0600, Tommy Reynolds wrote:
Agreed: filtering port 25 is an acceptable part of an over-all security scheme. Unfortunately, BellSouth appear to never have heard of authentication, certificates, and the like.
Surely if you're physically within their network you don't _need_ authentication? You just use their smarthost and you're authenticated by virtue of coming from one of their own customer IP addresses.
Tunneling could work if there were someone on the other end.
I wasn't thinking of tunnelling -- I was thinking of just connecting directly to the standard MSA port of a mail server elsewhere (cf. RFC2476).
Wanna volunteer?
I don't see why not. Use smtpauth.infradead.org -- username and password under separate cover.
Tommy Reynolds wrote:
Uttered "Paul W. Frields" stickster@gmail.com, spake thus:
<docrevision date="2005-12-31" version="0.15"> <author worker="me"/> <details>Style changes</details> </docrevision> <pkgrevision date="Sat Dec 31 2005" release="1"> <author worker="me"/> <details>Update to 0.15</details> </pkgrevision> This may be a case where it's good to sacrifice compactness on the altar of comprehensibility.
Since we simply disagree, let's just pick one, use it for a while, and then see which works better in the long run. Since we have multiple solutions, let evolution take its course. Pick one.
Saw this, thanks. For some reason I'm getting your mail to the list really late, which is why my response is so delayed in real time.
Yeah, I saw them arrive long after they were sent.
Thank BellSouth for blocking all 25/tcp and 25/udp access unless its to their very own SMTP server. They claim it is to help prevent relaying; they don't seem to believe there are other solutions. At that, they're better than the Charter setup that's my only alternative.
Cheers
You can call the customer service line and request an exemption from the port 25 blocking. I did this and had full port 25 privileges within minutes. In the case of their dynamic solutions, the exemption is connected to your PPPoE login, so it follows your IP changes.