How we deal with I/O from the task engine
by Michael DeHaan
As a consequence of the task engine, we need a more reliable way of
directing IO to each task to the appropriate logfiles.
For "actions" in cobbler, this means in api.py we will have a new way to
pass in stdin/stdout that defaults to the sys.stdin,sys.stdout.
When the API is invoked from remote.py, for background tasks, we will
override these with the new log handles for that object.
When we call subprocess, we will pass these log handles down into
subprocess so it too can log output to the right places.
We cannot globally modify sys.stdin and sys.stdout as that is shared
between threads, though we may trap everything from cobblerd that
is not otherwise trapped to /var/log/cobbler/some.log until we can make
sure it's all logged properly (as a result, any thing in this log should
be deemed to be logged to the wrong place).
For instance, in api.py reposync was:
def reposync(self, name=None, tries=1, nofail=False):
This will need to be:
def reposync(self, name=None, tries=1, nofail=False,
stdout=sys.stdout, stderr=sys.stderr):
--Michael
14 years, 11 months
Task Engine!
by Michael DeHaan
I have some code checked in on the master branch that is the initial
start of a task engine for background tasks in Cobbler 2.0
Currently there is a background_sync method, that would show the path to
doing nearly anything backgroundy (we need to tweak it a bit to allow
passing arguments).
It is very rough, but works, (and uses threads)
Eventually this can be used for:
* Reposyncs from the web app -- any specific repo or just one
* Calls to the deployment API from the web app
* Lots of batch actions.
* Cobbler replicate from the web app
When I say the web app, I also mean XMLRPC.
The next step for this is to make a task status page for the web
application, showing at least the most recent tasks. (We can worry
about paginating this later, I don't think it will be super-important
as no one should be kicking off /THAT/ many tasks and we'll still be
logging).
We are also going to modify things so that background tasks trap
stderr/stdout and send them to /var/log/cobbler/tasks/task_id.log
Task logs that are complete or failed for, say, 1 hr, will also be
auto-deleted by cobblerd.
Once the task engine is complete, we can work on implementing *all*
actions in cobblerd that cobbler can do, as well as working on remoting
the command line.
As a sidenote, remember that until this is done changes in cobbler's CLI
will /not/ show up in the web app. When we are done, there will be no
local CLI.
--Michael
14 years, 11 months
Some minor setting renaming... (yumreposync_flags -> reposync_flags) and why
by Michael DeHaan
Perhaps a first (very minor) casuality of the cleanup in 2.0, we had a
flag named
yumreposync_flags.
Ideally this should be yum_reposync_flags
Except the CLI option has always been reposync_flags
To keep it consistent, we have to name the flag in settings
"reposync_flags", which means upon upgrade folks will have to rename the
flag in their settings file.
Ditto for createrepo_flags.
If they don't rename the flags, nothing bad will happen, they'll just
get the new defaults versus whatever customizations.
14 years, 11 months
[PATCH] Auto-discover repo metadata
by Garrett Holmstrom
This proposed patch for cobbler makes the reposync action search for
repo metadata using the repo's main metadata file so we don't rely on
metadata files' having certain names like I describe in
https://fedorahosted.org/cobbler/ticket/454. After finding metadata
files the createrepo call can use it adds them using the appropriate
command line switches. Turn it off with "discover_repodata: 0" in
/etc/cobbler/settings if you don't want this behavior.
(This patch is based on commit b9380c592148eaa1af4531663b1f02e87a2f3321
on git://github.com/gholms/cobbler.git)
14 years, 11 months
Extremely limited Python 1.5 support ... and a branching decision
by Michael DeHaan
I've made some changes to koan to allow the main codepath (when executed
from Python only, for the purposes of Satellite Server managed systems
still running RHEL 2.1) to work under Python 1.5 (aka RHEL 2)... just
for the "--replace-self" functionality.
The code is rather ugly, so I was thinking I /could/ just fork it and
remove the non-Python 1.5 parts on a "rhel21" branch, but was worried a
bit about feature drift if the same
codepaths weren't available for both. I also think it's unfair to
expect people contributing patches to remember how Python 1.5 works or
to test things on Python 1.5.
So, anyway, right now there is an import test that sets ANCIENT_PYTHON =
1 if it can't import a module that is present in other python distributions.
Given this is a bit evil for folks to maintain, perhaps branching it
(making a rhel21 branch) is a better idea. This would be effectively
freezing EL2.1 support forever
at 1.6.x levels.
The only catch here is that we can't be certain we'll never have to
modify that branch again, when adding new features.
However, the code path for --replace-self has been stable. So I'm
leaning that way.
--Michael
14 years, 11 months
[PATCH] web app batch action fixes
by Vreman, Peter
0002-fix-row-name-to-be-item-name-instead-of-hash.patch
0003-domulti-show-error-page-instead-of-raising-exception.patch
0007-multi-netboot-fixed-and-added-additional-validation.patch
0008-netboot-disable-fixed.patch
Regards,
Peter
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
14 years, 11 months
[PATCH] remote API refactoring to use generic methods
by Vreman, Peter
Please find patches attached that refactor more remote methods to use the generic item functions.
The patch removes another 200 lines and 8KB of source code.
Regards,
Peter
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
14 years, 11 months
Further storage object planning
by Michael DeHaan
From talking with Scott Henson and John Eckersberg today, we were
working out the storage ideas a bit, and came to some conclusions that
we wanted to share here since we were talking about this recently, to
see what everyone thinks....
cobbler storage add ...
This command will create objects with either --type=nfs or --type=san
This represents the external filer.
This will also take various connection/login parameters
"cobbler sync" will connect to the device and make sure it is configured
according to the parameters.
Each cobbler interface will contain one or more "mounts" which will work
sort of in the same vein as interfaces.
A mount will have
a storage object reference
a WWN, if for a SAN
etc
When a kickstart file request is made, cobbler will use a background
task to talk to the remote storage configured for that system to
configure the remote storage.
In %post, a snippet will continue the remainder of the setup.
The actual implementations of the various actions are vendor specific,
but in general, the only user command to initiate action will be
"cobbler sync" (which we will tweak to allow for syncing just storage,
or just files, or just DHCP/DNS also)
In the UI, storage will be a new object type... and mounts will be on
the system page just like interfaces
We will need to tweak the UI to allow for expandable/collapseable
sections as there's a bit too much for one page of scrolling right now,
even if they are color coded and grouped together.
--Michael
14 years, 11 months
[PATCH] rpm build and noisy log item fix
by Vreman, Peter
- Remove kwargs log entry that was written for every object when loading web app list page:
0001-remove-noisy-log-for-find-item.patch
- Fix RPM build for new snippet:
0003-domulti-show-error-page-instead-of-raising-exception.patch
Regards,
Peter
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
14 years, 11 months
Further deployment API planning
by Michael DeHaan
Scott's going to share some on this, but we were discussing this as well.
(A) The deployment API should also be able to deploy physical systems,
knowing to first try Func (if installed and reachable), then SSH, then
PXE. Power management if enabled should be used
for the PXE piece.
(B) The deployment API should not need to take a method... we should
try Func first, then SSH.
(C) We need to expose this in the web app. Most likely what we will
do is have a button in the systems list page that pops up a floating DIV.
In that DIV, we can choose whether we want to deploy a system definition
physically or virtually, and if virtually to what host or what virtual
group.
Since this will use the new task engine the results will go to the task
page once the buttons are kicked off.
--Michael
14 years, 11 months