Bill Peck wrote:
Marian Csontos wrote:
Lets keep this discussion on beaker-devel if possible. :-)
> On 11/03/2009 04:40 PM, Bill Peck wrote:
>
>> Hi Marian,
>>
>> Below is an example of how to upload files in real time in beaker. I am
>> doing this for the console logging. It should be very similar for the
>> harness.
>>
>> Two methods for uploading files
>>
>> task_upload_file Uploads file to task specific location
>> (typically the stdout/stderr of the task)
>> recipe_upload_file Uploads file to recipe specific location
>> (typically console logs)
>>
>>
> Oh, I see. I ignored Watchdog class entirely and missed
> recipe_upload_file method.
>
> If I understand sources correctly, Watchdog is separate from proxy and
> does not handle XML-RPC`s at all. Why is the recipe_upload_file method
> in Watchdog class and not in ProxyHelper? Is it because task is not
> expected to upload recipe specific files? Are we sure it is not?
>
It doesn't handle proxies. It does its own authenticated xmlrpc back to
the scheduler.
> E.g. I would like harness to submit its logs in some cases after the
> recipe is done. Though this could be done by adding another task to
> XML, I am not sure it would work in all cases - e.g. when recipe is
> aborted. This would require a way to extend the recycle period, after
> recipe has finished.
>
I'm fine with moving the recipe_upload_file method to the ProxyHelper.
I didn't see a use case at first but your explanation makes sense.
> From *_upload_file doc-string:
>
>> path: the relative path to upload to
>>
> and it is used here:
>
>> self.recipe_upload_file(watchdog['recipe_id'],
>> "/",
>> "console.log",
>> size,
>> md5sum,
>> where,
>> data)
>>
> What is supposed to be relative address starts with a slash.
>
Let me show you what happens on the server side...
upload = Uploader(config.get("basepath.logs",
"/var/www/beaker/logs"))
recipe_path = "%02d/%s/%s/%s" % (int(str(job_id)[-2:]),
job_id,
recipe_id,
path)
return self.upload.uploadFile(recipe_path,
name,
size,
md5sum,
offset,
data)
so path is relative to
/var/www/beaker/logs/last2digitsofjobid/jobid/recipid/
There are checks to make sure path does not include ..
> Is it correct?
> Is the slash mandatory?
>
I think you could pass "" and it would be fine. But to me the
slash is
saying I'm uploading to the root of my recipe. (or task if you are
doing a task upload)
>
>> server = xmlrpclib.Server("http://%s:%s" % (server_hostname, port))
>>
>> if os.path.exists(filepath):
>> file = open(filepath, "r")
>> while True:
>> where = file.tell()
>> line = file.read(65536)
>> if not line:
>> time.sleep(1)
>> file.seek(where)
>> else:
>> size = len(line)
>> data = base64.encodestring(line)
>> md5sum = md5_constructor(line).hexdigest()
>> server.task_upload_file(task_id,
>> "/",
>> "taskoutput.log",
>> size,
>> md5sum,
>> where,
>> data)
>>
>>
>> Should be able to do the above from a pipe as well. May have to change
>> the seek
>> _______________________________________________
>> Beaker-devel mailing list
>> Beaker-devel(a)lists.fedorahosted.org
>>
https://fedorahosted.org/mailman/listinfo/beaker-devel
>>
>>
_______________________________________________
Beaker-devel mailing list
Beaker-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/beaker-devel