I'm trying to get plague set up and I'm running into compatibility problems that lead me to think I'm starting in the wrong place.
I picked FC7 and sqlite to start with for plague's home OS and a non-DB-server DB option. The production system will be RHEL4/5 once I'm comfortable I know how to get things running smoothly. I tried briefly with RHEL4 a few months back and didn't get this far.
I started with a fresh install of FC7 (incl. python-2.5, python-sqlite2) and plague 0.4.4.1-4.fc7 RPMs. After running into some of the first issues I switched to CVS MAIN with similar results.
Here's what I've done so far:
* server/DBManager.py - import sqlite3 as sqlite - remove encoding="utf-8" from sqlite.connect
* server/main.py * server/BuildMaster.py - sqlite hack: move dbm connection into BuildMaster so it can create one for each thread.
* common/XMLRPCServerProxy.py - set self._use_datetime = 0
* common/AuthedXMLRPCServer.py - pass allow_none=False and encoding=None to SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
* builder/Config.py server/Config.py - change BaseConfig.BaseConfig.ConfigError to BaseConfig.ConfigError
I have the builder and server on the same host, w/o SSL yet. With the 0.4.4.1 RPMs and some of the changes above the server was able to talk to the builder but I ran into the sqlite problem and switched to CVS at that point.
CVS has the new Active/Passive builders but the default port configs don't appear to be correct for having the builder and server on the same machine, but I can't match up the config sections and field names in the server and builder configs to point them at each other correctly. At this point, the server's not seeing any builders. I tried running 'plague-client list' anyway and get an exception "local variable 'curs' referenced before assignment" which I'm pretty sure is caused by my sqlite thread hack.
What OS+versions should I use to get plague up and running most quickly?
Once I'm up and running in a known-working environment I'd have a better idea of what changes are right for FC7 or RHEL4/5.
Thanks,
-- Ken
Hi Ken!
Am 2007-02-25 21:26, Ken MacLeod schrieb: [ ... ]
Here's what I've done so far:
server/DBManager.py
- import sqlite3 as sqlite
- remove encoding="utf-8" from sqlite.connect
server/main.py
server/BuildMaster.py
- sqlite hack: move dbm connection into BuildMaster so
it can create one for each thread.
common/XMLRPCServerProxy.py
- set self._use_datetime = 0
Why that?
- common/AuthedXMLRPCServer.py
SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
- pass allow_none=False and encoding=None to
Why that?
- builder/Config.py server/Config.py
BaseConfig.ConfigError
- change BaseConfig.BaseConfig.ConfigError to
Yes. I already submitted a patch to Dan for this.
I have the builder and server on the same host, w/o SSL yet. With the 0.4.4.1 RPMs and some of the changes above the server was able to talk to the builder but I ran into the sqlite problem and switched to CVS at that point.
So you' running 0.5.0 now. The same as I do. Right?
CVS has the new Active/Passive builders but the default port configs don't appear to be correct for having the builder and server on the same machine, but I can't match up the config sections and field names in the server and builder configs to point them at each other correctly. At this point, the server's not seeing any builders. I tried running 'plague-client list' anyway and get an exception "local variable 'curs' referenced before assignment" which I'm pretty sure is caused by my sqlite thread hack.
What OS+versions should I use to get plague up and running most quickly?
I have plg 0.5.0. mysql 4.1.20 running on ES4. But I also have an instance running on FC6.
Once I'm up and running in a known-working environment I'd have a better idea of what changes are right for FC7 or RHEL4/5.
Example config: builder.cfg: ==================================== [General] comm_type = active builder_user = plague-builder hostname = somehost.somecompany.com server = somehost.somecompany.com debug = True max_jobs = 3 builder_cmd = /usr/bin/mock
[Directories] target_configs_dir = /etc/plague/builder/targets builder_work_dir = /tmp/builder_work
[SSL] use_ssl = False
[Active] xmlrpc_port = 8889 fileserver_port= 8890
server.cfg: ==================================== [Database] engine = mysql
[Directories] repo_dir = /repodir mock_configs_dir = /etc/mock server_work_dir = /rpmbuild target_configs_dir = /etc/plague/server/targets tmpdir = /tmp
[Active Builders] xmlrpc_server_port = 8889 file_server_port = 8890 builder1 = 0 somehost2.somecompany.com builder2 = 50 somehost.somecompany.com
[General] traceback_server = no hostname = somehost.somecompany.com depsolve_jobs = no Debug = True allow_reenqueue = no
[SSL] server_key_and_cert = /etc/plague/server/certs/server/server_key_and_cert.pem ca_cert = /etc/plague/server/certs/ca/my_ca_ca_cert.pem
[UI] log_url = http://somehost.somecompany.com/buildlogs/ guest_allowed = no port = 8887 use_ssl = yes client_ca_cert = /etc/plague/server/certs/ca/client_ca_ca_cert.pem
[mysql Engine] timeout = 3 host = localhost user = root password = database = plague
[CVS] use_cvs = yes
[Builders] use_ssl = no
[Email] success_emails = email_from = buildsys@somecompany.com admin_emails = of@redhat.at
If you need help creating the ssl certificates or anything else, just mail me. :-) And don't use sqlite as database backend; It often runs into locking problems; That's why I switched to mysql. However, you do need to create the tables in the mysql database first. I have the statements ready if you need...
Best, Oliver
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
Hi Ken!
Am 2007-02-25 21:26, Ken MacLeod schrieb: [ ... ]
Here's what I've done so far:
- server/DBManager.py
- import sqlite3 as sqlite
- remove encoding="utf-8" from sqlite.connect
Why these?
- server/main.py
- server/BuildMaster.py
it can create one for each thread.
- sqlite hack: move dbm connection into BuildMaster so
sqlite sucks for threading, it's best to just setup a postgresql or mysql server.
- common/XMLRPCServerProxy.py
- set self._use_datetime = 0
Why that?
Probably due to Python 2.5.
- common/AuthedXMLRPCServer.py
SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
- pass allow_none=False and encoding=None to
Why that?
- builder/Config.py server/Config.py
BaseConfig.ConfigError
- change BaseConfig.BaseConfig.ConfigError to
Yes. I already submitted a patch to Dan for this.
Hmm, I thought I'd committed that. Will need to look again.
I have the builder and server on the same host, w/o SSL yet. With the 0.4.4.1 RPMs and some of the changes above the server was able to talk to the builder but I ran into the sqlite problem and switched to CVS at that point.
So you' running 0.5.0 now. The same as I do. Right?
CVS has the new Active/Passive builders but the default port configs don't appear to be correct for having the builder and server on the same machine, but I can't match up the config sections and field names in the server and builder configs to point them at each other correctly. At this point, the server's not seeing any builders. I tried running 'plague-client list' anyway and get an exception "local variable 'curs' referenced before assignment" which I'm pretty sure is caused by my sqlite thread hack.
What OS+versions should I use to get plague up and running most quickly?
I have plg 0.5.0. mysql 4.1.20 running on ES4. But I also have an instance running on FC6.
Once I'm up and running in a known-working environment I'd have a better idea of what changes are right for FC7 or RHEL4/5.
Example config: builder.cfg: ==================================== [General] comm_type = active builder_user = plague-builder hostname = somehost.somecompany.com server = somehost.somecompany.com debug = True max_jobs = 3 builder_cmd = /usr/bin/mock
[Directories] target_configs_dir = /etc/plague/builder/targets builder_work_dir = /tmp/builder_work
[SSL] use_ssl = False
[Active] xmlrpc_port = 8889 fileserver_port= 8890
server.cfg:
[Database] engine = mysql
[Directories] repo_dir = /repodir mock_configs_dir = /etc/mock server_work_dir = /rpmbuild target_configs_dir = /etc/plague/server/targets tmpdir = /tmp
[Active Builders] xmlrpc_server_port = 8889 file_server_port = 8890 builder1 = 0 somehost2.somecompany.com builder2 = 50 somehost.somecompany.com
[General] traceback_server = no hostname = somehost.somecompany.com depsolve_jobs = no Debug = True allow_reenqueue = no
[SSL] server_key_and_cert = /etc/plague/server/certs/server/server_key_and_cert.pem ca_cert = /etc/plague/server/certs/ca/my_ca_ca_cert.pem
[UI] log_url = http://somehost.somecompany.com/buildlogs/ guest_allowed = no port = 8887 use_ssl = yes client_ca_cert = /etc/plague/server/certs/ca/client_ca_ca_cert.pem
[mysql Engine] timeout = 3 host = localhost user = root password = database = plague
[CVS] use_cvs = yes
[Builders] use_ssl = no
[Email] success_emails = email_from = buildsys@somecompany.com admin_emails = of@redhat.at
If you need help creating the ssl certificates or anything else, just mail me. :-) And don't use sqlite as database backend; It often runs into locking problems; That's why I switched to mysql. However, you do need to create the tables in the mysql database first. I have the statements ready if you need...
Yeah, using a real database is the way to go.
Dan
Am 2007-02-26 13:39, Dan Williams schrieb:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
[ ... ]
Yeah, using a real database is the way to go.
Maybe someone should also have a look at the default ports and other defaults...
BUT... What's about plague? Will it live or will it die? I'm thinking about koji... :-)
Best, Oliver
On Mon, 2007-02-26 at 13:53 +0100, Oliver Falk wrote:
Am 2007-02-26 13:39, Dan Williams schrieb:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
[ ... ]
Yeah, using a real database is the way to go.
Maybe someone should also have a look at the default ports and other defaults...
BUT... What's about plague? Will it live or will it die? I'm thinking about koji... :-)
I'm not entirely sure. I had thought the course of action was (a) opening up most of Brew, and (b) grafting in the bits of plague that were useful.
Dan
Am 2007-02-26 17:48, Dan Williams schrieb:
On Mon, 2007-02-26 at 13:53 +0100, Oliver Falk wrote:
Am 2007-02-26 13:39, Dan Williams schrieb:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
[ ... ]
Yeah, using a real database is the way to go.
Maybe someone should also have a look at the default ports and other defaults...
BUT... What's about plague? Will it live or will it die? I'm thinking about koji... :-)
I'm not entirely sure. I had thought the course of action was (a) opening up most of Brew, and (b) grafting in the bits of plague that were useful.
OK. So it's better to stick with plague until some working >release< of koji is there - at least for me this is true :-) Let's see if it will be possible to run it without all the Kerberos stuff...
-of
Dan Williams dcbw@redhat.com writes:
On Mon, 2007-02-26 at 12:06 +0100, Oliver Falk wrote:
Am 2007-02-25 21:26, Ken MacLeod schrieb:
Here's what I've done so far:
- server/DBManager.py
- import sqlite3 as sqlite
- remove encoding="utf-8" from sqlite.connect
Why these?
I'm guessing Python 2.5 too. If RHEL4 has it different I should be able to work out a patch that works for both.
- builder/Config.py server/Config.py
BaseConfig.ConfigError
- change BaseConfig.BaseConfig.ConfigError to
Yes. I already submitted a patch to Dan for this.
Hmm, I thought I'd committed that. Will need to look again.
I'm not seeing it in CVS MAIN.
Yeah, using a real database is the way to go.
Yup, that appears to be the case!
-- Ken
Oliver Falk oliver@linux-kernel.at writes:
Am 2007-02-25 21:26, Ken MacLeod schrieb: [ ... ]
Here's what I've done so far:
- common/XMLRPCServerProxy.py
- set self._use_datetime = 0
Why that?
- common/AuthedXMLRPCServer.py
SimpleXMLRPCServer.SimpleXMLRPCDispatcher's
- pass allow_none=False and encoding=None to
Why that?
Yes, as Dan mentioned, both due to Python 2.5.
So you' running 0.5.0 now. The same as I do. Right?
It's more accurate to say I'm running CVS MAIN, but yes ;-)
Example config: builder.cfg: ==================================== [General] comm_type = active
[Active] xmlrpc_port = 8889 fileserver_port= 8890
server.cfg:
[Active Builders] xmlrpc_server_port = 8889 file_server_port = 8890
[UI]
port = 8887
If you need help creating the ssl certificates or anything else, just mail me. :-) And don't use sqlite as database backend; It often runs into locking problems; That's why I switched to mysql.
The above config (thanks!) and switching to MySQL has got the builder and server talking to each other and queueing jobs. Not that this changes anything, but I also backed out the sqlite changes that I can't verify. So I still have the XMLRPC change I made and DBManager change same as yours.
I don't remember if I created /tmp/builder_work or if the server or builder created it, but on the job I just queued the builder gets an error because /tmp/builder_work is owned by root and the builder can't get access to it.
I killed the job (noted a typo in the message about not being able to send mail, patch sent) and requeued. A couple minor glitches later...
Success!
However, you do need to create the tables in the mysql database first. I have the statements ready if you need...
I didn't see that, the tables got created.
Going to test it out some more...
-- Ken
buildsys@lists.fedoraproject.org