Yaakov Nemoy wrote:
On Nov 25, 2007 4:00 PM, Toshio Kuratomi <a.badger(a)gmail.com>
> Here's a short script to test our TG apps run via supervisor for
> excessive memory usage and restart them if necessary. We could run this
> via cron in alternate hours on each app server. Does this seem like a
> good or bad idea to people?
+1, but does it make sure all transactions are finished? I know smolt
does not have good transaction protection. If a transaction fails
halfway through, we might have a mess.
Not if the app doesn't. From a brief test, TG apps do not do this.
The script is asking supervisor to shutdown the application. supervisor
sends a TERM to the TG app (we can configure it to send something other
than TERM if we want but I don't see any documentation that leads me to
believe it will be different with a HUP or QUIT). At that point it
looks like a TG app will immediately shutdown and rollback any current
smolt is on shaky ground if it's not using transactions correctly... At
the beginning of the month when smolt was getting hit hard we did pretty
much this same thing except manually instead of via a script when we
noticed that smolt was giving timeouts and taking up 1G+ of RAM. I
think the current smolt code is using SQLAlchemy, correct? It's pretty
easy to use transactions so that you don't leave the db in an
inconsistent state with that configuration. Using the session's
implicit transaction flushed just before the return should do the safe
thing. You can look through the code later and find additional places
where you can safely flush the transaction if there's a need.