All,
I finished and (committed r637) enhancements to the suds caching. The previous approach
added caching to the Transport layer and was causing some problems. It /blindly/ cached
downloaded XML documents such as WSDLs and imported XSD files as XML text. The next time
the document is needed (at the same URL), the transport returned the local (cached) copy
of the XML file and it was SAX parsed as if it was downloaded via http. This actually
saved a lot of time because most servers returned these documents very slowly is many
cases. The problems with this approach are:
- Corrupt, incomplete or otherwise invalid XML document text was being
cached because the Transport layer was not content aware. It simply cached
whatever it downloaded.
- Invalid cache files caused SAX parsing exceptions until the cache file
expired or the user cleared the cache.
- Cached files had to be SAX parsed every time the were retrieved.
The approach taken by the r637 enhancement puts caching above the Transport layer and at
the Document Processing layer. By doing this, suds is now caching sax.document.Document
objects using python pickle. This ensures that cached XML documents are valid and allows
for proper handling of corrept, incomplete or otherwise invalid cached files. In the
event that a cached file fails to load, the document processing layer simply uses the
transport to download a good copy which is then SAX parsed and cached.
I benchmarked the performance difference between sax parsing and unpickling a 2M XML file
using cPickle. In most cases, I observed a ~2x performance gain using pickle.load() over
sax parsing. Yeah, I'm a little surprised too. I expected about a ~10x gain but at
least
it's significantly better.
I tried really hard to preserve backwards compatibility but some minor things did have to
change. These things should not affect most users:
- The caching classes (Cache, FileCache, NoCache) moved from the suds.transport
package
to a suds.cache.py module.
- All caching has been removed from the Transport classes.
- The get() & put() methods on the Cache interface have changed slightly to be
document oriented instead of file pointer (fp) or file-like oriented.
- Users disabling the cache using /NoCache/ should switch to the already
existing: Client(url, cache=None) syntax. Or, import NoCache from the suds.cache.
- Users implementing their own caches need to adapt to the changes to the
Cache interface.
I plan to continue toward caching the entire WSDL (wsdl.Definitions) object but this is a
good interim step and solves most of the caching problems for now.
This has been released in 0.3.9 beta which I believe to be very stable.
Hope this helps.
Regards,
Jeff
Attachments:
- smime.p7s
(application/pkcs7-signature — 5.0 KB)