Great news, thanks Jeff. I'll start testing this later in the week and will
report back.
-Matt
On Wed, Jan 13, 2010 at 9:20 AM, Jeff Ortel <jortel(a)redhat.com> wrote:
All,
I finished and (committed r637) enhancements to the suds caching. The
previous approach added caching to the Transport layer and was causing some
problems. It /blindly/ cached downloaded XML documents such as WSDLs and
imported XSD files as XML text. The next time the document is needed (at
the same URL), the transport returned the local (cached) copy of the XML
file and it was SAX parsed as if it was downloaded via http. This actually
saved a lot of time because most servers returned these documents very
slowly is many cases. The problems with this approach are:
- Corrupt, incomplete or otherwise invalid XML document text was being
cached because the Transport layer was not content aware. It simply
cached
whatever it downloaded.
- Invalid cache files caused SAX parsing exceptions until the cache file
expired or the user cleared the cache.
- Cached files had to be SAX parsed every time the were retrieved.
The approach taken by the r637 enhancement puts caching above the Transport
layer and at the Document Processing layer. By doing this, suds is now
caching sax.document.Document objects using python pickle. This ensures
that cached XML documents are valid and allows for proper handling of
corrept, incomplete or otherwise invalid cached files. In the event that a
cached file fails to load, the document processing layer simply uses the
transport to download a good copy which is then SAX parsed and cached.
I benchmarked the performance difference between sax parsing and unpickling
a 2M XML file using cPickle. In most cases, I observed a ~2x performance
gain using pickle.load() over sax parsing. Yeah, I'm a little surprised
too. I expected about a ~10x gain but at least it's significantly better.
I tried really hard to preserve backwards compatibility but some minor
things did have to change. These things should not affect most users:
- The caching classes (Cache, FileCache, NoCache) moved from the
suds.transport package
to a suds.cache.py module.
- All caching has been removed from the Transport classes.
- The get() & put() methods on the Cache interface have changed slightly
to be
document oriented instead of file pointer (fp) or file-like oriented.
- Users disabling the cache using /NoCache/ should switch to the already
existing: Client(url, cache=None) syntax. Or, import NoCache from the
suds.cache.
- Users implementing their own caches need to adapt to the changes to the
Cache interface.
I plan to continue toward caching the entire WSDL (wsdl.Definitions) object
but this is a good interim step and solves most of the caching problems for
now.
This has been released in 0.3.9 beta which I believe to be very stable.
Hope this helps.
Regards,
Jeff
_______________________________________________
suds mailing list
suds(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/suds