Hello,

I've run some (naive) benchmarks to compare suds performance with Netsuite Web Services, between 0.3.8 and 0.3.9 versions, and I've got disappointing results.

You'll find enclosed the benchmark_netsuite.py script I've used. I benchmarked three operations: parsing the wsdl, creating a Netsuite passport for authentication and the authentication itself. Of those, only the first is really interesting as it shows the impact of the new ObjectCache on wsdl parsing.

The tests have been made on an Ubuntu Server 9.10 64-bit, with 4GB of RAM, and Python 2.6.

To summarize:

- with 0.3.8: first wsdl parsing is ~165 seconds, next are ~107 seconds, cache directory is 1.7MB and top indicates RAM usage a bit over 1GB;

- with 0.3.9: first wsdl parsing is ~490 seconds, next are ~123 seconds, cache directory is 110MB and top indicates RAM usage over 2GB.

Details are below.

If you need more information on my test environment or if you have more testing suggestions, I'll be glad to help.

Regards,

Georges Martin

--

* with 0.3.8:

> rm /tmp/suds/*
> sudo easy_install -U "suds==0.3.8"
> python benchmark_netsuite.py
...
parsing Netsuite wsdl : 165.887 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.728 seconds
connected to Netsuite

parsing Netsuite wsdl : 108.711 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 2.537 seconds
connected to Netsuite

parsing Netsuite wsdl : 109.574 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 5.229 seconds
connected to Netsuite

parsing Netsuite wsdl : 107.834 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.434 seconds
connected to Netsuite

parsing Netsuite wsdl : 107.900 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.462 seconds
connected to Netsuite

parsing Netsuite wsdl : 107.398 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 4.364 seconds
connected to Netsuite

> du -h /tmp/suds

1.7M    /tmp/suds


* with 0.3.9:

> rm /tmp/suds/*
> sudo easy_install -U "suds==0.3.9"
> python benchmark_netsuite.py

parsing Netsuite wsdl : 494.904 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.301 seconds
connected to Netsuite

parsing Netsuite wsdl : 127.856 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 2.020 seconds
connected to Netsuite

parsing Netsuite wsdl : 123.874 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 2.566 seconds
connected to Netsuite

parsing Netsuite wsdl : 119.649 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.561 seconds
connected to Netsuite

parsing Netsuite wsdl : 123.335 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 1.669 seconds
connected to Netsuite

parsing Netsuite wsdl : 123.569 seconds
creating Netsuite passport : 0.001 seconds
authenticating to Netsuite : 2.026 seconds
connected to Netsuite

> du -h /tmp/suds

110M 
   /tmp/suds