Thanks Jeff,

I'm using pprofile because we're running cobblerd in daemon mode and cProfile doesn't follow threads.  But I hadn't thought to look at wsgi config - I'll investigate and post back.

On Wed, Aug 17, 2016 at 8:19 AM, Jeff Schroeder <jeffschroeder@computer.org> wrote:
At my previous employer we were able to increase concurrent API requests per second by using uwsgi and creating a wsgi file for cobbler vs using the existing hand rolled web buts. The bottleneck was then cobblerd. We didn't profile much further as it was "good enough TM" for us, but it certainly is a worthwhile exercise. Maybe cProfile + runsnake?


On Tuesday, August 16, 2016, <alincoln@ischool.berkeley.edu> wrote:
Hi all, we developed some automation to create Cobbler system records automatically (using the XMLRPC API), but we're finding that performance drops drastically as the number of records scales up.  Using a benchmark script to generate records sequentially, we see that the creation time goes from a starting average of 2 seconds to 20 seconds at about 500 records, and at the end of a long run (~1000 records) we can see creation times as high as 150s.

Logs indicate that the internal consistency check that Cobbler performs to prevent dupes is creating a *lot* of overhead.  The constant resizing of the dictionary that contains all system records seems to be the issue.  We saw no difference in performance between 2.4 and 2.6, despite the API request queuing improvements in 2.6.

Has anyone else come across this?  Currently I'm working with pprofile to try to get more detail on the bottleneck, and will follow up here.  Any/all experiences, suggestions welcome.

Thanks,

Tony
_______________________________________________
cobbler-devel mailing list
cobbler-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/admin/lists/cobbler-devel@lists.fedorahosted.org


--
Text by Jeff, typos by iPhone

_______________________________________________
cobbler-devel mailing list
cobbler-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/admin/lists/cobbler-devel@lists.fedorahosted.org