From: Chenxiong Qi cqi@redhat.com
Original implementation sends multiple times of calls to listPackages to gather the number of packages of each user, then multiple database queries and unnecessary operations in listPackages that are not related to the statistics itself will happen repeatedly while iterating users.
This patch simplifies this process by providing a higher level API, which requires only one database query against the tag_packages table, and any client is able to get the statistics data by a signle XMLRPC call. --- .gitignore | 2 ++ hub/kojihub.py | 12 ++++++++++++ www/kojiweb/index.py | 7 ++----- 3 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/.gitignore b/.gitignore index 4857165..3a8896f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ *.pyc *.pyo tests/test.py +*.swp +*~ diff --git a/hub/kojihub.py b/hub/kojihub.py index 92fc889..d5e9aa0 100644 --- a/hub/kojihub.py +++ b/hub/kojihub.py @@ -9332,6 +9332,18 @@ class RootExports(object): values=locals(), opts=queryOpts) return query.iterate()
+ def reportPackagesByUser(self): + '''Report of packages by user + + Return a mapping from user.id to the number of packages + ''' + fields = ('user_id', 'packages_count') + sql = ''' +SELECT owner, COUNT(package_id) FROM tag_packages +GROUP BY owner ORDER BY owner''' + rows = _multiRow(sql, None, fields) + return dict(((str(row['user_id']), row['packages_count']) for row in rows)) +
class BuildRoot(object):
diff --git a/www/kojiweb/index.py b/www/kojiweb/index.py index ff15a5f..279e6fc 100644 --- a/www/kojiweb/index.py +++ b/www/kojiweb/index.py @@ -1919,13 +1919,10 @@ def packagesbyuser(environ, start=None, order=None):
maxPackages = 1 users = server.listUsers() + data = server.reportPackagesByUser()
- server.multicall = True for user in users: - server.count('listPackages', userID=user['id'], with_dups=True) - packageCounts = server.multiCall() - - for user, [numPackages] in zip(users, packageCounts): + numPackages = data.get(str(user['id']), 0) user['packages'] = numPackages if numPackages > maxPackages: maxPackages = numPackages
Thanks for the patch, Chenxiong! A couple small things I noticed:
On Sat, Feb 7, 2015 at 1:50 AM, cqi@redhat.com wrote:
This patch simplifies this process by providing a higher level API, which requires only one database query against the tag_packages table, and any client is able to get the statistics data by a signle XMLRPC call.
signle -> single
diff --git a/.gitignore b/.gitignore index 4857165..3a8896f 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ *.pyc *.pyo tests/test.py +*.swp +*~
This .gitignore change is ok, but it should be broken out into a separate commit, rather than done in this commit.
buildsys@lists.fedoraproject.org