Change in vdsm[ovirt-3.4]: xmlrpc: Support HTTP 1.1

nsoffer at redhat.com nsoffer at redhat.com
Wed Feb 26 22:50:17 UTC 2014


Hello Adam Litke, Antoni Segura Puimedon, Dan Kenigsberg,

I'd like you to do a code review.  Please visit

    http://gerrit.ovirt.org/25124

to review the following change.

Change subject: xmlrpc: Support HTTP 1.1
......................................................................

xmlrpc: Support HTTP 1.1

Supporting HTTP 1.1 allow using keep-alive on the client side,
decreasing the latency and load on the engine and increasing the number
of host that engine can support.

This patch override do_POST and report_404 methods that wrongly shutdown
the connection at the end of the do_POST request. This is the
responsibility of the server, and not the request handler.  The fixed
methods are used only on Python 2.6. Python 2.7 already include these
changes. Then the protocol_version is set to HTTP/1.1, enabling
automatic keep-alive.

A new configuration option "xmlrpc_http11" can be used to disbale this
feature and use HTTP/1.0 as used before.

We can see in the logs that only few threads are created now for service
XMLRPC requests, and most requests are handled by the same thread.

Change-Id: Ie033d5c53c81c8db99d5c26697a1727be030e0b4
Signed-off-by: Nir Soffer <nsoffer at redhat.com>
Reviewed-on: http://gerrit.ovirt.org/24294
Reviewed-by: Antoni Segura Puimedon <asegurap at redhat.com>
Reviewed-by: Adam Litke <alitke at redhat.com>
Reviewed-by: Dan Kenigsberg <danken at redhat.com>
---
M lib/vdsm/config.py.in
M lib/vdsm/utils.py
2 files changed, 86 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/25124/1

diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in
index 0f84f16..a3be658 100644
--- a/lib/vdsm/config.py.in
+++ b/lib/vdsm/config.py.in
@@ -179,6 +179,9 @@
 
         ('xmlrpc_enable', 'true', 'Enable the xmlrpc server'),
 
+        ('xmlrpc_http11', 'true',
+            'Enable HTTP/1.1 keep-alive connections'),
+
         ('jsonrpc_enable', 'true', 'Enable the JSON RPC server'),
 
         ('report_host_threads_as_cores', 'false',
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 92d6f25..3c6bdcb 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -55,6 +55,7 @@
 import zombiereaper
 
 from cpopen import CPopen
+from .config import config
 from . import constants
 
 # Buffsize is 1K because I tested it on some use cases and 1K was fastest. If
@@ -146,7 +147,88 @@
         else:
             raise
 
-IPXMLRPCRequestHandler = SimpleXMLRPCRequestHandler
+
+class IPXMLRPCRequestHandler(SimpleXMLRPCRequestHandler):
+
+    if config.getboolean('vars', 'xmlrpc_http11'):
+        protocol_version = "HTTP/1.1"
+    else:
+        protocol_version = "HTTP/1.0"
+
+    # Override Python 2.6 version to support HTTP 1.1.
+    #
+    # This is the same code as Python 2.6, not shutting down the connection
+    # when a request is finished. The server is responsible for closing the
+    # connection, based on the http version and keep-alive and connection:
+    # close headers.
+    #
+    # Additionally, add "Content-Length: 0" header on internal errors, when we
+    # don't send any content. This is required by HTTP 1.1, otherwise the
+    # client does not have any clue that the response was finished.
+    #
+    # These changes were taken from Python 2.7 version of this class. If we are
+    # running on Python 2.7, these changes are not needed, hence we override
+    # the methods only on Python 2.6.
+
+    if sys.version_info[:2] == (2, 6):
+
+        def do_POST(self):
+            # Check that the path is legal
+            if not self.is_rpc_path_valid():
+                self.report_404()
+                return
+
+            try:
+                # Get arguments by reading body of request.
+                # We read this in chunks to avoid straining
+                # socket.read(); around the 10 or 15Mb mark, some platforms
+                # begin to have problems (bug #792570).
+                max_chunk_size = 10*1024*1024
+                size_remaining = int(self.headers["content-length"])
+                L = []
+                while size_remaining:
+                    chunk_size = min(size_remaining, max_chunk_size)
+                    chunk = self.rfile.read(chunk_size)
+                    if not chunk:
+                        break
+                    L.append(chunk)
+                    size_remaining -= len(L[-1])
+                data = ''.join(L)
+
+                # In previous versions of SimpleXMLRPCServer, _dispatch
+                # could be overridden in this class, instead of in
+                # SimpleXMLRPCDispatcher. To maintain backwards compatibility,
+                # check to see if a subclass implements _dispatch and dispatch
+                # using that method if present.
+                response = self.server._marshaled_dispatch(
+                    data, getattr(self, '_dispatch', None))
+            except Exception, e:
+                # This should only happen if the module is buggy
+                # internal error, report as HTTP server error
+                self.send_response(500)
+
+                # Send information about the exception if requested
+                if getattr(self.server, '_send_traceback_header', False):
+                    self.send_header("X-exception", str(e))
+                    self.send_header("X-traceback", traceback.format_exc())
+
+                self.send_header("Content-length", '0')
+                self.end_headers()
+            else:
+                # got a valid XML RPC response
+                self.send_response(200)
+                self.send_header("Content-type", "text/xml")
+                self.send_header("Content-length", str(len(response)))
+                self.end_headers()
+                self.wfile.write(response)
+
+        def report_404(self):
+            self.send_response(404)
+            response = 'No such page'
+            self.send_header("Content-type", "text/plain")
+            self.send_header("Content-length", str(len(response)))
+            self.end_headers()
+            self.wfile.write(response)
 
 
 class IPXMLRPCServer(SimpleXMLRPCServer):


-- 
To view, visit http://gerrit.ovirt.org/25124
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie033d5c53c81c8db99d5c26697a1727be030e0b4
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.4
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>
Gerrit-Reviewer: Adam Litke <alitke at redhat.com>
Gerrit-Reviewer: Antoni Segura Puimedon <asegurap at redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken at redhat.com>


More information about the vdsm-patches mailing list