| msg278291 - (view) |
Author: Attila Vangel (avangel) |
Date: 2016-10-08 11:53 |
| Go to https://docs.python.org/3/library/xmlrpc.client.html Under '21.26.8. Example of Client Usage' -> 'To access an XML-RPC server through a HTTP proxy, you need to define a custom transport. The following example shows how:' copy the example code to a .py file (for me it is easier than REPL), e.g. xmlrpc_client_http_proxy_test.py This is the example code: import xmlrpc.client, http.client class ProxiedTransport(xmlrpc.client.Transport): def set_proxy(self, proxy): self.proxy = proxy def make_connection(self, host): self.realhost = host h = http.client.HTTPConnection(self.proxy) return h def send_request(self, connection, handler, request_body, debug): connection.putrequest("POST", 'http://%s%s' % (self.realhost, handler)) def send_host(self, connection, host): connection.putheader('Host', self.realhost) p = ProxiedTransport() p.set_proxy('proxy-server:8080') server = xmlrpc.client.ServerProxy('http://time.xmlrpc.com/RPC2', transport=p) print(server.currentTime.getCurrentTime()) I changed the 'proxy-server:8080' to '10.144.1.11:8080' which is a valid HTTP/HTTPS proxy in company I work for. Try to run this code: $ python3 xmlrpc_client_http_proxy_test.py Traceback (most recent call last): File "xmlrpc_client_http_proxy_test.py", line 21, in print(server.currentTime.getCurrentTime()) File "/usr/lib/python3.5/xmlrpc/client.py", line 1092, in __call__ return self.__send(self.__name, args) File "/usr/lib/python3.5/xmlrpc/client.py", line 1432, in __request verbose=self.__verbose File "/usr/lib/python3.5/xmlrpc/client.py", line 1134, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib/python3.5/xmlrpc/client.py", line 1146, in single_request http_conn = self.send_request(host, handler, request_body, verbose) File "xmlrpc_client_http_proxy_test.py", line 13, in send_request connection.putrequest("POST", 'http://%s%s' % (self.realhost, handler)) AttributeError: 'str' object has no attribute 'putrequest' Personally I don't like the idea of putting this amount of code to documentation: - as it seems, without automated tests running it, the code seems to rot, and gets outdated - I need to paste this boilerplate code to my application if I want this functionality. IMHO it would be much better to move this ProxiedTransport example code (after fixing it) to e.g. xmlrpc.client.HttpProxyTransport (or similar name) class. Details about python3: $ python3 Python 3.5.2 (default, Sep 10 2016, 08:21:44) [GCC 5.4.0 20160609] on linux Type "help", "copyright", "credits" or "license" for more information. >>> |
|
|
| msg278292 - (view) |
Author: Attila Vangel (avangel) |
Date: 2016-10-08 12:02 |
| I tested it also on Python 3.4.3. I got the same error. |
|
|
| msg278297 - (view) |
Author: Berker Peksag (berker.peksag) *  |
Date: 2016-10-08 13:04 |
| Thanks for the report. Can you try the attached script? |
|
|
| msg278300 - (view) |
Author: Attila Vangel (avangel) |
Date: 2016-10-08 13:33 |
| Hi, thx for the quick turnaround. I tried the proxy.py (on python 3.5) of course replacing 'YOUR_PROXY' with '10.144.1.11:8080' according to my environment. python3 proxy.py Traceback (most recent call last): File "proxy.py", line 27, in print(server.examples.getStateName(41)) File "/usr/lib/python3.5/xmlrpc/client.py", line 1092, in __call__ return self.__send(self.__name, args) File "/usr/lib/python3.5/xmlrpc/client.py", line 1432, in __request verbose=self.__verbose File "/usr/lib/python3.5/xmlrpc/client.py", line 1134, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib/python3.5/xmlrpc/client.py", line 1147, in single_request resp = http_conn.getresponse() File "/usr/lib/python3.5/http/client.py", line 1197, in getresponse response.begin() File "/usr/lib/python3.5/http/client.py", line 297, in begin version, status, reason = self._read_status() File "/usr/lib/python3.5/http/client.py", line 258, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/lib/python3.5/socket.py", line 575, in readinto return self._sock.recv_into(b) ConnectionResetError: [Errno 104] Connection reset by peer However, meanwhile I studied a bit the http.client API on how to use HTTP proxy, and I found set_tunnel() can do it. I had success by only overriding make_connection() in ProxiedTransport: - copy current code of make_connection() from xmlrpc.client.Transport to ProxiedTransport (NOTE, this itself violates the DRY principle, but there is no better way to do it), change it slightly: - create HTTPSConnection to the proxy (as I wanted to access a https URL) - use .set_tunnel(chost) on this connection I did not want to paste the code here, because - I did not want to fill the 'PSF Contributor Agreement', at least yet - it may be Python version specific solution. |
|
|
| msg278304 - (view) |
Author: Berker Peksag (berker.peksag) * |
Date: 2016-10-08 15:14 |
| Thanks. I guess your solution was similar to the attached patch? |
|
|
| msg278305 - (view) |
Author: Attila Vangel (avangel) |
Date: 2016-10-08 15:29 |
| It's my pleasure. It was somewhat similar: - the set_proxy() is the same - for the make_connection() I gave the necessary clues, so one can create the code and you can use that in a way that I don't have to spend time on the PSF Contributor Agreement - overriding send_request() is not necessary at all, because the HTTP proxying is done in http.client level |
|
|
| msg278362 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2016-10-09 15:19 |
| New changeset 94c9c314f5d9 by Berker Peksag in branch '3.5': Issue #28389: Fix ProxiedTransport example in xmlrpc.client documentation https://hg.python.org/cpython/rev/94c9c314f5d9 New changeset 60c5c77c0190 by Berker Peksag in branch '3.6': Issue #28389: Merge from 3.5 https://hg.python.org/cpython/rev/60c5c77c0190 New changeset 5f9351bc317e by Berker Peksag in branch 'default': Issue #28389: Merge from 3.6 https://hg.python.org/cpython/rev/5f9351bc317e |
|
|
| msg278382 - (view) |
Author: Attila Vangel (avangel) |
Date: 2016-10-09 17:47 |
| Thanks for fixing this issue. I checked the changed documentation online, and I came up with a very similar solution. One difference is that although this example overrides the make_connection() method, but omits the following lines which are present in the xmlrpc.client.Transport code (I checked that in python 3.5.3): # return an existing connection if possible. This allows # HTTP/1.1 keep-alive. if self._connection and host == self._connection[0]: return self._connection[1] # create a HTTP connection object from a host descriptor chost, self._extra_headers, x509 = self.get_host_info(host) Please check xmlrpc.client.Transport.make_connection(). I am not sure about the what kind of effect this may have. I used chost as the parameter of set_tunnel(), rather than host (after briefly checking the code how it is calculated I felt that it's better to use chost than host, but I don't have a deep understanding of the http.client code). The proxy_headers is a new thing, I guess it's a good thing if someone needs that. I don't understand why the '*' character is needed in connection = http.client.HTTPConnection(*self.proxy) However I'm quite new to python. I will try this new code tomorrow. |
|
|