cpython: 0175883d9513 (original) (raw)

Mercurial > cpython

changeset 73575:0175883d9513

Closes #13297: use bytes type to send and receive binary data through XMLRPC. [#13297]

line wrap: on

line diff

--- a/Doc/library/xmlrpc.client.rst +++ b/Doc/library/xmlrpc.client.rst @@ -8,7 +8,7 @@ .. XXX Not everything is documented yet. It might be good to describe

• Marshaller, Unmarshaller, getparser, dumps, loads, and Transport.

• Marshaller, Unmarshaller, getparser and Transport.

Source code: :source:Lib/xmlrpc/client.py @@ -21,7 +21,12 @@ supports writing XML-RPC client code; it between conformable Python objects and XML on the wire. -.. class:: ServerProxy(uri, transport=None, encoding=None, verbose=False, allow_none=False, use_datetime=False) +.. class:: ServerProxy(uri, transport=None, encoding=None, verbose=False, [](#l1.17)

•                   allow_none=False, use_datetime=False, \[](#l1.18)

•                   use_builtin_types=False)[](#l1.19)

+

• .. versionchanged:: 3.3

•  The *use_builtin_types* flag was added.[](#l1.22)

A :class:ServerProxy instance is an object that manages communication with a remote XML-RPC server. The required first argument is a URI (Uniform Resource @@ -34,9 +39,13 @@ between conformable Python objects and X XML; the default behaviour is for None to raise a :exc:TypeError. This is a commonly-used extension to the XML-RPC specification, but isn't supported by all clients and servers; see http://ontosys.com/xml-rpc/extensions.php for a

• description. The use_datetime flag can be used to cause date/time values to
• be presented as :class:datetime.datetime objects; this is false by default.
• :class:datetime.datetime objects may be passed to calls.

• description. The use_builtin_types flag can be used to cause date/time values
• to be presented as :class:datetime.datetime objects and binary data to be
• presented as :class:bytes objects; this flag is false by default.
• :class:datetime.datetime and :class:bytes objects may be passed to calls. +
• The obsolete use_datetime flag is similar to use_builtin_types but it
• applies only to date/time values. Both the HTTP and HTTPS transports support the URL syntax extension for HTTP Basic Authentication: http://user:pass@host:port/path. The user:pass @@ -78,12 +87,12 @@ between conformable Python objects and X | | only their dict attribute is | | | transmitted. | +---------------------------------+---------------------------------------------+

• | :const:dates | in seconds since the epoch (pass in an |
• | | instance of the :class:DateTime class) or |

• | :const:dates | In seconds since the epoch. Pass in an |
• | | instance of the :class:DateTime class or | | | a :class:datetime.datetime instance. | +---------------------------------+---------------------------------------------+

• | :const:binary data | pass in an instance of the :class:Binary |
• | | wrapper class |

• | :const:binary data | Pass in an instance of the :class:Binary |
• | | wrapper class or a :class:bytes instance. | +---------------------------------+---------------------------------------------+ This is the full set of data types supported by XML-RPC. Method calls may also @@ -98,8 +107,9 @@ between conformable Python objects and X ensure that the string is free of characters that aren't allowed in XML, such as the control characters with ASCII values between 0 and 31 (except, of course, tab, newline and carriage return); failing to do this will result in an XML-RPC

• request that isn't well-formed XML. If you have to pass arbitrary strings via
• XML-RPC, use the :class:Binary wrapper class described below.

• request that isn't well-formed XML. If you have to pass arbitrary bytes
• via XML-RPC, use the :class:bytes class or the class:Binary wrapper class
• described below. :class:Server is retained as an alias for :class:ServerProxy for backwards compatibility. New code should use :class:ServerProxy. @@ -249,7 +259,7 @@ The client code for the preceding server Binary Objects --------------

-This class may be initialized from string data (which may include NULs). The +This class may be initialized from bytes data (which may include NULs). The primary access to the content of a :class:Binary object is provided by an attribute: @@ -257,15 +267,15 @@ attribute: .. attribute:: Binary.data The binary data encapsulated by the :class:Binary instance. The data is

• provided as an 8-bit string.

• provided as a :class:bytes object.

:class:Binary objects have the following methods, supported mainly for internal use by the marshalling/unmarshalling code: -.. method:: Binary.decode(string) +.. method:: Binary.decode(bytes)

• Accept a base64 string and decode it as the instance's new data.

• Accept a base64 :class:bytes object and decode it as the instance's new data.

.. method:: Binary.encode(out) @@ -471,14 +481,21 @@ Convenience Functions it via an extension, provide a true value for allow_none. -.. function:: loads(data, use_datetime=False) +.. function:: loads(data, use_datetime=False, use_builtin_types=False) Convert an XML-RPC request or response into Python objects, a (params,[](#l1.107) methodname). params is a tuple of argument; methodname is a string, or None if no method name is present in the packet. If the XML-RPC packet represents a fault condition, this function will raise a :exc:Fault exception.

• The use_datetime flag can be used to cause date/time values to be presented as
• :class:datetime.datetime objects; this is false by default.

• The use_builtin_types flag can be used to cause date/time values to be
• presented as :class:datetime.datetime objects and binary data to be
• presented as :class:bytes objects; this flag is false by default. +
• The obsolete use_datetime flag is similar to use_builtin_types but it
• applies only to date/time values. +
• .. versionchanged:: 3.3

•  The *use_builtin_types* flag was added.[](#l1.121)

.. _xmlrpc-client-example:

--- a/Lib/test/test_xmlrpc.py +++ b/Lib/test/test_xmlrpc.py @@ -24,6 +24,8 @@ alist = [{'astring': 'foo@bar.baz.spam', 'ashortlong': 2, 'anotherlist': ['.zyx.41'], 'abase64': xmlrpclib.Binary(b"my dog has fleas"),

•      'b64bytes': b"my dog has fleas",[](#l2.7)

•      'b64bytearray': bytearray(b"my dog has fleas"),[](#l2.8)
       'boolean': False,[](#l2.9)
       'unicode': '\u4000\u6000\u8000',[](#l2.10)
       'ukey\u4000': 'regular value',[](#l2.11)

@@ -44,27 +46,54 @@ class XMLRPCTestCase(unittest.TestCase): def test_dump_bare_datetime(self): # This checks that an unwrapped datetime.date object can be handled # by the marshalling code. This can't be done via test_dump_load()

•    # since with use_datetime set to 1 the unmarshaller would create[](#l2.16)

•    # since with use_builtin_types set to 1 the unmarshaller would create[](#l2.17)
     # datetime objects for the 'datetime[123]' keys as well[](#l2.18)
     dt = datetime.datetime(2005, 2, 10, 11, 41, 23)[](#l2.19)

•    self.assertEqual(dt, xmlrpclib.DateTime('20050210T11:41:23'))[](#l2.20)
     s = xmlrpclib.dumps((dt,))[](#l2.21)

•    (newdt,), m = xmlrpclib.loads(s, use_datetime=1)[](#l2.22)

+

•    result, m = xmlrpclib.loads(s, use_builtin_types=True)[](#l2.24)

•    (newdt,) = result[](#l2.25)

•    self.assertEqual(newdt, dt)[](#l2.26)

•    self.assertIs(type(newdt), datetime.datetime)[](#l2.27)

•    self.assertIsNone(m)[](#l2.28)

+

•    result, m = xmlrpclib.loads(s, use_builtin_types=False)[](#l2.30)

•    (newdt,) = result[](#l2.31)
     self.assertEqual(newdt, dt)[](#l2.32)

•    self.assertEqual(m, None)[](#l2.33)

•    self.assertIs(type(newdt), xmlrpclib.DateTime)[](#l2.34)

•    self.assertIsNone(m)[](#l2.35)

•    (newdt,), m = xmlrpclib.loads(s, use_datetime=0)[](#l2.37)

•    self.assertEqual(newdt, xmlrpclib.DateTime('20050210T11:41:23'))[](#l2.38)

•    result, m = xmlrpclib.loads(s, use_datetime=True)[](#l2.39)

•    (newdt,) = result[](#l2.40)

•    self.assertEqual(newdt, dt)[](#l2.41)

•    self.assertIs(type(newdt), datetime.datetime)[](#l2.42)

•    self.assertIsNone(m)[](#l2.43)

+

•    result, m = xmlrpclib.loads(s, use_datetime=False)[](#l2.45)

•    (newdt,) = result[](#l2.46)

•    self.assertEqual(newdt, dt)[](#l2.47)

•    self.assertIs(type(newdt), xmlrpclib.DateTime)[](#l2.48)

•    self.assertIsNone(m)[](#l2.49)

+ def test_datetime_before_1900(self): # same as before but with a date before 1900 dt = datetime.datetime(1, 2, 10, 11, 41, 23)

•    self.assertEqual(dt, xmlrpclib.DateTime('00010210T11:41:23'))[](#l2.55)
     s = xmlrpclib.dumps((dt,))[](#l2.56)

•    (newdt,), m = xmlrpclib.loads(s, use_datetime=1)[](#l2.57)

+

•    result, m = xmlrpclib.loads(s, use_builtin_types=True)[](#l2.59)

•    (newdt,) = result[](#l2.60)
     self.assertEqual(newdt, dt)[](#l2.61)

•    self.assertEqual(m, None)[](#l2.62)

•    self.assertIs(type(newdt), datetime.datetime)[](#l2.63)

•    self.assertIsNone(m)[](#l2.64)

•    (newdt,), m = xmlrpclib.loads(s, use_datetime=0)[](#l2.66)

•    self.assertEqual(newdt, xmlrpclib.DateTime('00010210T11:41:23'))[](#l2.67)

•    result, m = xmlrpclib.loads(s, use_builtin_types=False)[](#l2.68)

•    (newdt,) = result[](#l2.69)

•    self.assertEqual(newdt, dt)[](#l2.70)

•    self.assertIs(type(newdt), xmlrpclib.DateTime)[](#l2.71)

•    self.assertIsNone(m)[](#l2.72)

def test_bug_1164912 (self): d = xmlrpclib.DateTime() @@ -133,6 +162,25 @@ class XMLRPCTestCase(unittest.TestCase): xmlrpclib.loads(strg)[0][0]) self.assertRaises(TypeError, xmlrpclib.dumps, (arg1,))

• def test_dump_bytes(self):

•    sample = b"my dog has fleas"[](#l2.81)

•    self.assertEqual(sample, xmlrpclib.Binary(sample))[](#l2.82)

•    for type_ in bytes, bytearray, xmlrpclib.Binary:[](#l2.83)

•        value = type_(sample)[](#l2.84)

•        s = xmlrpclib.dumps((value,))[](#l2.85)

+

•        result, m = xmlrpclib.loads(s, use_builtin_types=True)[](#l2.87)

•        (newvalue,) = result[](#l2.88)

•        self.assertEqual(newvalue, sample)[](#l2.89)

•        self.assertIs(type(newvalue), bytes)[](#l2.90)

•        self.assertIsNone(m)[](#l2.91)

+

•        result, m = xmlrpclib.loads(s, use_builtin_types=False)[](#l2.93)

•        (newvalue,) = result[](#l2.94)

•        self.assertEqual(newvalue, sample)[](#l2.95)

•        self.assertIs(type(newvalue), xmlrpclib.Binary)[](#l2.96)

•        self.assertIsNone(m)[](#l2.97)

+ def test_get_host_info(self): # see bug #3613, this raised a TypeError transp = xmlrpc.client.Transport() @@ -140,9 +188,6 @@ class XMLRPCTestCase(unittest.TestCase): ('host.tld', [('Authorization', 'Basic dXNlcg==')], {}))

• def test_dump_bytes(self):

•    self.assertRaises(TypeError, xmlrpclib.dumps, (b"my dog has fleas",))[](#l2.107)

- def test_ssl_presence(self): try: import ssl

--- a/Lib/xmlrpc/client.py +++ b/Lib/xmlrpc/client.py @@ -386,8 +386,8 @@ class Binary: if data is None: data = b"" else:

•        if not isinstance(data, bytes):[](#l3.7)

•            raise TypeError("expected bytes, not %s" %[](#l3.8)

•        if not isinstance(data, (bytes, bytearray)):[](#l3.9)

•            raise TypeError("expected bytes or bytearray, not %s" %[](#l3.10)
                             data.__class__.__name__)[](#l3.11)
         data = bytes(data)  # Make a copy of the bytes![](#l3.12)
     self.data = data[](#l3.13)

@@ -559,6 +559,14 @@ class Marshaller: write("\n") dispatch[str] = dump_unicode

• def dump_bytes(self, value, write):

•    write("<value><base64>\n")[](#l3.19)

•    encoded = base64.encodebytes(value)[](#l3.20)

•    write(encoded.decode('ascii'))[](#l3.21)

•    write("</base64></value>\n")[](#l3.22)

• dispatch[bytes] = dump_bytes
• dispatch[bytearray] = dump_bytes

+ def dump_array(self, value, write): i = id(value) if i in self.memo: @@ -629,7 +637,7 @@ class Unmarshaller: # and again, if you don't understand what's going on in here, # that's perfectly ok.

• def init(self, use_datetime=False):

• def init(self, use_datetime=False, use_builtin_types=False): self._type = None self._stack = [] self._marks = []

@@ -637,7 +645,8 @@ class Unmarshaller: self._methodname = None self._encoding = "utf-8" self.append = self._stack.append

•    self._use_datetime = use_datetime[](#l3.42)

•    self._use_datetime = use_builtin_types or use_datetime[](#l3.43)

•    self._use_bytes = use_builtin_types[](#l3.44)

def close(self): # return response tuple and target method @@ -749,6 +758,8 @@ class Unmarshaller: def end_base64(self, data): value = Binary() value.decode(data.encode("ascii"))

•    if self._use_bytes:[](#l3.52)

•        value = value.data[](#l3.53)
     self.append(value)[](#l3.54)
     self._value = 0[](#l3.55)

  dispatch["base64"] = end_base64 @@ -860,21 +871,26 @@ FastMarshaller = FastParser = FastUnmars

#

return A (parser, unmarshaller) tuple.

-def getparser(use_datetime=False): +def getparser(use_datetime=False, use_builtin_types=False): """getparser() -> parser, unmarshaller Create an instance of the fastest available parser, and attach it to an unmarshalling object. Return both objects. """ if FastParser and FastUnmarshaller:

•    if use_datetime:[](#l3.69)

•    if use_builtin_types:[](#l3.70)
         mkdatetime = _datetime_type[](#l3.71)

•        mkbytes = base64.decodebytes[](#l3.72)

•    elif use_datetime:[](#l3.73)

•        mkdatetime = _datetime_type[](#l3.74)

•        mkbytes = _binary[](#l3.75)
     else:[](#l3.76)
         mkdatetime = _datetime[](#l3.77)

•    target = FastUnmarshaller(True, False, _binary, mkdatetime, Fault)[](#l3.78)

•        mkbytes = _binary[](#l3.79)

•    target = FastUnmarshaller(True, False, mkbytes, mkdatetime, Fault)[](#l3.80)
     parser = FastParser(target)[](#l3.81)

  else:

•    target = Unmarshaller(use_datetime=use_datetime)[](#l3.83)

•    target = Unmarshaller(use_datetime=use_datetime, use_builtin_types=use_builtin_types)[](#l3.84)
     if FastParser:[](#l3.85)
         parser = FastParser(target)[](#l3.86)
     else:[](#l3.87)

@@ -912,7 +928,7 @@ def dumps(params, methodname=None, metho encoding: the packet encoding (default is UTF-8)

• All 8-bit strings in the data structure are assumed to use the

• All byte strings in the data structure are assumed to use the packet encoding. Unicode strings are automatically converted, where necessary. """ @@ -971,7 +987,7 @@ def dumps(params, methodname=None, metho

(None if not present).

@see Fault

-def loads(data, use_datetime=False): +def loads(data, use_datetime=False, use_builtin_types=False): """data -> unmarshalled data, method name Convert an XML-RPC packet to unmarshalled data plus a method @@ -980,7 +996,7 @@ def loads(data, use_datetime=False): If the XML-RPC packet represents a fault condition, this function raises a Fault exception. """

• p, u = getparser(use_datetime=use_datetime)

• p, u = getparser(use_datetime=use_datetime, use_builtin_types=use_builtin_types) p.feed(data) p.close() return u.close(), u.getmethodname() @@ -1092,8 +1108,9 @@ class Transport:

  that they can decode such a request

  encode_threshold = None #None = don't encode

• def init(self, use_datetime=False):

• def init(self, use_datetime=False, use_builtin_types=False): self._use_datetime = use_datetime

•    self._use_builtin_types = use_builtin_types[](#l3.122)
     self._connection = (None, None)[](#l3.123)
     self._extra_headers = [][](#l3.124)

@@ -1154,7 +1171,8 @@ class Transport: def getparser(self): # get parser and unmarshaller

•    return getparser(use_datetime=self._use_datetime)[](#l3.130)

•    return getparser(use_datetime=self._use_datetime,[](#l3.131)

•                     use_builtin_types=self._use_builtin_types)[](#l3.132)

## # Get authorization info from host parameter @@ -1361,7 +1379,7 @@ class ServerProxy: """ def init(self, uri, transport=None, encoding=None, verbose=False,

•             allow_none=False, use_datetime=False):[](#l3.140)

•             allow_none=False, use_datetime=False, use_builtin_types=False):[](#l3.141)
     # establish a "logical" server connection[](#l3.142)

# get the url @@ -1375,9 +1393,11 @@ class ServerProxy: if transport is None: if type == "https":

•            transport = SafeTransport(use_datetime=use_datetime)[](#l3.149)

•            handler = SafeTransport[](#l3.150)
         else:[](#l3.151)

•            transport = Transport(use_datetime=use_datetime)[](#l3.152)

•            handler = Transport[](#l3.153)

•        transport = handler(use_datetime=use_datetime,[](#l3.154)

•                            use_builtin_types=use_builtin_types)[](#l3.155)
     self.__transport = transport[](#l3.156)

self.__encoding = encoding or 'utf-8'

--- a/Misc/NEWS +++ b/Misc/NEWS @@ -374,6 +374,8 @@ Core and Builtins Library ------- +- Issue #13297: Use bytes type to send and receive binary data through XMLRPC. +

• Issue #6397: Support "/dev/poll" polling objects in select module, under Solaris & derivatives.