[Python-Dev] Recent changes to TextIOWrapper and its tests (original) (raw)

Jeff Allen "ja...py" at farowl.co.uk
Wed Mar 20 22:27:05 CET 2013

On 19/03/2013 08:03, Serhiy Storchaka wrote:

On 18.03.13 22:26, Jeff Allen wrote:

The puzzle is that it requires t.read() to succeed.

When I insert a check for bytes type in all the places it seems necessary in my code, I pass the first two conditions, but since t.read() also raises TypeError, the overall test fails. Is reading the stream with read() intended to succeed? Why is this desired? An alternative option is to change C implementation of TextIOWrapper.read() to raise an exception in this case. However I worry that it can break backward compatibility. Thanks for this and the previous note. It is good to get it from the horse's mouth. I was surprised that the Python 3 version of the test was different here. I'd looked at the source of textio.c and found no test for bytes type in the n<0 branch of textiowrapper_read. Having tested it just now, I see that the TypeError is raised by the decoder in Py3k, because the input (when it is a unicode string) does not bear the buffer API, and not by a type test in TextIOWrapper.read() at all.

For Jython, I shall make TextIOWrapper raise TypeError and our version of test_io will check for it. Any incompatibility relates only to whether a particular mistake sometimes goes undetected, so I feel pretty compatible. Added to which, this is the behaviour of Python 3 and we feel safe anticipating Py3k in small ways.

Are there other tests (in other test files) which fail with a new Jython TextIOWrapper?

I don't think there is anything else specific to TextIOWrapper, but if there is I will first treat that as a fault in our implementation. This is the general approach, to emulate the CPython implementation unless a test is clearly specific to arbitrary implementation choices. (There's a general exclusion for garbage collection.) In this case the test appeared to reflect an accident of implementation, but might just have been deliberate.

Parts of the Jython implementation of io not yet implemented in Java are supplied by a Python module _jyio. This is essentially a copy of the corresponding parts of _pyio, except that it has to pass the C* tests, not the Py* tests. In places _jyio is therefore closer to _io than is _pyio. For example, it makes the type tests just discussed, and passes CTextIOWrapperTest.test_illegal_decoder and test_initialization. _jyio.StringIO has getstate and setstate methods lacking in _pyio counterparts to pass pickling tests in test_memoryio. This might be of interest to CPython for _pyio.

Jeff Allen

More information about the Python-Dev mailing list