Python Type-Testing Decorator (original) (raw)

Function Arguments/Return Type-Testing Decorator

[Jan-2015] As part of a recent project, I extended an arguments type-checking function+method decorator that appears in Chapter 39 of Learning Python, 5th Edition, to also perform return type testing, and use 3.X function attributes instead of decorator arguments. That makes it 3.X only, but it's simple to translate this code to use 2.X-compatible decorator arguments instead, as shown in the book.

Fetch the decorator module here:

⇨ <debugtypes.py>

Usage

See the decorator module's self-test code for usage examples, and the book for the logic underlying the decorator itself (its origins go back to this post from 2008/2009).

As another use case—and an example of bytes and bit-level processing—I used this decorator to verify that type annotations coded in simple file obfuscation functions were being respected. Here's a partial excerpt from this (not yet published) client:

""" [Scheme 2 of 4]

data = do_encode2(b'spam', ord('x')) 120 b'spam' bytearray(b'\x00\xeb\x00\xe8\x00\xd9\x00\xe5')

data = do_decode2(bytes(data), ord('x')) # need bytes() for decorator only! 120 bytearray(b'\x00\xeb\x00\xe8\x00\xd9\x00\xe5') bytearray(b'spam') """

from debugtypes import debugtypes

@debugtypes def do_encode2(data: bytes, adder: int) -> bytearray: trace(adder, data[:4]) newdata = bytearray() for byte in data: # bytes => int (or via bytes[i]) word = byte + adder # add scrambler byte1 = (word & 0xFF00) >> 8 # upper byte byte2 = (word & 0X00FF) # lower byte newdata.extend([byte1, byte2]) # 2 bytes for 1, to binary file trace(newdata[:8]) # int => bytes (or bytes([int])) return newdata

@debugtypes def do_decode2(data: bytes, adder: int) -> bytearray: trace(adder, data[:8]) newdata = bytearray() ix = 0 while ix < len(data): byte1, byte2 = data[ix], data[ix+1] # bytes => int word = (byte1 << 8) + byte2 # 1 word to 2 bytes word -= adder # remove scrambler newdata.append(word) # retain low byte ix += 2 trace(newdata[:4]) return newdata

... from encoder import * encode, decode = do_encode2, do_decode2 # choose your weapon data = open(filename, 'rb').read() # load from original name data = encode(data, adder) # scramble byte data newname = filename + encext # write to new enc name with open(newname, 'wb') as newfile: # guarantee closes newfile.write(data)

Caveats

This decorator might be useful during development to ensure coding-time expectations, but has some major downsides:

• As the book notes, manual type testing in general can limit code flexibility in most contexts. In this specific case, the decorator precludes processing other unrelated objects with compatible interfaces. In the example client below, for instance, a bytearray won't pass a bytes type test when the decorator is deployed (and requires extra manual conversion), but works fine if the decorator is removed because its interface is compatible with the code. The type test fails because bytes objects are not instances of bytearray, and vice-versa. Adding extra generic base types can address some such issues, but would add new complexity all their own for an artificial and arguably dubious cause. Python is about object interfaces, not type constraints.
• Besides limiting code scope, type constraints are also often fully pointless in Python—the run-time error checking already performed by the language will generally catch type mismatches automatically during testing. That is, it's better to write code expecting a compatible object interface, and let Python's own error checking detect cases of interface mismatch. See the self-test code in the decorator module listed above for a prime example; most type errors caught by the decorator's manual type testing generate normal Python errors if the decorator's type testing is disabled (via the "-O" command-line switch). The decorator's coding pattern may have valid use cases, but type checking may not be one of them.

Update Apr-2015: A similar modelis being proposed as standard type declarations for Python 3.5; it threatens to escalate the same caveats to best practice. For an arguably better use case for decorators, see the Private/Public attributes class decorator example from LP5E.