[Python-Dev] A Hygienic Macro System in Python? (original) (raw)

David Abrahams David Abrahams" <david.abrahams@rcn.com
Tue, 19 Mar 2002 21:27:22 -0500

----- Original Message ----- From: "Guido van Rossum" <guido@python.org>

I recently had a thought about how Python and C++ represent extreme ends of the spectrum of how much happens in the parser/compiler vs. how much happens at run time (although Lisp would be even more extreme than Python on this scale).

- In C++, the compiler is incredibly powerful (especially with the new templates). This means you can use the compiler for very sophisticated compile-time processing. But the downside is that if you don't know a lot about how compilers work, or how the specific C++ compiler you're using works, it can be quite baffling to figure out how you're supposed to accomplish even simple things (e.g. examples like "istreamiterator i(cin);" are quite baffling).

Is this baffling because of the syntactic use of "<" and ">" or something else? Is this really accomplishing such a simple thing? Wouldn't the equivalent thing in Python be equally baffling? How do you make a Python iterator which reads integers from a file?

You /can/ do some extremely sophisticated compile-time processing using the C++ compiler, and I wouldn't dream of arguing that it's always straightforward, but the above doesn't seem like a good example.

- In Python, the parser and compiler are extremely stupid -- e.g. the compiler doesn't even know the type of variables, and translates everything to very simple bytecode.

This means that (ideally) when using Python, you only need to learn the intricacies of one system: the runtime, and that one is relatively well-behaved: when something's wrong, all the context is available, and can be shown in the debugger or in the traceback. Contrast that to debugging C++: when the problem occurs at runtime, it may be difficult to correlate it to the relevant portions of the source (e.g. because of template expansion).

That doesn't follow for any compiler I know about. Debuggers step through template code just like any other code. When templates cause debugging woes it's generally at compile-time, because compilers /don't/ provide debugging tools for compile-time processing. The best most of us can hope for is a template instantiation backtrace.

When a C++ problem occurs at compile time, it may still be hidden by layers of template expansion, and even if it isn't, there are so many things to talk about that the compiler may have a hard time telling you what's wrong in a language you can understand

True enough, though more often than not compilers simply give you so much information that it's hard to sort out what's relevant from what isn't. Just imagine if Python were a pure functional language and all you had to debug it with was a stack backtrace which included the values of every function parameter. The situation with C++ compile-time debugging is analogous. Post-processing tools like STLFilt can help a (little) bit.

-- again, you often end up learning a lot about compiler internals if you want to understand its error messages.

I wouldn't say that. I don't know anything about the internals of the C++ compilers I use.

I guess I'm wary of anything that adds compile time complexity, especially if the user will be aware of it -- I'm not set against adding optimizations, as long as they don't change the semantics and don't add new error messages or warnings.

It's hard to understand why anyone would want this sort of thing in a (relatively) slow dynamic language like Python. Since there's no static type checking, and compile-time polymorphism isn't going to buy you a lot of speed, you can just use runtime polymorphism for most of the cases I can imagine using templates for in C++.

-Dave