| msg41006 - (view) |
Author: Jon Riehl (jriehl) |
Date: 2002-08-23 17:03 |
| The following are files for the implementation of PEP 269. The primary changes to the core involve moving required pgen modules from the pgen only module list to the Python library module list in Makefile.pre.in. Some of the modules required better memory allocation and management and the corresponding deallocators are in the Python extension module (maybe these should be moved into the pgen modules in Parser). Initially included are two implementations. The first is a basic implementation that follows the PEP API more or less. The second (currently unfinished) implementation provides a more object oriented interface to the extension module. Please note there are some commented out modifications to setup.py as I was unable to build the extension module(s) automagically. For some reason the linker I was using (on a BSD box) complained that it couldn't find the _Py_pgen symbol, even though I verified its existence in the new Python library. Maybe it is checking against an older Python library on the system? Things to be done (as of initial submission) include resolving on a single interface, documenting the one true interface, and finishing any unimplemented routines (such as are found in the OO implementation). In the final integration, a pgenmodule.c file should be added to the Modules directory in the main distribution. |
|
|
| msg41007 - (view) |
Author: Guido van Rossum (gvanrossum) *  |
Date: 2002-09-03 19:44 |
| Logged In: YES user_id=6380 I guess I'm going to hve to look at this to pronounce... |
|
|
| msg41008 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-09-09 12:09 |
| Logged In: YES user_id=6380 I looked a bit, and it's very raw - the author admits that. Jon, are you still working on this? Do you have a more polished version? Maybe we can separate out the memory allocation patches and commit these already? They don't break anything. |
|
|
| msg41009 - (view) |
Author: Jon Riehl (jriehl) |
Date: 2002-09-12 18:04 |
| Logged In: YES user_id=22448 Guido, as per my private message, I'll attempt to submit another patch by the end of the month, pending resumption of "work" on the 23rd. Commitment of the memory allocation patch is fine, and any future patches would be against the updated pgen code (I don't have commit permissions, so someone else will have to do this possibly.) |
|
|
| msg41010 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-09-13 04:35 |
| Logged In: YES user_id=6380 Cool. Maybe I'll get to it, maybe not. :-( |
|
|
| msg41011 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-11-14 17:32 |
| Logged In: YES user_id=6380 Lowering priority until Jon has his next version ready. |
|
|
| msg41012 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2003-02-05 16:42 |
| Logged In: YES user_id=6380 Closed. Jon, if you're still interested in this, upload something new and reopen the patch (or I can do that). |
|
|
| msg41013 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2003-04-14 20:44 |
| Logged In: YES user_id=6380 OK, reading the README now... |
|
|
| msg41014 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2003-04-17 15:12 |
| Logged In: YES user_id=6380 I've checked in the changes to the parser and Makefile etc. This means you should be able to distribute pgenmodule.c as a clean extension to Python 2.3. I've decided not to incorporate pgenmodule.c at this time. There are three problems with it as I see it: - The C coding style is really different, and makes it hard to read: 2-space indents instead of tabs, spaces before left parentheses in argument lists, space between * and name in things like "PyObject * name", placement of curly braces, reversal of arguments to comparisons (e.g. "NULL != x"), and long strings of underscores in comments. - The code isn't robust enough: I got a segfault (can't remember after what sequence of events, sorry), and it caused a SystemError (error return without exception set) when the grammar file couldn't be read. - The pgen API seems a bit strange -- some things that I would expect to be methods of AST objects are functions in the pgen module (even when a companion function *is* an AST method). - The unit test doesn't fit in the Python test framework. - I cooked up a little expression grammar and tried to parse a sample string, but all I got was exceptions. I guess I am missing something, but there's no tutorial that explains me how to do things (I was copying the code from test_pgen.py which works :-( ). Nevertheless, thanks for the work so far -- I'm sure at some point pgen will be usable from Python! |
|
|
| msg41015 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2003-10-31 13:54 |
| Logged In: YES user_id=21627 Jon, are you still interested in taking this further? |
|
|
| msg41016 - (view) |
Author: Jon Riehl (jriehl) |
Date: 2003-11-01 23:20 |
| Logged In: YES user_id=22448 I am to some extent. I've done several wacky things, namely putting the pgen extension module into my Basil project (basically I think PEP 269 is dead), as well as re-written pgen in "pure" Python (no C extension module calls) The pure Python pgen looks like it will be going into Pypy. I don't know if I am interested in the speed enough to warrant serious analysis of the pgen C extension and the segmentation fault Guido reported. Let me know if you had anything specific in mind. |
|
|
| msg41017 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2003-11-02 18:35 |
| Logged In: YES user_id=21627 I was wondering what the possible next steps could be. With your current analysis, it appears that 1. declare PEP 269 withdrawn 2. reject this patch would be a possibility. I would want to keep this open only if some volunteer would indicate interest in completing this work. |
|
|
| msg41018 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2004-08-18 11:57 |
| Logged In: YES user_id=21627 I have now marked the PEP as deferred, until some volunteer picks it up again. I'm rejecting this patch (but left a pointer in the PEP for reference). |
|
|
| msg41019 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2004-08-19 03:57 |
| Logged In: YES user_id=6380 FWIW, I *also* rewrote pgen in Python. Is the Pypy version real? If so, I ma not have to publish my code (which is owned by Elemental Security but if there's demand I can attempt to open-source it). |
|
|
| msg41020 - (view) |
Author: Jon Riehl (jriehl) |
Date: 2004-08-19 19:02 |
| Logged In: YES user_id=22448 It may not be super usable (I believe it is as usable as pgen is, which is just usable enough to automate Python parser generation), but I consider it finished. The SVN URL is now: http://codespeak.net/svn/basil/trunk/basil/common/python/ |
|
|