[Python-Dev] REMOVE !! (original) (raw)

adam adam@isdn.net.il
Sun, 2 Dec 2001 09:36:43 +0200

----- Original Message ----- From: <python-dev-request@python.org> To: <python-dev@python.org> Sent: Saturday, December 01, 2001 6:33 AM Subject: Python-Dev digest, Vol 1 #1748 - 15 msgs

Send Python-Dev mailing list submissions to python-dev@python.org

To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request@python.org You can reach the person managing the list at python-dev-admin@python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..."

Today's Topics: 1. ArtifactFile: I can't change my patch files (James C. Ahlstrom) 2. Re: problems with DBM nonuniformity (Jason R. Mastaler) 3. Re: -DINET6 in Makefile (Martin v. Loewis) 4. Re: -DINET6 in Makefile (M.-A. Lemburg) 5. RE: gc.garbage (Tim Peters) 6. RE: gc.garbage (Tim Peters) 7. RE: Subclassing varying length types (What's a PyStructSequence ?) (Tim Peters) 8. Re: -DINET6 in Makefile (Martin v. Loewis) 9. Re: gc.garbage (Martin v. Loewis) 10. testbuiltin failing? or just 64-bit platforms (Mark Favas) 11. RE: testbuiltin failing? or just 64-bit platforms (Tim Peters) 12. Re: testbuiltin failing? or just 64-bit platforms (Guido van Rossum) 13. RE: testbuiltin failing? or just 64-bit platforms (Tim Peters) 14. RE: testbuiltin failing? or just 64-bit platforms (Barry A. Warsaw) 15. RE: testbuiltin failing? or just 64-bit platforms (Tim Peters) ------ Message: 1 Date: Fri, 30 Nov 2001 12:07:03 -0500 From: "James C. Ahlstrom" <jim@interet.com> To: python-dev@python.org Subject: [Python-Dev] ArtifactFile: I can't change my patch files I am trying to replace files in my patch #483466, but I am getting File delete: ArtifactFile: Permission Denied on SourceForge. If I need to be a member of the Python project to do this, could someone please add me? Or tell me what's wrong? The new files implement the changes to sys.path[0] that we have been discussing. I still need to make further changes to import.c so that "" and relative paths are handled. So don't install this patch yet, but look it over for problems if you want. Jim ------ Message: 2 To: python-list@python.org, python-dev@python.org, Date: Fri, 30 Nov 2001 10:17:24 -0700 From: "Jason R. Mastaler" <jason-dated-1007831845.6c6a51@mastaler.com> Subject: [Python-Dev] Re: problems with DBM nonuniformity Skip Montanaro <skip@pobox.com> writes: > Seems to me the natural thing to do would be to add > "getdatafilename" and "getindexfilename" methods (or something > similar) to the underlying modules (dbhash, bsddb, dbm, etc) and > expose them through anydbm. I see. So you agree that with the current implementation, there isn't a reliable way to do what I'm trying to do with DBM? > It's too late for 2.2, but I suspect if you implemented something > and method name(s) could be settled on it would make it into CVS > early in the 2.3 cycle. This seems like a small enough change that > you just file a bug report on SourceForge with the proposal and add > an implementation when you have something workable. I'm not sure when I'll be able to get to this, but I'll put it on my TODO list. In the meantime, I think I'll just support the auto regeneration feature I mentioned with CDB[1] instead of DBM since its file interface is consistent across platforms. Footnotes: 1. python-cdb extension module (http://pilcrow.madison.wi.us/) -- (TMDA - http://tmda.sourceforge.net) (Python-based SPAM reduction system) ------ Message: 3 Date: Fri, 30 Nov 2001 21:22:24 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: mal@lemburg.com CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile > What's the reasoning behind putting -DINET6 into the default > compiler options of the generic Makefile ? I believe the sole reason is that the author of the patch didn't know how to get it into pyconfig.h. itojun recently confirmed that all uses of the INET6 can be replaced with ENABLEIPV6, and that the define may go away. I hesitate to change that, though, since some of the IPv6 implementations may require that INET6 is defined when processing the "system" headers (not all IPv6 implementations we support actually come with the operating system). > I'm just asking because such a define will be inherited by > all extensions being compiled with distutils and the Makefile.pre.in > setup process... sounds like trouble if you ask me. What kind of trouble? > Shouldn't the define be placed into the pyconfig.h file or > only in those extensions which need it ? Wouldn't it cause the same trouble there? Regards, Martin ------ Message: 4 Date: Fri, 30 Nov 2001 21:46:14 +0100 From: "M.-A. Lemburg" <mal@lemburg.com> Organization: eGenix.com Software GmbH To: "Martin v. Loewis" <martin@v.loewis.de> CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile "Martin v. Loewis" wrote: > > > What's the reasoning behind putting -DINET6 into the default > > compiler options of the generic Makefile ? > > I believe the sole reason is that the author of the patch didn't know > how to get it into pyconfig.h. itojun recently confirmed that all uses > of the INET6 can be replaced with ENABLEIPV6, and that the define may > go away. > > I hesitate to change that, though, since some of the IPv6 > implementations may require that INET6 is defined when processing > the "system" headers (not all IPv6 implementations we support actually > come with the operating system). For Python's own use, it should suffice defining the symbol in pyconfig.h. > > I'm just asking because such a define will be inherited by > > all extensions being compiled with distutils and the Makefile.pre.in > > setup process... sounds like trouble if you ask me. > > What kind of trouble? The symbol could enable some logic which may not be desired by the application, e.g. cause system includes to change, socket semantics of wrapped libs could also be affected etc. > > Shouldn't the define be placed into the pyconfig.h file or > > only in those extensions which need it ? > > Wouldn't it cause the same trouble there? No, because the pyconfig.h import is under extension control (e.g. you can first include the system or lib header files and only then import pyconfig.h). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH

Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/

------ Message: 5 From: "Tim Peters" <tim.one@home.com> To: "Neil Schemenauer" <nas@python.ca>, "PythonDev" <python-dev@python.org> Subject: RE: [Python-Dev] gc.garbage Date: Fri, 30 Nov 2001 18:30:43 -0500 [Neil Schemenauer] > Is there some way to prevent people from assigning to certain module > variables? [Tim] >> Not that I know of. If you're terribly concerned, gc could look up >> "garbage" in its dict on each access. That's what, e.g., >> PRINTITEM does with sys.stdout. ... [Neil] > What would happen if it's not a list? PRINTITEM raises RuntimeError. > I suppose the collector could do the same. Sure, that would be fine. >> But I'd be keener to see new words spelling out which parts of the gc >> interface are and aren't intended "to work" across releases ... > All of them? :-) Seriously, there could come a time when GC can no > longer be disabled. The debugging and threshold stuff is fairly > implementation dependent. getreferrers() and getobjects() are highly > implementation dependent. I suppose gc.collect() should always be > available. Anything else is fair game, IMHO. I meant "new words" in the docs, not on Python-Dev . > Incidentally, I can't say I'm happy with GC as it stands. Well, you're young and hopeful -- you'll get over both. I have, and am indeed happy with GC as it stands. > It uses too much memory now that so many objects are tracked. There I disagree, but subtly: it always used "too much" memory. The marginal memory cost in adding a gazillion new tracked types was minor, as very few programs have a gazillion frame objects or traceback objects or generator-iterator objects (etc) sitting around. The vast bulk of the damage was done the instant lists, tuples, dicts and instances got tracked. So it goes. > I had worked on the idea of a separate heap for GC objects for a while > but couldn't figure out how to make generational collection to work. Generational gimmicks are rare in non-copying collectors for this very reason, right? > As Don Beaudry's sig says: "so much code, so little time". :-) Time for Don to change his sig -- his young and hopeful days should be long gone by now too .

------ Message: 6 From: "Tim Peters" <tim.one@home.com> To: <python-dev@python.org> Subject: RE: [Python-Dev] gc.garbage Date: Fri, 30 Nov 2001 18:30:44 -0500 [Martin v. Loewis] > I wish the C API hadn't been changed for 2.2, rendering useless all > code that had been created to support GC in 2.0 and 2.1. Would we really need more than one hand to count all that code <0.9 wink>? not-aware-of-any-myself-ly y'rs - tim ------ Message: 7 From: "Tim Peters" <tim.one@home.com> To: <python-dev@python.org> Subject: RE: [Python-Dev] Subclassing varying length types (What's a PyStructSequence ?) Date: Fri, 30 Nov 2001 18:35:23 -0500 [M.-A. Lemburg] > ... > Hmm, this makes me wonder: perhaps we should start thinking > about phasing out varying length PyObjects in the interpreter... No chance, IMO: the memory savings is too great. > esp. the inability to subclass strings looks like a bummer for > future extensions of this particular type. Unicode doesn't have > this problem, BTW. OTOH, I know someone at Zope Corp who could testify with force about the memory burden of switching to Unicode strings -- if you've got gobs of 'em, it's much worse than a factor of 2 blowup. Moving to obmalloc.c should help that a lot (two malloc overheads per Unicode string, and obmalloc overheads are much lower). > Or we need to come up with a fairly nice way of making > subclassing varying length types a lot easier, e.g. by > adding a special pointer obext to PyObjectVARHEAD > which then allows declaring type extensions in an malloced > buffer. > > Thoughts ? Not a one . ------ Message: 8 Date: Sat, 1 Dec 2001 02:07:19 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: mal@lemburg.com CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile > > > Shouldn't the define be placed into the pyconfig.h file or > > > only in those extensions which need it ? > > > > Wouldn't it cause the same trouble there? > > No, because the pyconfig.h import is under extension control > (e.g. you can first include the system or lib header files > and only then import pyconfig.h). Of course, doing so would be really stupid. Python.h must be the first include, or things may break. Regards, Martin ------ Message: 9 Date: Sat, 1 Dec 2001 02:11:34 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: tim.one@home.com CC: python-dev@python.org Subject: Re: [Python-Dev] gc.garbage > [Martin v. Loewis] > > I wish the C API hadn't been changed for 2.2, rendering useless all > > code that had been created to support GC in 2.0 and 2.1. > > Would we really need more than one hand to count all that code <0.9_ wink>? Mine was among it (in pyexpat), and I really hate looking at the code now. Not only need I support two versions of Unicode (i.e. having or not having it), but also three versions of GC, all in a single file. Regards, Martin ------_ Message: 10 Date: Sat, 01 Dec 2001 09:55:49 +0800 From: Mark Favas <mark.favas@csiro.au> Organization: CSIRO To: python-dev@python.org Subject: [Python-Dev] testbuiltin failing? or just 64-bit platforms Anyone else getting testbuiltin failures with current CVS, or does it only show on 64-bit platforms? Changes in the past week seem to have caused the failure. Isolated to following (will post bug on SF): ***** older CVS works ***** 201 mark@gonzo.per.dem.csiro.au python Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> s=-1-sys.maxint >>> d=s >>> d '-9223372036854775808' >>> s -9223372036854775808 >>> ^D ***** current CVS fails ***** 202 mark@gonzo.per.dem.csiro.au cd dist/src 203 mark@gonzo.per.dem.csiro.au ./python Python 2.2b2+ (#541, Dec 1 2001, 08:04:58) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> s=-1-sys.maxint >>> d=s >>> d '8t\x10@\x01' >>> s -9223372036854775808 >>> ^D -- Mark Favas - m.favas@per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA ------ Message: 11 From: "Tim Peters" <tim.one@home.com> To: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] testbuiltin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 21:41:56 -0500 [Mark Favas] > Anyone else getting testbuiltin failures with current CVS, No. > or does it only show on 64-bit platforms? Changes in the past week > seem to have caused the failure. Isolated to following (will post bug > on SF): Thanks! > ***** older CVS works ***** > 201 mark@gonzo.per.dem.csiro.au python > Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> s=-1-sys.maxint > >>> d=s > >>> d > '-9223372036854775808' > >>> s > -9223372036854775808 > >>> ^D > > ***** current CVS fails ***** > 202 mark@gonzo.per.dem.csiro.au cd dist/src > 203 mark@gonzo.per.dem.csiro.au ./python > Python 2.2b2+ (#541, Dec 1 2001, 08:04:58) [C] on osf1V4 > Type "help", "copyright", "credits" or "license" for more information. > >>> import sys > >>> s=-1-sys.maxint > >>> d=s > >>> d > '8t\x10@\x01' > >>> s > -9223372036854775808 > >>> ^D It doesn't make sense, but it smells like a sprintf -> PyOSsnprintf screwup ... OK, our int repr code has always been wrong(!): static PyObject * intrepr(PyIntObject *v) { char buf[20]; PyOSsnprintf(buf, sizeof(buf), "%ld", v->obival); return PyStringFromString(buf); } 20 bytes isn't enough to hold the result on a 64-bit box (insert rant about the idiot practice of trying to make stack buffers as small as possible). You have 20 characters in your result, but need 21 to hold the trailing 0 byte too. I don't know what snprintf does when there's not enough room, but I think you just showed us what it does on Tru64 . Doing repr impllicitly instead works because that goes thru intprint instead of intrepr. Doing repr explicitly "worked" before by accident (the trailing null overwrite some random byte on the stack). Please change 20 to 200(*) (or 64 -- your choice) and see whether that fixes it? ------ Message: 12 To: "Tim Peters" <tim.one@home.com> cc: "Mark Favas" <mark.favas@csiro.au>, python-dev@python.org Subject: Re: [Python-Dev] testbuiltin failing? or just 64-bit platforms From: Guido van Rossum <guido@python.org> Date: Fri, 30 Nov 2001 22:01:24 -0500 > I don't know what snprintf does when there's not enough room, but > I think you just showed us what it does on Tru64 . Hm, snprintf is supposed to truncate the result, but it seems that here it refused to do anything. Maybe PyOSsnprintf should be a wrapper that checks the return value of snprintf? I noticed that none of the recently checked-in PyOSsnprintf calls have their return value checked, and I think it's better to leave it that way (since in many cases we really don't know what to do instead) -- maybe PyOSsnprintf should even return void (or does it already?). --Guido van Rossum (home page: http://www.python.org/~guido/) ------ Message: 13 From: "Tim Peters" <tim.one@home.com> To: "Guido van Rossum" <guido@python.org> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] testbuiltin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 22:42:58 -0500 > Hm, snprintf is supposed to truncate the result, According to C99, yes, but MS has its own pre-C99 snprintf semantics, and glibc has had at least two different versions. > but it seems that here it refused to do anything. Maybe PyOSsnprintf > should be a wrapper that checks the return value of snprintf? Check it for what? We can't (at least not yet) count on uniform behavior across platform snprintf implementations. > I noticed that none of the recently checked-in PyOSsnprintf calls > have their return value checked, They were all derived (and most mindlessly) from existing sprintf calls that didn't check either. A goal of this transformation was to change as little as possible. > and I think it's better to leave it that way (since in many cases we > really don't know what to do instead) -- maybe PyOSsnprintf should > even return void (or does it already?). The comments suggest it wants to return the C99-defined value (an int, < 0_ _for an encoding error, else the number of characters (excluding \0)_ written _if the size is big enough, else the number of characters that would have_ _been written (excluding \0) had size been big enough. So the output was_ _converted in full iff the return value is non-negative and strictly less_ _than the size passed in. That's fine by me, and I'll note that MS_ snprintf _meets that much too (if size isn't big enough, it returns a negative_ _result). So perhaps in a debug build PyOSsnprintf could assert that the_ _returned value is non-negative and less than the passed-in size ..._ _--_--_--_ _Message: 14_ _Date: Fri, 30 Nov 2001 23:17:31 -0500_ _To: "Tim Peters" <tim.one@home.com> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] testbuiltin failing? or just 64-bit platforms From: barry@zope.com (Barry A. Warsaw) >>>>> "TP" == Tim Peters <tim.one@home.com> writes: TP> It doesn't make sense, but it smells like a sprintf -> TP> PyOSsnprintf screwup ... OK, our int repr code has always TP> been wrong(!): | static PyObject * | intrepr(PyIntObject *v) | { | char buf[20]; | PyOSsnprintf(buf, sizeof(buf), "%ld", v->obival); | return PyStringFromString(buf); | } TP> 20 bytes isn't enough to hold the result on a 64-bit box TP> (insert rant about the idiot practice of trying to make stack TP> buffers as small as possible). You have 20 characters in your TP> result, but need 21 to hold the trailing 0 byte too. I don't TP> know what snprintf does when there's not enough room, but I TP> think you just showed us what it does on Tru64 . Heh, I was going to suggest that this might be a good place to substitute a call to PyStringFromFormat*() but then I read this little nugget: case 'd': case 'i': case 'x': (void) vaarg(count, int); /* 20 bytes should be enough to hold a 64-bit integer */ n += 20; break; love-ly y'rs, -Barry ------ Message: 15 From: "Tim Peters" <tim.one@home.com> To: "Barry A. Warsaw" <barry@zope.com> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] testbuiltin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 23:31:51 -0500 [Barry] > ... > Heh, I was going to suggest that this might be a good place to > substitute a call to PyStringFromFormat*() but then I read this > little nugget: > > case 'd': case 'i': case 'x': > (void) vaarg(count, int); > /* 20 bytes should be enough to > hold a 64-bit > integer */ > n += 20; > break; This use of 20 is fine (I checked all occurrences of "20" in the codebase, BTW); intrepr's use of 20 was wrong because it failed to allow for the trailing \0 byte too; the loop in PyStringFromFormatV doesn't have to worry about that (PyStringFromStringAndSize() later adds 1 of its own for the trailing \0 -- of course it doesn't actually add anything, but it adds 1 "in effect" <wink -- jeez, C string handling is obscure!>).

------

Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

End of Python-Dev Digest