Issue 464405: freeze doesn't like DOS files on Linux (original) (raw)

Issue464405

Created on 2001-09-24 13:41 by anonymous, last changed 2022-04-10 16:04 by admin. This issue is now closed.

Messages (18)
msg6653 - (view)	Author: Nobody/Anonymous (nobody)	Date: 2001-09-24 13:41
I've got a perfectly running Python program on Linux, importing some modules that were apparently written on Windows, since the files are in DOS format (carriage return + line feed at the end of lines). When I try to freeze this program, the module finder crashes on every module in DOS format claiming that there's a syntx error at the end of each line. This is not really serious, but quite annoying... Thanks.
msg6654 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2001-09-26 19:51
Logged In: YES user_id=21627 What Python version? Could you please submit an example of one of the files it is complaining about? Since you cannot attach it to this report, it would be good if you could open a new report. When doing so, please identify yourself.
msg6655 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2001-09-27 18:02
Logged In: YES user_id=6380 I can reproduce this with Python 2.2 quite easily: on Linux, in the Tools/freeze directory, use the ../scripts/lfcr.py script to give hello.py CRLF line endings. Then try to freeze it. The cause is the code in the tokenizer that parses from strings, which doesn't like the \r\n line endings and doesn't throw the \r out like the code that parses from a file does. I'm tempted to request a feature that the tokenizer should be smarter when it finds a \r (e.g. just ignore it).
msg6656 - (view)	Author: Neal Norwitz (nnorwitz) *	Date: 2002-10-18 20:56
Logged In: YES user_id=33168 I added "U" for the mode flag to open() for universal newlines. This fixed the problem. I modified Tools/freeze/modulefinder.py in run_script() and load_file() about lines 120 and 127. I didn't see any other open()s which seemed to need "U". Is that sufficient to close this bug?
msg6657 - (view)	Author: Neal Norwitz (nnorwitz) *	Date: 2002-11-12 23:21
Logged In: YES user_id=33168 I don't know if there's an easy way to fix this for 2.2.x, so I'm fixing this for 2.3 with universal newlines. Checked in as: * Tools/freeze/modulefinder.py 1.22
msg6658 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-14 18:28
Logged In: YES user_id=11105 Modulefinder is listed in PEP 0291 as being required to stay compatible with Python 1.5.2. A string method has crept in in line 167 which is easy to remove. Not so easy is this change for universal newline support. Since I am also listed as the maintainer, I would like to fix this. So, any idea how to detect if the current Python has universal newline support?
msg6659 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2002-11-14 18:31
Logged In: YES user_id=6380 try: f = open(filename, "U") except IOError: f = open(filename, "r") You can also check for sys.version >= "2.3"
msg6660 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-14 18:53
Logged In: YES user_id=11105 I like the first way better. Re the string methods: I'm getting tired fixing these, so I'd like to suggest to raise the PEP 291 version requirement for modulefinder to 2.0 (or 2.2?). What should I do? Contact the author (Neal, in this case)?
msg6661 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2002-11-14 18:56
Logged In: YES user_id=6380 I don't remenber whence the 1.5.2 requirement comes. I propose that all such requirements should be documented in a COMMENT at the TOP OF THE FILE, not just in PEP 291.
msg6662 - (view)	Author: Neal Norwitz (nnorwitz) *	Date: 2002-11-14 19:01
Logged In: YES user_id=33168 I can update the PEP if you want, or you can do it yourself.
msg6663 - (view)	Author: Neal Norwitz (nnorwitz) *	Date: 2002-11-14 19:11
Logged In: YES user_id=33168 Hmmm, I have an outstanding change to PEP 291 to remove modulefinder. Is there a requirement or does the PEP need to be updated? How's the following addition to the PEP. I put it as the last para in the Rationale section. Even if a package or module is listed in this PEP, authors should document a compatibility requirement in each file with a comment at the top of the file.
msg6664 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-14 19:18
Logged In: YES user_id=11105 Guido: We had some discussion about this when neal wrote the pep, and I suggested to list modulefinder there. Neal finally wrote the requirement into the PEP, and it was my impression that you accepted it, although you were not very happy with it. I will insert a comment into modulefinder.py listing the requirement.
msg6665 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-14 19:25
Logged In: YES user_id=11105 Neal: In this case (remove modulefinder from PEP 291) I won't insist on it. Let the BDFL decide, or has he already?
msg6666 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2002-11-14 19:26
Logged In: YES user_id=6380 That still doesn't explain to me *why* you wanted modulefinder.py listed. (Maybe my memory is bad.) If you were the source of that request and you're now revising the request, please go ahead.
msg6667 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2002-11-14 19:28
Logged In: YES user_id=6380 I have no need for modulefinder so I'm not deciding anything. It's up to Thomas.
msg6668 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-14 19:43
Logged In: YES user_id=11105 Yes, I requested to include modulefinder in PEP 291. There's no need to discuss this again, but I'll try to summarize the discussion we had in June: I use modulefinder in py2exe, a freeze like tool. Py2exe (currently) supports all python versions from 1.5.2 up to 2.2. Modulefinder is distributed together with py2exe, since it is not easily accessible in the Tools directory. So I requested it to be compatible with all these Python versions, but you doubted it would work because of byte-code changes. My experience is that it works well. Another approach would have been to move modulefinder into the standard lib, but you didn't want this because it is only useful for the freeze utility. So it finally ended up been listed in the pep, with me as the maintainer. Neal: I'll think about the Python versions which I want to be supported, and contact you again (or update the pep myself).
msg6669 - (view)	Author: Jack Jansen (jackjansen) *	Date: 2002-11-15 23:34
Logged In: YES user_id=45365 Just a quick note on testing for universal newlines: the idiom I use is "hasattr(sys.stdout, 'newlines')". Cleaner may be hasattr(file, 'newlines'), but this won't work as far back as 1.5.2, I think.
msg6670 - (view)	Author: Thomas Heller (theller) *	Date: 2002-11-18 08:29
Logged In: YES user_id=11105 Jack, this looks much nicer. Thanks.