Issue 18458: interactive interpreter crashes and test_readline fails on OS X 10.9 Mavericks due to libedit update (original) (raw)

Created on 2013-07-15 12:24 by ronaldoussoren, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
dynamic_editline_workaround_detection.txt ronaldoussoren,2013-07-15 12:24 review
issue18458.patch ned.deily,2013-09-06 22:10 review
patch_readline_issue_18458.sh ned.deily,2013-10-24 02:47
Messages (24)
msg193090 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-15 12:24
Modules/readline.c contains a workaround for a bug in the readline emulation of libedit: that emulation uses a different starting offset for the history than the real readline. In more recent versions of libedit (such as the one you can now download from <http://www.thrysoee.dk/editline/>) the bug has been fixed and the workaround causes problems. The attached patch dynamicly detects if the workaround is necessary. NOTE: Actually using the trysoee.dk libedit requires a patch to setup.py, that version of libedit does not install libreadline.dylib, only libedit.dylib.
msg193091 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-15 12:27
(set as "crash" because the current workaround causes a crash with recent versions of libedit)
msg193467 - (view) Author: Michael Kleehammer (mkleehammer) * Date: 2013-07-21 21:08
Just wanted to chime in that I reported this to Apple and their response was exactly this. (Perhaps Ronald works for Apple?) Is there a temporary work around in the meantime other than rebuilding everything?
msg193470 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-07-21 21:59
If you are using an existing binary installation, you could disable readline processing altogether by simply renaming the readline extension module, for example: cd /Library/Frameworks/Python.framework/Versions/3.3 cd ./lib/python3.3/lib-dynload sudo mv readline.so readline.so.disabled Of course, then you will not have interactive command line history and much of the editing features. But it shouldn't crash. If you don't mind using GNU readline (which is GPL licensed), you *might* be able to install the third-party readline module from PyPI: https://pypi.python.org/pypi/readline But extension module building on unreleased systems may be problematic. It may be easier to build on a currently supported system and move it. As necessary, we will likely need to provide updates for the OS X installers in conjunction with the official release of 10.9.
msg193500 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2013-07-22 05:43
Building just the readline extension shouldn't be that hard, using a small custom setup.py script. Then install by manually copying it to the right location. And no, I don't work for Apple.
msg197113 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-09-06 22:10
The original patch missed one spot. Here's an updated version. I'm going to apply this to default (for the imminent 3.4.0a2) in case people start running into this on new versions of OS X. It needs to be backported to 2.7 and 3.3; a test would be nice and it could be simplified a bit as part of the generalized support for libedit (Issue13501).
msg197116 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-09-06 22:17
New changeset b6b8a2171aa3 by Ned Deily in branch 'default': Issue #18458: Prevent crashes with newer versions of libedit. Its readline http://hg.python.org/cpython/rev/b6b8a2171aa3
msg199637 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-10-12 22:58
New changeset 1e03fd72e116 by Ned Deily in branch '2.7': Issue #18458: Prevent crashes with newer versions of libedit. Its readline http://hg.python.org/cpython/rev/1e03fd72e116 New changeset dfb7cab9f819 by Ned Deily in branch '3.3': Issue #18458: Prevent crashes with newer versions of libedit. Its readline http://hg.python.org/cpython/rev/dfb7cab9f819 New changeset 47a7313f079a by Ned Deily in branch 'default': Issue #18458: merge comments from 3.3 http://hg.python.org/cpython/rev/47a7313f079a
msg199638 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-12 23:04
Backported to 2.7 (for 2.7.6) and 3.3 (for 3.3.3) as well. The current test_readline already covers this case.
msg201068 - (view) Author: Robert Xiao (nneonneo) * Date: 2013-10-23 21:01
I've attached a fixed readline.so built from today's 2.7 branch, for the convenience of anyone who upgraded to 10.9 and now has crashing Python. Drop the file in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload, replacing the original readline.so. This file is compatible with both 32-bit and 64-bit python.org builds of Python 2.7.5.
msg201070 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-23 21:42
Robert, I appreciate your effort but I do not think it is good practice to recommend downloading and installing binary bits from the bug tracker from an uploaded file. Also, at a cursory examination, the readline.so supplied is not exactly a drop-in replacement for the one supplied by the python.org installer as it is linked with wrong ncurses library. We should have a new set of maintenance releases soon. My preference would be for you to delete that file.
msg201087 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-24 03:07
Until the updated readline module is available in python.org 2.7.6 and 3.3.3 maintenance releases, here is a script that will check python.org versions of Python 2.7.x, 3.2.x, 3.3.x, and 3.4.x for interactive crashes and will patch around the problem as needed. On most systems you will need to run it under an account with administrator privileges. To use it, open a terminal session in Terminal.app (or other shell), then enter: curl -O http://bugs.python.org/file32324/patch_readline_issue_18458.sh openssl sha1 patch_readline_issue_18458.sh # the digest should be 7cb0ff57820a027dd4ca242eb2418930f8f46b4c sh ./patch_readline_issue_18458.sh The output on a 10.9 system with python.org 2.7.5, 3.3.2, and 3.4.0a4 installed should look something like this: -- running on OS X 10.9 -- 2.7 crashed -- 2.7 moving readline extension to /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/readline.so.disabled -- need to patch. Enter password if prompted Password: -- 2.7 is now patched -- 3.2 not found - skipped -- 3.3 crashed -- 3.3 moving readline extension to /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/lib-dynload/readline.so.disabled -- 3.3 is now patched -- 3.4 does not need to be patched - skipped -- done
msg201129 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-10-24 13:43
Does this affect 2.6? Is there a patch we need to get into 2.6.9 final?
msg201176 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-24 20:14
Barry: yes, 2.6 is affected. See discussion on python-dev.
msg201184 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2013-10-24 20:43
On Oct 24, 2013, at 08:14 PM, Ned Deily wrote: >Barry: yes, 2.6 is affected. See discussion on python-dev. Thanks Ned for the background over on python-dev. Unless I hear objections otherwise, I am not going to apply this to 2.6.
msg201507 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-27 23:02
An update: release candidates for Python 3.3.3 and 2.7.6 are now available, including binary installers for OS X, that include this fix and fully support OS X 10.9. http://www.python.org/download/releases/3.3.3/ http://www.python.org/download/releases/2.7.6/
msg201748 - (view) Author: Mark Richman (mrichman) Date: 2013-10-30 16:55
I had to do `sudo sh ./patch_readline_issue_18458.sh` or the patch script itself would cause Python to crash. After applying this patch, I got the following output, and the problem is still *not* solved for me: -- running on OS X 10.9 -- 2.7 does not need to be patched - skipped -- 3.2 not found - skipped -- 3.3 not found - skipped -- 3.4 not found - skipped -- done
msg201772 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2013-10-30 19:32
Mark, I'm not sure I understand what you saw but the patch script will cause a Python crash as part of its testing so that is to be expected. You should not have to run the script using sudo. This script also only applies to Pythons installed from python.org (or otherwise installed into /Library/Frameworks). Please check which python you are using. Using whatever command name you enter to start the failing python, try the following (I'll assume you use "python2.7"): type python2.7 which python2.7 python2.7 -c "import sys;print(sys.version)" python2.7 -c "import sys;print(sys.executable)" The value for sys.executable should be: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python In any case, you could manually rename _readline.so as shown in one of the earlier messages (substituting "2.7" for "3.3"). Or you could install 2.7.6rc1.
msg201773 - (view) Author: Mark Richman (mrichman) Date: 2013-10-30 19:37
My mistake. I'm using /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -- *Mark Richman* web: www.markrichman.com email: mark@markrichman.com tel: (954) 234-9049 http://www.linkedin.com/in/mrichman On Wed, Oct 30, 2013 at 3:32 PM, Ned Deily <report@bugs.python.org> wrote: > > Ned Deily added the comment: > > Mark, I'm not sure I understand what you saw but the patch script will > cause a Python crash as part of its testing so that is to be expected. You > should not have to run the script using sudo. This script also only > applies to Pythons installed from python.org (or otherwise installed into > /Library/Frameworks). Please check which python you are using. Using > whatever command name you enter to start the failing python, try the > following (I'll assume you use "python2.7"): > > type python2.7 > which python2.7 > python2.7 -c "import sys;print(sys.version)" > python2.7 -c "import sys;print(sys.executable)" > > The value for sys.executable should be: > > /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python > > In any case, you could manually rename _readline.so as shown in one of the > earlier messages (substituting "2.7" for "3.3"). Or you could install > 2.7.6rc1. > > ---------- > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue18458> > _______________________________________ >
msg201888 - (view) Author: Russell Jurney (Russell.Jurney) Date: 2013-11-01 09:35
I have installed 2.7.6 RC1 on OS X 10.9, and this bug still exists. See http://stackoverflow.com/questions/19722580/segfault-11-with-pandas-with-python-v2-7-6-rc1-on-mac-os-x-10-9
msg201902 - (view) Author: Robert Xiao (nneonneo) * Date: 2013-11-01 14:36
Russell, that's not related to readline. Please file a new bug report.
msg244692 - (view) Author: Etienne Le Sueur (Etienne Le Sueur) Date: 2015-06-02 21:32
So what's the resolution here? All I see is a script that disables the readline module depending on whether the interactive session crashes... Has a fix been pushed into the 3.4 release such that linking the readline module against editline results in a working interactive interpreter?
msg244694 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-06-02 21:39
Yes, this is fixed, as the issue resolution says. If you are curious about the fix, follow the links to the commits starting in .
msg244701 - (view) Author: Etienne Le Sueur (Etienne Le Sueur) Date: 2015-06-02 22:04
Thanks, I had missed that message. On 6/2/15 4:39 PM, R. David Murray wrote: > R. David Murray added the comment: > > Yes, this is fixed, as the issue resolution says. If you are curious about the fix, follow the links to the commits starting in . > > ---------- > nosy: +r.david.murray > > _______________________________________ > Python tracker <report@bugs.python.org> > <http://bugs.python.org/issue18458> > _______________________________________
History
Date User Action Args
2022-04-11 14:57:48 admin set github: 62658
2021-11-04 14:27:39 erlendaasland set nosy: + jcea, ronaldoussoren, nneonneo, mkleehammer, ned.deily, python-dev, mrichman, Russell.Jurney, Etienne Le Sueur, - barry, r.david.murray, ahmedsayeed1982components: - emailversions: - Python 3.11
2021-11-04 14:26:29 erlendaasland set messages: -
2021-11-04 12:12:54 ahmedsayeed1982 set versions: + Python 3.11, - Python 2.7, Python 3.3, Python 3.4nosy: + ahmedsayeed1982, - jcea, ronaldoussoren, nneonneo, mkleehammer, ned.deily, python-dev, mrichman, Russell.Jurney, Etienne Le Sueurmessages: + components: + email
2015-06-02 22:04:04 Etienne Le Sueur set messages: +
2015-06-02 21:39:23 r.david.murray set nosy: + r.david.murraymessages: +
2015-06-02 21:32:49 Etienne Le Sueur set nosy: + Etienne Le Sueurmessages: +
2014-05-05 05:29:24 ned.deily link issue21433 superseder
2014-04-15 16:44:11 ned.deily link issue21241 superseder
2014-04-02 03:19:47 zach.ware link issue21129 superseder
2014-03-25 16:44:25 ned.deily link issue21064 superseder
2014-02-04 21:51:03 ned.deily link issue20512 superseder
2013-12-03 05:19:31 ned.deily link issue19868 superseder
2013-11-18 12:45:45 ezio.melotti link issue19644 superseder
2013-11-12 13:31:31 ezio.melotti link issue19559 superseder
2013-11-07 03:51:08 ned.deily link issue19516 superseder
2013-11-01 17:10:13 ned.deily unlink issue19471 superseder
2013-11-01 14:36:27 nneonneo set messages: +
2013-11-01 14:35:55 nneonneo set files: - readline.so
2013-11-01 09:35:12 Russell.Jurney set nosy: + Russell.Jurneymessages: +
2013-11-01 09:26:01 ezio.melotti link issue19471 superseder
2013-10-30 19:37:17 mrichman set messages: +
2013-10-30 19:32:25 ned.deily set messages: +
2013-10-30 16:55:33 mrichman set nosy: + mrichmanmessages: +
2013-10-27 23:02:44 ned.deily set messages: +
2013-10-26 00:33:57 ned.deily link issue19401 superseder
2013-10-24 21🔞57 benjamin.peterson link issue19381 superseder
2013-10-24 20:43:23 barry set messages: +
2013-10-24 20:14:07 ned.deily set messages: +
2013-10-24 13:43:32 barry set nosy: + barrymessages: +
2013-10-24 07:53:56 ned.deily link issue19374 superseder
2013-10-24 03:08:00 ned.deily set messages: + title: interactive interpreter crashes and test_readline fails with newer versions of libedit -> interactive interpreter crashes and test_readline fails on OS X 10.9 Mavericks due to libedit update
2013-10-24 02:47:33 ned.deily set files: + patch_readline_issue_18458.sh
2013-10-23 22:06:17 ned.deily link issue19370 superseder
2013-10-23 21:42:54 ned.deily set messages: +
2013-10-23 21:01:50 nneonneo set files: + readline.sonosy: + nneonneomessages: +
2013-10-23 19:48:27 ned.deily link issue19368 superseder
2013-10-23 18:05:33 ned.deily link issue19366 superseder
2013-10-12 23:04:06 ned.deily set status: open -> closedtitle: libedit history offset workaround -> interactive interpreter crashes and test_readline fails with newer versions of libeditmessages: + keywords: - needs reviewresolution: fixedstage: patch review -> resolved
2013-10-12 22:58:12 python-dev set messages: +
2013-09-06 22:17:06 python-dev set nosy: + python-devmessages: +
2013-09-06 22:10:12 ned.deily set files: + issue18458.patchmessages: +
2013-07-23 14🔞02 jcea set nosy: + jcea
2013-07-22 05:43:46 ronaldoussoren set messages: +
2013-07-21 21:59:41 ned.deily set messages: +
2013-07-21 21:08:17 mkleehammer set nosy: + mkleehammermessages: +
2013-07-16 22:55:29 ned.deily link issue18289 superseder
2013-07-15 12:27:21 ronaldoussoren set keywords: + patch, needs reviewtype: crashmessages: + stage: patch review
2013-07-15 12:24:20 ronaldoussoren create