Issue 30574: Document why embedding the interpreter can sometimes crash. (original) (raw)

Created on 2017-06-05 13:47 by Decorater, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (13)
msg295187 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 13:47
According to this example at https://docs.python.org/3/extending/embedding.html#very-high-level-embedding you can use something similar on embedding the interpreter. However for some reason it can crash on Windows and probably some other platforms no matter what python code you enter in the string. I am currently getting that issue on Windows even with the python DLL's being in the same folder of the embedded executable it seems (when I open the crash dump in VS2010 express) that it tried to access an memory address that it has no permissions to. However I have been thinking as to why that might be. The system should have plenty of memory left for it to allocate and run just fine unless it wants to use an already active python process (why would that be when other applications that embed the interpreter seem to work just fine). I think documenting there on when and iff the compiled code crashes with suggestions on how to fix them would be best to do on the page. it could be that the example actually compiles but does not work anymore too. But why would it be on the page if it would compile but crash on any string put into it? Something makes me think the crash is on the Py_DecodeLocale line with the size_t * param being NULL.
msg295188 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-06-05 13:55
I'm a bit confused what you're asking for here. Certainly if you embed Python, you can get crashes, just as with any other C program. Pass an invalid pointer to a Python API, build your application and the Python DLL with incompatible C runtimes, free memory you don't own, ... That's not really something we need to cover in the Python documentation, though, it's just things you need to understand if you're doing anything beyond the most trivial C programming. If there's a specific issue that you're having, we'd need more details to say anything else.
msg295190 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 14:22
Well the example I linked if I compile it and try to run it it instantly crashes with no explainable reason. I am using the same compiler used to build the core too and the normal interpreter works just fine with the code I am trying to embed. So it seems to definitely be the Py_DecodeLocale or the Py_SetProgramName line.
msg295191 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-06-05 14:31
Worked fine for me: >cl .\pyc.c /I env:LOCALAPPDATAProgramsPythonPython36includeenv:LOCALAPPDATA\Programs\Python\Python36\include env:LOCALAPPDATAProgramsPythonPython36includeenv:LOCALAPPDATA\Programs\Python\Python36\libs\python36.lib Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24213.1 for x64 Copyright (C) Microsoft Corporation. All rights reserved. pyc.c Microsoft (R) Incremental Linker Version 14.00.24213.1 Copyright (C) Microsoft Corporation. All rights reserved. /out:pyc.exe pyc.obj ...\Programs\Python\Python36\libs\python36.lib >.\pyc.exe Today is Mon Jun 5 15:27:45 2017 So it doesn't look like the problem is with the code, so much as with how you've done the build. The problem here is that there are so many possibilities for C compilation environments, that there's simply no way we could give a "this is how you set up to build an embedding application". If you have specific suggestions for how to improve the documentation, a patch would be welcome. But in the absence of specifics, I don't think there's anything actionable in this report.
msg295193 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 14:39
hmm, maybe it was because I was compiling it inside of the VS2015 IDE in an project file and not with cl.exe directly.
msg295194 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 14:48
Well looks like with cl.exe it is still crashing.
msg295195 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2017-06-05 14:53
My bet is still on something like you compiling your code 32-bit and linking to 64-bit Python, or something like that.
msg295196 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-06-05 14:54
Agreed, there's nothing for us to do here. It sounds like you may need to go to a help forum to work through your crash. If you learn things that you think it would be valuable to add to the docs, please do submit a doc enhancement issue/PR.
msg295197 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 14:56
I am compiling and linking to 32 bit python based on the folder I entered for the lib file. Unless the lib file in the lib folder is somehow for the 64 bit one.
msg295198 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 15:12
Wait a minute, I think it is because the 32 and 64 bit python36.lib files are in the path environment variable so that might be the reason for the crash.
msg295199 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 15:19
E:\Users\Elsword\Documents\GitHub\TinyURL\app>call "E:\Program Files (x86)\Micro soft Visual Studio 14.0\Common7\Tools\VsDevCmd.bat" E:\Users\Elsword\Documents\GitHub\TinyURL\app>cd "E:\Users\Elsword\Documents\Git Hub\TinyURL\app\" E:\Users\Elsword\Documents\GitHub\TinyURL\app>RC.exe /r TinyURL.rc Microsoft (R) Windows (R) Resource Compiler Version 10.0.10011.16384 Copyright (C) Microsoft Corporation. All rights reserved. E:\Users\Elsword\Documents\GitHub\TinyURL\app>cl.exe /Tc "TinyURL.c" /I E:\Python360\include E:\Python360\libs\python36.lib /link /OUT:"TinyURL.exe" /SUBSYSTEM:CONSOLE "TinyURL.res" /FIXED Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x86 Copyright (C) Microsoft Corporation. All rights reserved. TinyURL.c TinyURL.c(204): warning C4129: '/': unrecognized character escape sequence Microsoft (R) Incremental Linker Version 14.00.24215.1 Copyright (C) Microsoft Corporation. All rights reserved. /out:TinyURL.exe /OUT:TinyURL.exe /SUBSYSTEM:CONSOLE TinyURL.res /FIXED TinyURL.obj E:\Python360\libs\python36.lib Wait a minute, it seems the build tools are a bit out of date.
msg295202 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 16:26
I found where the crash was at. It was at my call to PyRun_SimpleString. Something in the string caused it to crash.
msg295204 - (view) Author: Decorater (Decorater) * Date: 2017-06-05 16:45
turns out it was crashing because I forgot to add in Py_Initialize.
History
Date User Action Args
2022-04-11 14:58:47 admin set github: 74759
2017-06-05 19:58:07 paul.moore set nosy: - paul.moore
2017-06-05 19:39:47 r.david.murray set nosy: - r.david.murray
2017-06-05 16:45:31 Decorater set messages: +
2017-06-05 16:26:18 Decorater set messages: +
2017-06-05 15:25:56 zach.ware set assignee: docs@python -> nosy: - docs@python, zach.ware
2017-06-05 15:19:25 Decorater set messages: +
2017-06-05 15:12:17 Decorater set messages: +
2017-06-05 14:56:38 Decorater set messages: +
2017-06-05 14:54:15 r.david.murray set status: open -> closednosy: + r.david.murraymessages: + resolution: not a bugstage: resolved
2017-06-05 14:53:28 paul.moore set messages: +
2017-06-05 14:48:47 Decorater set messages: +
2017-06-05 14:39:47 Decorater set messages: +
2017-06-05 14:31:57 paul.moore set messages: +
2017-06-05 14:22:56 Decorater set messages: +
2017-06-05 13:55:46 paul.moore set messages: +
2017-06-05 13:47:58 Decorater create