Issue 33902: entry_points/console_scripts is too slow (original) (raw)

Created on 2018-06-19 14:34 by gwk, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg319971 - (view) Author: George King (gwk) * Date: 2018-06-19 14:34
On my newish macOS laptop using Python 3.6 or 3.7, a no-op script takes 3 times as long to invoke using the entry_points machinery as it does to invoke directly. Here are some exemplary times (best times after several tries). $ time python3.6 entrypoint.py real 0m0.047s user 0m0.033s sys 0m0.010s $ time python3.6 /Library/Frameworks/Python.framework/Versions/3.6/bin/entrypoint real 0m0.153s user 0m0.129s sys 0m0.020s In this example, entrypoint.py consists of `def main(): pass`; the second script is that generated by `pip3 install ...`. I have confirmed that the `-e` flag to pip is not the culprit. I look at a cprofile trace of the invocation and I do not see an obvious single culprit, but I'm not an expert at reading such traces either. I know that there has been some previous debate about "how slow is slow", whether to try to improve import times, etc. I'm not trying to fan any flames here, but from a practical perspective, this discrepancy is showing up in developer tools that I'm writing, and is driving me to abandon entry_points in favor of less standard, less cross-platform install hackery in my setup.py scripts. An extra 0.1s per invocation is visibly detrimental to shell scripts calling out to installed python programs.
msg319989 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-06-19 17:54
This really isn't enough information to know exactly what's going on in your case, but there are things that are known to be slow for CLI startups. Probably the biggest contributor is pkg_resources. Of course, that's a third party library that's popular for handling entry points, so it's not really a Python problem. There are some other third party libraries on PyPI that may be more performant for you. Entry point handling is something that I'd like to try to address in 3.8 in a manner similar to importlib.resources.
msg319994 - (view) Author: George King (gwk) * Date: 2018-06-19 19:21
Thanks Barry. My question then is, what relationship does cpython have with pip, setuptools, distutils and pkg_resources? Since pip comes bundled with Python now it seems a little bit closer than "3rd party". Thanks!
msg319995 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-06-19 20:00
"It's complicated" :) But technically speaking they are all separate projects, so while there is synergy, CPython itself *imports* some of those to provide various tasks, but it doesn't lead their development. You'll find a lot of the same players in all those projects though. This might be a good resource: https://www.pypa.io/en/latest/ I hope you don't mind, but I am going to close this issue since it's not directly related to Python itself. If you feel otherwise and/or can provide a reproducible test case, please do reopen it. Thanks for your contribution to Python!
msg319996 - (view) Author: George King (gwk) * Date: 2018-06-19 20:07
OK, thanks. I agree that this is best pursued with the developers of the relevant modules. I appreciate your quick and detailed responses!
History
Date User Action Args
2022-04-11 14:59:01 admin set github: 78083
2018-06-19 20:07:46 gwk set messages: +
2018-06-19 20:00:40 barry set status: open -> closedresolution: third partymessages: + stage: resolved
2018-06-19 19:21:48 gwk set messages: +
2018-06-19 17:54:20 barry set nosy: + barrymessages: +
2018-06-19 15:15:36 ericvw set nosy: + ericvw
2018-06-19 14:34:17 gwk create