Issue 1284670: Allow to restrict ModuleFinder to get "direct" dependencies (original) (raw)

process

Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: JelleZijlstra, eric.araujo, eric.snow, ethan.furman, iritkatriel, jvr, mikecmcleod, misc, ncoghlan, nnorwitz, theller
Priority: normal Keywords: easy, patch

Created on 2005-09-08 09:28 by misc_from_metz, last changed 2022-04-11 14:56 by admin.

Files
File name Uploaded Description Edit
modulefinder.no_recurse.diff misc_from_metz,2005-09-08 09:28 Add a parameter to ModuleFinder, to not recurse when searching module.
modulefinder-no-recurse.diff eric.araujo,2011-07-29 13:53 review
1.html mikecmcleod,2021-12-01 16:31 bpo-1284670
2.html mikecmcleod,2021-12-01 16:33 fix non_recursive_import_test_1 and set debug to 0
Pull Requests
URL Status Linked Edit
PR 30228 closed mikecmcleod,2021-12-22 11:25
Messages (16)
msg48725 - (view) Author: Scherer Michael (misc_from_metz) Date: 2005-09-08 09:28
this patch add a parameter to ModuleFinder, to not recurse when searching module. This way, someone can compute the modules needed by a specific file, without having to compute all other imported module, which may take time for result that are no t needed. With the current system, you cannot create a clean graph of modules dependancy, since they are all on the same level. This patch do not change the default behavior.
msg48726 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2005-10-01 03:39
Logged In: YES user_id=33168 Thomas are you the defacto maintainer of modulefinder? Any comments?
msg48727 - (view) Author: Thomas Heller (theller) * (Python committer) Date: 2007-06-08 19:48
I have no time or interest in this patch, so unassigning.
msg114551 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-21 19:19
Anyone interested in this or can it be closed?
msg114553 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-21 19:33
Adding jvr and people interested in import machinery, per Misc/maintainers.rst Michael, can you refresh your patch against the py3k branch? Please also generate a unified diff from the top level of the source distribution, as detailed in http://www.python.org/dev/patches/ The feature seems useful to me and the patch is straightforward. Tests and doc updates are needed; do you want to add them? Mark, I think that feature requests are never closed just because they’re old; they have to be rejected, implemented or obsoleted. (Thank you for all your triage work.)
msg114645 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2010-08-22 08:42
The generalist in me is inclined to suggest a "depth" parameter (with depth=1 equivalent to direct dependencies only, and depth = None meaning all dependencies), but I must admit I don't have a concrete use case for the extra generality. So the simpler, recurse/don't recurse approach is probably a better option (building a depth-limited search on top of the recursion flag wouldn't be difficult anyway). Aside from missing docs and unit test updates, the idea seems sound.
msg114646 - (view) Author: Alyssa Coghlan (ncoghlan) * (Python committer) Date: 2010-08-22 08:45
Although I do find it a little concerning that there is no mention of sys.path_hooks or sys.meta_path in the modulefinder source code. I suspect this module only works correctly with vanilla filesystem based imports and can't handle anything imported via PEP 302.
msg124928 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-30 18:37
The depth parameter idea sounds like YAGNI, so let’s stay with a recurse boolean :)
msg140602 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-18 16:23
I applied the patch, added a test and found a bug. Here’s my progress so far; someone can start from it to write more tests and fix the code.
msg141381 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-29 13:53
I modernized modulefinder a bit in 1521d9837d16; here’s a refreshed patch.
msg406579 - (view) Author: mike mcleod (mikecmcleod) * Date: 2021-11-19 12:17
I would like to help with this issue. I note the history (using Git Dag) of the modulefinder.py shows that it has been modified with changes related here: Author: Éric Araujo <merwok@netwok.org> 2011-07-28 22:35:29 Committer: Éric Araujo <merwok@netwok.org> 2011-07-28 22:35:29 Parent: 867754e3e3c907f87d1bb0282df628deb392cd62 (merge 11164) Child: cee6bb549946bb096aa380a49f3b11daf2d8713e (Fix bug I unwittingly added in 1521d9837d16 (found by Ezio Melotti)) Branches: main, remotes/origin/3.10, remotes/origin/3.6, remotes/origin/3.7, remotes/origin/3.8, remotes/origin/3.9, remotes/origin/bpo-45514/deprecate-importlib-resources-legacy, remotes/origin/buildbot-custom, remotes/origin/main, remotes/upstream/3.10, remotes/upstream/3.6, remotes/upstream/3.7, remotes/upstream/3.8, remotes/upstream/3.9, remotes/upstream/bpo-45514/deprecate-importlib-resources-legacy, remotes/upstream/buildbot-custom, remotes/upstream/main, remotes/upstream/windeepfreeze Follows: v3.2.1 Precedes: v3.3.0a1 Modernize modulefinder module and tests a bit. The tests don’t use an internal distutils function anymore, and use regular assertEqual with sorted lists instead of a convoluted manual diff. and Author: Éric Araujo <merwok@netwok.org> 2011-08-01 14:29:07 Committer: Éric Araujo <merwok@netwok.org> 2011-08-01 14:29:07 Parent: 1e3a68d36b08cd9d59084a37c8cb6c2d911868ce (Modernize modulefinder module and tests a bit.) Child: cb66eb0deca1d5cd232f97c76a215ecaab958d30 (Issue #13959: Deprecate imp.get_suffixes() for new attributes on) Branches: main, remotes/origin/3.10, remotes/origin/3.6, remotes/origin/3.7, remotes/origin/3.8, remotes/origin/3.9, remotes/origin/bpo-45514/deprecate-importlib-resources-legacy, remotes/origin/buildbot-custom, remotes/origin/main, remotes/upstream/3.10, remotes/upstream/3.6, remotes/upstream/3.7, remotes/upstream/3.8, remotes/upstream/3.9, remotes/upstream/bpo-45514/deprecate-importlib-resources-legacy, remotes/upstream/buildbot-custom, remotes/upstream/main, remotes/upstream/windeepfreeze Follows: v3.2.1 Precedes: v3.3.0a1 Fix bug I unwittingly added in 1521d9837d16 (found by Ezio Melotti) And the modulefinder.py file has been subsequently modified by another 28 changes. So I am wondering if this issue is still relevant?
msg406981 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-11-25 09:04
Mike, from looking at the code the change proposed here is not there, so while the patch may not apply cleanly anymore, the commits you mention do not make this issue irrelevant.
msg407139 - (view) Author: mike mcleod (mikecmcleod) * Date: 2021-11-27 11:10
Ok, I will work on this soon and make further comments.
msg407470 - (view) Author: mike mcleod (mikecmcleod) * Date: 2021-12-01 16:31
I have made the changes as indicated in the diff files. I have tested against the latest from my GitHub copy. The result is: ./python -m test -j0 == CPython 3.11.0a2+ (heads/main:3a91617590, Dec 1 2021, 15:11:41) [GCC 11.2.0] == Linux-5.13.0-22-generic-x86_64-with-glibc2.34 little-endian == cwd: /home/dougal/Documents/cpython/build/test_python_300237æ == CPU count: 8 == encodings: locale=UTF-8, FS=utf-8 ... 0:01:34 load avg: 8.77 [221/430/1] test_modulefinder passed == Tests result: FAILURE == 409 tests OK. 1 test failed: test_embed 20 tests skipped: test_curses test_devpoll test_epoll test_ioctl test_kqueue test_msilib test_ossaudiodev test_smtpnet test_socketserver test_startfile test_tix test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winconsoleio test_winreg test_winsound test_xmlrpc_net test_zipfile64 Total duration: 5 min 30 sec Tests result: FAILURE The first diff is attached. But there was a bug in the code and I will upload in next comment.
msg407471 - (view) Author: mike mcleod (mikecmcleod) * Date: 2021-12-01 16:33
Minor fix after test.
msg411038 - (view) Author: Jelle Zijlstra (JelleZijlstra) * (Python committer) Date: 2022-01-20 17:22
Mike's fix unfortunately didn't work out. What are the rules about closing old enhancement requests? modulefinder is an old and rarely used package, and I feel like the use case is better served by a PyPI package.
History
Date User Action Args
2022-04-11 14:56:12 admin set github: 42354
2022-01-20 17:22:41 JelleZijlstra set nosy: + JelleZijlstramessages: +
2021-12-22 11:25:43 mikecmcleod set pull_requests: + <pull%5Frequest28450>
2021-12-01 16:33:24 mikecmcleod set files: + 2.htmlmessages: +
2021-12-01 16:31:56 mikecmcleod set files: + 1.htmlmessages: +
2021-11-27 11:10:48 mikecmcleod set messages: +
2021-11-25 09:05:00 iritkatriel set nosy: + iritkatrielmessages: +
2021-11-19 17:16:06 ethan.furman set nosy: + ethan.furman
2021-11-19 12:17:36 mikecmcleod set nosy: + mikecmcleodmessages: +
2013-01-25 19🔞16 brett.cannon set nosy: - brett.cannon
2011-07-29 13:53:58 eric.araujo set files: + modulefinder-no-recurse.diffmessages: +
2011-07-29 13:52:42 eric.araujo set files: - modulefinder-no-recurse.diff
2011-07-18 16:32:20 eric.snow set nosy: + eric.snow
2011-07-18 16:23:56 eric.araujo set files: + modulefinder-no-recurse.diffnosy: + miscmessages: +
2010-12-30 18:37:02 eric.araujo set versions: + Python 3.3, - Python 3.2nosy: - misc_from_metz, BreamoreBoymessages: + assignee: eric.araujostage: test needed -> patch review
2010-08-22 08:45:21 ncoghlan set messages: +
2010-08-22 08:42:18 ncoghlan set messages: +
2010-08-21 19:33:56 eric.araujo set nosy: + eric.araujo, jvr, brett.cannon, ncoghlanmessages: +
2010-08-21 19:19:00 BreamoreBoy set nosy: + BreamoreBoymessages: + versions: + Python 3.2, - Python 2.7
2009-04-22 05:08:10 ajaksu2 set keywords: + easy
2009-03-31 01:19:22 ajaksu2 set stage: test neededtype: enhancementversions: + Python 2.7
2005-09-08 09:28:31 misc_from_metz create