Issue 2919: Merge profile/cProfile in 3.n+1 (original) (raw)

process

Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: alexandre.vassalotti Nosy List: BreamoreBoy, MrJean1, alexandre.vassalotti, asvetlov, barry, belopolsky, benjamin.peterson, brett.cannon, eric.araujo, eric.snow, flox, georg.brandl, hdiogenes
Priority: critical Keywords: patch

Created on 2008-05-19 20:02 by brett.cannon, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
profile_module2.tgz MrJean1,2009-06-02 18:40 merged profile/cProfile module, rev'd
pyprofile.py alexandre.vassalotti,2009-06-28 17:01 Profiler interface for lsprof.
lsprof.py alexandre.vassalotti,2009-07-07 00:37 Python version of the _lsprof module (updated).
Messages (22)
msg67073 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2008-05-19 20:02
cProfile should go away in 3.0 (as a public module) and just be what profile uses when available.
msg67896 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-06-10 04:56
I will try to fix issue. I cannot promise that I will get it done before the beta though.
msg68043 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-06-12 02:09
This didn't get done for the betas, but we're still going to release. Knocking it down to critical.
msg71312 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2008-08-18 02:59
Can you get this done before beta 3?
msg71392 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2008-08-18 23:26
Sorry, I can't. I would love to get this done, but I am just too busy this week (and next week doesn't look any better). Furthermore, the amount of work that the profile/cProfile merge will require is quite considerable. So, I don't expect someone else could get this done before the beta.
msg71393 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2008-08-18 23:27
Let's retarget 3.1 then.
msg88553 - (view) Author: Humberto Diógenes (hdiogenes) * Date: 2009-05-29 23:42
How is this going?
msg88613 - (view) Author: Jean Brouwers (MrJean1) Date: 2009-05-31 21:56
Attached is *an* attempt to combine the old profile/cProfile modules into a single one called profile. Only the C and Python source and test files are included, not any documentation files. More details are in the README file, copied below. Hopefully, this is useful. /Jean Brouwers These are the modified files to replace the old profile module with the fast cProfile one and rename the latter. A complete list and description follow. ./README ./profileNEW/setup.py ./profileNEW/Lib/profile.py ./profileNEW/Lib/test/profilee.py ./profileNEW/Lib/test/test_profile.py ./profileNEW/Modules/_profile.c ./profilePy3k/setup.py ./profilePy3k/Lib/profile.py ./profilePy3k/Lib/test/profilee.py ./profilePy3k/Lib/test/test_cprofile.py ./profilePy3k/Lib/test/test_profile.py ./profilePy3k/Modules/_lsprof.c ./diff_-u_profilePy3k_Modules_lsprof.c_profileNEW_Modules_profile.c.txt ./diff_-ur_profilePy3k_profileNEW.txt ./_profile_hires.c Directory profileNEW contains all the source and test files in their corresponding directories. Directory profilePy3k contains the old, original files from the py3k trunk taken on May 31, 2009. Basically, the old _lsprof.c file has been modified and renamed to _profile.c. Both old test_profile.py and test_cprofile.py files have been combined into a single new file test_profile.py. The file diff_-ur_profilePy3k_profileNEW.txt contains the forward diffences between all the new and orginal files. File diff_-u_profilePy3k_Modules_lsprof.c_profileNEW_Modules_profile.c.txt contains just the differences between the old _lsprof.c and new _profile.c files. Lastly, for your consideration, file _profile_hires.c is the new _profile.c file enhanced to use a high resolution timer, see <http://bugs.python.org/issue2281>. This file would replace the _profile.c file in the profileNEW/Modules directory. The new files and tests were installed in a Python 3.0.1 build and compiled and tested on MacOS X 10.4.11 Tiger (Intel). The patches from issue 5330 were also installed in this build, see <http://bugs.python.org/issue5330>. All tests passed after regenerating the expected results % python.exe Lib/test/test_profile.py -r Regenerating Lib/test/test_profile.py... % python.exe Lib/test/test_profile.py test_bad_counter_during_dealloc (__main__.ProfileTest) ... ok test_calling_conventions (__main__.ProfileTest) ... ok test_profile (__main__.ProfileTest) ... ok ---------------------------------------------------------------------- Ran 3 tests in 0.017s OK
msg88614 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2009-05-31 22:05
Thanks for the code, Jean. With Python 3.1 about to go out the door this will have to be a 3.2 thing. But I plan to start looking at this module merge some time in July so I should get to looking at what you did then (unless someone beats me to it).
msg88624 - (view) Author: Jean Brouwers (MrJean1) Date: 2009-05-31 23:50
I just added another comment about the high-resolution timer in <http://bugs.python.org/issue2281>.
msg88762 - (view) Author: Jean Brouwers (MrJean1) Date: 2009-06-02 18:40
Fixed a couple of typos in the README and _profile_hires.c files of the attached profile_module2.tgz tar ball.
msg89171 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-06-09 22:08
Jean Brouwers wrote: > All tests passed after regenerating the expected results The tests for profile are output test. If you regenerate them, they pass for sure. Merging cProfile/profile will require a lot more work than just renaming the module. The designs of cProfile and profile are significantly different. So a successful merge will require settling one profiler design (cProfile is the better designed one, IMHO) and rewrite the other one to match the chosen design.
msg89778 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-06-28 16:58
Yesterday, I ported the _lsprof module to Python as an experiment. I still have a few issues to iron out, but the core of the functionality is there. I am not sure yet how this experiment will fit in the profile/cProfile merge. I wrote the port mainly to get a better idea of what is needed for the merge and how the profiler works.
msg90205 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2009-07-07 00:37
Here's an updated version of my lsprof.py module. I fixed a few bugs, includes the awful reference leak that was present in the previous version (i.e., the profiler code was keeping a reference to every frame executed). I consider the lsprof.py code functionally equivalent to _lsprof. Now, I am wondering if I should add the features from profile.py missing in _lsprof, e.g., the bias argument and the calibration code.
msg97612 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2010-01-11 22:50
Any comment about the code I posted?
msg97647 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2010-01-12 18:35
I'll see if I can have a look sometime soon, but my profile experience is practically non-existent so I am not the best person for a code review.
msg110190 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-07-13 13:36
Is there anyone who could review this, it seems a shame that someone's work has been sitting in limbo for over a year?
msg110216 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-13 18:54
Is this a case of pure python implementation with a C speedup? Can you upload the patch to Rietveld? I'll take a look once it is there.
msg110217 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2010-07-13 18:55
Alexandre has commit privs so he could just go ahead and commit if he wants. I do hope to look at this code at some point in my life. I will see if I can't find time at EuroPython next week, especially if Alexander has fixes to it and still feels the need for a review.
msg165958 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-07-20 17:52
So I didn't find time at EuroPython. =) Is this dead, Alexandre, or do you want to see this happen for Python 3.4?
msg199897 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-10-14 14:28
Last desperate ping for 3.4?
msg199974 - (view) Author: Alexandre Vassalotti (alexandre.vassalotti) * (Python committer) Date: 2013-10-15 04:39
Let's close this. There is no point keeping this languishing any further. The profile and cProfile modules are different enough that we can't merge them without breaking compatibility. They should treated as two separate profilers. Maybe in the future we should include a drop-in module written in Python for cProfile to be used for another Python implementation. But for now, I don't see the benefit, since the profile module is good enough and alternate implementations should be able to re-do the _lsprof accelerator in their target language without too much trouble if desired.
History
Date User Action Args
2022-04-11 14:56:34 admin set github: 47168
2013-10-15 04:39:36 alexandre.vassalotti set status: open -> closedassignee: alexandre.vassalottiresolution: wont fixmessages: +
2013-10-14 14:28:35 georg.brandl set nosy: + georg.brandlmessages: +
2012-11-17 15:05:49 brett.cannon unlink issue2775 dependencies
2012-07-21 10:10:34 flox set nosy: + flox
2012-07-20 17:52:13 brett.cannon set messages: +
2012-06-24 20:11:53 pitrou set title: Merge profile/cProfile in 3.0 -> Merge profile/cProfile in 3.n+1
2011-12-09 19:32:16 eric.snow set nosy: + eric.snow
2010-10-14 23:12:31 eric.araujo set nosy: + eric.araujo
2010-07-13 18:55:41 brett.cannon set messages: +
2010-07-13 18:54:10 belopolsky set nosy: + belopolskymessages: +
2010-07-13 13:36:47 BreamoreBoy set nosy: + BreamoreBoymessages: +
2010-03-23 05:57:51 asvetlov set nosy: + asvetlov
2010-01-12 18:35:48 brett.cannon set messages: +
2010-01-11 22:50:47 alexandre.vassalotti set messages: +
2009-07-07 00:37:16 alexandre.vassalotti set files: - lsprof.py
2009-07-07 00:37:06 alexandre.vassalotti set files: + lsprof.pymessages: +
2009-06-28 17:01:52 alexandre.vassalotti set files: + pyprofile.py
2009-06-28 16:58:55 alexandre.vassalotti set files: + lsprof.pymessages: +
2009-06-09 22:08:10 alexandre.vassalotti set messages: +
2009-06-02 18:40:27 MrJean1 set files: - profile_module.tgz
2009-06-02 18:40:15 MrJean1 set files: + profile_module2.tgzmessages: +
2009-05-31 23:50:44 MrJean1 set messages: +
2009-05-31 22:13:15 rhettinger set versions: + Python 3.2, - Python 3.1
2009-05-31 22:05:15 brett.cannon set messages: + stage: patch review
2009-05-31 21:56:31 MrJean1 set files: + profile_module.tgznosy: + MrJean1messages: +
2009-05-29 23:42:42 hdiogenes set nosy: + hdiogenesmessages: +
2008-08-18 23:27:40 benjamin.peterson set priority: release blocker -> criticalnosy: + benjamin.petersonmessages: + versions: + Python 3.1, - Python 3.0
2008-08-18 23:26:07 alexandre.vassalotti set keywords: + patchmessages: +
2008-08-18 02:59:48 barry set messages: +
2008-07-31 02:16:40 benjamin.peterson set priority: critical -> release blocker
2008-06-12 02:09:22 barry set priority: release blocker -> criticalnosy: + barrymessages: +
2008-06-10 04:56:46 alexandre.vassalotti set nosy: + alexandre.vassalottimessages: +
2008-05-19 20:02:29 brett.cannon link issue2775 dependencies
2008-05-19 20:02:13 brett.cannon create