| msg41371 - (view) |
Author: Brett Cannon (brett.cannon) *  |
Date: 2002-10-13 07:07 |
| As mentioned in Zack Weinberg's Python-dev thread, "A few lessons from the tempfile.py rewrite", it was brought up that not having thread in all distributions is a pain because you have to write around instances of it not existing. Guido suggested the compromise of having a dummy_thread.py so that one could do: try: import thread as _thread except ImportError: import dummy_thread as _thread This is specific comment can be found at http://mail.python.org/pipermail/python-dev/2002-August/027985.html . Well, I decided to implement dummy_thread.py . I have tested it against test_thread.py and test_threading.py by editing the files locally to import dummy_thread as thread. All tests did pass. The only thing that is not necessarily straight-forward in the implementation is that for non-blocking lock.acquire() calls, the code actually checks to see if the lock is available; blocking calls just automatically acquire the lock since blocking when there is not real threading would not big good. =) It was also to appease the assert demons in threading.py for Condition objects since Condition._is_owned() did not like having lock.acquire() always succeed. As for adding regression tests, one could just run test_thread.py specifically against dummy_thread.py . The tougher test, though, was getting test_threading.py to work with threading.py importing dummy_thread. I am not, as of this exact moment, uploading any patches to add regression tests since I wrote this on a whim and I am leaving for vacation on 2002-10-14. I will be back by November 1, though, and I can try to come up with how to integrate regression tests then if no one has beaten me to it. |
|
|
| msg41372 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2002-11-28 03:07 |
| Logged In: YES user_id=357491 So all patches and files needed for dummy_thread to be accepted into the stdlib are now uploaded. The various parts of this patch (which all don't need to be accepted which is why I did it the way I have) are as follows: The module: dummy_thread.py is the module. test_dummy_thread.py is the testing suite. New test files: Since I had to basically simulate threads, I wrote the tests as if I was going to test the thread module. So I wrote test_dummy_thread.py in such a way that ``test_main()`` can accept a module to use as the thread implementation. The included test_thread.py uses test_dummy_thread.py to test the thread module. It is more in-depth then the current implementation. If my version of test_thread.py is used, then Lib/test/output/test_thread will need to be removed (my testing suite uses PyUnit). As an aside, dummy_thread.py does pass the current test_thread.py testing suite. Changed modules: added_dummy.diff is a diff that changes Queue, threading, and tempfile to use dummy_thread when thread is not available. They all still pass their tests when using dummy_thread (dummy_thread has one design decision just to pass threading's ``assert`` statements; all info is in the doc strings). pydoc can be changed so as to not use threading if the thread module is not available, but there are no tests for the module, so I will let the person who checks this in make the call whether it should be added. I tried it and everything seemed to work fine so I personally think it is safe for pydoc to stay as it is. Documentation: libdummythread.tex is my attempt at writing the docs for dummy_thread. Since it basically mirrors thread, I state to use thread's docs for API info and explain what dummy_thread is for. doc_dummy.tex adds notes to libthread.tex and libthreading.tex mentioning dummy_thread and in threading's case how to make sure that you only import threading if you want only true threading. Mentioning Queue's use of dummy_thread as backup could be useful, but since there is no lock-up problems I don't see the need. dummy_thread has only one issue with replacing thread in all cases: blocking I/O. Both test_asynchat and test_socket block on their threads because they spawn both server and client threads that block waiting for each other. But both modules import thread explicitly, so there is no worry with these modules running with dummy_thread as their thread implementation. I also couldn't run test_socketserver because I can't run test_normalization. But other then those three tests, all other tests pass both under a regular debug compile and a --without-thread compile; things just run a little slower. =) The 2002-10-13 copy of dummy_thread.py can be deleted (wish SF would let the patch creator delete files). This is my first big patch that touches docs, test, stdlib modules and has a new doc file. Just thought I should warn you. =) |
|
|
| msg41373 - (view) |
Author: Tim Peters (tim.peters) * |
Date: 2002-12-05 02:01 |
| Logged In: YES user_id=31435 Deleted all attachments, at Brett's request. |
|
|
| msg41374 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2002-12-08 03:37 |
| Logged In: YES user_id=357491 OK, so now both ``dummy_thread`` and ``dummy_threading`` are done. ``dummy_threading`` was requested by python-dev so as to not have ``threading`` be imported even if ``thread`` is not available. It actually uses ``threading`` itself and through some trickery (read below) just subs out the ``thread`` code and have it use ``dummy_thread``. Here is the file breakdown: dummy_thread.py : Implementation of dummy_thread (I tried to not go overboard with the name of the file =) dummy_threading : I think you know what this is. =) The trick to making this work was a lot of ``sys.modules`` manipulation. It adds an extra module named ``_dummy_threading``; I figured naming it this would be okay since other modules like ``re`` have other modules with a similar naming scheme. added_dummy.diff : Patches Lib/Queue.py and Lib/tempfile.py to use ``dummy_thread``. As I have mentioned in another post, ``pydoc`` probably can be safely patched as well, but since there are no tests I don't want to risk it. doc_dummy.diff : Patches Doc/lib/libthread.text and Doc/lib/libthreading.tex to mention their corresponding dummy twins. libdummythread.tex : Documentation for ``dummy_thread``. libdummythreading.tex : Documentation fo ``dummy_threading``. test_dummy_thread.py : I think it is obvious from the name. test_dummy_threading.py : Ditto for this file as well. It is basically just ``test_threading`` with a ``test_main()`` function instead of being run on import. I think a Lib/test/output/ file needs to be created (should be able to just copy ``test_threading``'s file). test_thread.py : I wrote the tests for ``dummy_thread`` as if I was writing tests for ``thread``. Since I went to that trouble I made sure that ``test_dummy_thread`` could be run while using ``thread`` instead of ``dummy_thread``. This testing suite is much more in-depth than the one currently in the stdlib. I wasn't able to pull this off for ``test_threading``. I think the docs and ``dummy_threading`` are the biggest things to look at and check. ``dummy_threading`` because of the amount of ``sys.modules`` fiddling it does (although I tested it rather thoroughly to make sure nothing got messed up after the file is imported). The docs just because they are my first attempt and they are rather thin (although that might be okay). ``dummy_thread`` is completely independent and can be added without ``dummy_threading``, so if there is any hold-ups with ``dummy_threading``, please at least get ``dummy_thread`` in. |
|
|
| msg41375 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2002-12-08 08:48 |
| Logged In: YES user_id=21627 A couple of comments: - please leave the thread tests intact, and have the dummy tests make use of the thread tests. You should be able to use the same technique, just reverse. - please provide the next version of the patch as an archive file (.zip/.tgz), which unpacks all files into the right place, and includes a single patch to apply with patch -pX (ideally for X=0). - Feel free to use the \moduleauthor and \sectionauthor markups. - Consider providing threading.__all__ (and copying this into dummy_threading). Perhaps python-dev needs to comment on whether this is desirable. - Clean up holding_* - What is the purpose of _dummy_threading? Can't you just do "from threading import *"? - Why is it necessary to override _start_new_thread? Doesn't the "import thread" in threading.py already pick up dummy_threading? |
|
|
| msg41376 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2002-12-08 21:31 |
| Logged In: YES user_id=357491 In response to Martin's comments: - The problem with doing it in reverse is that would require ``thread`` to be available for the initial import and that is not guaranteed like ``dummy_thread`` would be. I could do some more ``sys.modules`` fiddling but I would like to keep that to a minimum since it is tricky and error-prone. If it is not desired to replace the ``thread`` testing suite then I propose leave it be but still use my new tests for ``dummy_thread``. - OK. Do realize, though, that I did it this way so as to allow ``dummy_thread`` to not be held up by ``dummy_threading``. Would it be okay if I did two patch files, one for ``dummy_thread`` and one for ``dummy_threading``? Considering that all of your comments are against ``dummy_threading`` (short of the testing suite issue) I know ``dummy_threading`` is going to be what gets the most comments and revisions and if this gets dragged out I would like to at least get ``dummy_thread`` in. - I assume those are for the docs. Where should I put them? - Actually ``threading`` is very careful to not pollute the namespace. I will have a look, though, and see if it is called for. I will also make sure all of my temp variables get deleted so as to not pollute ``dummy_threading``'s namespace. - Clean up how? The only offensive thing I see are the # comments but since I am playing around with stuff that normally shouldn't be I felt ample comments was justified. If you mean make sure to delete the variables I am going to add that. - The original thinking was that I needed a copy of the code to be referenced from; I wasn't thinking that doing the ``import *`` would make references for ``threading``'s code. I just quickly tried commenting out all of that code and did what you suggested and I get an error about NoneType not being callable. I will look at it some more, but I wouldn't hold my breath on it working without it. - You're right, it isn't needed. Once again I was not thinking in terms of references to modules and how the code would be pointing to the actual module code and not what was in ``sys.modules['thread']``. I will wait about a day or two before I post the next revision to get more comments. Thanks for the help, Martin. |
|
|
| msg41377 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2002-12-08 23:07 |
| Logged In: YES user_id=357491 This is a follow-up to Martin's question of why I need to have ``_dummy_threading``. I tried ``from threading import *`` and ran into the issue of getting an error of trying to use NoneType as callable. The problem was that I was deleting ``threading`` from ``sys.modules`` so as to allow future imports to use ``thread`` and not use the already imported ``dummy_thread`` version of ``threading`` that I set up. But with ``threading`` deleted the module code was being returned as ``None`` objects. And even if I leave it in and don't delete the module from ``threading`` I still have issues because subsequent ``threading`` import calls mess with ``threading`` and I still get the error; I think it reloads the module and that is not good in this case. So I don't see how I can get rid of it. I don't think it is a big issue considering you will only know about it if you probe ``sys.modules`` and if you are doing that you had better know what you are doing. Besides, ``re`` imports a total of 3 modules named ``re``, ``sre``, and ``_sre``, so I don't think having this module add one more is going to be that big of a deal. =) |
|
|
| msg41378 - (view) |
Author: Brett Cannon (brett.cannon) * |
Date: 2002-12-11 02:52 |
| Logged In: YES user_id=357491 Files dummy_thread.zip and dummy_threading.zip (use the "2002-12-10b" version of dummy_threading.zip!) have been uploaded. I followed Martin's advise and created zip files (using ``zipfile.ZipFile``, uncompressed) to be unzipped from the ``python/dist/src`` directory of your copy of CVS. Everything should unzip to their proper places from there (used relative directories from ``src``). The diffs are ``cvs diff -u``, so that ``-p`` stuff Martin was mentioning should not be an issue. The diffs are unpacked into ``src`` and are obviously named. Hopefully I didn't botch anything. =) Compared to the old files I cleaned up ``dummy_threading.py`` by removing the redundant assigning I was doing and I made sure I cleaned up the module's namespace. I also added ``__all__`` to ``Lib/threading.py`` (initially forgot I had done that which is why there is a "2002-12-10b" version of dummy_threading.zip). Can someone please delete all files *except* dummy_thread.zip ("2002-12-10"), dummy_threading.zip ("2002-12-10b"), and test_thread.py? The reason I want test_thread.py left is because I think that the current test_thread.py is not very thorough, and thus I am disagreeing with Martin by wanting to replace it. And I still think that ``Lib/pydoc.py`` can be patched to use ``dummy_thread``, but without a testing suite for that module I didn't want to risk patching it myself. |
|
|
| msg41379 - (view) |
Author: Guido van Rossum (gvanrossum) * |
Date: 2002-12-30 22:30 |
| Logged In: YES user_id=6380 Thanks! Checked in now. |
|
|