| msg279439 - (view) |
Author: Mariatta (Mariatta) *  |
Date: 2016-10-25 18:32 |
| Deprecation warning was added to asyncore in https://bugs.python.org/issue25002 asyncore is still used in several tests and should be replaced. |
|
|
| msg291907 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-04-19 21:40 |
| > asyncore is still used in several tests and should be replaced. Here is the list of Python files using asyncore: haypo@selma$ grep -l asyncore $(find -name "*.py") ./Lib/asyncore.py ./Lib/asynchat.py ./Lib/test/test_smtpd.py ./Lib/test/test_asyncore.py ./Lib/test/test_smtplib.py ./Lib/test/test_os.py ./Lib/test/test_ssl.py ./Lib/test/test_logging.py ./Lib/test/libregrtest/save_env.py ./Lib/test/test_asynchat.py ./Lib/test/test_pyclbr.py ./Lib/test/test_support.py ./Lib/test/test_poplib.py ./Lib/test/test_ftplib.py ./Lib/smtpd.py smtpd has been rewritten with asyncio: http://aiosmtpd.readthedocs.io/ The documentation of the smtpd module already points to aiosmtpd. libregrtest/save_env.py only checks that asyncore.socket_map is not modified, we can keep it until asyncore is really removed. test_support.py only tests that it's possible to have a "clean" import of asyncore, it can be easily be replaced later. The list can reduced to: ./Lib/test/test_smtplib.py ./Lib/test/test_os.py ./Lib/test/test_ssl.py ./Lib/test/test_logging.py ./Lib/test/test_pyclbr.py ./Lib/test/test_poplib.py ./Lib/test/test_ftplib.py |
|
|
| msg291908 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-04-19 21:41 |
| asyncore was modified to emit a DeprecationWarning: http://bugs.python.org/issue25002#msg279417 But then a lot of code starts with to fail -Werror. Copy of Martin Panter's : I normally run the tests with -Werror, and the failures I get are: * test_ssl, test_smtplib, test_poplib, test_logging, test_ftplib, test_support: tests use asyncore * test_os: test uses asynchat (When you import asynchat, the first error complains about importing asyncore, there are actually two warnings) * test_all: This seems to ignore DeprecationWarning; perhaps an exception for PendingDeprecationWarning should also be added? * test_asyncore and test_asynchat: Obviously these have to still test the modules, so they should anticipate the warnings, perhaps using Serhiy’s code * test_smtpd: smtpd module itself uses asyncore; see Issue 25008 |
|
|
| msg294710 - (view) |
Author: Grzegorz Grzywacz (grzgrzgrz3) * |
Date: 2017-05-29 19:46 |
| I would like to work on this issue. I think it's a good idea to split this task into few parts/PR. Let me start from ./Lib/test/test_poplib.py. What about rewriting pop3 server stub using asyncio, i think requests could be handled synchronously, there will be no benefits from asynchronous. Please let me know what you think. |
|
|
| msg294712 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2017-05-29 20:34 |
| > I think it's a good idea to split this task into few parts/PR. Let me start from ./Lib/test/test_poplib.py. Once you have a patch, open a new issue and mention it here. |
|
|
| msg294771 - (view) |
Author: Grzegorz Grzywacz (grzgrzgrz3) * |
Date: 2017-05-30 17:25 |
| ./Lib/test/test_poplib.py sub-issue |
|
|
| msg294773 - (view) |
Author: Grzegorz Grzywacz (grzgrzgrz3) * |
Date: 2017-05-30 17:29 |
| ./Lib/test/test_poplib.py sub-issue Fixed issue number from previous comment |
|
|
| msg356355 - (view) |
Author: (dankreso) * |
Date: 2019-11-11 09:27 |
| Hi, is this still open? I'm happy to work on replacing asyncore usage in one of the other test files. |
|
|
| msg356364 - (view) |
Author: Joannah Nanjekye (nanjekyejoannah) * |
Date: 2019-11-11 14:38 |
| @dankreso, the issue is still open and no one has claimed it yet. So feel free to open a pull request. |
|
|
| msg356369 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2019-11-11 16:54 |
| > I'm happy to work on replacing asyncore usage in one of the other test files. Sounds good, just let us know which one(s) you're working on. (: |
|
|
| msg357088 - (view) |
Author: Jackson Riley (jacksonriley) * |
Date: 2019-11-20 17:28 |
| Lib/test/test_pyclbr.py - subissue A trivial one! |
|
|
| msg379342 - (view) |
Author: Kyle Stanley (aeros) * |
Date: 2020-10-22 20:39 |
| Since this issue is now a significant blocker for PEP 594 (remove stdlib dead batteries, which includes asyncore and asynchat), I'm going to prioritize working on this and assign it to myself. |
|
|
| msg396295 - (view) |
Author: Irit Katriel (iritkatriel) * |
Date: 2021-06-21 22:46 |
| If the tests are the only thing blocking removal, does it make sense to move asyncore and asynchat to test.support and get on with removing them from the stdlib? |
|
|
| msg396297 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-06-21 23:02 |
| Irit: > If the tests are the only thing blocking removal, does it make sense to move asyncore and asynchat to test.support and get on with removing them from the stdlib? Yes, it makes sense and it can be done right now, since asynchat/asyncore is deprecated since Python 3.6. |
|
|
| msg396319 - (view) |
Author: Irit Katriel (iritkatriel) * |
Date: 2021-06-22 10:45 |
| There is actually one usage outside of the tests: Lib/smtpd.py There is a note in the doc saying " The aiosmtpd package is a recommended replacement for this module. It is based on asyncio and provides a more straightforward API. smtpd should be considered deprecated." https://docs.python.org/3/library/smtpd.html |
|
|
| msg396323 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-06-22 11:32 |
| So the first step is to decide if it's ok to remove smtpd right now. |
|
|
| msg404575 - (view) |
Author: Irit Katriel (iritkatriel) * |
Date: 2021-10-21 09:31 |
| asyncore, asynchat and smtpd are all deprecated now, for removal in 3.12. While the tests are not blocking their removal (we can move them to test.support) it would still be better to rewrite these tests. |
|
|
| msg406159 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-11 12:12 |
| I forgot about this issue and I created a duplicate: bpo-45785. Copy of my messages. I propose to remove asyncore, asynchat and smtpd modules from the Python standard library to remove the Python maintenance burden. These modules are deprecated since Python 3.6. -- The asyncore module is a very old module of the Python stdlib for asynchronous programming, usually to handle network sockets concurrently. It's a common event loop, but its design has many flaws. The asyncio module was added to Python 3.4 with a well designed architecture. Twisted developers who have like 10 to 20 years of experience in asynchronous programming helped to design the asyncio API. By design, asyncio doesn't have flaws which would be really hard to fix in asyncore and asynchat. It was decided to start deprecating the asyncore and asynchat module in Python 3.6 released in 2016, 5 years ago. Open issues in asyncore and asynchat have been closed as "wont fix" because the module is deprecated. The two modules are basically no longer maintained. Documentation: * https://docs.python.org/dev/library/asyncore.html * https://docs.python.org/dev/library/asynchat.html I propose to remove the two modules right now in the Python stdlib. They were removed for 4 Python releases (3.6-3.10), it's long enough to respect the PEP 387. The PEP requires 2 Python releases at least before considering removing code. Since there are still 7 tests of the Python test suite still uses asyncore and asynchat, I propose to move these modules in Lib/test/ directory and make them private: * Rename Lib/asyncore.py to Lib/test/support/_asyncore.py * Rename Lib/asynchat.py to Lib/test/support/_asynchat.py Projects using asyncore and asynchat should use asyncio. If someone really wants to continue using asyncore and asynchat, it's trivial to copy asyncore.py and asynchat.py in their project, and maintain these files on their side. -- About the smtpd module, it is also deprecated since Python 3.6 (deprecated for 4 Python releases). It's used by a single test (test_logging). I also propose to remove it from the stdlib. I proposed to rename Lib/smtpd.py to Lib/test/support/_smtpd.py. Projects using smtpd can consider using aiosmtpd which is based on asyncio: https://aiosmtpd.readthedocs.io/ Or again, they can copy Python 3.10 smtpd.py in their project and maintain this file on their side. -- Ah, a DeprecationWarning warning is only emitted at runtime since Python 3.10. What's New in Python 3.10: "asynchat, asyncore, smtpd: These modules have been marked as deprecated in their module documentation since Python 3.6. An import-time DeprecationWarning has now been added to all three of these modules." https://docs.python.org/dev/whatsnew/3.10.html#asynchat-asyncore-smtpd |
|
|
| msg406160 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-11 12:17 |
| I wrote PR 29521 to remove the asyncore, asynchat and smtpd modules. > libregrtest/save_env.py only checks that asyncore.socket_map is not modified, we can keep it until asyncore is really removed. I chose to remove it instead of my PR. > test_support.py only tests that it's possible to have a "clean" import of asyncore, it can be easily be replaced later. This code has been removed in the meanwhile. > But then a lot of code starts with to fail -Werror. While tests using asyncore, asynchat and smtpd have been modified to catch the DeprecationWarning warning, I chose to remove this warning in my PR, since I made the 3 modules private. I don't think that test.support private modules must emit deprecation warnings. > So the first step is to decide if it's ok to remove smtpd right now. IMO it's ok to remove it since it is deprecated since Python 3.6. > asyncore, asynchat and smtpd are all deprecated now, for removal in 3.12. Did you see an explicit mention of Python 3.12? I only saw mention of "Python 3.6" which is the version when the 3 modules were deprecated. |
|
|
| msg406161 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-11 12:19 |
| Since Christian Heimes decided to mark my issue bpo-45785 "Remove asyncore, asynchat and smtpd modules" as a duplicate of this issue, I changed this issue title to "Remove asyncore, asynchat and smtpd modules". |
|
|
| msg406162 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-11 12:31 |
| I announced my plan to remove the 3 modules on python-dev: https://mail.python.org/archives/list/python-dev@python.org/thread/LZOOLX5EKOITW55TW7JQYKLXJUPCAJB4/ |
|
|
| msg406170 - (view) |
Author: Senthil Kumaran (orsenthil) * |
Date: 2021-11-11 18:43 |
| +1 to these modules removal. One concern I have is when users follow internet examples and look out for these modules or examples. What is the best way to show them the modern usage? - Should Python docs show some example snippet of the most common usage of aiosmtpd ? - Echo server / client using asyncio. A stdlib page dedicated to removal, and showing examples using these modules, especially aiostmpd as it is not a part of stdlib, might be a good idea. |
|
|
| msg406173 - (view) |
Author: Andrew Svetlov (asvetlov) * |
Date: 2021-11-11 19:26 |
| > One concern I have is when users follow internet examples and look out for these modules or examples. There is an option: keep removed modules but replace each module content with 'raise ImportError("Please use instead")' stub. The actual module removal can be postponed for years until most internet resources reflect this fact. On Thu, Nov 11, 2021 at 8:43 PM Senthil Kumaran <report@bugs.python.org> wrote: > > Senthil Kumaran <senthil@python.org> added the comment: > > +1 to these modules removal. > > One concern I have is when users follow internet examples and look out for > these modules or examples. > > What is the best way to show them the modern usage? > > - Should Python docs show some example snippet of the most common usage of > aiosmtpd ? > - Echo server / client using asyncio. > > A stdlib page dedicated to removal, and showing examples using these > modules, especially aiostmpd as it is not a part of stdlib, might be a good > idea. > > ---------- > nosy: +orsenthil > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue28533> > _______________________________________ > |
|
|
| msg406208 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-12 13:21 |
| > There is an option: keep removed modules but replace each module content > with 'raise ImportError("Please use instead")' stub. > The actual module removal can be postponed for years until most internet resources reflect this fact. That prevents someone to provide asyncore for Python 3.11 under the same name, "asyncore". The stdlib module has the priority over the third-party module. |
|
|
| msg406210 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-12 13:23 |
| Senthil: "+1 to these modules removal" Would you mind to formally approve the PR https://github.com/python/cpython/pull/29521 ? |
|
|
| msg406213 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-12 13:29 |
| > One concern I have is when users follow internet examples and look out for these modules or examples. Well, we cannot modify printed books :-) > What is the best way to show them the modern usage? Buy new books? :-) > Should Python docs show some example snippet of the most common usage of aiosmtpd ? I don't think that we should start providing tutorial for 3rd party modules. There are enough places on the Internet for that. For example, what is aiosmtpd deprecated a function and our doc uses it? Once a Python version reaches its "security-only" fixes, we cannot modify the doc anymore, and you fall into the same problem again. > A stdlib page dedicated to removal, and showing examples using these modules, especially aiostmpd as it is not a part of stdlib, might be a good idea. IMO it's way better to maintain such documentation outside Python itself. It's simpler to keep it up to date. |
|
|
| msg406214 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-12 13:41 |
| For example, what *if* aiosmtpd deprecated a function |
|
|
| msg406215 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-12 14:05 |
| See also the (dfaft) PEP 594: "Removing dead batteries from the standard library" https://www.python.org/dev/peps/pep-0594/ |
|
|
| msg406372 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-15 23:29 |
| New changeset 9bf2cbc4c498812e14f20d86acb61c53928a5a57 by Victor Stinner in branch 'main': bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521) https://github.com/python/cpython/commit/9bf2cbc4c498812e14f20d86acb61c53928a5a57 |
|
|
| msg406377 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-15 23:50 |
| asyncore, asynchat, smtpd modules have been removed. I close this issue. > asyncore is still used in several tests and should be replaced This issue is not solved. My PR only copied asyncore as a private module in test.support (test.support._asyncore). If someone cares, please open a new issue. |
|
|
| msg406434 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-11-16 20:58 |
| About the PEP 387 process, Brett wrote on python-dev that "the modules got yanked prematurely", so I formally requested a SC exception: https://github.com/python/steering-council/issues/86 |
|
|
| msg407559 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-12-02 23:55 |
| According to a code search in the PyPI top 5000 projects: the source code of 21 projects contains "import asyncore", "import asynchat" or "import smtpd": * ansible-5.0.0 * cassandra-driver-3.25.0 * django-extensions-3.1.5 * eth_abi-2.1.1 * eth-account-0.5.6 * eth-hash-0.3.2 * eth-utils-2.0.0 * gevent-21.8.0 * h5py-3.6.0 * hexbytes-0.2.2 * jedi-0.18.1 * M2Crypto-0.38.0 * mercurial-6.0 * mypy-0.910 * plac-1.3.3 * pyftpdlib-1.5.6 * pyinotify-0.9.6 * pysnmp-4.4.12 * pytest-localserver-0.5.1 * pytype-2021.11.29 * tlslite-0.4.9 I ignored false positives like "from eventlet(...) import asyncore". |
|
|
| msg407560 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-12-03 00:00 |
| > ansible-5.0.0 Reported upstream (it's a single file, used for tests): https://github.com/ansible/ansible/issues/76452 > gevent-21.8.0 It's https://github.com/gevent/gevent/tree/master/src/greentest directory which contains subsets of the Python test suite. I guess that when they will copy the Python 3.11 tests, it will be fine. |
|
|
| msg407897 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-12-07 08:13 |
| I reopen the issue: the Steering Council asked to revert the change removing the 3 modules, since the DeprecationWarning was only emitted in a single Python release (Python 3.10) and so the PEP 387 process was not respected. I created the PR 29951 to add again the 3 modules to Python 3.11. |
|
|
| msg407900 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-12-07 08:20 |
| https://github.com/python/steering-council/issues/86 |
|
|
| msg407915 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2021-12-07 11:31 |
| New changeset cf7eaa4617295747ee5646c4e2b7e7a16d7c64ab by Victor Stinner in branch 'main': Revert "bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)" (GH-29951) https://github.com/python/cpython/commit/cf7eaa4617295747ee5646c4e2b7e7a16d7c64ab |
|
|
| msg413909 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2022-02-24 13:39 |
| Follow-up: PEP 594 schedules the removal of these 4 modules in Python 3.12, but the PEP remains a draft. |
|
|