Issue 38605: [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.11 (original) (raw)
Messages (40)
Author: STINNER Victor (vstinner) * 
Date: 2019-10-27 16:07
The PEP 563: Postponed evaluation of annotations was introduced an opt-in feature using "from future import annotations". It is scheduled to become the default in Python 4.0.
I would prefer to limit the number of incompatible changes in Python 4.0: it should just a "regular" release, with a regular number of incompatible changes. The version number change is going to cause enough troubles...
Would it be possible possible to enable postponed evaluation of annotations either before or after Python 4.0? For example, can we imagine to enable it by default in Python 3.9? If not, what about Python 3.10?
See also the PEP 608 (Coordinated Python release) and bpo-38604 (Schedule Py_UNICODE API removal).
Author: Ivan Levkivskyi (levkivskyi) *
Date: 2019-10-27 19:17
IMO 3.10 would be better, since 3.9 would be too soon (it would be like a schedule for a normal deprecation).
Also if we are really doing this, I think it is better to announce this soon.
Also we should try to fix relevant issues related to string annotations (in typing and dataclasses), like https://github.com/python/typing/issues/508, https://github.com/python/typing/issues/574, https://bugs.python.org/issue37838, https://bugs.python.org/issue34776 and https://bugs.python.org/issue37948.
Author: Guido van Rossum (gvanrossum) *
Date: 2019-10-27 23:23
We never should have mentioned 4.0 as the target date to make this the default (and only) behavior -- who knows whether there will ever even be a Python 4.0? Even 3.10 might be on the early side (assuming we'll switch to a year-long release cycle per PEP 602 -- we will then make deprecations in general take two release cycles).
I do agree that we should start the process of deprecating the non-future behavior here in 3.9. I know of one project with a private fork of Python (for other reasons) that has made this default.
Author: STINNER Victor (vstinner) *
Date: 2019-10-28 11:03
We never should have mentioned 4.0 as the target date to make this the default (and only) behavior
I am fine with modifying future documentation to only modify the "Mandatory" column to remove Python 4.0, and then close this issue: https://docs.python.org/dev/library/future.html
--
But I like the "PEP 563: Postponed evaluation of annotations", IMHO it would be nice to get it as the default behavior :-) It's just a matter of properly organize the transition ;-)
Author: Guido van Rossum (gvanrossum) *
Date: 2019-10-28 14:43
You can bring the deprecation schedule up on discourse or python-dev so more folks can let us know whether they'd be okay with 3.9 or 3.10.
Author: STINNER Victor (vstinner) *
Date: 2020-01-28 02:19
I started a thread on typing-sig: https://mail.python.org/archives/list/typing-sig@python.org/thread/JG4IWHO3TZORNESR6ARXFL6OUZU2BEBX/
Author: Brett Cannon (brett.cannon) *
Date: 2020-04-17 00:46
I personally like 3.10 as the target as that means users had at least 3 years to move to move over. Plus we can put a warning in the What's New for 3.9 about our plans for 3.10.
Author: STINNER Victor (vstinner) *
Date: 2020-04-17 00:50
This issue has been discussed during the Language Summit. A quick poll showed that the majority is in favor of changing the default in Python 3.9.
Lukasz proposed a PEP update to propose to switch the default in Python 3.9: https://github.com/python/peps/pull/1371/
For me, the unclear part is which projects would be impacted if the default changes?
Someone mentioned attrs, but it seems like attrs is fine: https://github.com/python-attrs/attrs/issues/288#issuecomment-587265961
In term of workflow, I would prefer to get such incompatible in the very beginning of a devcycle, rather than just before the feature freeze. But I don't think that it's a blocker issue. Technically, changes are allowed until 3.9.0 beta1. Moreover, Lukasz is the 3.9 release manager, the author of the PEP 563 and he is in favor of changing the default in 3.9 :-)
Author: Guido van Rossum (gvanrossum) *
Date: 2020-05-18 16:30
Too bad nobody took any action here after the positive outcome of the discussion at the summit.
Author: STINNER Victor (vstinner) *
Date: 2020-05-18 22:20
Too bad nobody took any action here after the positive outcome of the discussion at the summit.
I didn't understand it this way. I understood that some people were not 100% comfortable to target 3.9. The question was 3.9 or 3.10. Since the release cycle is now shorter (1 years), only targeting 3.10 is not a big deal ;-)
Author: Łukasz Langa (lukasz.langa) *
Date: 2020-05-18 22:26
We'll make this an announced 3.10 feature early on. The discussion at the Summit wasn't as clear cut to me: 35% of participants would rather see this default later than 3.9.
Author: STINNER Victor (vstinner) *
Date: 2020-05-26 15:16
Is there anyone interested to implement this change in Python 3.10?
Author: Batuhan Taskaya (BTaskaya) *
Date: 2020-05-26 17:24
I opened the PR 20434 as draft, but from what I understand, there is going to be some breakage (on our test suite). I'll try to narrow it down (currently ~4 tests instead of ~20) but I dont want to prevent anyone else from working on this, so feel free to ignore my PR if you have a working test suite with a low breakage level.
Author: Batuhan Taskaya (BTaskaya) *
Date: 2020-05-26 20:27
After trying to complete a patch, there are a few issues that immediately showed itself (and this might lead to not to do this in 3.10, I dont know);
First one is double-forward-ref, which is usage of string-annotations when there is postponed evaluatation of annotations:
import typing from future import annotations def x(a: 'int'): pass ... typing.get_type_hints(x) {'a': ForwardRef('int')}
If we make annoatations feature default, this would be default behavior. The solution would be a workaround to the compiler; static int compiler_visit_annexpr(struct compiler *c, expr_ty annotation) { if (annotation->kind == Constant_kind && PyUnicode_CheckExact(annotation->v.Constant.value)) { PyObject *text = annotation->v.Constant.value; Py_INCREF(text); ADDOP_LOAD_CONST_NEW(c, text); } else { ADDOP_LOAD_CONST_NEW(c, _PyAST_ExprAsUnicode(annotation)); } return 1; } But I am not sure if this is too silly or not.
The second problem is inspect.signature. If we don't resolve annotations there and continue it is definitely going to break some code. If we resolve, that would mean that annotations must able to point something real (and this might not be the real case if the user uses a string annotation etc.) and will break code. (both tried and both breaks different modules on the stdlib tests)
The third problem is various dataclass hacks. Like _type_{field.name} etc. annotations and how ClassVar/InitVar parsed.
There are also some little parts that need to change.
Any thoughts on these issues?
Author: STINNER Victor (vstinner) *
Date: 2020-05-26 20:45
The second problem is
inspect.signature. If we don't resolve annotations there and continue it is definitely going to break some code.
Would you mind to elaborate why would it break some code? Consumers of annotations should already be prepared to get directly types or strings, no?
If we resolve, that would mean that annotations must able to point something real (and this might not be the real case if the user uses a string annotation etc.) and will break code. (both tried and both breaks different modules on the stdlib tests)
I expect that resolving has an impact on performance, whereas the caller may not use annotations at all but only cares of the number of parameters or their name.
It would be resonable to not resolve annotations in signature() by default. If someone cares, maybe a new parameter can be added to resolve annotations?
Author: Guido van Rossum (gvanrossum) *
Date: 2020-05-26 22:18
Double forward ref: IMO this can be resolved in the get_type_hints() functions. (Łukasz do you agree?)
inspect.signature(): Maybe this could switch to using typing.get_type_hints()? Then again if performance is important here maybe we cannot change anything.
dataclasses hacks: these should just be resolved.
Another thought: maybe some of these issues can be considered bugs in 3.9 as well, and we should fix them there too? That might help us decide the right path forward. After all we should really encourage people to start using from __future__ import annotations in their code, to help them get ready for these issues in 3.10.
Final thought: I know at least the Dropbox client team, and possibly also Instagram, has already turned on from __future__ import annotations by default in their local fork of Python. Maybe we can ask them if they ever felt the need to change inspect.signature or typing.get_type_hints.
Author: Eric V. Smith (eric.smith) *
Date: 2020-05-26 23:13
To my knowledge, dataclasses works with from __future__ import annotations. If there are specific examples of problems, I'd like to hear about it: please open a separate issue.
There is a hack (discussed at PyCon 2018 with all of the relevant players) where it avoids importing typing to look at typing.ClassVar, but I think that code is all correct. Maybe I should just bite the bullet and import typing, since I believe importing it is faster than it used to be.
Author: Batuhan Taskaya (BTaskaya) *
Date: 2020-05-28 10:51
From now on, should typing.get_type_hints automatically resolve arguments too? An example would be this;
import typing T = typing.TypeVar("T") class Loop(typing.Generic[T]): subloop: typing.Final["Loop[int]"] print(typing.get_type_hints(Loop))
{'subloop': typing.Final[main.Loop[int]]} If we run the same code under future annotations {'subloop': typing.Final[ForwardRef('Loop[int]')]}
Author: Guido van Rossum (gvanrossum) *
Date: 2020-05-28 14:02
I think in general it is more insightful to discuss the behavior of get_type_hints() given specific things in annotations.
We generally don't write forward refs inside forward refs, like "SomeClass['int']". So maybe that code was wrong? Where did you find it?
Author: Batuhan Taskaya (BTaskaya) *
Date: 2020-05-28 14:31
An example would be this https://github.com/python/cpython/blob/24bddc1b3b58f6899b2d412e51b37f68536e4fe2/Lib/test/test_typing.py#L2744-L2745. Either I can change tests in order to reflect now everything is a forward ref by default class Loop: attr: Final['Loop'] to class Loop: attr: Final[Loop] or resolve everything on get_type_hints.
Author: Guido van Rossum (gvanrossum) *
Date: 2020-05-28 15:47
There will still be a lot of code written that way, because people need compatibility with earlier versions of Python. So I think it should be fixed in get_type_hints().
Author: Terry J. Reedy (terry.reedy) *
Date: 2020-08-11 14:29
Issue 41314 changed the future annotations default version to 3.10. 3.10.0a1 is scheduled for next Oct 5, less than 2 months from now. It would be good if PR 20434 were merged before that.
Author: STINNER Victor (vstinner) *
Date: 2020-10-01 22:36
Issue 41314 changed the future annotations default version to 3.10. 3.10.0a1 is scheduled for next Oct 5, less than 2 months from now. It would be good if PR 20434 were merged before that.
Python 3.9.0 final is also around the corner. Any update on this issue?
Author: Batuhan Taskaya (BTaskaya) *
Date: 2020-10-06 20:02
@gvanrossum should I draft a patch to convert this inspect.signature(callable, *, follow_wrapped=True) (and also things like inspect.Signature.from_callable) into inspect.signature(callable, *, follow_wrapped=True, globalns=None, localns=None) in order to pass them to get_type_hints (for resolving in different namespaces)?
Author: Guido van Rossum (gvanrossum) *
Date: 2020-10-06 20:03
New changeset 044a1048ca93d466965afc027b91a5a9eb9ce23c by Batuhan Taskaya in branch 'master': bpo-38605: Make 'from future import annotations' the default (GH-20434) https://github.com/python/cpython/commit/044a1048ca93d466965afc027b91a5a9eb9ce23c
Author: Guido van Rossum (gvanrossum) *
Date: 2020-10-06 20:06
I'm marking this fixed, but there are some subtle edge cases. That's what alpha releases are for.
@BTaskaya, adding those arguments to various inspect functions sounds like a nice thing to do. You could either open a new issue or just reference this (closed) issue in your PR -- up to you.
Author: STINNER Victor (vstinner) *
Date: 2020-10-06 23:22
Good! I'm happy to see yet another "Python 4.0 change" going away :-) Let's make "Python 4.0" as compatible as possible ;-)
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2020-10-10 10:01
test_grammar and test_pydoc became failed on my machine until I removed all .pyc files. We need to bump the magic number for .pyc files.
Author: Guido van Rossum (gvanrossum) *
Date: 2020-10-10 22:19
New changeset 22220ae216b11644b23511c6287a52e7d28aeb9f by Batuhan Taskaya in branch 'master': bpo-38605: bump the magic number for 'annotations' future (#22630) https://github.com/python/cpython/commit/22220ae216b11644b23511c6287a52e7d28aeb9f
Author: miss-islington (miss-islington)
Date: 2021-04-07 05:02
New changeset 1be456ae9d53bb1cba2b24fc86175c282d1c2169 by Saiyang Gou in branch 'master': bpo-38605: Update "Future statements" docs since PEP 563 is always enabled (GH-25236) https://github.com/python/cpython/commit/1be456ae9d53bb1cba2b24fc86175c282d1c2169
Author: Pablo Galindo Salgado (pablogsal) *
Date: 2021-04-21 11:41
New changeset b0544ba77cf86074fb1adde00558c67ca75eeea1 by Pablo Galindo in branch 'master': bpo-38605: Revert making 'from future import annotations' the default (GH-25490) https://github.com/python/cpython/commit/b0544ba77cf86074fb1adde00558c67ca75eeea1
Author: STINNER Victor (vstinner) *
Date: 2021-04-21 12:18
New changeset b0544ba77cf86074fb1adde00558c67ca75eeea1 by Pablo Galindo in branch 'master':
Copy of the NEWS entry:
Revert making from __future__ import annotations the default. This follows
the Steering Council decision to postpone PEP 563 changes to at least Python
3.11. See the original email for more information regarding the decision:
https://mail.python.org/archives/list/python-dev@python.org/thread/CLVXXPQ2T2LQ5MP2Y53VVQFCXYWQJHKZ/.
Patch by Pablo Galindo.
Author: Guido van Rossum (gvanrossum) *
Date: 2021-04-23 15:24
I haven't followed this precisely, but I recall that when we first made PEP 563 the default, we had to fix a number of bugs in various library modules (e.g. inspect) that were only apparent when from __future__ import annotations was used. I hope we're not throwing away those bugfixes. Hopefully Batuhan has a recollection of what I am thinking of, there was some significant delay while we figured out what to do about some of these.
Author: Batuhan Taskaya (BTaskaya) *
Date: 2021-04-23 20:35
Hopefully Batuhan has a recollection of what I am thinking of, there was some significant delay while we figured out what to do about some of these.
The major one that I'd recall is that inspect.signature() just uses whatever is in annotations instead of resolving those. Now that future.annotations is not the default one, we can add a new option named 'resolve_annotations' and call typing.get_type_hints when activated. Here is a quick demo;
from future import annotations
import inspect
def foo(a: int, b: int) -> str: ...
def _get_annotations(func, **signature_opts):
signature = inspect.signature(func, **signature_opts)
return {
param.name: param.annotation
for param in signature.parameters.values()
}
print('bare: ', _get_annotations(foo)) print('annotations resolved: ', _get_annotations(foo, resolve_annotations=True))
bare: {'a': 'int', 'b': 'int'} annotations resolved: {'a': <class 'int'>, 'b': <class 'int'>}
This would be a clear feature for both PEP 563 users + people who are still using string annotations mixed with normal ones. What do you think Guido?
Author: Guido van Rossum (gvanrossum) *
Date: 2021-04-24 03:38
I don't like adding a flag. In the PEP-563-by-default world we didn't need a flag. Perhaps once Larry implements inspect.get_annotations() () we can make inspect.signature() always call that?
Author: miss-islington (miss-islington)
Date: 2021-04-25 19:49
New changeset f84f1b5c638eeb6e13c287fe5ebf3a7d2fdb60e9 by Saiyang Gou in branch 'master':
bpo-38605: Update future module doc as annotations is now "mandatory in 3.11" (GH-25602)
https://github.com/python/cpython/commit/f84f1b5c638eeb6e13c287fe5ebf3a7d2fdb60e9
Author: Vedran Čačić (veky) *
Date: 2021-05-21 12:48
May I ask, is this going forward? I installed 3.11-dev, it's not there (though future claims it should be). I understand if it isn't done yet, just want to know if there's risk it will be postponed again (or even given up).
Author: Batuhan Taskaya (BTaskaya) *
Date: 2021-05-21 12:53
May I ask, is this going forward? I installed 3.11-dev, it's not there (though future claims it should be). I understand if it isn't done yet, just want to know if there's risk it will be postponed again (or even given up).
We are still waiting a ruling on PEP 649. If it gets rejected, and no more ideas arises (before beta cut for 3.11), I guess we could move on and resolve this issue again.
Author: Steven D'Aprano (steven.daprano) *
Date: 2021-10-17 05:47
Now that we're in 3.11, people are starting to notice that stringy annotations are not the default (see #45499 for example).
What can we do to get PEP 649 moving forward?
Author: Serhiy Storchaka (serhiy.storchaka) *
Date: 2021-10-17 07:28
I think it would help if we could enable some future feature globally by command line option or environment variable, without modifying all source files. It would allow users to quickly test their code base for compatibility with future changes. The problem currently is that nobody bothers to add "from future import ...", so we have surprises every time when try to make it by default.
A tool which automatically adds or removes "from future import ..." in files could help too.
History
Date
User
Action
Args
2022-04-11 14:59:22
admin
set
github: 82786
2022-02-07 11:12:54
jwilk
set
nosy: + jwilk
2021-11-04 14:17:28
eryksun
set
messages: -
2021-11-04 14:17:12
eryksun
set
nosy: + gvanrossum, barry, brett.cannon, terry.reedy, eric.smith, steven.daprano, methane, lukasz.langa, serhiy.storchaka, veky, levkivskyi, Anthony Sottile, pablogsal, miss-islington, BTaskaya, gousaiyang, AlexWaygood, - vstinner, ezio.melotti, ahmedsayeed1982
versions: + Python 3.10, - Python 3.8
2021-11-04 12:09:31
ahmedsayeed1982
set
versions: + Python 3.8, - Python 3.10
nosy: + ezio.melotti, vstinner, ahmedsayeed1982, - gvanrossum, barry, brett.cannon, terry.reedy, eric.smith, steven.daprano, methane, lukasz.langa, serhiy.storchaka, veky, levkivskyi, Anthony Sottile, pablogsal, miss-islington, BTaskaya, gousaiyang, AlexWaygood
messages: +
components: + Unicode, - Interpreter Core
2021-10-17 08:48:32
AlexWaygood
set
nosy: + AlexWaygood
2021-10-17 07:28:36
serhiy.storchaka
set
messages: +
2021-10-17 05:47:50
steven.daprano
set
nosy: + steven.daprano
messages: +
2021-05-21 12:53:50
BTaskaya
set
messages: +
2021-05-21 12:48:39
veky
set
nosy: + veky
messages: +
2021-04-27 14:22:08
vstinner
set
nosy: - vstinner
2021-04-25 19:49:50
miss-islington
set
messages: +
2021-04-25 19:12:59
gousaiyang
set
pull_requests: + <pull%5Frequest24318>
2021-04-25 15:08:23
Anthony Sottile
set
nosy: + Anthony Sottile
pull_requests: + <pull%5Frequest24316>
stage: resolved -> patch review
2021-04-24 03:38:51
gvanrossum
set
messages: +
2021-04-23 20:35:29
BTaskaya
set
messages: +
2021-04-23 15:24:04
gvanrossum
set
messages: +
2021-04-21 12🔞51
vstinner
set
messages: +
2021-04-21 12:12:11
vstinner
set
title: [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.10 -> [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.11
2021-04-21 11:41:23
pablogsal
set
messages: +
2021-04-20 21:22:15
pablogsal
set
status: closed -> open
resolution: fixed -> postponed
2021-04-20 21🔞26
pablogsal
set
nosy: + pablogsal
pull_requests: + <pull%5Frequest24214>
2021-04-07 05:02:28
miss-islington
set
nosy: + miss-islington
messages: +
2021-04-07 00:12:22
gousaiyang
set
nosy: + gousaiyang
pull_requests: + <pull%5Frequest23974>
2020-10-10 22:19:53
gvanrossum
set
messages: +
2020-10-10 10:35:37
BTaskaya
set
pull_requests: + <pull%5Frequest21607>
2020-10-10 10:01:45
serhiy.storchaka
set
nosy: + serhiy.storchaka
messages: +
2020-10-06 23:22:54
vstinner
set
messages: +
2020-10-06 20:06:36
gvanrossum
set
status: open -> closed
resolution: fixed
messages: +
stage: patch review -> resolved
2020-10-06 20:03:29
gvanrossum
set
messages: +
2020-10-06 20:02:12
BTaskaya
set
messages: +
2020-10-01 22:36:47
vstinner
set
nosy: + vstinner
messages: +
2020-08-11 14:29:45
terry.reedy
set
nosy: + terry.reedy
messages: +
2020-08-02 01:23:39
methane
set
nosy: + methane
2020-05-28 15:47:53
gvanrossum
set
messages: +
2020-05-28 14:41:50
vstinner
set
nosy: - vstinner
2020-05-28 14:31:04
BTaskaya
set
messages: +
2020-05-28 14:02:42
gvanrossum
set
messages: +
2020-05-28 10:51:15
BTaskaya
set
messages: +
2020-05-26 23:13:14
eric.smith
set
messages: +
2020-05-26 22🔞57
gvanrossum
set
messages: +
2020-05-26 20:45:55
vstinner
set
messages: +
2020-05-26 20:27:06
BTaskaya
set
messages: +
2020-05-26 17:24:25
BTaskaya
set
messages: +
2020-05-26 17:21:47
BTaskaya
set
messages: -
2020-05-26 16:06:27
BTaskaya
set
keywords: + patch
stage: needs patch -> patch review
pull_requests: + <pull%5Frequest19691>
2020-05-26 15:42:15
BTaskaya
set
messages: +
2020-05-26 15:16:59
vstinner
set
versions: + Python 3.10
2020-05-26 15:16:53
vstinner
set
messages: +
2020-05-18 22:26:31
lukasz.langa
set
messages: +
2020-05-18 22:20:54
vstinner
set
messages: +
2020-05-18 16:30:06
gvanrossum
set
versions: - Python 3.9
messages: +
components: + Interpreter Core, - Library (Lib)
type: behavior
stage: needs patch
2020-05-18 15:40:30
vstinner
set
title: [typing] PEP 563: Postponed evaluation of annotations: enable it by default before Python 4.0 -> [typing] PEP 563: Postponed evaluation of annotations: enable it by default in Python 3.10
2020-04-17 06:49:29
BTaskaya
set
nosy: + BTaskaya
2020-04-17 00:50:37
vstinner
set
messages: +
2020-04-17 00:46:45
brett.cannon
set
nosy: + brett.cannon
messages: +
2020-01-28 02:19:04
vstinner
set
messages: +
2019-10-28 17:06:14
barry
set
nosy: + barry
2019-10-28 14:43:52
gvanrossum
set
messages: +
2019-10-28 11:03:22
vstinner
set
messages: +
2019-10-27 23:23:41
gvanrossum
set
messages: +
2019-10-27 22:51:45
eric.smith
set
nosy: + eric.smith
2019-10-27 19:17:48
levkivskyi
set
nosy: + gvanrossum
messages: +
2019-10-27 16:16:48
xtreak
set
nosy: + levkivskyi
2019-10-27 16:07:10
vstinner
create