Typing spec gap about properties in Protocols (original) (raw)

Can a Protocol property be implemented by an attribute? I.e., is this type-valid code?

from typing import Protocol

class HasField(Protocol):
    @property
    def my_field(self) -> int:  ...

class C(HasField):
    def __init__(self):
        self.my_field = 42

Currently type-checkers disagree. pyright thinks it’s a typing error “Literal[42] is not assignable to property”, while mypy and pyre are ok with it.
This looks intentional on at least 2 of the 3 checkers:

  1. Pyright maintains that

For class inheritance, attributes and protocols cannot be intermixed (i.e. you can’t override a property with a normal attribute or vice versa).

2.Mypy holds that :

Since properties are used to emulate attributes, it would certainly be very annoying if MyPy where to separate fields and properties as separate kinds of attributes.

It might be beneficial for the spec to step in and settle this.

Personal opinion: while pep 544 fails to address this directly, it is unequivocally motivated by the need for structural typing. The example above aims exactly to use structural typing to express that any function accepting an argument x of a class conforming to HasField is free to use x.my_field without worrying about where it came from or how it is implemented. All checkers agree with this when the argument is decorated directly -

def func(x: HasField): ...

and as a user I think the same should hold when the decoration is indirect and involves inheritance, i.e.,

def func(x: C)

The semantics I would like to see is that a protocol property

    @property
    def my_field(self) -> int:  ...

is interpreted as "users of a class C conforming to the protocol can read C.my_field, and a property setter in the protocol -

    @my_field.setter
    def my_field(self) -> int:  ...

mean that "users of a class C conforming to the protocol can assign to C.my_field. And the way to express that “a class conforms to the protocol” is of course for a class to inherit it.

WDYT?

carljm (Carl Meyer) April 9, 2025, 5:09am 2

Hi Ofek,

I find your example a bit confusing, because it mixes structural typing (protocols) with actual inheritance, which I think muddies the question.

If we want to discuss only structural typing, all three type checkers are fine with this code:

from typing import Protocol

class HasField(Protocol):
    @property
    def my_field(self) -> int:  ...

class C:
    def __init__(self):
        self.my_field = 42

def f(x: HasField): ...

f(C())

In other words, all the type checkers agree (I think correctly) that structurally, an instance of C is a subtype of the HasField protocol.

The situation changes when you actually have C inherit HasField at runtime, as in your example, because then we have to not only consider HasField as expressing a structural type, but we also have to consider its inherited concrete implementation. (There is no magic whereby inherited Protocols are erased at runtime.) And in this scenario, I believe pyright is correct to error. Given the code in your initial example, trying to create an instance of C at runtime leads to AttributeError: property 'my_field' of 'C' object has no setter. It is not safe for a subclass to assign to an attribute that is a read-only property in a base class.

If we want to discuss standardizing the latter behavior, I think it will be clearer to leave Protocol out of it entirely. It doesn’t change the behavior of any of the type checkers if we look at this example instead:

class HasField:
    @property
    def my_field(self) -> int:
        return 1

class C(HasField):
    def __init__(self):
        self.my_field = 42

If we were revisiting core design decisions of Protocol, I think it might make sense to clarify that Protocols are only structural types, and consider a design where inherited Protocols are erased at runtime. This would mean that inheriting a protocol could become purely an assertion for a static type checker to verify that the type implements the protocol, and we wouldn’t have to consider it as an actual runtime base class. But it is probably too late to make this change.

OfekShilon (Ofek Shilon) April 10, 2025, 6:28pm 3

Thank you for this answer! I agree that indeed this can be narrowed down to a core issue not involving protocols: can properties and attributes override each other? I was hoping the answer is ‘yes’, as properties aim to be drop-in replacements for attributes, but apparently type checkers don’t agree even on this.

Pyright rejects it (intentionally so). Mypy accepts it since 2017 (again, very much intentionally), as does pyre.

Hopefully it can be agreed that this deserves spec guidance. Suggested direction:

carljm (Carl Meyer) April 10, 2025, 8:39pm 4

I do think this is an area where specification would be useful, as some library APIs are designed for subclassing, and whether this particular form of subclassing is allowed would be relevant to designers of such APIs, whose clients may use different type checkers.

The unsoundness in the mypy/pyre approach is the one @erictraut points out in the linked issue: we have covariant type[], so if B is a subclass of C, type[B] is a subtype of type[C]. But if we allow properties to override attributes, and vice versa, it breaks substitutability of the type[] types, because accessing a property on the class returns the property object, whereas accessing an attribute might give you the attribute type itself or might not be bound at all. So any code that accesses an attribute on a type[] type could be broken under these rules, and the type checker would not be able to catch it.

I’m not sure how to go about reaching consensus on this.

OfekShilon (Ofek Shilon) April 11, 2025, 3:46pm 5

So it seems these two natural wishes collide -
(1) Make properties and attributes interchangeable,
(2) Have covariant type[ X ].

And perhaps explicitly prefix that with -
(0) Make different type checkers consistent.

I think (0) is an obvious prerequisite to any discussion, and since this is a discussion about types and not runtime behavior may I suggest the choice between (1) and (2) is best approached as a pragmatic one:
How often would users benefit from having properties and attributes interchangeable in the type system? Versus, how often would users benefit from having A < B → type[A] < type[B]?
Put another way: which choice, between (1) and (2), would lead to less # type ignores in code, worldwide?

I dare guess the answer is (1). A proxy test in case anyone thinks otherwise would be to grep your code base for @property and for type(*) and compare the counts.

The sacrifice in soundness seems very much tolerable - other languages make much worse sacrifices in type systems for usability (I believe eg Eiffel made argument-types covariant instead of contravariant, just to be more intuitive)

beauxq (Doug Hoskisson) April 11, 2025, 6:08pm 6

As far as this issue goes,
I think (1) and (2) don’t collide, and we can have both
if we sacrifice (3)
(3) Support getting the property object from a type

I’ve seen a lot of code make good use of (1) and a lot of code make good use of (2).
I think I’ve seen more (2) than (1). (So if we have to choose between them, choose (2).)
I’ve never seen any code make good use of (3).

I think this is not a good proxy test.
In one project I just looked in: 494 @property and and 259 type(*) - and this project definitely makes a lot more use of (2). The vast majority of the @property are not overridden at all, much less overridden with attributes.

I dont think attributes and descriptors are interchangable at all, and I wouldn’t want to sacrifice 3, I’d want type checkers to be able to eventually understand descriptors in specific, and metaprogramming techniques in general. There are good uses both of getting and setting property objects, most of them are metaprogramming that significantly save on repeated code being written out by hand: example.

beauxq (Doug Hoskisson) April 11, 2025, 9:39pm 8

That’s not an example of using (3), because it’s using setattr - sacrificing (3) wouldn’t make any difference to that.

I’m not denying that there are examples out there. I’m sure there are. But I still haven’t seen any.

I’m also not saying that we should sacrifice (3) - just pointing it out as another option.
I don’t have much opinion between sacrificing (1) and (3), but I think (2) is the most valuable, and the most used, and should not go away.

OfekShilon (Ofek Shilon) April 12, 2025, 1:42pm 9

IIUC what you’re suggesting is a change to runtime, and I wouldn’t go there for sake of type-checker consistency. Can’t say if indeed there’s no good usage for this out there, but somehow disabling it would break existing code (perhaps in subtle ways) with no easy workarounds.

OfekShilon (Ofek Shilon) April 12, 2025, 2:01pm 10

I dare guess that in the vast majority of cases appearance of type(*) doesn’t imply relying on covariance either - as said this is just a proxy, and I’d welcome any suggestion of a better one. Also, I’m curious - can you point to examples of good use of type(*) covariance?

beauxq (Doug Hoskisson) April 12, 2025, 2:14pm 11

No, I’m not suggesting a change to runtime. I’m talking about type-checker support.
Runtime can still get the property object from a type. Type checkers just don’t need to allow it.

beauxq (Doug Hoskisson) April 12, 2025, 3:05pm 12

The vast majority of type[X] annotations are making use of type covariance (because if they weren’t, they could just use a hard-coded type that would have no need for an annotation).
Not all of those are good uses though. Many of them should change to a Callable that returns an instance of the type, if that’s all they’re used for.
If you find a type[X] annotation that isn’t just used as a Callable that returns an instance of the type, I would probably say that’s a good use of it.

carljm (Carl Meyer) April 13, 2025, 12:03am 13

If type checkers didn’t allow accessing properties as an attribute on the class, then it would be safe for properties to override instance-only attributes (and vice versa), where an instance-only attribute is one that also can’t be accessed on the class (i.e. it isn’t assigned a value in the class namespace.)

OfekShilon (Ofek Shilon) April 13, 2025, 1:09pm 14

That does seem like a step that is easier to agree on, but I’m afraid it’s impact wouldn’t be very wide - as I believe for dataclasses every instance attribute is backed by a class attribute.

Tinche (Tin Tvrtković) April 13, 2025, 2:12pm 15

This isn’t true for dataclasses, but it is true for all slotted classes (that’s how __slots__ work).

Funnily enough, both pyright and Mypy are completely wrong here:

from dataclasses import dataclass


@dataclass
class Test:
    a: int


reveal_type(Test.a)

Both claim it’s an int, but in reality Test.a will raise an AttributeError.

Eneg (Eneg) April 13, 2025, 3:18pm 16

FWIW this is one of the places PEP 767 aims to address.

I imagine this stems from annotations at class level being currently treated as “instance attribute which can also be a class variable”.

jorenham (Joren Hammudoglu) April 25, 2025, 9:02pm 17

For getter-only properties, PEP 767’s ReadOnly sounds like a safe solution to me.

But for properties with an fset or fdel, another solution is needed. For instance (pun intended), something like an InstanceVar that prohibits the attribute from being used as a class-attribute.