Python's Self Type: How to Annotate Methods That Return self (original) (raw)

Have you ever found yourself lost in a big repository of Python code, struggling to keep track of the intended types of variables? Without the proper use of type hints and annotations, uncovering variable types can become a tedious and time-consuming task. Perhaps you’re an avid user of type hints but aren’t sure how to annotate methods that return self or other instances of the class itself. That’s the issue that you’ll tackle in this tutorial.

First, though, you’ll need to understand what type hints are and how they work. Type hints allow you to explicitly indicate variable types, function arguments, and return values. This can make your code more readable and maintainable, especially as it grows in size and complexity.

You specify variable and function argument types with a colon (:) then the data type, while return value annotations use a dash–greater than symbol (->) then the return type. To see an example, you can write a function that accepts as input the number of pies that you bought and the price per pie then outputs a string summarizing your transaction:

In buy_pies(), you type the num_pies variable with int and price_per_pie with float. You annotate the return value with the str type because it returns a string.

Types and annotations in Python usually don’t affect the functionality of the code, but many static type checkers and IDEs recognize them. For instance, if you hover over buy_pies() in VS Code, then you can see the type of each argument or return value:

You can also use annotations when working with classes. This can help other developers understand which return type to expect from a method, which can be especially useful when working with complex class hierarchies. You can even annotate methods that return an instance of the class.

One use case for types and annotations is to annotate methods that return an instance of their class. This is particularly useful for class methods and can prevent the confusion that arises when working with inheritance and method chaining. Unfortunately, annotating these methods can be confusing and cause unexpected errors. A natural way to annotate such a method is to use the class name, but the following won’t work:

In the example above, .enqueue() from Queue appends an item to the queue and returns the class instance. While it might seem intuitive to annotate .enqueue() with the class name, this causes both static type checking and runtime errors. Most static type checkers realize that Queue isn’t defined before it’s used, and if you try to run the code, then you’ll get the following NameError:

There would also be issues when inheriting from Queue. In particular, a method like .enqueue() would return Queue even if you called it on a subclass of Queue. Python’s Self type can handle these situations, offering a compact and readable annotation that takes care of the subtleties of annotating methods returning an instance of the enclosing class.

In this tutorial, you’ll explore the Self type in detail and learn how to use it to write more readable and maintainable code. In particular, you’ll see how to annotate a method with the Self type and make sure your IDE will recognize this. You’ll also examine alternative strategies for annotating methods that return a class instance and explore why the Self type is preferred.

How to Annotate a Method With the Self Type in Python

Due to its intuitive and concise syntax, as defined by PEP 673, the Self type is the preferred annotation for methods that return an instance of their class. The Self type can be imported directly from Python’s typing module in version 3.11 and beyond. For Python versions less than 3.11, the Self type is available in typing_extensions.

As an example, you’ll annotate a stack data structure. Pay particular attention to your .push() annotation:

You import the Self type from typing in line 3 and annotate .push() with -> Self in line 9. This tells static type checkers that .push() returns a Stack instance, allowing you to confidently chain multiple pushes together. Note that it’s usually not necessary to annotate self and cls parameters, as discussed in PEP 484.

For Python versions less than 3.11, you can use the typing_extensions module to import the Self type, and you can leave the remaining code unchanged:

By importing Self from typing_extensions, you can use Self to annotate methods in the same way that you would use the typing module in Python 3.11.

As you learned, .push() appends items to the stack and returns the updated stack instance, necessitating the Self annotation. This allows you to sequentially chain .push() methods to a stack instance, making your code more concise and readable:

In the above example, you instantiate a Stack instance, push three elements to the stack in sequence, and pop one element. By including Self as the annotation, you can examine .push() directly from an instantiated object to see what it returns:

VS Code recognizes the return type of .push()

When you hover over .push() in VS Code, you can see that the return type is Stack, as the annotation notes. With this annotation, others who read your code won’t have to look at the Stack definition to know that .push() returns the class instance.

Next, you’ll look at a class that represents the state and logic of a bank account. The BankAccount class supports several actions, such as depositing and withdrawing funds, that update the state of the account and return the class instance. For example, .deposit() takes a dollar amount as input, increments the internal balance of the account, and returns the instance so that you can chain other methods:

In BankAccount, you annotate .display_balance(), .deposit(), and .withdraw() with Self to return an instance of the class. You can instantiate BankAccount and deposit or withdraw funds an arbitrary number of times:

Here, you define a BankAccount instance with an account number and initial balance. You then chain multiple methods that perform deposits, withdrawals, and balance displays, each of which returns self. The REPL automatically prints the return value of the last expression in the method chain, .display_balance(). Its output, BankAccount(account_number=1534899324, balance=50), provides a nice representation of the class.

Other use cases for the Self type are class methods and inheritance hierarchies. For instance, if a parent and its child class have methods that return self, then you can annotate both with the Self type.

Interestingly, when a child object calls a parent method that returns self, type checkers will indicate that the method returns an instance of the child class. You can see this idea by creating a SavingsAccount class that inherits from BankAccount:

SavingsAccount has a .from_application() method that creates a class instance from applicant parameters rather than through the regular constructor. It also has .add_interest(), which deposits interest to the account balance and returns the class instance. You can boost the readability and maintainability of both these methods with the Self type.

Next, create a SavingsAccount object from a new application, make deposits and withdrawals, and add interest:

VS Code recognizes that .from_application() returns an instance of SavingsAccount:

VS Code recognizes the return type of .from_application()

When you hover over .from_application(), the type checker indicates that the return type is SavingsAccount. VS Code also recognizes that the return type of .deposit() is SavingsAccount, despite this method’s being defined in the BankAccount parent class:

VS Code recognizes the return type of an inherited method

Overall, the Self type is an intuitive and Pythonic choice for annotating methods that return self or, more generally, a class instance. Static type checkers recognize Self, and you can import the symbol so running the code doesn’t cause name errors.

In the next sections, you’ll explore alternatives to the Self type and see their implementations. Self is quite new, and several alternative approaches existed before Self was added. You may encounter these other annotations while reading through old code, so it’s important to understand how they work and what their limitations are.

Annotating With TypeVar

Another way to annotate methods that return an instance of their class is to use TypeVar. A type variable is a type that can function as a placeholder for a specific type during type checking. Type variables are often used for generic types, such as lists of specific objects like list[str] and list[BankAccount].

TypeVar allows you to declare parameters for generic types and function definitions, making it a valid candidate for annotating methods that return a class instance. To use TypeVar in this context, you can import it from Python’s typing module and give a name to your type in the constructor:

In the example above, you create the TStack type variable, which you can use to annotate .push() in the Stack class. In this case, TStack is bound by Stack, allowing the type variable to materialize as Stack or a subtype of Stack. You can now annotate methods with TStack:

In line 11, you annotate .push() with the TStack type. Also notice that you’ve annotated the self parameter with TStack. This is required for the static type checker to properly materialize TStack as Stack. A TypeVar that’s bound by a class can materialize as any subclass. This comes in handy in your BankAccount and SavingsAccount examples:

Here, TBankAccount is bound by BankAccount, allowing you to properly annotate the methods that return self in BankAccount:

You annotate .display_balance() with TBankAccount to specify that it’ll return a class instance. It’s important to remember that TBankAccount isn’t the same as BankAccount. Rather, it’s a type variable that represents the BankAccount type during type checking.

TBankAccount serves no other purpose than to represent the BankAccount type in annotations where you can’t use BankAccount directly. You can also use this type to annotate methods in the SavingsAccount child class:

You annotate SavingsAccount.from_application() with the TBankAccount type variable, and you annotate the cls parameter with type[TBankAccount]. Most static type checkers should recognize this as valid type hinting for both BankAccount and SavingsAccount.

The main drawback is that TypeVar is verbose, and a developer can easily forget to instantiate a TypeVar instance or properly bind the instance to a class. It’s also important to note that not all IDEs recognize TypeVar when inspecting methods. These are the primary reasons why the Self type is preferred over TypeVar.

In the next section, you’ll explore another alternative to Self and TypeVar, the __future__ module. You’ll see how this method overcomes the verbosity of TypeVar but still isn’t preferred over the Self type because it doesn’t support inheritance well.

Using the `future` Module

Python’s __future__ module offers a different approach to annotating methods that return the enclosing class. The __future__ module is sometimes used to introduce incompatible changes that are intended to become part of a future Python version.

For Python versions greater than 3.7, you can import the annotations feature from the __future__ module at the top of a script and use the class name directly as the annotation. You can see this with the Stack class:

In line 3, you import annotations from __future__, allowing you to use annotation features that might not be available in the version of Python you’re using. In line 11, you use the class name directly as the annotation for .push(). You can see the annotation when inspecting .push() in the same way as earlier.

Under the hood, the annotations aren’t executed but get stored as strings that can be executed later. This way of evaluating annotations has raised some discussion about potentially better ways to do this in future Python versions.

While the __future__ module works for annotating methods with the class name, this isn’t a best practice because the Self type is more intuitive and Pythonic. Plus, remembering to import from __future__ at the top of the script can be a hassle. More importantly, inheritance isn’t properly supported when annotating with __future__. Have a look at what happens to SavingsAccount methods when you use __future__ annotations:

The code above redefines SavingsAccount, which inherits from BankAccount. Notice how you’ve annotated .deposit() in BankAccount with BankAccount, and you’ve annotated the methods returning self in SavingsAccount with SavingsAccount. Everything appears fine, but look what happens when you check the type of .deposit() from an instance of SavingsAccount:

Inherited methods from SavingsAccount are incorrectly annotated with BankAccount

The return type of .deposit() is displaying as BankAccount even though this object is an instance of SavingsAccount. This happens because SavingsAccount inherits from BankAccount, and the __future__ annotation doesn’t properly support inheritance. This creates even more type checking issues when you inspect .add_interest():

The type check of .add_interest() fails because .deposit() is incorrectly annotated

The type check of .add_interest() fails because the type checker thinks that deposit() returns a BankAccount instance, but BankAccount has no methods named .add_interest(). This illustrates the most prominent flaw of __future__ annotations. While __future__ annotations may work for many classes, they’re not appropriate when typing inherited methods.

In the next section, you’ll explore an annotation that’s functionally similar but more direct than __future__ annotations. This will help you understand what __future__ is doing under the hood. Keep in mind that all the alternative annotations for methods that return class instances are no longer considered best practices. You should opt for the Self type, but it’s good to understand the alternatives because you might come across them in code.

Type Hinting With Strings

Lastly, you can use strings to annotate methods that return class instances. You should use the string annotation for Python versions less than 3.7, or when none of the other approaches work. String annotation doesn’t require any imports, and most static type checkers recognize it:

In this case, the string annotation should contain the name of the class. Otherwise, the static type checker won’t recognize the return type as a valid Python object. String annotations directly accomplish something similar to what __future__ annotations do behind the scenes.

One major drawback of string annotations is that they’re not preserved with inheritance. When a subclass inherits a method from a superclass, the annotations specified as strings in the superclass don’t automatically propagate to the subclass. This means that if you rely on string annotations for type hinting or documentation purposes, then you’ll need to redeclare the annotations in each subclass, which can be error-prone and time-consuming.

Many developers also find the syntax of string annotations to be unusual or non-idiomatic compared to other features of Python. In the early versions of Python 3, when type hints were introduced, string annotations were the only available option. However, with the introduction of the typing module and the type hints syntax, you now have a more standard and expressive way to annotate types.

Conclusion

Using type hints and annotations in Python can make your code more readable and maintainable, especially as it grows in size and complexity. By indicating the types of variables, function arguments, and return values, you can help other developers understand the intended types of variables and what to expect from function calls.

The Self type is a special type hint that you can use to annotate a method that returns an instance of its class. This makes the return type explicit and can help prevent subtle bugs that might arise when working with inheritance and subclassing. While you can use other options like TypeVar, the __future__ module, and strings to annotate methods returning class instances, you should use the Self type when possible.