Python 3.12 (original) (raw)

Python 3.12’s beta’s are out, which means the features are locked in. The theme this year has been cleanup and typing. distutils has been removed, and setuptools is no longer present in default environments.

Faster CPython

The faster CPython project is still working on features, though most of the changes this time around don’t seem to affect daily performance as much as 3.11 did. A lot of work went into the per interpreter GIL, which will be covered in it’s own section later. There still are some nice user-facing improvements though.

Comprehensions are now inlined, making them up to 2x faster. This does affect scoping inside comprehensions; just in case you are using stacklevel=n to go up through a comprehension in a warning, you’d need to reduce n by 1 in 3.12.

inspect.getattr_static has been made much faster (2-6x!); and this now is used in the implementation of typing.runtime_checkable, so that checkingisinstance on a Protocol is much faster for smallish Protocols. This is a huge win for readability and typing, as this matters in tight loops in libraries like Rich; this should reduce the need for the manual performance-oriented workarounds they employ today.

os.stat() on Windows is more accurate, and also faster. asyncio.current_taskis 4-6x faster. And f-strings tokenize faster, as well, though I’ll cover f-strings in more detail below.

Error messages

The suggestion features keep getting better, based heavily on reports from people teaching Python about common mistakes. Trying to use a stdlib module without importing it, reversing the order of from and import, and forgettingself. in front of an attempted attribute access now have customized error messages. And now when you try to import something that doesn’t exist from a module, you’ll get suggestions based on what’s actually in the module! F-strings also get much better error messages, but more on those below.

Typing

Generics

There were a lot of typing improvements this round, including a brand new syntax for generics! This is common in other statically-typed languages, but might seem a bit odd if you’ve not seen one of those before. Here’s an example, compared with the classic TypeVar method, and C++:

def f[T](x: T) -> T:
    return 2 * x

from typing import TypeVar

T = TypeVar("T")


def f(x: T) -> T:
    return 2 * x

template<typename T>
T f(T x) {
    return 2*x;
}

Bounds and constraints are supported too:

def f[T: numbers.Real](x: T) -> T:
    return 2 * x

from typing import TypeVar

T = TypeVar("T", bound=numbers.Real)


def f(x: T) -> T:
    return 2 * x

template <typename Data>
concept Numeric = std::is_arithmetic_v<Data>;

template<Numeric T>
T f(T x) {
    return 2*x;
}

And you can also make generic type aliases in one line:

type Vector3D[T] = tuple[T, T, T]

This also works if the type alias is not generic, so most usage of TypeVar andTypeAlias (and ParamSpec and TypeVarTuple, as * and ** are both supported) are no longer needed.

Unlike most typing improvements, this is not available in older Python versions, even with from __future__ import annotations.

Other typing improvements

You can now use a TypedDict for **kwargs! To do so, you need the newtyping.Unpack[T] wrapper, since the normal syntax for **kwargs just includes the values.

There’s now a typing.override decorator, to indicate that you intended to override a method.

array.array is now Generic.

Native f-strings

F-strings are now a native part of the syntax rather than a thousand-plus line hack on strings. This means the following is now valid:

The nested quotes are fine, because the stuff inside the brackets is normally parsed Python! Multiple lines, escape codes, all that stuff now works like normal Python, and not like string contents. This also means error messages now look like normal Python and can access the correct places in the line(s). And tokenizing the f-strings for parsing is 64% faster!

Per-interpreter GIL

One of the biggest features, and one with possibly the most performance potential, is the per-interpreter GIL. This is a rather odd new feature, though, because it landed without a Python interface; you have to use the C API for now. Though don’t worry, there will be a first-party PyPI module providing a Python API, and that will help guide the development of a proper standard library API, probably in 3.13.

The API will probably look something like this:

# Get "interpreters" from somewhere for now
interp = interpreters.create()
script = "print('Hello world')"
interp.run(script)

This would also integrate with threading:

t = Thread(target=interp.run, args=(script,))

The interpreters-3-12 module on PyPI contains initial work for this. There are other plans and hopes, like a dedicated API for data passing, but that’s the core idea for now. These interpreters each have their own GIL, so that means that with effort, you can run Python fully multithreaded in a single process! See the examples inericsnowcurrently/interpretersfor more.

Other features

The buffer protocol can now be used from Python, with __buffer__, making it a proper (and static-typable!) Protocol (available as collections.abc.Buffer). And a few other things:

Removals

Distutils has been removed from the standard library. Setuptools is no longer installed by default when you make a new venv (or use the third-partyvirtualenv on Python 3.12), or when you run ensurepip. You really should be providing at least a pyproject.toml with the following content:

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

Dropping this in with your current setup.py is enough to be ready for the future, though you should also look into modern Python backends like hatchling or C++ backends like scikit-build-core for a simpler and more elegant packaging experience.

Other removed libraries include asynchat and asyncore. There were also removals from unittest, configparser, sqlite, importlib, and more.

Other Developer changes

As usual, there are some more technical changes that will excite some people:

Final Words

If you are using GitHub Actions, the new and best way to add 3.12 is to use this:

- uses: actions/setup-python@v4
  with:
    python-version: "3.12"
    allow-prereleases: true

This works in a matrix, etc. too. Also, note that setup-python recently started supporting setting up multiple Python’s at once, with a range - very useful if you are using nox or tox, for example.

And if you are using cibuildwheel, we’ve supported this since beta 1 with the following flag:

- uses: pypa/cibuildwheel@v2.14
  with:
    CIBW_ALLOW_PRERELEASES: true

If you are using NumPy, you might be in for a wait. The just-released version of numpy (1.25) does not support Python 3.12; the next release (1.26) will complete the migration to a new build backend, and is supposed to come up “when 3.12 is released”. I don’t know if that means final release, RC 1, or some future beta.