[Python] Release 4.21.0 broke multiple Google Cloud client libraries ("TypeError: Descriptors cannot not be created directly.") (original) (raw)

Example error for the google-cloud-logging==3.1.1 library using protobuf==4.21.0:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/google/cloud/logging/__init__.py", line 18, in <module>
    from google.cloud.logging_v2 import __version__
  File "/usr/local/lib/python3.10/site-packages/google/cloud/logging_v2/__init__.py", line 25, in <module>
    from google.cloud.logging_v2.client import Client
  File "/usr/local/lib/python3.10/site-packages/google/cloud/logging_v2/client.py", line 25, in <module>
    from google.cloud.logging_v2._helpers import _add_defaults_to_filter
  File "/usr/local/lib/python3.10/site-packages/google/cloud/logging_v2/_helpers.py", line 25, in <module>
    from google.cloud.logging_v2.entries import LogEntry
  File "/usr/local/lib/python3.10/site-packages/google/cloud/logging_v2/entries.py", line 31, in <module>
    import google.cloud.audit.audit_log_pb2  # noqa: F401
  File "/usr/local/lib/python3.10/site-packages/google/cloud/audit/audit_log_pb2.py", line 62, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/lib/python3.10/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Note that this works fine with the previous protobuf==3.20.1 release.

To reproduce:

docker run -it python:3.10 /bin/bash

Within the Docker container:

pip3 install google-cloud-logging==3.1.1 protobuf==4.21.0 python3 -c "import google.cloud.logging"