Table of Contents Hide
A breaking upgrade in protobuf from version 4.21.0 broke multiple Google cloud client libraries and leads to typeerror: descriptors cannot not be created directly.
The solutions to this problem are –
- Downgrade the protobuf package to 3.20.x or lower.
- Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
This new version offers significantly better parsing performance than previous releases, especially for large payloads. These changes are mentioned in Protocol Buffer Guide.
Here is the list of affected packages from this change which leads to
typeerror: descriptors cannot not be created directly –
- Onnx – Open Neural Network Exchange
- TensorFlow 1.15.5
Most of the google-cloud libraries are prone to this error.
Solution 1: Downgrade protobuf package
It’s not always evident if there is
protobuf installed in your project. This is because different libraries have dependencies and they install them.
If you are using google-cloud libraries then probably
protobuf is installed by them.
One of the solution is to downgrade protobuf package to version 3.20.x or lower. You can use pip for that –
pip install protobuf==3.20.1
You can also downgrade the package using
requirements.txt file. Append the protobuf attribute at the end of this file. When the parser read
requirements.txt file, it will override the previously mentioned protobuf versions and choose the last one.
... ... protobuf==3.20.1
Solution 2: Setting PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION Flag
Another solution to resolve “typeerror: descriptors cannot not be created directly” is to set
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION flag to be equal to
python. It’s an environment variable which can be set using this command –
This flag is also used to share messages between Python and C++. In that case we set
The problem with this approach is that it is much slower. This is because it uses pure-Python parsing. While the new version of protobuf is upgraded to improve significant performance in parsing and we couldn’t use that.
Due to the breaking changes in protobuf version 4.21.0 most of the libraries which were dependent on it broke on update. This version is better in parsing performance but is not backward compatible. So, the libraries which are not upgraded with this change are still dependent on previous versions of protobuf and throwing typeerror: descriptors cannot not be created directly.