[Python] Downloading sources of pyarrow and its requirements from pypi takes several minutes starting from 0.16.0 · Issue #25105 · apache/arrow (original) (raw)

It appears that

      python -m pip download --dest /tmp pyarrow==0.17.1 --no-binary :all:

takes several minutes to execute.

There seems to be an increase in runtime starting from 0.16.0: on Python 2

 python -m pip download --dest /tmp pyarrow==0.15.1 --no-binary :all:

appears to be somewhat faster, but the same command is still slow on Py3.

The command is stuck for a while with "Installing build dependencies ... ", and increased CPU usage.

The intent of this command is to download source tarball for a package and its dependencies.

Some investigation was started on the mailing list: https://lists.apache.org/thread.html/r9baa48a9d1517834c285f0f238f29fcf54405cb7cf1e681314239d7f%40%3Cdev.arrow.apache.org%3E

Reporter: Valentyn Tymofieiev

Note: This issue was originally created as ARROW-8983. Please see the migration documentation for further details.