GitHub - ofek/pypinfo: Easily view PyPI download statistics via Google's BigQuery. (original) (raw)

pypinfo: View PyPI download statistics with ease.

pypinfo is a simple CLI to access PyPI download statistics via Google's BigQuery.

Click to unfold usage

$ pypinfo Usage: pypinfo [OPTIONS] [PROJECT] [FIELDS]... COMMAND [ARGS]...

Valid fields are:

project | version | file | pyversion | percent3 | percent2 | impl | impl-version |

openssl | date | month | year | country | installer | installer-version |

setuptools-version | system | system-release | distro | distro-version | cpu |

libc | libc-version

Options: -a, --auth TEXT Path to Google credentials JSON file. --run / --test --test simply prints the query. -j, --json Print data as JSON, with keys rows and query. -i, --indent INTEGER JSON indentation level. -t, --timeout INTEGER Milliseconds. Default: 120000 (2 minutes) -l, --limit INTEGER Maximum number of query results. Default: 10 -d, --days INTEGER Number of days in the past to include. Default: 30 -sd, --start-date TEXT Must be negative or YYYY-MM[-DD]. Default: -31 -ed, --end-date TEXT Must be negative or YYYY-MM[-DD]. Default: -1 -m, --month TEXT Shortcut for -sd & -ed for a single YYYY-MM month. -w, --where TEXT WHERE conditional. Default: file.project = "project" -o, --order TEXT Field to order by. Default: download_count --all Show downloads by all installers, not only pip. -pc, --percent Print percentages. -md, --markdown Output as Markdown. -v, --verbose Print debug messages to stderr. --version Show the version and exit. -h, --help Show this message and exit.

pypinfo accepts 0 or more options, followed by exactly 1 project, followed by 0 or more fields. By default only the last 30 days are queried. Let's take a look at some examples!

Tip: If queries are resulting in NoneType errors, increase timeout.

Downloads for a project

$ pypinfo requests Served from cache: False Data processed: 2.83 GiB Data billed: 2.83 GiB Estimated cost: $0.02

All downloads

$ pypinfo "" Served from cache: False Data processed: 116.15 GiB Data billed: 116.15 GiB Estimated cost: $0.57

Downloads for a project by Python version

$ pypinfo django pyversion Served from cache: False Data processed: 967.33 MiB Data billed: 968.00 MiB Estimated cost: $0.01

All downloads by country code

$ pypinfo "" country Served from cache: False Data processed: 150.40 GiB Data billed: 150.40 GiB Estimated cost: $0.74

Downloads for a project by system and distribution

$ pypinfo cryptography system distro Served from cache: False Data processed: 2.52 GiB Data billed: 2.52 GiB Estimated cost: $0.02

Most popular projects in the past year

$ pypinfo --days 365 "" project Served from cache: False Data processed: 1.69 TiB Data billed: 1.69 TiB Estimated cost: $8.45

Downloads between two YYYY-MM-DD dates

$ pypinfo --start-date 2018-04-01 --end-date 2018-04-30 setuptools Served from cache: False Data processed: 571.37 MiB Data billed: 572.00 MiB Estimated cost: $0.01

Downloads between two YYYY-MM dates

• A yyyy-mm --start-date defaults to the first day of the month
• A yyyy-mm --end-date defaults to the last day of the month

$ pypinfo --start-date 2018-04 --end-date 2018-04 setuptools Served from cache: False Data processed: 571.37 MiB Data billed: 572.00 MiB Estimated cost: $0.01

Downloads for a single YYYY-MM month

$ pypinfo --month 2018-04 setuptools Served from cache: False Data processed: 571.37 MiB Data billed: 572.00 MiB Estimated cost: $0.01

Percentage of Python 3 downloads of the top 100 projects in the past year

Let's use --test to only see the query instead of sending it.

$ pypinfo --test --days 365 --limit 100 "" project percent3 SELECT file.project as project, ROUND(100 * SUM(CASE WHEN REGEXP_EXTRACT(details.python, r"^([^.]+)") = "3" THEN 1 ELSE 0 END) / COUNT(), 1) as percent_3, COUNT() as download_count, FROM bigquery-public-data.pypi.file_downloads WHERE timestamp BETWEEN TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -366 DAY) AND TIMESTAMP_ADD(CURRENT_TIMESTAMP(), INTERVAL -1 DAY) AND details.installer.name = "pip" GROUP BY project ORDER BY download_count DESC LIMIT 100

Downloads for a given version

pypinfo supports PEP 440 version matching.

We can use it to query stats on a given major version.

$ pypinfo -pc 'pip==21.*' pyversion version Served from cache: False Data processed: 34.45 MiB Data billed: 35.00 MiB Estimated cost: $0.01

We can also use it to query stats on an exact version:

$ pypinfo -pc 'numpy==1.23rc3' pyversion version Served from cache: False Data processed: 34.01 MiB Data billed: 35.00 MiB Estimated cost: $0.01

Check how many downloads came from continuous integration servers:

❯ pypinfo --percent --days 5 pillow ci Served from cache: False Data processed: 384.22 MiB Data billed: 385.00 MiB Estimated cost: $0.01