GitHub - scikit-hep/scikit-hep-testdata: A common package to provide example files (e.g., ROOT) for testing and developing packages against. (original) (raw)

scikit-hep-testdata

A common package to provide example files (e.g. ROOT) for testing and developing packages against. The sample of files is representative of typical files found "in the wild".

In addition to including some root files directly, this package adds some simple helper methods to get larger files from common open-access data repositories.

Installing and usage

To install:

python -m pip install scikit-hep-testdata

Once installed, absolute file paths can be resolved using the helper methods:

from skhep_testdata import data_path

filename = data_path("some_file.root")

By default, if an unknown file is requested an exception is raised but this can be skipped by passing the above method raise_missing=False:

filename = data_path("unknown_file.root", raise_missing=False)

The files are not stored on PyPI, so if installed from SDist/wheel, the "local" files will not be present, but will be downloaded from GitHub and cached in the~/.local/skhepdata directory. If you make an editable install from the Git repo, or if you set SKHEP_DATA=1 when building/installing from the Git repo, you will have the data files locally.

You can see all "local" files with skhep_testdata.known_files, and you can download all files at once with skhep_testdata.download_all(), optionally selecting the download cache directory.

Remote vs. Local files

Some files, particularly large ones, for example, are not stored within this package and instead live on a remote server; we call these "remote files". To obtain these use the same data_path method as above, however this will trigger the code to download and configure the remote file. This might be slow the first time round but will subsequently be as fast as for a local file. WARNING: the local file caching system has not yet been applied to remote files.

Command-line invocation

You can also interact with this package from the command-line:

Print a path (download if needed)

python -m skhep_testdata cms_hep_2012_tutorial/data.root

Show all "local" files

python -m skhep_testdata --list

Download all files to an existing directory

python -m skhep_testdata --all --dir local

You can also use pipx run scikit-hep-testdata to access the above CLI without installing.

Adding new files

We're on the look out for new, interesting files!

Large files: If the file is particularly large, for example > 25 MB, it might be worth adding to an external open access data repository and adding a configuration here so that the internal helper methods can pull this down.
Experiment data policies: Please make sure you have permissions to add the file to this collection, and that any private or sensitive data has been appropriately masked, salted, or scrambled.

List of files

The following lists describe the files known by this package.

Files stored in this package

Listed here

Known remote files

CMS HEP public tutorial from 2012

Contributors

We hereby acknowledge the contributors that made this project possible (emoji key):

benkrikler💻 📖	Jim Pivarski🚧 🔣 📖	Henry Schreiner🚧 🔣 💻 📖	Eduardo Rodrigues🚧 🔣 💻	Matthew Feickert🔣 💻	Pratyush Das🔣 💻	Jerry Ling🔣 💻
Jonas Eschle💻	Giordon Stark🔣 💻	Dmitry Kalinkin🔣	Michele Peresano🔣	Luis Antonio Obis Aparicio🔣	Oksana Shadura🔣	Nicholas Smith🔣
Beojan Stanislaus🔣	Lukas🔣	Johannes Schumann🔣	Elliott Kauffman🔣	Tom Eichlersmith🔣	Alexander Puck Neuwirth🔣	ioanaif🔣
Andres Rios Tascon🔣	Artur Cordeiro Oudot Choi🔣	Samantha Abbott🔣

This project follows the all-contributors specification.

Acknowledgements

Many of the files collected directly within this package were collated originally by Jim Pivarski for uproot

Running the tests

This package uses pytest to run the unit tests. Install with pip install scikit-hep-testdata[test] or pip install -e .[test] (dev) to get the testing requirements. then run: