Contributing (original) (raw)

This project is a community effort, and everyone is welcome to contribute. It is hosted on scikit-learn/scikit-learn. The decision making process and governance structure of scikit-learn is laid out in Scikit-learn governance and decision-making.

Scikit-learn is somewhat selective when it comes to adding new algorithms, and the best way to contribute and to help the project is to start working on known issues. See Issues for New Contributors to get started.

In case you experience issues using this package, do not hesitate to submit a ticket to theGitHub issue tracker. You are also welcome to post feature requests or pull requests.

Ways to contribute#

There are many ways to contribute to scikit-learn, with the most common ones being contribution of code or documentation to the project. Improving the documentation is no less important than improving the library itself. If you find a typo in the documentation, or have made improvements, do not hesitate to create a GitHub issue or preferably submit a GitHub pull request. Full documentation can be found under the doc/ directory.

But there are many other ways to help. In particular helping toimprove, triage, and investigate issues andreviewing other developers’ pull requests are very valuable contributions that decrease the burden on the project maintainers.

Another way to contribute is to report issues you’re facing, and give a “thumbs up” on issues that others reported and that are relevant to you. It also helps us if you spread the word: reference the project from your blog and articles, link to it from your website, or simply star to say “I use it”:

In case a contribution/issue involves changes to the API principles or changes to dependencies or supported versions, it must be backed by aEnhancement proposals (SLEPs), where a SLEP must be submitted as a pull-request toenhancement proposalsusing the SLEP templateand follows the decision-making process outlined in Scikit-learn governance and decision-making.

Automated Contributions Policy#

Please refrain from submitting issues or pull requests generated by fully-automated tools. Maintainers reserve the right, at their sole discretion, to close such submissions and to block any account responsible for them.

Ideally, contributions should follow from a human-to-human discussion in the form of an issue.

Submitting a bug report or a feature request#

We use GitHub issues to track all bugs and feature requests; feel free to open an issue if you have found a bug or wish to see a feature implemented.

In case you experience issues using this package, do not hesitate to submit a ticket to theBug Tracker. You are also welcome to post feature requests or pull requests.

It is recommended to check that your issue complies with the following rules before submitting:

How to make a good bug report#

When you submit an issue to GitHub, please do your best to follow these guidelines! This will make it a lot easier to provide you with good feedback:

If you want to help curate issues, read about Bug triaging and issue curation.

Contributing code#

Note

To avoid duplicating work, it is highly advised that you search through theissue tracker and the PR list. If in doubt about duplicated work, or if you want to work on a non-trivial feature, it’s recommended to first open an issue in the issue trackerto get some feedbacks from core developers.

One easy way to find an issue to work on is by applying the “help wanted” label in your search. This lists all the issues that have been unclaimed so far. In order to claim an issue for yourself, please comment exactly/take on it for the CI to automatically assign the issue to you.

To maintain the quality of the codebase and ease the review process, any contribution must conform to the project’s coding guidelines, in particular:

Video resources#

These videos are step-by-step introductions on how to contribute to scikit-learn, and are a great companion to the following text guidelines. Please make sure to still check our guidelines below, since they describe our latest up-to-date workflow.

Note

In January 2021, the default branch name changed from master to mainfor the scikit-learn GitHub repository to use more inclusive terms. These videos were created prior to the renaming of the branch. For contributors who are viewing these videos to set up their working environment and submitting a PR, master should be replaced to main.

How to contribute#

The preferred way to contribute to scikit-learn is to fork the main repository on GitHub, then submit a “pull request” (PR).

In the first few steps, we explain how to locally install scikit-learn, and how to set up your git repository:

  1. Create an account on GitHub if you do not already have one.
  2. Fork the project repository: click on the ‘Fork’ button near the top of the page. This creates a copy of the code under your account on the GitHub user account. For more details on how to fork a repository see this guide.
  3. Clone your fork of the scikit-learn repo from your GitHub account to your local disk:
    git clone git@github.com:YourLogin/scikit-learn.git # add --depth 1 if your connection is slow
    cd scikit-learn
  4. Follow steps 2-6 in Building from source to build scikit-learn in development mode and return to this document.
  5. Install the development dependencies:
    pip install pytest pytest-cov ruff mypy numpydoc black==24.3.0
  6. Add the upstream remote. This saves a reference to the main scikit-learn repository, which you can use to keep your repository synchronized with the latest changes:
    git remote add upstream git@github.com:scikit-learn/scikit-learn.git
  7. Check that the upstream and origin remote aliases are configured correctly by running git remote -v which should display:
    origin git@github.com:YourLogin/scikit-learn.git (fetch)
    origin git@github.com:YourLogin/scikit-learn.git (push)
    upstream git@github.com:scikit-learn/scikit-learn.git (fetch)
    upstream git@github.com:scikit-learn/scikit-learn.git (push)

You should now have a working installation of scikit-learn, and your git repository properly configured. It could be useful to run some test to verify your installation. Please refer to Useful pytest aliases and flags for examples.

The next steps now describe the process of modifying code and submitting a PR:

  1. Synchronize your main branch with the upstream/main branch, more details on GitHub Docs:
    git checkout main
    git fetch upstream
    git merge upstream/main
  2. Create a feature branch to hold your development changes:
    git checkout -b my_feature
    and start making changes. Always use a feature branch. It’s good practice to never work on the main branch!
  3. (Optional) Install pre-commit to run code style checks before each commit:
    pip install pre-commit
    pre-commit install
    pre-commit checks can be disabled for a particular commit withgit commit -n.
  4. Develop the feature on your feature branch on your computer, using Git to do the version control. When you’re done editing, add changed files usinggit add and then git commit:
    git add modified_files
    git commit
    to record your changes in Git, then push the changes to your GitHub account with:
    git push -u origin my_feature
  5. Follow theseinstructions to create a pull request from your fork. This will send an notification to potential reviewers. You may want to consider sending an message to the discord in the development channel for more visibility if your pull request does not receive attention after a couple of days (instant replies are not guaranteed though).

It is often helpful to keep your local feature branch synchronized with the latest changes of the main scikit-learn repository:

git fetch upstream git merge upstream/main

Subsequently, you might need to solve the conflicts. You can refer to theGit documentation related to resolving merge conflict using the command line.

Pull request checklist#

Before a PR can be merged, it needs to be approved by two core developers. An incomplete contribution – where you expect to do more work before receiving a full review – should be marked as a draft pull requestand changed to “ready for review” when it matures. Draft PRs may be useful to: indicate you are working on something to avoid duplicated work, request broad review of functionality or API, or seek collaborators. Draft PRs often benefit from the inclusion of a task list in the PR description.

In order to ease the reviewing process, we recommend that your contribution complies with the following rules before marking a PR as “ready for review”. Thebolded ones are especially important:

  1. Give your pull request a helpful title that summarizes what your contribution does. This title will often become the commit message once merged so it should summarize your contribution for posterity. In some cases “Fix ” is enough. “Fix #” is never a good title.
  2. Make sure your code passes the tests. The whole test suite can be run with pytest, but it is usually not recommended since it takes a long time. It is often enough to only run the test related to your changes: for example, if you changed something insklearn/linear_model/_logistic.py, running the following commands will usually be enough:
    • pytest sklearn/linear_model/_logistic.py to make sure the doctest examples are correct
    • pytest sklearn/linear_model/tests/test_logistic.py to run the tests specific to the file
    • pytest sklearn/linear_model to test the wholelinear_model module
    • pytest doc/modules/linear_model.rst to make sure the user guide examples are correct.
    • pytest sklearn/tests/test_common.py -k LogisticRegression to run all our estimator checks (specifically for LogisticRegression, if that’s the estimator you changed).
      There may be other failing tests, but they will be caught by the CI so you don’t need to run the whole test suite locally. For guidelines on how to use pytest efficiently, see the Useful pytest aliases and flags.
  3. Make sure your code is properly commented and documented, and make sure the documentation renders properly. To build the documentation, please refer to our Documentation guidelines. The CI will also build the docs: please refer to Generated documentation on GitHub Actions.
  4. Tests are necessary for enhancements to be accepted. Bug-fixes or new features should be provided withnon-regression tests. These tests verify the correct behavior of the fix or feature. In this manner, further modifications on the code base are granted to be consistent with the desired behavior. In the case of bug fixes, at the time of the PR, the non-regression tests should fail for the code base in the main branch and pass for the PR code.
  5. If your PR is likely to affect users, you need to add a changelog entry describing your PR changes, see the following README <https://github.com/scikit-learn/scikit-learn/blob/main/doc/whats_new/upcoming_changes/README.md>for more details.
  6. Follow the Coding guidelines.
  7. When applicable, use the validation tools and scripts in the sklearn.utilsmodule. A list of utility routines available for developers can be found in theUtilities for Developers page.
  8. Often pull requests resolve one or more other issues (or pull requests). If merging your pull request means that some other issues/PRs should be closed, you should use keywords to create link to them(e.g., Fixes #1234; multiple issues/PRs are allowed as long as each one is preceded by a keyword). Upon merging, those issues/PRs will automatically be closed by GitHub. If your pull request is simply related to some other issues/PRs, or it only partially resolves the target issue, create a link to them without using the keywords (e.g., Towards #1234).
  9. PRs should often substantiate the change, through benchmarks of performance and efficiency (see Monitoring performance) or through examples of usage. Examples also illustrate the features and intricacies of the library to users. Have a look at other examples in the examples/directory for reference. Examples should demonstrate why the new functionality is useful in practice and, if possible, compare it to other methods available in scikit-learn.
  10. New features have some maintenance overhead. We expect PR authors to take part in the maintenance for the code they submit, at least initially. New features need to be illustrated with narrative documentation in the user guide, with small code snippets. If relevant, please also add references in the literature, with PDF links when possible.
  11. The user guide should also include expected time and space complexity of the algorithm and scalability, e.g. “this algorithm can scale to a large number of samples > 100000, but does not scale in dimensionality:n_features is expected to be lower than 100”.

You can also check our Code Review Guidelines to get an idea of what reviewers will expect.

You can check for common programming errors with the following tools:

Bonus points for contributions that include a performance analysis with a benchmark script and profiling output (see Monitoring performance). Also check out the How to optimize for speed guide for more details on profiling and Cython optimizations.

Note

The current state of the scikit-learn code base is not compliant with all of those guidelines, but we expect that enforcing those constraints on all new contributions will get the overall code base quality in the right direction.

Continuous Integration (CI)#

Commit message markers#

Please note that if one of the following markers appear in the latest commit message, the following actions are taken.

Note that, by default, the documentation is built but only the examples that are directly modified by the pull request are executed.

Build lock files#

CIs use lock files to build environments with specific versions of dependencies. When a PR needs to modify the dependencies or their versions, the lock files should be updated accordingly. This can be done by adding the following comment directly in the GitHub Pull Request (PR) discussion:

@scikit-learn-bot update lock-files

A bot will push a commit to your PR branch with the updated lock files in a few minutes. Make sure to tick the Allow edits from maintainers checkbox located at the bottom of the right sidebar of the PR. You can also specify the options --select-build,--skip-build, and --select-tag as in a command line. Use --help on the scriptbuild_tools/update_environments_and_lock_files.py for more information. For example,

@scikit-learn-bot update lock-files --select-tag main-ci --skip-build doc

The bot will automatically add commit message markers to the commit for certain tags. If you want to add more markers manually, you can do so using the --commit-marker option. For example, the following comment will trigger the bot to update documentation-related lock files and add the [doc build] marker to the commit:

@scikit-learn-bot update lock-files --select-build doc --commit-marker "[doc build]"

Resolve conflicts in lock files#

Here is a bash snippet that helps resolving conflicts in environment and lock files:

pull latest upstream/main

git pull upstream main --no-rebase

resolve conflicts - keeping the upstream/main version for specific files

git checkout --theirs build_tools//.lock build_tools//environment.yml
build_tools//lock.txt build_tools//requirements.txt git add build_tools//.lock build_tools/*/environment.yml
build_tools//lock.txt build_tools//*requirements.txt git merge --continue

This will merge upstream/main into our branch, automatically prioritising theupstream/main for conflicting environment and lock files (this is good enough, because we will re-generate the lock files afterwards).

Note that this only fixes conflicts in environment and lock files and you might have other conflicts to resolve.

Finally, we have to re-generate the environment and lock files for the CIs, as described in Build lock files, or by running:

python build_tools/update_environments_and_lock_files.py

Stalled pull requests#

As contributing a feature can be a lengthy process, some pull requests appear inactive but unfinished. In such a case, taking them over is a great service for the project. A good etiquette to take over is:

Stalled and Unclaimed Issues#

Generally speaking, issues which are up for grabs will have a“help wanted”. tag. However, not all issues which need contributors will have this tag, as the “help wanted” tag is not always up-to-date with the state of the issue. Contributors can find issues which are still up for grabs using the following guidelines:

Issues for New Contributors#

New contributors should look for the following tags when looking for issues. We strongly recommend that new contributors tackle “easy” issues first: this helps the contributor become familiar with the contribution workflow, and for the core devs to become acquainted with the contributor; besides which, we frequently underestimate how easy an issue is to solve!

Documentation#

We are glad to accept any sort of documentation:

SelectKBest : Select features based on the k highest scores.
SelectFpr : Select features based on a false positive rate test.

It is important to keep a good compromise between mathematical and algorithmic details, and give intuition to the reader on what the algorithm does.

You can edit the documentation using any text editor, and then generate the HTML output by following Building the documentation. The resulting HTML files will be placed in _build/html/ and are viewable in a web browser, for instance by opening the local _build/html/index.html file or by running a local server

python -m http.server -d _build/html

Building the documentation#

Before submitting a pull request check if your modifications have introduced new sphinx warnings by building the documentation locally and try to fix them.

First, make sure you have properly installed the development version. On top of that, building the documentation requires installing some additional packages:

pip install sphinx sphinx-gallery numpydoc matplotlib Pillow pandas
polars scikit-image packaging seaborn sphinx-prompt
sphinxext-opengraph sphinx-copybutton plotly pooch
pydata-sphinx-theme sphinxcontrib-sass sphinx-design
sphinx-remove-toctrees

To build the documentation, you need to be in the doc folder:

In the vast majority of cases, you only need to generate the web site without the example gallery:

The documentation will be generated in the _build/html/stable directory and are viewable in a web browser, for instance by opening the local_build/html/stable/index.html file. To also generate the example gallery you can use:

This will run all the examples, which takes a while. You can also run only a few examples based on their file names. Here is a way to run all examples with filenames containing plot_calibration:

EXAMPLES_PATTERN="plot_calibration" make html

You can use regular expressions for more advanced use cases.

Set the environment variable NO_MATHJAX=1 if you intend to view the documentation in an offline setting. To build the PDF manual, run:

Sphinx version

While we do our best to have the documentation build under as many versions of Sphinx as possible, the different versions tend to behave slightly differently. To get the best results, you should use the same version as the one we used on CircleCI. Look at thisGitHub searchto know the exact version.

Generated documentation on GitHub Actions#

When you change the documentation in a pull request, GitHub Actions automatically builds it. To view the documentation generated by GitHub Actions, simply go to the bottom of your PR page, look for the item “Check the rendered docs here!” and click on ‘details’ next to it:

../_images/generated-doc-ci.png

Testing and improving test coverage#

High-quality unit testingis a corner-stone of the scikit-learn development process. For this purpose, we use the pytestpackage. The tests are functions appropriately named, located in testssubdirectories, that check the validity of the algorithms and the different options of the code.

Running pytest in a folder will run all the tests of the corresponding subpackages. For a more detailed pytest workflow, please refer to thePull request checklist.

We expect code coverage of new features to be at least around 90%.

To test code coverage, you need to install the coverage package in addition to pytest.

  1. Run pytest --cov sklearn /path/to/tests. The output lists for each file the line numbers that are not tested.
  2. Find a low hanging fruit, looking at which lines are not tested, write or adapt a test specifically for these lines.
  3. Loop.

Monitoring performance#

This section is heavily inspired from the pandas documentation.

When proposing changes to the existing code base, it’s important to make sure that they don’t introduce performance regressions. Scikit-learn usesasv benchmarks to monitor the performance of a selection of common estimators and functions. You can view these benchmarks on the scikit-learn benchmark page. The corresponding benchmark suite can be found in the asv_benchmarks/ directory.

To use all features of asv, you will need either conda or virtualenv. For more details please check the asv installation webpage.

First of all you need to install the development version of asv:

pip install git+https://github.com/airspeed-velocity/asv

and change your directory to asv_benchmarks/:

The benchmark suite is configured to run against your local clone of scikit-learn. Make sure it is up to date:

In the benchmark suite, the benchmarks are organized following the same structure as scikit-learn. For example, you can compare the performance of a specific estimator between upstream/main and the branch you are working on:

asv continuous -b LogisticRegression upstream/main HEAD

The command uses conda by default for creating the benchmark environments. If you want to use virtualenv instead, use the -E flag:

asv continuous -E virtualenv -b LogisticRegression upstream/main HEAD

You can also specify a whole module to benchmark:

asv continuous -b linear_model upstream/main HEAD

You can replace HEAD by any local branch. By default it will only report the benchmarks that have change by at least 10%. You can control this ratio with the -f flag.

To run the full benchmark suite, simply remove the -b flag :

asv continuous upstream/main HEAD

However this can take up to two hours. The -b flag also accepts a regular expression for a more complex subset of benchmarks to run.

To run the benchmarks without comparing to another branch, use the runcommand:

asv run -b linear_model HEAD^!

You can also run the benchmark suite using the version of scikit-learn already installed in your current Python environment:

It’s particularly useful when you installed scikit-learn in editable mode to avoid creating a new environment each time you run the benchmarks. By default the results are not saved when using an existing installation. To save the results you must specify a commit hash:

asv run --python=same --set-commit-hash=

Benchmarks are saved and organized by machine, environment and commit. To see the list of all saved benchmarks:

and to see the report of a specific run:

When running benchmarks for a pull request you’re working on please report the results on github.

The benchmark suite supports additional configurable options which can be set in the benchmarks/config.json configuration file. For example, the benchmarks can run for a provided list of values for the n_jobs parameter.

More information on how to write a benchmark and how to use asv can be found in the asv documentation.

Issue Tracker Tags#

All issues and pull requests on theGitHub issue trackershould have (at least) one of the following tags:

Bug:

Something is happening that clearly shouldn’t happen. Wrong results as well as unexpected errors from estimators go here.

Enhancement:

Improving performance, usability, consistency.

Documentation:

Missing, incorrect or sub-standard documentations and examples.

New Feature:

Feature requests and pull requests implementing a new feature.

There are four other tags to help new contributors:

Good first issue:

This issue is ideal for a first contribution to scikit-learn. Ask for help if the formulation is unclear. If you have already contributed to scikit-learn, look at Easy issues instead.

Easy:

This issue can be tackled without much prior experience.

Moderate:

Might need some knowledge of machine learning or the package, but is still approachable for someone new to the project.

Help wanted:

This tag marks an issue which currently lacks a contributor or a PR that needs another contributor to take over the work. These issues can range in difficulty, and may not be approachable for new contributors. Note that not all issues which need contributors will have this tag.

Maintaining backwards compatibility#

Deprecation#

If any publicly accessible class, function, method, attribute or parameter is renamed, we still support the old one for two releases and issue a deprecation warning when it is called, passed, or accessed.

Deprecating a class or a function

Suppose the function zero_one is renamed to zero_one_loss, we add the decoratorutils.deprecated to zero_one and call zero_one_loss from that function:

from ..utils import deprecated

def zero_one_loss(y_true, y_pred, normalize=True): # actual implementation pass

@deprecated( "Function zero_one was renamed to zero_one_loss in 0.13 and will be " "removed in 0.15. Default behavior is changed from normalize=False to " "normalize=True" ) def zero_one(y_true, y_pred, normalize=False): return zero_one_loss(y_true, y_pred, normalize)

One also needs to move zero_one from API_REFERENCE toDEPRECATED_API_REFERENCE and add zero_one_loss to API_REFERENCE in thedoc/api_reference.py file to reflect the changes in API Reference.

Deprecating an attribute or a method

If an attribute or a method is to be deprecated, use the decoratordeprecated on the property. Please note that thedeprecated decorator should be placed before the property decorator if there is one, so that the docstrings can be rendered properly. For instance, renaming an attribute labels_ to classes_ can be done as:

@deprecated( "Attribute labels_ was deprecated in 0.13 and will be removed in 0.15. Use " "classes_ instead" ) @property def labels_(self): return self.classes_

Deprecating a parameter

If a parameter has to be deprecated, a FutureWarning warning must be raised manually. In the following example, k is deprecated and renamed to n_clusters:

import warnings

def example_function(n_clusters=8, k="deprecated"): if k != "deprecated": warnings.warn( "k was renamed to n_clusters in 0.13 and will be removed in 0.15", FutureWarning, ) n_clusters = k

When the change is in a class, we validate and raise warning in fit:

import warnings

class ExampleEstimator(BaseEstimator): def init(self, n_clusters=8, k='deprecated'): self.n_clusters = n_clusters self.k = k

def fit(self, X, y):
    if self.k != "deprecated":
        warnings.warn(
            "`k` was renamed to `n_clusters` in 0.13 and will be removed in 0.15.",
            FutureWarning,
        )
        self._n_clusters = self.k
    else:
        self._n_clusters = self.n_clusters

As in these examples, the warning message should always give both the version in which the deprecation happened and the version in which the old behavior will be removed. If the deprecation happened in version 0.x-dev, the message should say deprecation occurred in version 0.x and the removal will be in 0.(x+2), so that users will have enough time to adapt their code to the new behaviour. For example, if the deprecation happened in version 0.18-dev, the message should say it happened in version 0.18 and the old behavior will be removed in version 0.20.

The warning message should also include a brief explanation of the change and point users to an alternative.

In addition, a deprecation note should be added in the docstring, recalling the same information as the deprecation warning as explained above. Use the.. deprecated:: directive:

.. deprecated:: 0.13 k was renamed to n_clusters in version 0.13 and will be removed in 0.15.

What’s more, a deprecation requires a test which ensures that the warning is raised in relevant cases but not in other cases. The warning should be caught in all other tests (using e.g., @pytest.mark.filterwarnings), and there should be no warning in the examples.

Change the default value of a parameter#

If the default value of a parameter needs to be changed, please replace the default value with a specific value (e.g., "warn") and raiseFutureWarning when users are using the default value. The following example assumes that the current version is 0.20 and that we change the default value of n_clusters from 5 (old default for 0.20) to 10 (new default for 0.22):

import warnings

def example_function(n_clusters="warn"): if n_clusters == "warn": warnings.warn( "The default value of n_clusters will change from 5 to 10 in 0.22.", FutureWarning, ) n_clusters = 5

When the change is in a class, we validate and raise warning in fit:

import warnings

class ExampleEstimator: def init(self, n_clusters="warn"): self.n_clusters = n_clusters

def fit(self, X, y):
    if self.n_clusters == "warn":
        warnings.warn(
            "The default value of `n_clusters` will change from 5 to 10 in 0.22.",
            FutureWarning,
        )
        self._n_clusters = 5

Similar to deprecations, the warning message should always give both the version in which the change happened and the version in which the old behavior will be removed.

The parameter description in the docstring needs to be updated accordingly by adding a versionchanged directive with the old and new default value, pointing to the version when the change will be effective:

.. versionchanged:: 0.22 The default value for n_clusters will change from 5 to 10 in version 0.22.

Finally, we need a test which ensures that the warning is raised in relevant cases but not in other cases. The warning should be caught in all other tests (using e.g., @pytest.mark.filterwarnings), and there should be no warning in the examples.

Code Review Guidelines#

Reviewing code contributed to the project as PRs is a crucial component of scikit-learn development. We encourage anyone to start reviewing code of other developers. The code review process is often highly educational for everybody involved. This is particularly appropriate if it is a feature you would like to use, and so can respond critically about whether the PR meets your needs. While each pull request needs to be signed off by two core developers, you can speed up this process by providing your feedback.

Note

The difference between an objective improvement and a subjective nit isn’t always clear. Reviewers should recall that code review is primarily about reducing risk in the project. When reviewing code, one should aim at preventing situations which may require a bug fix, a deprecation, or a retraction. Regarding docs: typos, grammar issues and disambiguations are better addressed immediately.

Here are a few important aspects that need to be covered in any code review, from high-level questions to a more detailed check-list.

Standard replies for reviewing includes some frequent comments that reviewers may make.

Reviewing open pull requests (PRs) helps move the project forward. It is a great way to get familiar with the codebase and should motivate the contributor to keep involved in the project. [1]

Reading the existing code base#

Reading and digesting an existing code base is always a difficult exercise that takes time and experience to master. Even though we try to write simple code in general, understanding the code can seem overwhelming at first, given the sheer size of the project. Here is a list of tips that may help make this task easier and faster (in no particular order).