cpython: d22ef969cb82 (original) (raw)

Mercurial > cpython

changeset 89629:d22ef969cb82

Close #19407: New installation & distribution guides - based on pip and other PyPA tools - includes references to the new Python Packaging User Guide where appropriate (and the relevant section is at least partially filled in) - started new FAQ sections - both guides aim to introduce users to basic open source concepts if they aren't aware of them - existing guides have been relocated (now linked from the distutils docs) rather then removed, since there is some needed material that has yet to be relocated to the distutils docs as a reference for the legacy formats [#19407]

Nick Coghlan ncoghlan@gmail.com
date	Thu, 13 Mar 2014 22:13:45 +1000
parents	69d503b12432
children	f5be4ea5b43e
files	Doc/conf.py Doc/contents.rst Doc/distributing/index.rst Doc/distutils/index.rst Doc/extending/index.rst Doc/install/index.rst Doc/installing/index.rst Doc/library/distutils.rst Doc/library/ensurepip.rst Doc/tools/sphinxext/indexcontent.html Doc/tutorial/whatnow.rst Doc/using/venv-create.inc Doc/using/windows.rst Doc/whatsnew/3.4.rst Misc/NEWS
diffstat	15 files changed, 460 insertions(+), 33 deletions(-)[+] [-] Doc/conf.py 4 Doc/contents.rst 12 Doc/distributing/index.rst 143 Doc/distutils/index.rst 6 Doc/extending/index.rst 42 Doc/install/index.rst 6 Doc/installing/index.rst 211 Doc/library/distutils.rst 21 Doc/library/ensurepip.rst 2 Doc/tools/sphinxext/indexcontent.html 8 Doc/tutorial/whatnow.rst 4 Doc/using/venv-create.inc 5 Doc/using/windows.rst 4 Doc/whatsnew/3.4.rst 19 Misc/NEWS 6

line wrap: on

line diff

--- a/Doc/conf.py +++ b/Doc/conf.py @@ -120,11 +120,11 @@ latex_font_size = '10pt' latex_documents = [ ('c-api/index', 'c-api.tex', 'The Python/C API', _stdauthor, 'manual'),

('distutils/index', 'distutils.tex',

('distributing/index', 'distributing.tex', 'Distributing Python Modules', _stdauthor, 'manual'), ('extending/index', 'extending.tex', 'Extending and Embedding Python', _stdauthor, 'manual'),

('install/index', 'install.tex',

('installing/index', 'installing.tex', 'Installing Python Modules', _stdauthor, 'manual'), ('library/index', 'library.tex', 'The Python Library Reference', _stdauthor, 'manual'),

--- a/Doc/contents.rst +++ b/Doc/contents.rst @@ -11,8 +11,8 @@ library/index.rst extending/index.rst c-api/index.rst

distutils/index.rst
install/index.rst

distributing/index.rst
installing/index.rst howto/index.rst faq/index.rst glossary.rst @@ -21,3 +21,11 @@ bugs.rst copyright.rst license.rst +

+.. include legacy packaging docs in build + +.. toctree::

🥷 +
distutils/index.rst
install/index.rst

new file mode 100644 --- /dev/null +++ b/Doc/distributing/index.rst @@ -0,0 +1,143 @@ +.. _distributing-index: + +###############################

Distributing Python Modules +###############################

+ +:Email: distutils-sig@python.org + + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their software +available for other Python developers to use under open source license terms. + +This allows Python users to share and collaborate effectively, benefiting +from the solutions others have already created to common (and sometimes +even rare!) problems, as well as potentially contributing their own +solutions to the common pool. + +This guide covers the distribution part of the process. For a guide to +installing other Python projects, refer to the +:ref:installation guide <installing-index>. + +.. note:: +

For corporate and other institutional users, be aware that many
organisations have their own policies around using and contributing to
open source software. Please take such policies into account when making
use of the distribution and installation tools provided with Python. +

+ +Key terms +========= + +* the Python Package Index <https://pypi.python.org/pypi>__ is a public

repository of open source licensed packages made available for use by
other Python users +* the `Python Packaging Authority
http://packaging.python.org/en/latest/future.html`__ are the group of
developers and documentation authors responsible for the maintenance and
evolution of the standard packaging tools and the associated metadata and
file format standards. They maintain a variety of tools, documentation
and issue trackers on both GitHub <https://github.com/pypa> and
BitBucket <https://bitbucket.org/pypa/>__. +* distutils is the original build and distribution system first added to
the Python standard library in 1998. While direct use of distutils is
being phased out, it still laid the foundation for the current packaging
and distribution infrastructure, and it not only remains part of the
standard library, but its name lives on in other ways (such as the name
of the mailing list used to coordinate Python packaging standards
development). + + +Open source licensing and collaboration +======================================= + +In most parts of the world, software is automatically covered by copyright. +This means that other developers require explicit permission to copy, use, +modify and redistribute the software. + +Open source licensing is a way of explicitly granting such permission in a +relatively consistent way, allowing developers to share and collaborate +efficiently by making common solutions to various problems freely available. +This leaves many developers free to spend more time focusing on the problems +that are relatively unique to their specific situation. + +The distribution tools provided with Python are designed to make it +reasonably straightforward for developers to make their own contributions +back to that common pool of software if they choose to do so. + +The same distribution tools can also be used to distribute software within +an organisation, regardless of whether that software is published as open +source software or not. + + +Installing the tools +==================== + +The standard library does not include build tools that support modern +Python packaging standards, as the core development team has found that it +is important to have standard tools that work consistently, even on older +versions of Python. + +The currently recommended build and distribution tools can be installed +using pip:: +
pip install setuptools wheel twine

+ + +Reading the guide +================= + +The Python Packaging User Guide covers the various key steps and elements +involved in creating a project + +* Project structure +* Building and packaging the project +* Uploading the project to the Python Package Index_ + +.. _Project structure: [](#l3.103)

http://packaging.python.org/en/latest/tutorial.html#creating-your-own-project[](#l3.104) +.. _Building and packaging the project: [](#l3.105)
http://packaging.python.org/en/latest/tutorial.html#building-packaging-your-project[](#l3.106) +.. _Uploading the project to the Python Package Index: [](#l3.107)
http://packaging.python.org/en/latest/tutorial.html#building-uploading-your-project-to-pypi[](#l3.108) +

+ +How do I...? +============ + +These are quick answers or links for some common tasks. + +... choose a name for my project? +--------------------------------- + +This isn't an easy topic, but here are a few tips: + +* check the Python Package Index to see if the name is already in use +* check popular hosting sites like GitHub, BitBucket, etc to see if there

is already a project with that name +* check what comes up in a web search for the name you're considering +* avoid particularly common words, especially ones with multiple meanings,
as they can make it difficult for users to find your software when
searching for it + + +... create and distribute binary extensions? +-------------------------------------------- + +This is actually quite a complex topic, with a variety of alternatives +available depending on exactly what you're aiming to achieve. See the +Python Packaging User Guide for more information and recommendations. + +.. seealso:: +
`Python Packaging User Guide: Binary Extensions
http://packaging.python.org/en/latest/extensions.html`__ +

+.. other topics: +

Once the Development & Deployment part of PPUG is fleshed out, some of
those sections should be linked from new questions here (most notably,
we should have a question about avoiding depending on PyPI that links to
http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)[](#l3.147)

--- a/Doc/distutils/index.rst +++ b/Doc/distutils/index.rst @@ -1,8 +1,8 @@ .. _distutils-index: -###############################

Distributing Python Modules -###############################

+##############################################

Distributing Python Modules (Legacy version) +##############################################

:Authors: Greg Ward, Anthony Baxter :Email: distutils-sig@python.org

--- a/Doc/extending/index.rst +++ b/Doc/extending/index.rst @@ -21,14 +21,31 @@ Python) that give the language its wide For a detailed description of the whole Python/C API, see the separate :ref:c-api-index. -.. note:: + +Recommended third party tools +============================= + +This guide only covers the basic tools for creating extensions provided +as part of this version of CPython. Third party tools like Cython, +cffi, SWIG and Numba offer both simpler and more sophisticated +approaches to creating C and C++ extensions for Python. + +.. seealso::

This guide only covers the basic tools for creating extensions provided
as part of this version of CPython. Third party tools may offer simpler
alternatives. Refer to the `binary extensions section
https://python-packaging-user-guide.readthedocs.org/en/latest/extensions.html`__
in the Python Packaging User Guide for more information.

Python Packaging User Guide: Binary Extensions <https://packaging.python.org/en/latest/extensions.html>_

 The Python Packaging User Guide not only covers several available[](#l5.25)

 tools that simplify the creation of binary extensions, but also[](#l5.26)

 discusses the various reasons why creating an extension module may be[](#l5.27)

 desirable in the first place.[](#l5.28)

+ +Creating extensions without third party tools +============================================= + +This section of the guide covers creating C and C++ extensions without +assistance from third party tools. It is intended primarily for creators +of those tools, rather than being a recommended way to create your own +C extensions. .. toctree:: :maxdepth: 2 @@ -38,4 +55,17 @@ For a detailed description of the whole newtypes.rst building.rst windows.rst + +Embedding the CPython runtime in a larger application +===================================================== + +Sometimes, rather than creating an extension that runs inside the Python +interpreter as the main application, it is desirable to instead embed +the CPython runtime inside a larger application. This section covers +some of the details involved in doing that successfully. + +.. toctree::

:maxdepth: 2
:numbered: + embedding.rst

--- a/Doc/install/index.rst +++ b/Doc/install/index.rst @@ -2,9 +2,9 @@ .. _install-index: -*****************************

Installing Python Modules - +***************

Installing Python Modules (Legacy version) +******************************************** :Author: Greg Ward

new file mode 100644 --- /dev/null +++ b/Doc/installing/index.rst @@ -0,0 +1,211 @@ +.. highlightlang:: none + +.. _installing-index: + +*****************************

Installing Python Modules +***************************** + +:Email: distutils-sig@python.org + +As a popular open source development project, Python has an active +supporting community of contributors and users that also make their software +available for other Python developers to use under open source license terms. + +This allows Python users to share and collaborate effectively, benefiting +from the solutions others have already created to common (and sometimes +even rare!) problems, as well as potentially contributing their own +solutions to the common pool. + +This guide covers the installation part of the process. For a guide to +creating and sharing your own Python projects, refer to the +:ref:distribution guide <distributing-index>. + +.. note:: +
For corporate and other institutional users, be aware that many
organisations have their own policies around using and contributing to
open source software. Please take such policies into account when making
use of the distribution and installation tools provided with Python. +

+ +Key terms +========= + +* pip is the preferred installer program. Starting with Python 3.4, it

is included by default with the Python binary installers.
installed into virtual environments +* a virtual environment is a semi-isolated Python environment that allows
packages to be installed for use by a particular application, rather than
being installed system wide +* pyvenv is the standard tool for creating virtual environments, and has
been part of Python since Python 3.3. Starting with Python 3.4, it
defaults to installing pip into all created virtual environments +* the Python Package Index <https://pypi.python.org/pypi>__ is a public
repository of open source licensed packages made available for use by
other Python users +* the `Python Packaging Authority
http://packaging.python.org/en/latest/future.html`__ are the group of
developers and documentation authors responsible for the maintenance and
evolution of the standard packaging tools and the associated metadata and
file format standards. They maintain a variety of tools, documentation
and issue trackers on both GitHub <https://github.com/pypa> and
BitBucket <https://bitbucket.org/pypa/>__. +* distutils is the original build and distribution system first added to
the Python standard library in 1998. While direct use of distutils is
being phased out, it still laid the foundation for the current packaging
and distribution infrastructure, and it not only remains part of the
standard library, but its name lives on in other ways (such as the name
of the mailing list used to coordinate Python packaging standards
development). + + +Basic usage +=========== + +The standard packaging tools are all designed to be used from the command +line. For Windows users, the examples below assume that the option to +adjust the system PATH environment variable was selected when installing +Python. For Linux users, the command to install into the system version of +Python 3 is likely to be pip3 rather than pip. + +The following command will install the latest version of a module and its +dependencies from the Python Package Index:: +
pip install SomePackage

+ +It's also possible to specify an exact or minimum version directly on the +command line:: +

pip install SomePackage==1.0.4 # specific version
pip install 'SomePackage>=1.0.4' # minimum version

+ +Normally, if a suitable module is already installed, attempting to install +it again will have no effect. Upgrading existing modules must be requested +explicitly:: +

pip install --upgrade SomePackage

+ +More information and resources regarding pip and its capabilities can be +found in the Python Packaging User Guide <http://packaging.python.org>__. + +pyvenv has its own documentation at :ref:scripts-pyvenv. Installing +into an active virtual environment uses the commands shown above. + +.. seealso:: +

`Python Packaging User Guide: Installing Python packages
http://packaging.python.org/en/latest/tutorial.html#installing-python-packages`__

+ + +How do I ...? +============= + +These are quick answers or links for some common tasks. + +... install pip in versions of Python prior to Python 3.4? +-------------------------------------------------------------- + +Python only started bundling pip with Python 3.4. For earlier versions, +pip needs to be "bootstrapped" as described in the Python Packaging +User Guide. + +.. seealso:: +

`Python Packaging User Guide: Installing the Tools
http://packaging.python.org/en/latest/tutorial.html#installing-the-tools`__ +

+ +.. installing-per-user-installation: + +... install packages just for the current user? +----------------------------------------------- + +Passing the --user option to pip install will install a package +just for the current user, rather than for all users of the system. + + +... install scientific Python packages? +--------------------------------------- + +A number of scientific Python packages have complex binary dependencies, and +aren't currently easy to install using pip directly. At this point in +time, it will often be easier for users to install these packages by +other means[](#l7.138) +<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>__ +rather than attempting to install them with pip. + +.. seealso:: +

`Python Packaging User Guide: Installing Scientific Packages
http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages`__ +

+ +... work with multiple versions of Python installed in parallel? +---------------------------------------------------------------- + +On Linux, Mac OS X and other POSIX systems, use the versioned Python commands +in combination with the -m switch to run the appropriate copy of +pip:: +

python2 -m pip install SomePackage # default Python 2
python2.7 -m pip install SomePackage # specifically Python 2.7
python3 -m pip install SomePackage # default Python 3
python3.4 -m pip install SomePackage # specifically Python 3.4 +

+(appropriately versioned pip commands may also be available) + +On Windows, use the py Python launcher in combination with the -m +switch:: +

py -2 -m pip install SomePackage # default Python 2
py -2.7 -m pip install SomePackage # specifically Python 2.7
py -3 -m pip install SomePackage # default Python 3
py -3.4 -m pip install SomePackage # specifically Python 3.4 +

+.. other questions: +

Once the Development & Deployment part of PPUG is fleshed out, some of
those sections should be linked from new questions here (most notably,
we should have a question about avoiding depending on PyPI that links to
http://packaging.python.org/en/latest/deployment.html#pypi-mirrors-and-caches)[](#l7.175) +

+ +Common installation issues +========================== + +Installing into the system Python on Linux +------------------------------------------ + +On Linux systems, a Python installation will typically be included as part +of the distribution. Installing into this Python installation requires +root access to the system, and may interfere with the operation of the +system package manager and other components of the system if a component +is unexpectedly upgraded using pip. + +On such systems, it is often better to use a virtual environment or a +per-user installation when installing packages with pip. + + +Installing binary extensions +---------------------------- + +Python has typically relied heavily on source based distribution, with end +users being expected to compile extension modules from source as part of +the installation process. + +With the introduction of support for the binary wheel format, and the +ability to publish wheels for at least Windows and Mac OS X through the +Python Package Index, this problem is expected to diminish over time, +as users are more regularly able to install pre-built extensions rather +than needing to build them themselves. + +Some of the solutions for installing scientific software[](#l7.207) +<http://packaging.python.org/en/latest/platforms.html#installing-scientific-packages>__ +that is not yet available as pre-built wheel files may also help with +obtaining other binary extensions without needing to build them locally. + +.. seealso:: +

`Python Packaging User Guide: Binary Extensions
http://packaging.python.org/en/latest/extensions.html`__

--- a/Doc/library/distutils.rst +++ b/Doc/library/distutils.rst @@ -12,14 +12,15 @@ additional modules into a Python install 100%-pure Python, or may be extension modules written in C, or may be collections of Python packages which include modules coded in both Python and C. - -User documentation and API reference are provided in another document: - -.. seealso:: +Most Python users will not want to use this module directly, but instead +use the cross-version tools maintained by the Python Packaging Authority. +Refer to the Python Packaging User Guide <http://packaging.python.org>_ +for more information.

:ref:distutils-index

 The manual for developers and packagers of Python modules.  This describes[](#l8.17)

 how to prepare :mod:`distutils`\ -based packages so that they may be[](#l8.18)

 easily installed into an existing Python installation.  It also contains[](#l8.19)

 instructions for end-users wanting to install a distutils-based package,[](#l8.20)

```
 :ref:`install-index`.[](#l8.21)
```

+For the benefits of packaging tool authors and users seeking a deeper +understanding of the details of the current packaging and distribution +system, the legacy :mod:distutils based user documentation and API +reference remain available: + +* :ref:install-index +* :ref:distutils-index

--- a/Doc/library/ensurepip.rst +++ b/Doc/library/ensurepip.rst @@ -28,7 +28,7 @@ when creating a virtual environment) or .. seealso::

:ref:install-index

:ref:installing-index The end user guide for installing Python packages :pep:453: Explicit bootstrapping of pip in Python installations

--- a/Doc/tools/sphinxext/indexcontent.html +++ b/Doc/tools/sphinxext/indexcontent.html @@ -16,14 +16,14 @@

<a class="biglink" href="{{ pathto("howto/index") }}">Python HOWTOs
in-depth documents on specific topics

 <p class="biglink"><a class="biglink" href="{{ pathto("installing/index") }}">Installing Python Modules</a><br/>[](#l10.7)

    <span class="linkdescr">installing from the Python Package Index &amp; other sources</span></p>[](#l10.8)

 <p class="biglink"><a class="biglink" href="{{ pathto("distributing/index") }}">Distributing Python Modules</a><br/>[](#l10.9)

    <span class="linkdescr">publishing modules for installation by others</span></p>[](#l10.10)
 <p class="biglink"><a class="biglink" href="{{ pathto("extending/index") }}">Extending and Embedding</a><br/>[](#l10.11)
    <span class="linkdescr">tutorial for C/C++ programmers</span></p>[](#l10.12)
 <p class="biglink"><a class="biglink" href="{{ pathto("c-api/index") }}">Python/C API</a><br/>[](#l10.13)
    <span class="linkdescr">reference for C/C++ programmers</span></p>[](#l10.14)

 <p class="biglink"><a class="biglink" href="{{ pathto("install/index") }}">Installing Python Modules</a><br/>[](#l10.15)

    <span class="linkdescr">information for installers &amp; sys-admins</span></p>[](#l10.16)

 <p class="biglink"><a class="biglink" href="{{ pathto("distutils/index") }}">Distributing Python Modules</a><br/>[](#l10.17)

    <span class="linkdescr">sharing modules with others</span></p>[](#l10.18)
 <p class="biglink"><a class="biglink" href="{{ pathto("faq/index") }}">FAQs</a><br/>[](#l10.19)
    <span class="linkdescr">frequently asked questions (with answers!)</span></p>[](#l10.20)

[](#l10.21)

--- a/Doc/tutorial/whatnow.rst +++ b/Doc/tutorial/whatnow.rst @@ -21,8 +21,8 @@ the set are: and many other tasks. Skimming through the Library Reference will give you an idea of what's available. -* :ref:install-index explains how to install external modules written by other

Python users. +* :ref:installing-index explains how to install additional modules written

by other Python users.

:ref:reference-index: A detailed explanation of Python's syntax and semantics. It's heavy reading, but is useful as a complete guide to the

--- a/Doc/using/venv-create.inc +++ b/Doc/using/venv-create.inc @@ -11,6 +11,11 @@ containing a copy of the python bina Windows). It also creates an (initially empty) lib/pythonX.Y/site-packages subdirectory (on Windows, this is Lib\site-packages). +.. seealso:: +

`Python Packaging User Guide: Creating and using virtual environments
http://packaging.python.org/en/latest/tutorial.html#creating-and-using-virtual-environments`__ +

.. highlight:: none On Windows, you may have to invoke the pyvenv script as follows, if you

--- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -11,6 +11,10 @@ This document aims to give an overview of Windows-specific behaviour you should know about when using Python on Microsoft Windows. +.. XXX (ncoghlan) +

This looks rather stale to me... +

Installing Python =================

--- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -180,6 +180,9 @@ New Expected Features for Python Impleme PEP 453: Explicit Bootstrapping of PIP in Python Installations -------------------------------------------------------------- +Bootstrapping pip by default +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The new :mod:ensurepip module (defined in :pep:453) provides a standard cross-platform mechanism to bootstrap the pip installer into Python installations and virtual environments. @@ -207,6 +210,22 @@ clear and simple directions on how to in __ http://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors[](#l14.15) + +Documentation changes +~~~~~~~~~~~~~~~~~~~~~ + +As part of this change, the :ref:installing-index and +:ref:distributing-index sections of the documentation have been +completely redesigned as short getting started and FAQ documents. Most +packaging documentation has now been moved out to the Python Packaging +Authority maintained Python Packaging User Guide[](#l14.25) +<http://packaging.python.org>__ and the documentation of the individual +projects. + +However, as this migration is currently still incomplete, the legacy +versions of those guides remaining available as :ref:install-index +and :ref:distutils-index. + .. note:: To avoid conflicts between parallel Python 2 and Python 3 installations,

--- a/Misc/NEWS +++ b/Misc/NEWS @@ -59,6 +59,12 @@ Documentation

Issue #20765: Add missing documentation for PurePath.with_name() and PurePath.with_suffix(). +- Issue #19407: New package installation and distribution guides based on

the Python Packaging Authority tools. Existing guides have been retained
as legacy links from the distutils docs, as they still contain some
required reference material for tool developers that isn't recorded
anywhere else. + Tests -----