Quick Reference | EDM Documentation (original) (raw)
This document provides a brief introduction to the most common set of commands that users will need when using EDM from the command line (CLI).
EDM is a multi-language environment manager, but we tried hard to present a CLI that is simple to use for the simple cases of people wanting to just get a python environment with Enthought binary packages
Setting up a default configuration
To create a default configuration, use the configure subcommand:
The configure command will create a configuration in ~/.edm.yaml.
The configure command will ask you whether you want to use an API token. You should generally say yes. An API token should be kept secret like a password, but they are revocable and hence more secure to store on disk. You can also use a token as created by the hatcher tool.
Look at the configuration help topic for more information about settings:
Installing and updating packages
A simple way to install packages is through the install command, e.g. to install ipython and numpy:
$ edm install ipython numpy
This will install ipython and numpy in the default environment. You can list installed packages with the list command:
`$ edm list
List of packages installed in the current environment
`
Use the update command to update a package, e.g. to update numpy:
To update all installed packages to the latest version and build number:
To create a subshell with this ipython activated, you should use the shell command:
You can exit this shell at any time by typing exit.
Using the EDM with Visual Studio Code
We recommend using Microsoft’s Python (IntelliSense) extension, and the other Microsoft extensions which it automatically installs (currently several for Jupyter and PyLance.)
The virtual environments created by EDM are recognized by VS Code. In many cases VS Code will automatically detect your EDM environments and make them available. If they are not, open the Settings menu, and find the setting “Python: Venv Path” (you can quickly find this by typing “venv” in the search bar). Enter in the path to the edm environments folder (usually~/.edm/envs/).
Once you have the interpreters available, to use them, go to the file you want to load the environment for, open the command palette, and choose “Select Python Interpreter”. You should see a list of all your EDM environments which can be filtered by typing in the search bar. Select one and the IntelliSense should now be aware of packages installed in that environment.
For a general guide on using virtual environments with VS Code, consultthe VS Code documentation.
Creating new environments
By default, basic EDM commands interact with the default environment. EDM supports the installation of multiple, isolated environments. EDM environments allow you to have multiple self-contained python installations. Typical usecases include having test and production environments, testing multiple python versions (2 or 3), implementations (cpython or pypy), etc.
There are multiple ways to create a new environment. The simple one is through the environments subcommand. To create a new environment calledtutorial:
$ edm environments create tutorial
This will install a new python interpreter, which will be completely independent of your other environments. In particular, if for some reason you break this test environment by installing development packages, it will not interfere with your existing environments.
You can list every environment with the command edm environments list:
$ edm environments list ... tutorial
A environment by itself is not very useful, you often want to install packages in it. You can install packages as usual with the edm install command, with an additional argument to specify the environment:
$ edm install ipython --environment tutorial
The above command will automatically create a new environment “tutorial” if EDM detects the environment does not exist yet.
Most commands acting on environments take an –environment parameter to specify a non-default environment.
Note
The environment name must be a valid filename that is supported by the filesystem. For example, on a POSIX filesystem, the environment name cannot contain / (slash) and the \x00 (null) characters.
Activating environments
To create a new shell with the newly created environment configured as the default, you should use the shell command with the environment name:
$ edm shell --environment tutorial
If you prefer using activation scripts ala virtualenv, this is also possible, as each environment contains virtualenv-like activation scripts.
At this point, typing python will bring you the python from the specified environment tutorial. Once you are in an environment shell, every subsequent command will use that environment by default. For example, edm install will automatically install packages in this environment.
At any point, you can check the currently activated environment by simply typing edm, and look at the first line:
`$ edm
Should print something like:
EDM environment manager (currently activated environment: 'tutorial') ... `
To exit the shell with this environment activated, simply type exit:
If you used the virtualenv-like activation scripts instead of the EDM shell command, you should type deactivate instead.
If you now type edm, you should see:
`$ edm
Should print something like:
Enthought Deployment Manager (no environment active - use "edm shell" to activate) ... `
Once an environment is activated, it becomes the default environment for every command taking an optional –environment option. E.g.:
`$ edm shell --environment tutorial
This will install ipython in the tutorial environment
$ edm install ipython `
Removing environments
Environments may be removed with the environments subcommand followed byremove. To remove an environment called tutorial:
$ edm environments remove tutorial
The default behavior in edm version 1.12.0 and later is to always remove the environment’s directory, even if it is “frozen” or contains files that are not under edm’s control. To keep the earlier versions’ behavior (not removing the directory if such files exist), use the option --no-purge.
Cloning environments
Cloning/replication of EDM environments is possible using environment bundles.
Environment Bundles
An EDM environment bundle is essentially a JSON serialization of an environment’s metadata. Bundles are the most reliable way of sharing EDM environments as they not only allow the user to replicate the set of installed dependencies but also support persistence for constraint modifiers, the list of manually installed packages, and the runtime version and implementation.
The following command creates a bundle from the BASE_ENV environment and stores it in bundled_env.bundle:
$ edm environments export BASE_ENV -f bundled_env.bundle
In the above example bundled_env.bundle will be overwritten without warning if it already exists.
Environment bundles are not meant to be user-editable. Making changes to the bundle files can have unexpected consequences and is strongly discouraged.
Enthought convention is to use the .bundle extension for environment bundle files.
The following command creates an environment called NEW_ENV from thebundled_env.bundle bundle:
$ edm environments import NEW_ENV -f bundled_env.bundle
The platform on which a bundle is imported must be the same as the one on which the bundle was created. For example, a bundle created on Windows cannot be used for creating an environment on Mac OS.
Installing alternative runtimes
Environments are created from so-called runtimes, which are packaged interpreters. By default, EDM will create environments from a runtime with the latest python version available. EDM allows to install different versions of python (e.g. python 2.7`).
To see the available runtimes on your platform, simply do:
$ edm available-runtimes cpython 2.7.10+1 enthought/free cpython 3.6.0+1 enthought/free
To create a 2.7 python environment, for example, simply do:
$ edm environments create test-python3 --version 2.7
Installing requirements
The edm install command can be used to install packages in a given environment. For instance:
$ edm install numpy -e my_env
would install the latest possible version of the numpy package in an environment called my_env. If no such environment exists, EDM will create one for you.
By default, edm install creates an environment with the latest CPython 3 runtime executable. The user can override this behavior using the--version and --implementation options.
Specifying version numbers
All available versions for a package can be determined using theedm search command. To request a specific version of a package we can simply do:
$ edm install "pkg == x.y.z-a"
Here pkg is the package name, x.y.z is the version, and a is the build number. EDM understands several relational operators that can be used in specfiying package versions as detailed by the examples in the table below.
| Operator | Sample Usage | Meaning |
|---|---|---|
| == | A == x.y.z-a | install exactly the given version of A |
| ^= | A ^= x.y.z | install latest build of the given version |
| <= | A <= x.y.z-a | install latest build of a version earlier than or equal to the given version |
| < | A < x.y.z-a | install latest build of an earlier version than the given version |
| >= | A >= x.y.z-a | install latest build of a version later than or equal to the given version |
| > | A > x.y.z-a | install latest build of a later version than the given version |
The <=, <, >=, and > operators accept partial version numbers. x.y.z-a, x.y.z, x.y and x are all valid version specifications for these operators.
EDM will always install the package with the latest possible version and build number that satisfies the user criteria while maintaining consistency in the target environment.
Constraint modifiers
EDM is currently equipped with the following constraint modifiers:
| Modifier | Action |
|---|---|
| --allow-older | permit an earlier version of the package |
| --allow-newer | permit a later version of the package |
| --allow-any | permit any version of the package |
Improper use of constraint modifiers can result in broken environments.
Constraint modifiers are special flags that can be used to change the EDM dependency solver’s behavior. They provide a way for the user to explicitly override the dependency resolution logic, thereby facilitating side-by-side installation of otherwise incompatible package versions.
The syntax for using constraint modifiers with an install command is as follows:
$ edm install <requirement> <modifier> <name>
Here requirement is the package that is to be installed, and modifieris one of the constraint modifiers from the table at the beginning of this section, that is to be applied to all requirements named name. Note that while requirement can specify partial or complete version information, name must contain the package name only.
Another way to use constraint modifiers is via the constraintscommand group. For instance, to add a constraint modifier we can simply do:
$ edm constraints add <modifier> <name>
Here, modifier and name retain their meaning from the previous example. In addition to add, the constraints command group provides functionality for listing (list) and deleting (forget) modifiers as well.
Constraint modifiers are persistent. EDM will remember the user-defined constraints on a particular package until it is either uninstalled from the environment, or the constraint is explicitly removed or modified by the user.
Example usage
Consider an environment with numpy-1.6.0 installed:
`$ edm list
Packages in environment 'edm'
prefix: '/Users/username/.edm/envs/edm'
mkl 10.3-1 enthought/free numpy 1.6.0-5 enthought/free `
Attempting to install the latest scipy version prompts an update of numpy:
`$ edm install "scipy ^= 0.17.1" Fetching indices for package repositories.. done
The following packages will be installed:
scipy 0.17.1-1 14.71 MiBThe following packages will be updated:
mkl 10.3-1 --> 11.1.4-1 78.26 MiB
numpy 1.6.0-5 --> 1.10.4-1 3.57 MiBDo you want to continue ? [Y/n]: `
Constraint modifiers can be used to suppress this behavior:
`$ edm install "scipy ^= 0.17.1" --allow-older mkl,numpy Fetching indices for package repositories.. done The following constraint modifications are applied to all requirements:
allow older : mkl, numpy
The following packages will be installed:
scipy 0.17.1-1 14.71 MiBDo you want to continue ? [Y/n]: `
Packages and repacking
Packages managed by EDM are based on the EGG file format, with the addition of Enthought-specific metadata.
The metadata format used in Enthought eggs is publicly available as part of the okonomiyaki library (okonomiyaki.file_formats.EggMetaData). EDM supports metadata version up to 1.3.
When users want to add new packages to an EDM-managed environment, the packages must be repacked to provide appropriate metadata to EDM. The repacking operation is done using edm repack-egg or edm repack-wheel, depending on the original package format. These commands require a standard set of inputs:
- Enthought-specific metadata for the package. The metadata includes a list of the direct dependencies of the package, and optionally may list files to add to the package (e.g. license files, data files).
- A build number which is required to differentiate updates to a package based on dependencies, rather than version changes. For example, package A version 1.0.0 build 1 depends on package B version 2.3.4. When package B is updated to 2.3.5 and no new version of package A is needed, package A will need to be rebuilt as build 2, specifying the updated dependency.
It is important to use the repack operations by specifying the EDM environment that must be used for executing the repacking operation. A Python 2 egg can’t be repacked using a Python 3 environment!
Once repacked, eggs can be uploaded to the Enthought Deployment Server using the hatcher utility.
Dependencies
While the EDM dependency solver supports complex expressions, the current Enthought egg format supports only a restricted set of dependency expressions in the metadata.
During the repack operations, users must provide a list of package dependencies, which can optionally include exact version and build number restrictions, such as:
packages = [ "package_A 1.9.0-1", "package_B 2.3.5", "package_C 4.2-3", "package_D 5", "package_C" ]
packages must be a valid Python list. Each item is a package name string, optionally including a version specifier. When no version is provided, the dependency is an open dependency on any version of the package. When a package version is specified, that exact version must be found (for example, package version 1.9.0 does not satisfy a dependency specification of 1.9). When a build number is also specified (following a dash), then that exact build number must also be found, in addition to the exact version number.
It is recommended to provide a version number without a build for each package dependency in order to constrain the dependency set to expected working solutions. It is not currently possible to express dependency ranges.
Providing metadata to repack operations
The metadata file is a text file that contains a set of valid Python statements. The two primary variables that can be defined in this file are:
- packages: the list of dependencies of this packages (see Dependencies)
- add_files: the files to be added to the Enthought packages while repacking the original content. “add_files” must be list of tuples, each tuple containing three values: a directory relative to the egg, a regular expression to match existing files within that directory, and a target archive directory, relative to the package installation subdirectory of site-packages.
By default, metadata are expected to be found in a file named endist.datwithin the current working directory. Users can change the location of the metadata file using the -f, –metadata-overrides-path option of the command line arguments.
Egg repacking
A setuptools egg can be repacked directly into an Enthought egg using:
` $ edm repack-egg --help Usage: edm repack-egg [OPTIONS] EGG
Repack a setuptools egg into an Enthought egg.
Options: -e, --environment TEXT The environment to repack the egg for -d, --output-directory DIRECTORY Directory where to write the repacked egg. Default to egg directory. -b, --build INTEGER Build number. Default to 1. -a, --architecture, --platform TEXT Target architecture (e.g. 'rh6-x86_64'). repack-egg will try to guess if not specified. --python TEXT PEP425 python tag. repack-egg will try to guess if not specified. --abi TEXT PEP425 abi tag. repack-egg will try to guess if not specified. -m, --metadata-version TEXT Metadata version to use when generating the egg. Default to 1.3 -f, --metadata-overrides-path PATH Path to an endist.dat-like file to override metadata. Default to current working directory. --extract-dependencies Extract dependencies from the setuptools egg. Default to False. -h, --help Show this message and exit. `
Wheel repacking
A wheel can be repacked into an Enthought egg using:
`$ edm repack-wheel --help Usage: edm repack-wheel [OPTIONS] WHEEL
Repack a wheel into an Enthought egg.
Options: -e, --environment TEXT The environment to repack the wheel for -d, --output-directory DIRECTORY Directory where to write the repacked egg. Defaul to the wheel directory -b, --build INTEGER Build number. Default to 1. -p, --platform TEXT Target architecture (e.g. 'rh6-x86_64'). repack-wheel will try to guess it if not specified. --python TEXT PEP425 python tag. repack-wheel will try to guess it if not specified. --abi TEXT PEP425 abi tag. repack-wheel will try to guess it if not specified. -m, --metadata-version TEXT Metadata version to use when generating the egg. Default to 1.3. -f, --metadata-overrides-path PATH Path to an endist.dat-like file to override metadata. Default to current working directory. --extract-dependencies Extract dependencies from the wheel. Default to False. -h, --help Show this message and exit. `
endist.dat file format
The endist.dat file is a configuration file used within the Enthought environment to customize metadata for Python eggs, which are distribution packages that contain all the files necessary to execute a program.
Purpose
The primary function of the endist.dat file is to allow users to override or add specific metadata when repacking eggs using the repack-egg orrepack-wheel tools. This customization is crucial when converting standard packages into a format that is compatible with Enthought’s edm, which may require additional metadata for dependencies, versioning, or inclusion of extra files.
Structure
The endist.dat file is a Python script that is executed (exec-ed) during the repacking process. It supports several variables that can be defined to customize the package metadata.
Variables
- `name`: Overrides the package name.
name = "custom_package_name"
- `version`: Overrides the package version.
- `build`: Sets a specific build number for the package.
4. `packages`: Specifies runtime dependencies. This is a list of strings where each string is a package with its required version. Valid specifications for the package version is either no version number (meaning compatible with any version of the package), a given version (“numpy 1.7.1”) or a given version with build number (“numpy 1.7.1-2”). There is currently no way to provide version ranges (>, >=, etc.)
packages = [ "dependency1 1.0", "dependency2 2.0" ]
5. `add_files`: Allows the inclusion of additional files not present in the original egg. This is defined as a list of tuples, where each tuple contains the directory path, a regular expression to match files, and the target directory within the egg.
add_files = [("source_dir", "*.txt", "EGG-INFO/docs")]
Usage Example
Below is an example of a endist.dat file:
`# endist.dat
name = "my_custom_package" version = "0.1.0" build = 1
packages = [ "numpy 1.20.0", "scipy", ]
add_files = [("docs", "*.md", "EGG-INFO/docs")] `
In this example: - The package name is set to my_custom_package. - The version is 0.1.0. - The build number is 1. - The package depends on specific versions of numpy and scipy. - Markdown documentation files from the docs directory are included in theEGG-INFO/docs directory within the egg.
Notes
- The endist.dat file is a powerful tool but should be used carefully since it directly influences how the egg is repacked and subsequently installed via edm.
- It is primarily intended for advanced users who need to customize package distribution within the Enthought environment.
EDM bundles
EDM bundles allow users to reproduce an environment into a different machine, e.g. to share an environment with somebody else. Bundle files are a JSON serialization of the environment’s metadata. Enthought convention is to give these bundle files the .bundle extension.To create a bundle from the environment foo, simply do as follows:
$ edm env export -f my-env.bundle foo
This will create the file my-env.bundle, which contains all the information required by EDM to create an environment with the exact same packages as foo:
# This will create a new environment called bar $ edm env import -f my-env.bundle bar
Exported bundles are platform-specific, and cannot be used to reproduce a similar environment on a different platform, e.g. you cannot use anosx-x86_64 bundle to create an environment on a Linux machine. See below for how to create bundles for other platforms.
Bundles are designed for exact reproduction: if edm envs import is successful, it is guaranteed that the newly created environment will have the exact same packages as the exported environment, even if the EDM configuration is different:
- The bundle stores the sha256 checksum and the source repository of each asset (runtime, packages).
- The environment creation will fail if the same assets are not found in the specified repository.
This protects against one repository’s package set overwriting another’s (e.g. your environment contains numpy 1.12.3-2 but multiple repositories have this exact version of numpy).
Data bundles
By default, bundles only contain metadata, that is they contain only a description of the exported environment. In this case, it is expected that EDM running on the machine importing the bundle will have access to the packages on the deployment server.
Starting with EDM 1.6.0, one can also create “data bundles”, which contain everything, including packages, needed to re-create the environment on a new machine. In this case, EDM import will not require access to the server, and the bundle can be imported on a disconnected machine. Enthought convention is to give data bundle files the .zbundle extension both to distinguish them from standard bundles, and to indicate that these files are zipped.
To create a data bundle, simply pass the --include-assets (or shortcut -i) in the export subcommand:
# my-env.zbundle will now contain every package from foo $ edm env export -i -f my-env.zbundle foo
Importing works exactly as before. EDM will automatically detect the data bundle, and work in disconnected mode.
Creating bundles from scratch
In the examples above bundles are created by exporting an existing runtime environment. Starting with EDM 1.10.0 it is now possible to generate a bundle from a list of requirements and other metadata, even if the target platform the bundle is created for is different than the platform where it is created. For example:
edm bundle generate --platform osx-x86_64 --version 3.6 \ -f my-env.bundle matplotlib-2.0.0-5 numpy-1.13.3-4
This command will generate a bundle which can be used on OSX to create an environment with Python 3.6, numpy and matplotlib, and their dependencies. Please see edm bundle generate --help for information about additional command-line options.
EDM applications
EDM applications are normal EDM Python environments that include the Enthought Application Manager (EAM) package. These applications are distributed as normal EDM environment bundles, which specify the environment’s requirements. These bundles are stored in the bundlerepositories of an Enthought Deployment Server (EDS) and can be downloaded and installed using EDM’s application command group or the EDM GUI frontend.
- Bundle repositories are supported by EDS version 0.17.0 and above.
- The
appgroup is an alias forapplication. - To learn more about bundles please see EDM bundles.
- Bundles can be uploaded using the hatcher tool.
List available applications
The available application bundles can be listed using the list-available subcommand of the application command group:
By default, only applications in the enthought/free repository are available and listed. To make available applications from other repositories, you will need to modify the edm configuration file. SeeAdvanced configuration.
Install application
Application bundles that are available in EDS can be installed using the install subcommand of the application command group.
$ edm app install mayavi_demo
EDM will download the bundle and create an EDM environment to install the Python runtime and the application’s package dependencies. EDM will ask the EAM tool to configure the application (e.g. create any necessary shortcuts).
- If an environment already exists, then the command will exit with an error. To override this behavior please use the
--forceoption. - By default, the name of the application bundle will be used as the EDM environment name. To override this default, please use the
-e/--environmentoption. - Please see
edm app install --helpfor information about additional command-line options.
Start application
The installed application can be started directly from the EDM command line interface using the start subcommand of the application command group. Options may be passed in by placing them after two dashes.
$ edm app start mayavi_demo -- --option-one -o
- Installed applications can be also started it using an application shortcut, if these were created during installation.
- The argument passed to the command is the EDM environment name where the application has been installed. By default this is the same as the application bundle name, but this is not necessarily true (see note on the
installcommand).
Remove application
To remove an installed application, we use the remove subcommand of the application command group. EDM will ask the EAM tool to remove the application configuration (e.g. desktop shortcuts) and then will delete the associated EDM Python environment.
$ edm app remove mayavi_demo
Please see edm app remove --help for information about additional command-line options.
Executing scripts in a specific environment
When you have multiple edm-managed environments, and want to execute a Python script or tool, you need to take care in specifying which environment it will be executed in.
Let’s assume that the script is to be executed in an environment named tutorial:
Use the run command
The run command will temporarily update the environment variables to activate the desired edm Python environment, then execute the specified command:
$ edm run -e tutorial – python myscript.py
Use the shell command
The shell command will create a new shell as a child process and activate the edm environment. You can then execute the script in that environment:
$ edm shell -e tutorial $ python myscript.py $ exit
Because this starts a child shell, you use the exit command when you are finished. This returns you to the parent shell where you were before typing the edm shell command.
Specifying the Python executable
An alternative that does not manipulate the environment variables is to call the Python executable directly.
To find the prefix of an edm Python environment, use edm prefix:
This will print out the folder path of the desired edm environment. Using the prefix we can then execute our script by calling python directly:
$ <prefix-to-the-python-environment>/bin/python myscript.py
or on Windows systems:
$ <prefix-to-the-python-environment>/python.exe myscript.py
Environment variables such as PATH are not modified with this method.
Use the activate script
All edm environments provide an activate script, which updates the environment variables and activates the Python environment:
$ source <prefix-to-the-python-environment>/bin/activate $ python myscript.py $ deactivate
or on Windows systems:
$ <prefix-to-the-python-environment>/Scripts/activate.bat $ python myscript.py $ deactivate
Because this modifies the environment within the current shell, use the deactivate command (not exit) when you are finished. This resets the environment to where it was beforeactivate.
Advanced configuration
A user’s EDM configuration is stored as a YAML file in ~/.edm.yaml``(default) or pointed to by the ``EDM_CONFIG environment variable or the-c <config_file_path command line option. The configuration file can be used to customize EDM’s command line behavior. Among other things, the user can:
- Specify the Enthought Deployment Server (EDS)instance to connect to.
- Provide credentials for authenticating to EDS.
- Specify which repositories in EDS should be queried and searched for eggs.
- Change the location of EDM runtimes and environments.
Using token based authentication
EDM supports the use of simple authentication which stores your username and password in plain-text in the .edm.yaml configuration file. While this is acceptable for temporary setups, it poses a significant security risk in the long run. We recommend using API tokens which are revokable, and do not divulge any password. To obtain a token, use the create-token subcommand and enter your username and password when prompted:
$ edm create-token --inject
The --inject flag instructs EDM to replace the existing authentication in your configuration with the generated token.
If there is already a token present that EDM can find, it will not prompt you for a username and password, and instead just use the token to authorize the request. EDM can pull the token from an argument (-t), the .edm.yaml file or the environment variable EDM_API_TOKEN. If for some reason you have multiple, different tokens in each of those locations, they are listed in the order EDM will check them, and the first one it finds will be used.
Examples
A simple configuration, close to the one created by EDM by default:
`# Change this if you are using on-site deployment servers store_url: "https://packages.enthought.com"
Packages' repositories
repositories:
- enthought/free
- enthought/gpl
Directory to use to cache downloaded files.
files_cache: "~/.cache/edm" `
Using token based authentication (recommended):
`# authentication is required with on-site deployment servers, or to
access commercial packages on packages.enthought.com
authentication: api_token: "Insert your actual token here" `
Using username/password authentication:
authentication: kind: simple username: "your enthought account, usually your registered email" password: "associated password"
Configuration properties
Listed below are the EDM properties that be controlled through the configuration file:
`# Path where all environments and runtimes will be stored
Default: ~/.edm/
root: string
Path to the logfile where log messages will be written to. If null, no
logging data is written to the filesystem.
Default: null
log_file:
- string or null
Maximum number of tries to retry connecting to a remote server or re-fetching data with invalid checksum
Default: 0
max_retries:
- integer
The name of the shell to use by default in edm shell.
Default:
- cmd for Windows.
- the SHELL environment variable or /bin/sh for POSIX.
shell:
- string
Whether to override system default PS1 in EDM sub-shells
Default: True
override_ps1:
- boolean
URLs for basic proxy authentication
Default: null
proxies: https: http://proxyserver.com:80 http: http://proxyserver.com:80
Whether to check SSL CA certificate or not
Default: True
verify_ssl: boolean
Custom CA certificate bundle used for verifying SSL
certificates. If null, the standard root CA certificates are
used, except on Windows, the system certificate store will also be used to
provide CA certificates.
Default: null
ca_certificates:
- string
The url of the store to connect to
Default: https://packages.enthought.com
store_url:
- string
Path where downloaded files will be stored
Default: ~/.cached_eggs/{PLATFORM}
files_cache:
- string
List of repository names in organization/repository format
Default: enthought/free
repositories:
- array
List of repository names in organization/repository format
Default: enthought/free
runtime_repositories:
- array
List of repository names in organization/repository format
Default: enthought/free
bundle_repositories:
- array
Authentication credentials can be 'simple' or 'token-based' (see example configurations above)
authentication:
- object `
Simple proxy configuration
By default, if you don’t use the on-site Enthought Deployment Server, EDM will pick up the standard proxy variables HTTPS_PROXY. For example, in a UNIX shell:
`$ export HTTPS_PROXY=http://proxy.com:3128
EDM should use the configured proxy at that point
$ edm check-connection `
In a Windows CMD prompt (not Powershell), it will look as follows:
C:\>set HTTPS_PROXY=http://proxy.com:3128 C:\>edm check-connection
If you use a server which is not using TLS (i.e. the server starts withhttp and not https), you need to define HTTP_PROXY instead. Note that it is the protocol used by the packages server that matters for theHTTP_PROXY vs HTTPS_PROXY choice, not the proxy scheme.
If the https proxy requires a CA certificate provided by your organization, a CA certificate bundle can be provided in either theREQUESTS_CA_BUNDLE or CURL_CA_BUNDLE environment variable:
$ export REQUESTS_CA_BUNDLE=/etc/ssl/bundle.pem
If not specified, the default port is 3128, i.e.HTTPS_PROXY=http://proxy.com and HTTPS_PROXY=http://proxy.com:3128are equivalent.
Proxy setup from the CLI
EDM also accepts the --proxy option to setup the proxy from the command line. If given, it overrides HTTP(S)_PROXY environment variables. In its simple form, you simply give the proxy url:
$ edm --proxy http://proxy.com:3218 check-connection
If you want to specify the protocol the proxy should be used for, you should use the form protocol:url, e.g.:
`# Equivalent to setting up HTTPS_PROXY=http://proxy.com:3128 $ edm --proxy https:http://proxy.com:3128
Equivalent to setting up HTTP_PROXY=http://proxy.com:3128
$ edm --proxy http:http://proxy.com:3128 `
If required for the https proxy server, a custom CA certificate can also be provided on the command line:
$ edm --proxy https:http://proxy.com:3128 --ca-certificates /etc/ssl/bundle.pem
If not specified, the default protocol is https, i.e.edm --proxy http://proxy.com andedm --proxy https:http://proxy.com are equivalent.
If the edm.yaml file and the environment both have proxy information in them, the settings in the environment variable will be used.
Proxy setup from configuration file
If you have to use a proxy every time you use EDM, passing the proxy information from the CLI is not convenient. You can setup proxies in your file~/.edm.yaml configuration as shown below:
`proxies: https: http://proxy.com:3128 # If you use a server that is not using TLS, you should use this: http: http://proxy.com:3128
If the HTTPS proxy server requires custom SSL certificates provided by
your organization, specify the file here:
ca_certificates: /etc/ssl/bundle.pem
If the HTTP(S) proxy server requires that SSL certificates NOT be checked,
then use this:
verify_ssl: False `
Proxies requiring authentication
EDM currently only supports basic authentication for proxies. If your credentials are “john” and “doe” for your username and password, you would specify the proxy URL as follows:
proxies: https: http://john:doe@proxy.com:3128
This form is also supported through the HTTP(S)_PROXY environment variables and the --proxy option.
The passwords stored in edm.yaml are not kept in plaintext. The exact mechanism of storage will depend on the user’s operating system. On Windows and MacOS, the system keystore will be used. In that case, the value in the “password” field will not contain the password and if this value gets captured by a log there is no cause for concern. On Linux systems, the password is encrypted, but not stored, which means it is less secure. It is unlikely that someone who simply had the encrypted string would be able to crack it, but Linux users should take care not to capture their proxy settings in logging.
If for some reason EDM cannot use one of these storage methods, it can be forced to store the password in plaintext, but this will require explicit confirmation from the user and comes with a warning message.
If a username and password have been written in plaintext in the edm.yaml file the next time edm checks the proxy configuration it will attempt to store the password in the keychain after which it will replace the password with either the encrypted string or the keychain reference.
Example: plaintext password is converted to keychain:
`proxies: https: http://john:doe@proxy.com:3128
becomes
proxies: https: http://john:@proxy.com:3128 `
Note
Username and password characters that are not alphanumeric or - (hyphen) or _ (underscore) or . (dot) or ~ (tilde) should be percent encoded using its numeric value. For example, the @ character would be percent encoded as %26 if using ASCII encoding. See RFC 3986 for details.
Proxy configuration using configure proxy
EDM can create or update proxy configurations in an EDM configuration file using the edm configure proxy command. This command allows users to store sensitive credentials in the configuration file without leaving them in plaintext.
Configure a proxy with manual password entry:
$ edm configure proxy -h https://proxy.address.com:1234 -u username
You will then be prompted for a password.
Configure a proxy entering password through stdin:
$ cat password.txt | edm configure proxy -h https://proxy.address.com:1234 -u username --password-stdin
Configure a proxy through interactive prompts:
$ edm configure proxy Enter the URL for this proxy, including the port and protocol. If no port is provided, it will default to 3128: http://proxy.address.com:1234 Does the proxy require credentials? [y/N]: Y Please enter the proxy username: username Please enter the proxy password: **** Proxy configuration updated.
Not all platforms provide an appropriate system credential store for EDM to use. If the user’s platform does not support secure credential storage, EDM will display a warning and store the credentials in plaintext.
Listing proxies
To see a list of the currently configured proxies for EDM, run:
$ edm configure proxy --list-proxies
This will print a list to console in the following format:
$ edm configure proxy --list-proxies { "protocol_version": 1, "msg_type": "configure.proxy.list", "payload": { "http": { "host": <host>, "port": <port>. "scheme": "https", "username": <username>, "password": <password> } }
The passwords printed by proxy list are not plaintext, but they are not safe to log or store. These will simply be the base 64 encoded versions of the password string.
This list will not include any proxy that has been set via environment variables.
Environment variables used by EDM
- EDM_API_TOKEN: specify the token to use for authenticating to the server
- EDM_CA_CERTIFICATES: Path to CA Certificate bundle for certificate verification
- EDM_CONFIG: path to the EDM configuration file.
- EDM_DEBUG: if set to non zero value, do not hide tracebacks
- EDM_INSECURE: if set to non zero value, disables SSL certificate verification
- EDM_LOG_FILE: specify the path to a logfile where logging output will be written
- EDM_MAX_RETRIES: maximum retries for network connection
- EDM_NO_DEBUG: if set to non zero value, force hiding tracebacks
- EDM_PROXY: Proxy settings (see Proxy setup from the CLI)
- EDM_QUIET: if set to non zero value, output will be minimal
- EDM_ROOT_DIRECTORY: specify the root directory to manipulate
- EDM_SHELL: specify the shell to be used in edm run, e.g. ‘zsh’ or ‘bash’ on Unix, ‘cmd’ or ‘powershell’ on Windows
Configuring repositories in edm.yaml
The edm.yaml file has three possible headings that can contain lists of repositories; repositories, bundle_repositories, andruntime_repositories. repositories are the repositories that EDM will search for packages, bundle_repositories for bundles, andruntime_repositories for runtimes respectively. These entries can be configured using any text editing program, but EDM also supports adding and removing entries using the command line. The base command is edm configure repositories, and it has three sub commands, list, add, andremove.
Listing repositories
To see the list of repositories currently in your edm.yaml file:
`$ edm configure repositories list repositories:
- enthought/free `
To see the list of runtime repositories currently in your edm.yaml file:
`$ edm configure repositories list --runtime runtime_repositories:
- enthought/free `
To see the list of bundle repositories currently in your edm.yaml file:
`$ edm configure repositories list --bundle bundle_repositories:
- enthought/free `
To see a combined list of all repositories:
`$ edm configure repositories list --list-all repositories:
- enthought/free bundle_repositories:
- enthought/repo1
- enthought/repo2 runtime_repositories:
- enthought/repo3
- enthought/repo4 `
Adding repositories
Adding repositories follows the same basic command structure as the listcommand. Simply specify a list of repositories to add. Do note that repository names must follow the format <organization name>/<repository name>.
To add a repository:
$ edm configure repositories add enthought/repo1 enthought/repo2 Updated repositories. Added repositories: enthought/repo1, enthought/repo2
To add a repository to the list of runtime repositories:
$ edm configure repositories add enthought/repo1 enthought/repo2 --runtime Updated runtime_repositories. Added repositories: enthought/repo1, enthought/repo2
To add a repository to the list of bundle repositories:
$ edm configure repositories add enthought/repo1 enthought/repo2 --bundle Updated bundle_repositories. Added repositories: enthought/repo1, enthought/repo2
Removing repositories
Removing repositories follows the same basic command structure as the listcommand. Simply specify a list of repositories to remove. Do note that the command will not remove any repositories if any of the repositories you are trying to remove cannot be found.
To remove repositories:
$ edm configure repositories remove enthought/repo1 enthought/repo2 Updated repositories. Removed repositories: enthought/repo1, enthought/repo2
To remove repositories from the list of runtime repositories:
$ edm configure repositories remove enthought/repo1 enthought/repo2 --runtime Updated runtime_repositories. Removed repositories: enthought/repo1, enthought/repo2
To remove repositories from the list of bundle repositories:
$ edm configure repositories remove enthought/repo1 enthought/repo2 --bundle Updated bundle_repositories. Removed repositories: enthought/repo1, enthought/repo2