Comparing Python Command-Line Parsing Libraries – Argparse, Docopt, and Click (original) (raw)
About a year ago I began a job where building command-line applications was a common occurrence. At that time I had used argparse quite a bit and wanted to explore what other options were available.
I found that the most popular alternatives available were click and docopt. During my exploration I also found that other than each libraries “why use me” section there was not much available for a complete comparison of the three libraries. Now there is—this blog post!
If you want to, you can head directly to the source though it really won’t do much good without the comparisons and step-by-step construction presented in this article.
This article uses the following versions of the libraries:
(Ignore invoke for now, it’s a special surprise for later!)
Command-Line Example
The command-line application that we are creating will have the following interface:
python [file].py [command] [options] NAME
Basic Usage
Usage w/ Options (Flags)
This article will compare each libraries method for implementing the following features:
- Commands (
hello,goodbye) - Arguments (name)
- Options/Flags (
--greeting=<str>,--caps)
Additional features:
- Version Printing (
-v/--version) - Automated Help Messages
- Error Handling
As you would expect argparse, docopt, and click implement all of these features (as any complete command-line library would). This fact means that the actual implementation of these features is what we will compare. Each library takes a very different approach that lends to a very interesting comparison - argparse=standard, docopt=docstrings, click=decorators.
Bonus Sections
- I’ve been curious about using task-runner libraries like fabric and it’s python3 replacement invoke to create simple command-line interfaces, so I will try and put the same interface together with invoke.
- A few extra steps are needed when packaging command-line applications, so I’ll cover those steps as well!
Commands
Let’s begin by setting up the basic skeleton (no arguments or options) for each library.
Argparse
With this we now have two commands (hello and goodbye) and a built-in help message. Notice that the help message changes when run as an option on the command hello.
Docopt
With this we, again, have two commands (hello, goodbye) and a built-in help message. Notice that the help message DOES NOT change when run as an option on the hello command. In addition we do not need to explicitly specify the commands.py -h | --help in the Options section to get a help command. However, if we don’t they will not show up in the output help message as options.
Click
With this we now have two commands (hello, goodbye) and a built-in help message. Notice that the help message changes when run as an option on the hello command.
Even at this point you can see that we have very different approaches to constructing a basic command-line application. Next let’s add the NAME argument, and the logic to ouput the result from each tool.
Arguments
In this section we will be adding new logic to the same code shown in the previous section. We’ll add comments to new lines stating their purpose. Arguments (aka positional arguments) are required inputs to a command-line application. In this case we are adding a required name argument so that the tool can greet a specific person.
Argparse
To add an argument to a subcommand we use the add_argument method. And in order to execute the correct logic, when a command is called, we use the set_defaults method to set a default function. Finally we execute the default function by calling args.func(args) after we parse the arguments at runtime.
Docopt
In order to add an option, we add a <name> to the docstring. The <> are used to designate a positional argument. In order to execute the correct logic we must check if the command (treated as an argument) is True at runtime if arguments['hello']:, then call the correct function.
Note that the help message is not specific to the subcommand, rather it is the entire docstring for the program.
Click
In order to add an argument to a click command we use the @click.argument decorator. In this case we are just passing the argument name, but there are many more options some of which we’ll use later. Since we are decorating the logic (function) with the argument we don’t need to do anything to set or make a call to the correct logic.
Flags/Options
In this section we will again be adding new logic to the same code shown in the previous section. We’ll add comments to new lines stating there purpose. Options are non-required inputs that can be given to alter the execution of a command-line application. Flags are a Boolean only (True/False) subset of options. For example: --foo=bar will pass bar as the value for the foo option and --baz (if defined as a flag) will pass the value of True is the option is given, or False if not.
For this example we are going to add the --greeting=[greeting] option, and the --caps flag. The greeting option will have default values of Hello and Goodbye and allow the user to pass in a custom greeting. For example given --greeting=Wazzup the tool will respond with Wazzup, [name]!. The --caps flag will uppercase the entire response if given. For example given --caps the tool will respond with HELLO, [NAME]!.
Argparse
Docopt
Once we hit the case of adding options with defaults, we hit a snag with the basic implementation of commands in docopt. Let’s continue just to illustrate the issue.
Now, see what happens when we run the following commands:
What?! Because we can only set a single default for the --greeting option both of our Hello and Goodbye commands now respond with Hello, Kyle!. In order for us to make this work we’ll need to follow the git example docopt provides. The refactored code is shown below:
As you can see the hello|goodbye subcommands are now there own docstrings tied to the variables HELLO and GOODBYE. When the tool is executed it uses a new argument, command, to decide which to parse. Not only does this correct the problem we had with only one default, but we now have subcommand specific help messages as well.
In addition all of our new options/flags are working as well:
Click
To add the greeting and caps options we use the @click.option decorator. Again, since we have default greetings now we have pulled the logic out into a single function (def greeter(**kwargs):).
Version Option (--version)
In this section we’ll be showing how to add a --version argument to each of our tools. For simplicity we’ll just hardcode the version number to 1.0.0. Keep in mind that in a production application, you will want to pull this from the installed application. One way to achieve this is with this simple process:
A second option for determining the version would be to have an automated version-bumping software change the version number defined in the file when a new version is released. This is possible with bumpversion. But this approach is not recommended as it’s easy to get out of sync. Generally, it’s best practice to keep a version number in as few places as possible.
Since the implementation of adding a hard-coded version option is fairly simple we will use ... to denote skipped sections of the code from the last section.
Argparse
For argparse we again need to use the add_argument method, this time with the action='version' parameter and a value for version passed in. We apply this method to the root parser (instead of the hello or goodbye subparsers).
docopt
In order to add --version to docopt we add it as an option to our primary docstring. In addition we add the version parameter to our first call to docopt (parsing the primary docstring).
Click
Click provides us with a convenient @click.version_option decorator. To add this we decorate our greet function (main @click.group function).
Improving Help (-h/--help)
The final step to completing our application is to improve the help documentation for each of the tools. We’ll want to make sure that we can access help with both -h and --help and that each argument and option have some level of description.
Argparse
By default argparse provides us with both -h and --help so we don’t need to add anything for that. However our current help documentation for the subcommands is lacking information on what --caps and --greeting do and what the name argument is.
In order to add more information we use the help parameter of the add_argument method.
Now when we provide the help flag we get a much more complete result:
Docopt
This section is where docopt shines. Because we wrote the documentation as the definition of the command-line interface itself, we already have the help documentation completed. In addition -h and --help are already provided.
Click
Adding help documentation to click is very similar to argparse. We need to add the help parameter to all of our @click.option decorators.
However, click DOES NOT provide us -h by default. We need to use the context_settings parameter to override the default help_option_names.
Now the click help documentation is complete.
Error Handling
Error handling is an important part of any application. This section will explore the default error handling of each application and implement additional logic if needed. We’ll explore three error cases:
- Not enough required arguments given.
- Invalid options/flags given.
- A flag with a value is given.
Argparse
Not very exciting, as argparse handles all of our error cases out of the box.
Docopt
Unfortunatly, we have a bit of work to get docopt to an acceptable minimum level of error handling. The reccomended method for validation in docopt is the schema module. *Make sure to install - pip install schema. In addition they provide a very basic validation example. The following is our application with schema validation:
With this validation in place we now get some error messages.
While these messages are not very descriptive and may be hard to debug for larger applications, it’s better than no validation at all. The schema module does provide other mechanisms for adding more descriptive error messages but we won’t cover those here.
Click
Just like argparse, click handles error input by default.
With that, we have completed the construction of the command-line application we set out to build. Before we conclude let’s take a look at another possible option.
Invoke
Can we use invoke, a simple task running library, to build the greeter command-line application? Let’s find out!
To start let’s begin with the simplest version of the greeter:
tasks.py
With this very simple file we get a two tasks and very minimal help. From the same directory as tasks.py we get the following results:
Now let’s add in our options/flags - --greeting and --caps. In addition, we can pull out the greeting logic into it’s own function, just as we did with the other tools.
Now we actually have the complete interface we designated in the beginning!
Help Documentation
In order to compete with argparse, docopt, and click, we’ll also need to be able to add complete help documentation. Luckily this is also available in invoke by using the help parameter of the @task decorator and adding docstrings to the decorated functions.
Version Option
Implementing a --version option is not quite as simple and comes with a caveat. The basics are that we will add version=False as an option to each of the tasks that calls a new print_version function if True. In order to make this work we cannot have any positional arguments without defaults or we get:
Also note that we are calling --version on our commands hello and goodbye because invoke itself has a version command:
The completed implementation of a version command follows:
Now we are able to ask invoke for the version of our tool:
Conclusion
To review, let’s take a look at the final version of each of the tools we created.
Argparse
Docopt
Click
Invoke
My Recomendation
Now, to get this out of the way, my personal go-to library is click. I have been using it on large, multi-command, complex interfaces for the last year. (Credit goes to @kwbeam for introducing me to click). I prefer the decorator approach and think it lends a very clean, composable interface. That being said, let’s evaluate each option fairly.
Argparse
Arparse is the standard library (included with Python) for creating command-line utilities. For that fact alone, it is arguably the most used of the tools examined here. Argparse is also very simple to use as lots of magic (implicit work that happens behind the scenes) is used to construct the interface. For example, both arguments and options are defined using the add_arguments method and argparse figures out which is which behind the scenes.
Docopt
If you think writing documentation is great, docopt is for you! In addition docopt has implementations for many other languages - meaning you can learn one library and use it across many languages. The downside of docopt is that it is very structured in the way you have to define your command-line interface. (Some might say this is a good thing!)
Click
I’ve already said that I really like click and have been using it in production for over a year. I encourage you to read the very complete Why Click? documentation. In fact, that documentation is what inspired this blog post! The decorator style implementation of click is very simple to use and since you are decorating the function you want executed, it makes it very easy to read the code and figure out what is going to be executed. In addition, click supports advanced features like callbacks, command nesting, and more. Click is based on a fork of the now deprecated optparse library. To learn more about Click, check out Click and Python: Build Extensible and Composable CLI Apps.
Invoke
Invoke surprised me in this comparison. I thought that a library designed for task execution might not be able to easily match full command-line libraries - but it did! That being said, I would not recommend using it for this type of work as you will certainly run into limitations for anything more complex than the example presented here.
Bonus: Packaging
Since not everyone is packaging up there python source with setuptools (or other solutions), we decided not to make this a core component of the article. In addition, we don’t want to cover packaging as a complete topic. If you want to learn more about packaging with setuptools go here or with conda go here or you can read my previous blog post on conda packaging. What we will cover here is how to use the entry_points option to make a command-line application an executable command on install.
Entry Point Basics
An entry_point is essentially a map to a single function in your code that will be given a command on your system PATH. An entry_point has the form - command = package.module:function
The best way to explain this is to just look at our click example and add an entry point.
Packaging Click Commands
Click makes packaging simple, as by default we are calling a single function when we execute our program:
In addition to the rest of the setup.py (not covered here), we would add the following to create an entry_point for our click application.
Assuming the following directory structure-
greeter/ ├── greet │ ├── __init__.py │ └── cli.py <-- the same as our final.py └── setup.py
-we will create the following entry_point:
When a user installs the package created with this entry_point, setuptools will create the following executable script (called greet) and place it on the PATH of the user’s system.
After installation the user will now be able to run the following:
Packaging Argparse Commands
The only thing we need to do differently from click is to pull all of the application initialization into a single function that we can call in our entry_point.
This:
Becomes:
Now we can use the same pattern for the entry_point we defined for click.
Packaging Docopt Commands
Packaging docopt commands requires the same process as argparse.
This:
Becomes: