ExUnit.DocTest — ExUnit v1.18.3 (original) (raw)

Extract test cases from the documentation.

Doctests allow us to generate tests from code examples found in @moduledoc and @doc attributes. To do this, invoke thedoctest/1 macro from within your test case and ensure your code examples are written according to the syntax and guidelines below.

Syntax

Every new test starts on a new line, with an iex> prefix. Multiline expressions can be used by prefixing subsequent lines with either ...> (recommended) or iex>.

The expected result should start the line after the iex>and ...> line(s) and be terminated by a newline.

Examples

To run doctests include them in an ExUnit case with a doctest macro:

defmodule MyModuleTest do
  use ExUnit.Case, async: true
  doctest MyModule
end

The doctest macro loops through all functions and macros defined in MyModule, parsing their documentation in search of code examples.

A very basic example is:

iex> 1 + 1
2

Expressions on multiple lines are also supported:

iex> Enum.map([1, 2, 3], fn x ->
...>   x * 2
...> end)
[2, 4, 6]

Multiple results can be checked within the same test:

iex> a = 1
1
iex> a + 1
2

If you want to keep any two tests separate, add an empty line between them:

iex> a = 1
1

iex> a + 1 # will fail with a `undefined variable "a"` error
2

If you don't want to assert for every result in a doctest, you can omit the result. You can do so between expressions:

iex> pid = spawn(fn -> :ok end)
iex> is_pid(pid)
true

As well as at the end:

iex> Mod.do_a_call_that_should_not_raise!(...)

This is useful when the result is something variable (like a PID in the example above) or when the result is a complicated data structure and you don't want to show it all, but just parts of it or some of its properties.

Similarly to IEx you can use numbers in your "prompts":

iex(1)> [1 + 2,
...(1)>  3]
[3, 3]

This is useful in two cases:

being able to refer to specific numbered scenarios
copy-pasting examples from an actual IEx session

You can also select or skip functions when callingdoctest. See the documentation on the :except and :only options below for more information.

Opaque types

Some types' internal structures are kept hidden and instead show a user-friendly structure when inspected. The idiom in Elixir is to print those data types in the format #Name<...>. Because those values are treated as comments in Elixir code due to the leading# sign, they require special care when being used in doctests.

Imagine you have a map that contains a DateTime and is printed as:

%{datetime: #DateTime<2023-06-26 09:30:00+09:00 JST Asia/Tokyo>}

If you try to match on such an expression, doctest will fail to compile. There are two ways to resolve this.

The first is to rely on the fact that doctest can compare internal structures as long as they are at the root. So one could write:

iex> map = %{datetime: DateTime.from_naive!(~N[2023-06-26T09:30:00], "Asia/Tokyo")}
iex> map.datetime
#DateTime<2023-06-26 09:30:00+09:00 JST Asia/Tokyo>

Whenever a doctest starts with "#Name<", doctest will perform a string comparison. For example, the above test will perform the following match:

inspect(map.datetime) == "#DateTime<2023-06-26 09:30:00+09:00 JST Asia/Tokyo>"

Alternatively, since doctest results are actually evaluated, you can have the DateTime building expression as the doctest result:

iex> %{datetime: DateTime.from_naive!(~N[2023-06-26T09:30:00], "Asia/Tokyo")}
%{datetime: DateTime.from_naive!(~N[2023-06-26T09:30:00], "Asia/Tokyo")}

The downside of this approach is that the doctest result is not really what users would see in the terminal.

Exceptions

You can also showcase expressions raising an exception, for example:

iex(1)> raise "some error"
** (RuntimeError) some error

Doctest will look for a line starting with ** ( and it will parse it accordingly to extract the exception name and message. The exception parser will consider all following lines part of the exception message until there is an empty line or there is a new expression prefixed with iex>. Therefore, it is possible to match on multiline messages as long as there are no empty lines on the message itself.

When not to use doctest

In general, doctests are not recommended when your code examples contain side effects. For example, if a doctest prints to standard output, doctest will not try to capture the output.

Similarly, doctests do not run in any kind of sandbox. So any module defined in a code example is going to linger throughout the whole test suite run.