Rootless mode (original) (raw)

Rootless mode allows running the Docker daemon and containers as a non-root user to mitigate potential vulnerabilities in the daemon and the container runtime.

Rootless mode does not require root privileges even during the installation of the Docker daemon, as long as theprerequisites are met.

Rootless mode executes the Docker daemon and containers inside a user namespace. This is very similar touserns-remap mode, except that with userns-remap mode, the daemon itself is running with root privileges, whereas in rootless mode, both the daemon and the container are running without root privileges.

Rootless mode does not use binaries with SETUID bits or file capabilities, except newuidmap and newgidmap, which are needed to allow multiple UIDs/GIDs to be used in the user namespace.

Distribution-specific hint

Tip

We recommend that you use the Ubuntu kernel.

Note

If the system-wide Docker daemon is already running, consider disabling it:

Should you choose not to shut down the docker service and socket, you will need to use the --forceparameter in the next section. There are no known issues, but until you shutdown and disable you're still running rootful Docker.

If you installed Docker 20.10 or later withRPM/DEB packages, you should have dockerd-rootless-setuptool.sh in /usr/bin.

Run dockerd-rootless-setuptool.sh install as a non-root user to set up the daemon:

If dockerd-rootless-setuptool.sh is not present, you may need to install the docker-ce-rootless-extras package manually, e.g.,

If you do not have permission to run package managers like apt-get and dnf, consider using the installation script available athttps://get.docker.com/rootless. Since static packages are not available for s390x, hence it is not supported for s390x.

The binaries will be installed at ~/bin.

SeeTroubleshooting if you faced an error.

To remove the systemd service of the Docker daemon, run dockerd-rootless-setuptool.sh uninstall:

Unset environment variables PATH and DOCKER_HOST if you have added them to ~/.bashrc.

To remove the data directory, run rootlesskit rm -rf ~/.local/share/docker.

To remove the binaries, remove docker-ce-rootless-extras package if you installed Docker with package managers. If you installed Docker withhttps://get.docker.com/rootless (Install without packages), remove the binary files under ~/bin:

Daemon

The systemd unit file is installed as ~/.config/systemd/user/docker.service.

Use systemctl --user to manage the lifecycle of the daemon:

To launch the daemon on system startup, enable the systemd service and lingering:

Starting Rootless Docker as a systemd-wide service (/etc/systemd/system/docker.service) is not supported, even with the User= directive.

To run the daemon directly without systemd, you need to run dockerd-rootless.sh instead of dockerd.

The following environment variables must be set:

Remarks about directory paths:

Client

You need to specify either the socket path or the CLI context explicitly.

To specify the socket path using $DOCKER_HOST:

To specify the CLI context using docker context:

Rootless Docker in Docker

To run Rootless Docker inside "rootful" Docker, use the docker:<version>-dind-rootlessimage instead of docker:<version>-dind.

The docker:<version>-dind-rootless image runs as a non-root user (UID 1000). However, --privileged is required for disabling seccomp, AppArmor, and mount masks.

Expose Docker API socket through TCP

To expose the Docker API socket through TCP, you need to launch dockerd-rootless.shwith DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="-p 0.0.0.0:2376:2376/tcp".

Expose Docker API socket through SSH

To expose the Docker API socket through SSH, you need to make sure $DOCKER_HOSTis set on the remote host.

Routing ping packets

On some distributions, ping does not work by default.

Add net.ipv4.ping_group_range = 0 2147483647 to /etc/sysctl.conf (or/etc/sysctl.d) and run sudo sysctl --system to allow using ping.

Exposing privileged ports

To expose privileged ports (< 1024), set CAP_NET_BIND_SERVICE on rootlesskit binary and restart the daemon.

Or add net.ipv4.ip_unprivileged_port_start=0 to /etc/sysctl.conf (or/etc/sysctl.d) and run sudo sysctl --system.

Limiting resources

Limiting resources with cgroup-related docker run flags such as --cpus, --memory, --pids-limitis supported only when running with cgroup v2 and systemd. SeeChanging cgroup version to enable cgroup v2.

If docker info shows none as Cgroup Driver, the conditions are not satisfied. When these conditions are not satisfied, rootless mode ignores the cgroup-related docker run flags. SeeLimiting resources without cgroup for workarounds.

If docker info shows systemd as Cgroup Driver, the conditions are satisfied. However, typically, only memory and pids controllers are delegated to non-root users by default.

To allow delegation of all controllers, you need to change the systemd configuration as follows:

Note

Delegating cpuset requires systemd 244 or later.

Limiting resources without cgroup

Even when cgroup is not available, you can still use the traditional ulimit andcpulimit, though they work in process-granularity rather than in container-granularity, and can be arbitrarily disabled by the container process.

For example:

Unable to install with systemd when systemd is present on the system

rootlesskit cannot detect systemd properly if you switch to your user via sudo su. For users which cannot be logged-in, you must use the machinectl command which is part of the systemd-container package. After installing systemd-container switch to myuser with the following command:

Where myuser@ is your desired username and @ signifies this machine.

Errors when starting the Docker daemon

[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted

This error occurs mostly when the value of /proc/sys/kernel/unprivileged_userns_clone is set to 0:

To fix this issue, add kernel.unprivileged_userns_clone=1 to/etc/sysctl.conf (or /etc/sysctl.d) and run sudo sysctl --system.

[rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: no space left on device

This error occurs mostly when the value of /proc/sys/user/max_user_namespaces is too small:

To fix this issue, add user.max_user_namespaces=28633 to/etc/sysctl.conf (or /etc/sysctl.d) and run sudo sysctl --system.

[rootlesskit:parent] error: failed to setup UID/GID map: failed to compute uid/gid map: No subuid ranges found for user 1001 ("testuser")

This error occurs when /etc/subuid and /etc/subgid are not configured. SeePrerequisites.

could not get XDG_RUNTIME_DIR

This error occurs when $XDG_RUNTIME_DIR is not set.

On a non-systemd host, you need to create a directory and then set the path:

Note

You must remove the directory every time you log out.

On a systemd host, log into the host using pam_systemd (see below). The value is automatically set to /run/user/$UID and cleaned up on every logout.

systemctl --user fails with "Failed to connect to bus: No such file or directory"

This error occurs mostly when you switch from the root user to a non-root user with sudo:

Instead of sudo -iu <USERNAME>, you need to log in using pam_systemd. For example:

The daemon does not start up automatically

You need sudo loginctl enable-linger $(whoami) to enable the daemon to start up automatically. SeeUsage.

iptables failed: iptables -t nat -N DOCKER: Fatal: can't open lock file /run/xtables.lock: Permission denied

This error may happen with an older version of Docker when SELinux is enabled on the host.

The issue has been fixed in Docker 20.10.8. A known workaround for older version of Docker is to run the following commands to disable SELinux for iptables:

docker pull errors

docker: failed to register layer: Error processing tar file(exit status 1): lchown : invalid argument

This error occurs when the number of available entries in /etc/subuid or/etc/subgid is not sufficient. The number of entries required vary across images. However, 65,536 entries are sufficient for most images. SeePrerequisites.

docker: failed to register layer: ApplyLayer exit status 1 stdout: stderr: lchown : operation not permitted

This error occurs mostly when ~/.local/share/docker is located on NFS.

A workaround is to specify non-NFS data-root directory in ~/.config/docker/daemon.json as follows:

docker run errors

docker: Error response from daemon: OCI runtime create failed: ...: read unix @->/run/systemd/private: read: connection reset by peer: unknown.

This error occurs on cgroup v2 hosts mostly when the dbus daemon is not running for the user.

To fix the issue, run sudo apt-get install -y dbus-user-session or sudo dnf install -y dbus-daemon, and then relogin.

If the error still occurs, try running systemctl --user enable --now dbus (without sudo).

--cpus, --memory, and --pids-limit are ignored

This is an expected behavior on cgroup v1 mode. To use these flags, the host needs to be configured for enabling cgroup v2. For more information, seeLimiting resources.

Networking errors

This section provides troubleshooting tips for networking in rootless mode.

Networking in rootless mode is supported via network and port drivers in RootlessKit. Network performance and characteristics depend on the combination of network and port driver you use. If you're experiencing unexpected behavior or performance related to networking, review the following table which shows the configurations supported by RootlessKit, and how they compare:

Network driver Port driver Net throughput Port throughput Source IP propagation No SUID Note
slirp4netns builtin Slow Fast ✅ ❌ ✅ Default in a typical setup
vpnkit builtin Slow Fast ✅ ❌ ✅ Default when slirp4netns isn't installed
slirp4netns slirp4netns Slow Slow ✅ ✅
pasta implicit Slow Fast ✅ ✅ ✅ Experimental; Needs pasta version 2023_12_04 or later
lxc-user-nic builtin Fast ✅ Fast ✅ ❌ ❌ Experimental
bypass4netns bypass4netns Fast ✅ Fast ✅ ✅ ✅ Note: Not integrated to RootlessKit as it needs a custom seccomp profile

For information about troubleshooting specific networking issues, see:

docker run -p fails with cannot expose privileged port

docker run -p fails with this error when a privileged port (< 1024) is specified as the host port.

When you experience this error, consider using an unprivileged port instead. For example, 8080 instead of 80.

To allow exposing privileged ports, seeExposing privileged ports.

Ping doesn't work

Ping does not work when /proc/sys/net/ipv4/ping_group_range is set to 1 0:

For details, seeRouting ping packets.

IPAddress shown in docker inspect is unreachable

This is an expected behavior, as the daemon is namespaced inside RootlessKit's network namespace. Use docker run -p instead.

--net=host doesn't listen ports on the host network namespace

This is an expected behavior, as the daemon is namespaced inside RootlessKit's network namespace. Use docker run -p instead.

Network is slow

Docker with rootless mode usesslirp4netns as the default network stack if slirp4netns v0.4.0 or later is installed. If slirp4netns is not installed, Docker falls back toVPNKit. Installing slirp4netns may improve the network throughput.

For more information about network drivers for RootlessKit, seeRootlessKit documentation.

Also, changing MTU value may improve the throughput. The MTU value can be specified by creating ~/.config/systemd/user/docker.service.d/override.conf with the following content:

And then restart the daemon:

docker run -p does not propagate source IP addresses

This is because Docker in rootless mode uses RootlessKit's builtin port driver by default, which doesn't support source IP propagation. To enable source IP propagation, you can:

The pasta network driver is experimental, but provides improved throughput performance compared to the slirp4netns port driver. The pasta driver requires Docker Engine version 25.0 or later.

To change the RootlessKit networking configuration:

  1. Create a file at ~/.config/systemd/user/docker.service.d/override.conf.
  2. Add the following contents, depending on which configuration you would like to use:
    • slirp4netns
    • pasta network driver with implicit port driver
  3. Restart the daemon:

For more information about networking options for RootlessKit, see:

Tips for debugging

Entering into dockerd namespaces

The dockerd-rootless.sh script executes dockerd in its own user, mount, and network namespaces.

For debugging, you can enter the namespaces by runningnsenter -U --preserve-credentials -n -m -t <span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mo stretchy="false">(</mo><mi>c</mi><mi>a</mi><mi>t</mi></mrow><annotation encoding="application/x-tex">(cat </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:1em;vertical-align:-0.25em;"></span><span class="mopen">(</span><span class="mord mathnormal">c</span><span class="mord mathnormal">a</span><span class="mord mathnormal">t</span></span></span></span>XDG_RUNTIME_DIR/docker.pid).