GitHub - mvdan/sh: A shell parser, formatter, and interpreter with bash support; includes shfmt (original) (raw)

sh

A shell parser, formatter, and interpreter. Supports POSIX Shell, Bash, andmksh. Requires Go 1.23 or later.

Quick start

To parse shell scripts, inspect them, and print them out, see the syntax package examples.

For high-level operations like performing shell expansions on strings, see theshell package examples.

shfmt

go install mvdan.cc/sh/v3/cmd/shfmt@latest

shfmt formats shell programs. See canonical.sh for a quick look at its default style. For example:

For more information, see its manpage, which can be viewed directly as Markdown or rendered with scdoc.

Packages are available on Alpine, Arch, Debian, Docker, Fedora, FreeBSD,Homebrew, MacPorts, NixOS, OpenSUSE, Scoop, Snapcraft, Void and webi.

gosh

go install mvdan.cc/sh/v3/cmd/gosh@latest

Proof of concept shell that uses interp. Note that it's not meant to replace a POSIX shell at the moment, and its options are intentionally minimalistic.

Fuzzing

We use Go's native fuzzing support. For instance:

cd syntax
go test -run=- -fuzz=ParsePrint

Caveats

When indexing Bash associative arrays, always use quotes. The static parser will otherwise have to assume that the index is an arithmetic expression.

$ echo '${array[spaced string]}' | shfmt 1:16: not a valid arithmetic operator: string $ echo '${array[dash-string]}' | shfmt ${array[dash - string]}

$(( and (( ambiguity is not supported. Backtracking would complicate the parser and make streaming support via io.Reader impossible. The POSIX spec recommends to space the operands if $( ( is meant.

$ echo '$((foo); (bar))' | shfmt 1:1: reached ) without matching $(( with ))

export, let, and declare are parsed as keywords. This allows statically building their syntax tree, as opposed to keeping the arguments as a slice of words. It is also required to support declare foo=(bar). Note that this means expansions like declare {a,b}=c are not supported.
The entire library is written in pure Go, which limits how closely the interpreter can follow POSIX Shell and Bash semantics. For example, Go does not support forking its own process, so subshells use a goroutine instead, meaning that real PIDs and file descriptors cannot be used directly.

JavaScript

The parser and formatter are available as a third party npm package called sh-syntax, which bundles a version of this library compiled to WASM.

Previously, we maintained an npm package called mvdan-sh which used GopherJS to bundle a JS version of this library. That npm package is now archived given its poor performance and GopherJS not being as actively developed. Any existing or new users should look at sh-syntax instead.

Docker

All release tags are published via Docker, such as v3.5.1. The latest stable release is currently published as v3, and the latest development version as latest. The images only include shfmt; -alpine variants exist on Alpine Linux.

To build a Docker image, run:

docker build -t my:tag -f cmd/shfmt/Dockerfile .

To use a Docker image, run:

docker run --rm -u "$(id -u):$(id -g)" -v "$PWD:/mnt" -w /mnt my:tag <shfmt arguments>

The following editor integrations wrap shfmt:

BashSupport-Pro - Bash plugin for JetBrains IDEs
dockerfmt - Dockerfile formatter using shfmt
intellij-shellcript - Intellij Jetbrains shell script plugin
micro - Editor with a built-in plugin
neoformat - (Neo)Vim plugin
shell-format - VS Code plugin
vscode-shfmt - VS Code plugin
shfmt.el - Emacs package
Sublime-Pretty-Shell - Sublime Text 3 plugin
Trunk - Universal linter, available as a CLI, VS Code plugin, and GitHub action
vim-shfmt - Vim plugin

Other noteworthy integrations include:

modd - A developer tool that responds to filesystem changes
prettier-plugin-sh - Prettier plugin using sh-syntax
sh-checker - A GitHub Action that performs static analysis for shell scripts
mdformat-shfmt - mdformat plugin to format shell scripts embedded in Markdown with shfmt
pre-commit-shfmt - pre-commit shfmt hook