sh

A shell parser, formatter, and interpreter with bash support; includes shfmt

Programming language: Go

License: BSD 3-clause "New" or "Revised" License

Tags: Text Processing     Specific Formats     Blackfriday    

Latest version: v3.3.1

README

sh

A shell parser, formatter, and interpreter. Supports POSIX Shell, Bash, and mksh. Requires Go 1.16 or later.

Quick start

To parse shell scripts, inspect them, and print them out, see the syntax examples.

For high-level operations like performing shell expansions on strings, see the shell examples.

shfmt

go install mvdan.cc/sh/v3/cmd/[email protected]

shfmt formats shell programs. See [canonical.sh](syntax/canonical.sh) for a quick look at its default style. For example:

shfmt -l -w script.sh

For more information, see [its manpage](cmd/shfmt/shfmt.1.scd), which can be viewed directly as Markdown or rendered with scdoc.

Packages are available on Alpine, Arch, Docker, FreeBSD, Homebrew, MacPorts, NixOS, Scoop, Snapcraft, Void and webi.

gosh

go install mvdan.cc/sh/v3/cmd/[email protected]

Proof of concept shell that uses interp. Note that it's not meant to replace a POSIX shell at the moment, and its options are intentionally minimalistic.

Fuzzing

This project makes use of go-fuzz to find crashes and hangs in both the parser and the printer. The fuzz-corpus branch contains a corpus to get you started. For example:

git checkout fuzz-corpus
cd syntax
./fuzz

Caveats

• When indexing Bash associative arrays, always use quotes. The static parser will otherwise have to assume that the index is an arithmetic expression.

$ echo '${array[spaced string]}' | shfmt
1:16: not a valid arithmetic operator: string
$ echo '${array[dash-string]}' | shfmt
${array[dash - string]}

• $(( and (( ambiguity is not supported. Backtracking would complicate the parser and make streaming support via io.Reader impossible. The POSIX spec recommends to space the operands if $( ( is meant.

$ echo '$((foo); (bar))' | shfmt
1:1: reached ) without matching $(( with ))

• Some builtins like export and let are parsed as keywords. This allows statically building their syntax tree, as opposed to keeping the arguments as a slice of words. It is also required to support declare foo=(bar). Note that this means expansions like declare {a,b}=c are not supported.

JavaScript

A subset of the Go packages are available as an npm package called mvdan-sh. See the [_js](_js) directory for more information.

Docker

To build a Docker image, checkout a specific version of the repository and run:

docker build -t my:tag -f cmd/shfmt/Dockerfile .

This creates an image that only includes shfmt. Alternatively, if you want an image that includes alpine, add --target alpine. To use the Docker image, run:

docker run --rm -v $PWD:/mnt -w /mnt my:tag <shfmt arguments>

pre-commit

It is possible to use shfmt with pre-commit and a local repo configuration like:

  - repo: local
    hooks:
      - id: shfmt
        name: shfmt
        minimum_pre_commit_version: 2.4.0
        language: golang
        additional_dependencies: [mvdan.cc/sh/v3/cmd/[email protected]]
        entry: shfmt
        args: [-w]
        types: [shell]

Related projects

The following editor integrations wrap shfmt:

• format-shell - Atom plugin
• intellij-shellcript - Intellij Jetbrains shell script plugin
• micro - Editor with a built-in plugin
• shell-format - VS Code plugin
• shfmt.el - Emacs package
• Sublime-Pretty-Shell - Sublime Text 3 plugin
• vim-shfmt - Vim plugin

Other noteworthy integrations include:

• Alternative docker image by PeterDaveHello
• modd - A developer tool that responds to filesystem changes
• prettier-plugin-sh - Prettier plugin using mvdan-sh
• sh-checker - A GitHub Action that performs static analysis for shell scripts
• mdformat-shfmt - mdformat plugin to format shell scripts embedded in Markdown with shfmt