GitHub - bytecodealliance/wit-bindgen: A language binding generator for WebAssembly interface types (original) (raw)

About

This project is a suite of bindings generators for languages that are compiled to WebAssembly and use the component model. Bindings are described with*.wit files which specify imports, exports, and facilitate reuse between bindings definitions.

The wit-bindgen repository is currently focused on guest programs which are those compiled to WebAssembly. Executing a component in a host is not managed in this repository, and some options of how to do so are described below. Languages developed in this repository are Rust, C, Java (TeaVM Java), Go (TinyGo), and C#. If you encounter any problems feel free to open an issue or chat with us on Zulip.

WIT as an IDL

The wit-bindgen project extensively uses WIT definitions to describe imports and exports. The items supported by WIT directly map to the component model which allows core WebAssembly binaries produced by native compilers to be transformed into a component. All imports into a WebAssembly binary and all exports must be described with WIT. An example file looks like:

package example:host;

world host { import print: func(msg: string);

export run: func(); }

This describes a "world" which describes both imports and exports that the WebAssembly component will have available. In this case the host will provide aprint function and the component itself will provide a run function.

Functionality in WIT can also be organized into interfaces:

package example:my-game;

interface my-plugin-api { record coord { x: u32, y: u32, }

get-position: func() -> coord; set-position: func(pos: coord);

record monster { name: string, hp: u32, pos: coord, }

monsters: func() -> list; }

world my-game { import print: func(msg: string); import my-plugin-api;

export run: func(); }

Here the my-plugin-api interface encapsulates a group of functions, types, etc. This can then be imported wholesale into the my-game world via themy-plugin-api namespace. The structure of a WIT document and world will affect the generated bindings per-language.

For more information about WIT and its syntax see the online documentation for WIT as well as its upstream reference.

Creating a Component

The end-goal of wit-bindgen is to facilitate creation of acomponent. Once a component is created it can then be handed off to any one of a number of host runtimes for execution. Creating a component is not supported natively by any language today, however, sowit-bindgen is only one of the pieces in the process of creating a component. The general outline for the build process of a component for a compiled language is:

  1. Using wit-bindgen source code for the language is generated representing bindings to the specified APIs. This source code is then compiled by the native compiler and used by user-written code as well.
  2. The native language toolchain is used to emit a core WebAssembly module. This core wasm module is the "meat" of a component and contains all user-defined code compiled to WebAssembly. The most common native target to use for compilation today is the wasm32-wasip1 target.
  3. The output core wasm module is transformed into a component using thewasm-tools project, notably the wasm-tools component new subcommand. This will ingest the native core wasm output and wrap the output into the component model binary format.

The precise tooling and commands at each of these steps differs language by language, but this is the general idea. With a component in-hand the binary can then be handed off to a host runtimes for execution.

Creating components: WASI

An important consideration when creating a component today is WASI. All current native toolchains for languages which have WASI support are using thewasi_snapshot_preview1 version of WASI. This definition of WASI was made with historical *.witx files and is not compatible with the component model. There is, however, a means by which to still create components from modules that are using wasi_snapshot_preview1 APIs.

The wasm-tools component new subcommand takes an --adapt argument which acts as a way to polyfill non-component-model APIs, like wasi_snapshot_preview1, with component model APIs. The Wasmtime runtime publishes adapter modules with each release that are suitable to use with--adapt to implement wasi_snapshot_preview1 in terms of WASI 0.2. On Wasmtime's releases page you'll see three modules to choose from:

Only one adapter is necessary and be sure to look for the latest versions as well.

Supported Guest Languages

The wit-bindgen project is primarily focused on guest languages which are those compiled to WebAssembly. Each language here already has native support for execution in WebAssembly at the core wasm layer (e.g. targets the current core wasm specification). Brief instructions are listed here for each language of how to use it as well.

Each project below will assume the following *.wit file in the root of your project.

// wit/host.wit package example:host;

world host { import print: func(msg: string);

export run: func(); }

Guest: Rust

The Rust compiler since version 1.82 supports a native wasm32-wasip2 target and can be added to any rustup-based toolchain with:

rustup target add wasm32-wasip2

In order to compile a wasi dynamic library, the following must be added to theCargo.toml file:

[lib] crate-type = ["cdylib"]

Projects can then depend on wit-bindgen by executing:

WIT files are currently added to a wit/ folder adjacent to your Cargo.tomlfile. Example code using this then looks like:

// src/lib.rs

// Use a procedural macro to generate bindings for the world we specified in // host.wit wit_bindgen::generate!({ // the name of the world in the *.wit input file world: "host", });

// Define a custom type and implement the generated Guest trait for it which // represents implementing all the necessary exported interfaces for this // component. struct MyHost;

impl Guest for MyHost { fn run() { print("Hello, world!"); } }

// export! defines that the MyHost struct defined below is going to define // the exports of the world, namely the run function. export!(MyHost);

By using cargo expand or cargo doc you can also explore the generated code. If there's a bug in wit-bindgenand the generated bindings do not compile or if there's an error in the generated code (which is probably also a bug in wit-bindgen), you can useWIT_BINDGEN_DEBUG=1 as an environment variable to help debug this.

This project can then be built with:

cargo build --target wasm32-wasip2

This creates a ./target/wasm32-wasip2/debug/my-project.wasm file which is suitable to execute in any component runtime. Using wasm-tools you can inspect the binary as well, for example inferring the WIT world that is the component:

wasm-tools component wit ./target/wasm32-wasip2/debug/my-project.wasm

world my-component {

import print: func(msg: string)

export run: func()

}

which in this case, as expected, is the same as the input world.

Guest: C/C++

C and C++ code can be compiled for the wasm32-wasip1 target using the WASI SDK project. The releases on that repository have precompiled clang binaries which are pre-configured to compile for WebAssembly.

To start in C and C++ a *.c and *.h header file is generated for your project to use. These files are generated with the wit-bindgen CLI command in this repository.

wit-bindgen c ./wit

Generating "host.c"

Generating "host.h"

Generating "host_component_type.o"

Some example code using this would then look like

// my-component.c

#include "host.h"

void host_run() { host_string_t my_string; host_string_set(&my_string, "Hello, world!");

host_print(&my_string);

}

This can then be compiled with clang from the WASI SDK and assembled into a component with:

clang host.c host_component_type.o my-component.c -o my-core.wasm -mexec-model=reactor wasm-tools component new ./my-core.wasm -o my-component.wasm

Like with Rust, you can then inspect the output binary:

wasm-tools component wit ./my-component.wasm

Guest C#

To generate the bindings:

wit-bindgen csharp -w command -r native-aot --generate-stub wit/

Now you create a c# project file:

dotnet new console -o MyApp
cd MyApp
dotnet new nugetconfig

In the nuget.config after <clear />make sure you have:

<add key="dotnet-experimental" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-experimental/nuget/v3/index.json" />
<add key="nuget" value="https://api.nuget.org/v3/index.json" />

In the MyApp.csproj add the following to the property group:

<RuntimeIdentifier>wasi-wasm</RuntimeIdentifier>
<UseAppHost>false</UseAppHost>
<PublishTrimmed>true</PublishTrimmed>
<InvariantGlobalization>true</InvariantGlobalization>
<SelfContained>true</SelfContained>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<WASI_SDK_PATH>path/to/wasi-sdk</WASI_SDK_PATH>

Add the native-aot compiler (substitute win-x64 for linux-x64 on Linux):

dotnet add package Microsoft.DotNet.ILCompiler.LLVM --prerelease
dotnet add package runtime.win-x64.Microsoft.DotNet.ILCompiler.LLVM --prerelease

Now you can build with:

Checkout out componentize-dotnet for a simplified experience.

Guest: Java

This project historically had some support forTeaVM-WASI, but it was unmaintained for a long time and never was at feature parity with other generators, so it was removed. The last commit with support for TeaVM-WASI washttps://github.com/bytecodealliance/wit-bindgen/commit/86e8ae2b8b97f11b73b273345b0e00340f017270.

Guest: TinyGo

The new TinyGo WIT bindings generator is currently in development at thego.bytecodealliance.org repository.

To install the wit-bindgen-go CLI, run:

go install go.bytecodealliance.org/cmd/wit-bindgen-go@latest

Note: it requires wasm-tools to be installed.

Then, you can generate the bindings for your project:

wit-bindgen-go generate

Guest: MoonBit

MoonBit can be compiled to WebAssembly using its toolchain:

moon build --target wasm # --debug to keep symbols

The generated core wasm will be found under target/wasm/release/build/gen/gen.wasm by default. Then you can use wasm-tools to componentize the module:

wasm-tools component embed wit target/wasm/release/build/gen/gen.wasm -o target/gen.wasm
wasm-tools component new target/gen.wasm -o target/gen.component.wasm

You may use --gen-dir to specify which package should be responsible for the exportation. The default is gen as mentioned above. This can be useful having one project that exports multiple worlds.

When using wit-bindgen moonbit, you may use --derive-show or --derive-eq to derive Show or Eq traits for all types. You may also use --derive-error, which will make types containing Error as error types in MoonBit.

You will find the files to be modified with the name **/stub.mbt. To avoid touching the files during regeneration (including moon.pkg.json or moon.mod.json) you may use --ignore-stub.

/!\ MoonBit is still evolving, so please check out the Weekly Updates for any breaking changes or deprecations.

Guest: Other Languages

Guest component support for JavaScript and Python is available incomponentize-js andcomponentize-py, respectively. See alsoThe WebAssembly Component Model developer's guidefor examples of how to build components using various languages.

Other languages such as Ruby, etc, are hoped to be supported one day with wit-bindgen or with components in general. It's recommended to reach out on zulip if you're intersted in contributing a generator for one of these langauges. It's worth noting, however, that turning an interpreted language into a component is significantly different from how compiled languages currently work (e.g. Rust or C/C++). It's expected that the first interpreted language will require a lot of design work, but once that's implemented the others can ideally relatively quickly follow suit and stay within the confines of the first design.

CLI Installation

To install the CLI for this tool (which isn't the only way it can be used), run the following cargo command. This will let you generate the bindings for any supported language.

cargo install wit-bindgen-cli

This CLI IS NOT stable and may change, do not expect it to be or rely on it being stable. Please reach out to us on zulip if you'd like to depend on it, so we can figure out a better alternative for your use case.

Host Runtimes for Components

The wit-bindgen project is intended to facilitate in generating a component, but once a component is in your hands the next thing to do is to actually execute that somewhere. This is not under the purview of wit-bindgen itself but these are some resources and runtimes which can help you work with components:

Note that the runtimes above are generally intended to work with arbitrary components, not necessarily only those created by wit-bindgen. This is also not necessarily an exhaustive listing of what can execute a component.

Building and Testing

To build the cli:

Learn more how to run the tests in the testing document.

Versioning and Releases

This repository's crates and CLI are all currently versioned at 0.X.Y whereY is frequently 0 and X increases most of the time with publishes. This means that changes are published as possibly-API-breaking changes as development continues here.

Also, this repository does not currently have a strict release cadence. Releases are done on an as-needed basis. If you'd like a release done please feel free to reach out on Zulip, file an issue, leave a comment on a PR, or otherwise contact a maintainer.

For maintainers, the release process looks like:

That should be it, but be sure to keep an eye on CI in case anything goes wrong.

License

This project is triple licenced under the Apache 2/ Apache 2 with LLVM exceptions/ MIT licences. The reasoning for this is:

For more details see

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache 2/ Apache 2 with LLVM exceptions/ MIT licenses, shall be licensed as above, without any additional terms or conditions.