Issue 46315: Add support for WebAssembly System Interface (wasm32-wasi) (original) (raw)

WASI is another WebAssembly platform similar to Emscripten (bpo-40280). Simply speaking Emscripten binaries (wasm32-emscripten) run inside a browser while WASI binaries target standalone runtimes like wasmtime [2][3] on the host. The lines are a bit blurry, as it is possible to run WASI binaries in the browser with help of JS-polyfills. WASI provides compile once, run anyway with JIT/AOT and sandboxing.

WASI is still under development and is lacking several core features:

• threading support and pthread API
• sockets
• signals are emulated (_WASI_EMULATED_SIGNAL and -lwasi-emulated-signal)
• DAC APIs like chmod, umask
• user-related APIs like pwd, grp
• dup(), dup2(), F_DUPFD

For 3.11 I plan to fix our use of #ifdef HAVE_FEATURE to make it easier to experiment with WASI. The pthread APIs need stubs, which I won't commit to 3.11 upstream yet.

[1] https://wasi.dev/ [2] https://github.com/bytecodealliance/wasmtime [3] https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

dup() is required by _PyTokenizer_FindEncodingFilename(). I came up with this hack:

// from wasi-libc libc-top-half/musl/src/internal/stdio_impl.h struct _IO_FILE { unsigned flags; unsigned char *rpos, *rend; int (*close)(FILE *); unsigned char *wend, *wpos; // incomplete };

static int dummy_close(FILE *fp) { return 0; };

static FILE * _Py_fdopen_borrow(int fd, const char *mode) { FILE fp = fdopen(fd, mode); ((struct _IO_FILE)fp)->close = dummy_close; return fp; }

keithw on #wasi pointed out that fopencookie() can archive the same outcome without resorting to ABI-specific hack. A trivial implementation is straight forward:

typedef union { void *cookie; int fd; } borrowed;

static ssize_t borrow_read(void *cookie, char *buf, size_t size) { borrowed b; b.cookie = cookie; return read(b.fd, (void *)buf, size); }

static ssize_t borrow_write(void *cookie, const char *buf, size_t size) { errno = ENOTSUP; return -1; }

static int borrow_seek(void *cookie, off_t *off, int whence) { borrowed b; b.cookie = cookie; off_t pos; pos = lseek(b.fd, *off, whence); if (pos == (off_t)-1) { return -1; } else { *off = pos; return 0;
} }

static int borrow_close(void *cookie) { // does not close(fd) return 0; }

FILE * _Py_fdopen_borrow(int fd, const char *mode) { // only support read for now if (strcmp(mode, "r") != 0) { return NULL; } cookie_io_functions_t cookie_io = { borrow_read, borrow_write, borrow_seek, borrow_close }; // cookie is just the fd borrowed b; b.fd = fd; return fopencookie(b.cookie, "r", cookie_io); }