Skip to content

gh-100956: Update directions for how to run python + wasm #101003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brendandburns wants to merge 1 commit into from
Closed

gh-100956: Update directions for how to run python + wasm #101003

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion Tools/wasm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@ Users and developers are encouraged to use the script
`Tools/wasm/wasm_build.py`. The tool automates the build process and provides
assistance with installation of SDKs.

## wasm32-emscripten build
* [Build emscripten](https://github.com/python/cpython/tree/main/Tools/wasm#wasm32-emscripten-build)
* [Serve Python WASM](https://github.com/python/cpython/tree/main/Tools/wasm#hosting-python-wasm-builds)
* [Build WASI-SDK](https://github.com/python/cpython/tree/main/Tools/wasm#wasi-wasm32-wasi)
* [Running Python in WASI]((https://github.com/python/cpython/tree/main/Tools/wasm#running-python-wasi)

Comment on lines +15 to +19

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should either use relative anchor links so this works offline or rely on GitHub's automatic table of contents for all Markdown files.

# wasm32-emscripten build

For now the build system has two target flavors. The ``Emscripten/browser``
target (``--with-emscripten-target=browser``) is optimized for browsers.
Expand Down Expand Up @@ -281,6 +286,50 @@ make -j$(nproc)
popd
```

## Running Python in WASI

Here is a simple script that can run a python script, it expects that the Python installation
is at the root of the filesystem (and it appears that currently that is [required](https://github.com/python/cpython/issues/100307).
Comment on lines +291 to +292

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue #100307 is not about where you run the WASI binary from, but how you have to mount the lib directory that contains the stdlib.

Suggested change
Here is a simple script that can run a python script, it expects that the Python installation
is at the root of the filesystem (and it appears that currently that is [required](https://github.com/python/cpython/issues/100307).
Here is a script for running a Python file. It helps demonstrate how WASI
follows a capabilities security model, requiring that you explicitly clear which
directories are accessible from within the WASI runtime. You can specify the
_host directories_ that you want to allow in and the _guest directories_ that
they map to.
Make sure to update the `PYTHON_WASI_ROOT` variable in the script to the
directory where you are storing your WASI binary and `lib` directories with the
stdlib.


```shell
#!/bin/bash

if [ $# -eq 0 ]; then
FILE=""
HOST_DIR=$PWD
GUEST_DIR=$PWD
elif [[ "$1" = /* ]]; then
REAL_PATH=$(realpath $1)
HOST_DIR=$(dirname $REAL_PATH)
GUEST_DIR=$HOST_DIR
FILE=${REAL_PATH}

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe to help make things clearer around the difference between host and guest directories it's worth mapping to a /workspace guest directory?

else
HOST_DIR=$PWD
GUEST_DIR=$PWD
FILE="/$PWD/${@#}"
fi

PYTHON_WASI_ROOT=/Python-3.11.0-wasm32-wasi-16

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably move this to the top and leave a comment about updating it.

wasmtime run --dir ${PYTHON_WASI_ROOT} \
--mapdir /::${PYTHON_WASI_ROOT} \
--dir ${HOST_DIR} \
--mapdir ${GUEST_DIR}::${HOST_DIR} \
-- ${PYTHON_WASI_ROOT}/python.wasm $FILE
Comment on lines +313 to +317

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--mapdir automatically adds the host directories as accessible.

Suggested change
wasmtime run --dir ${PYTHON_WASI_ROOT} \
--mapdir /::${PYTHON_WASI_ROOT} \
--dir ${HOST_DIR} \
--mapdir ${GUEST_DIR}::${HOST_DIR} \
-- ${PYTHON_WASI_ROOT}/python.wasm $FILE
wasmtime run --mapdir /::${PYTHON_WASI_ROOT} \
--mapdir ${GUEST_DIR}::${HOST_DIR} \
-- ${PYTHON_WASI_ROOT}/python.wasm $FILE

```

Once you have that script installed somewhere and executable (e.g. `/usr/bin/python3-wasm`)

```
# Launch a REPL
python3-wasm

# Launch a python file
python3-wasm some-python.py

# Launch a directory
python3-wasm some/directory/
```

@brettcannon brettcannon Jan 13, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you just want to quickly launch the WASI binary, you can run
`wasmtime run --dir . python.wasm`. This does require putting any code you
want in the same directory as `python.wasm`. If you want to pass any
arguments, use `--`, e.g. `wasmtime run --dir . python.wasm -- -c "print('hello, world!')"`.

## WASI limitations and issues (WASI SDK 15.0)

A lot of Emscripten limitations also apply to WASI. Noticeable restrictions
Expand Down