Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add example and docs about using Python libraries in a project #190

Merged
merged 3 commits into from
Jul 31, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions docs/docs/languages/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,83 @@ def worker(req):

If you prefer, you can configure the environment variable value dynamically by following [these instructions](../features/environment-variables.md#inject-existing-environment-variables).

## Python libraries

The Python ecosystem has a huge number of packages. Developers like you usually rely on other libraries to accomplish different goals. In this example, you will use the [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/#quick-start) to parse a HTML document and read its text. You have all the code available in the [examples/python-libs](https://github.com/vmware-labs/wasm-workers-server/tree/main/examples/python-libs) folder.
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved

To add a new Python library to your project, follow these steps:

1. First, create an `index.py` file with the content below. This worker reads a HTML document and return the text using the `bs4` library:
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved

```python title="./index.py"
from bs4 import BeautifulSoup

html_doc = """<!DOCTYPE html>
<head>
<title>Wasm Workers Server</title>
</head>
<body>
<main>
<p>This page was generated by a Python file running in WebAssembly.</p>
</main>
</body>
"""

def worker(req):
soup = BeautifulSoup(html_doc, 'html.parser')

res = Response(soup.get_text(". ", True))
res.headers["x-generated-by"] = "wasm-workers-server"

return res
```

1. Install the Beautiful Soup library with `pip` and save it in a new `_libs` folder:

```plain
pip3 install -t ./_libs beautifulsoup4
```

1. Create a `index.toml` file with the content below. Note the name of the TOML file must match the name of the worker (`index.py` and `index.toml`). The current configuration mounts the `_libs` folder and sets the `PYTHONPATH` environment variable to the mount path:
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved

```toml title="./index.toml"
name = "libs"
version = "1"

[vars]
PYTHONPATH = "/opt/python/libs"

[[folders]]
from = "./_libs"
to = "/opt/python/libs"
```

1. If you didn't download the `wws` server yet, check our [Getting Started](../get-started/quickstart.md) guide. You also need to install the Python runtime with the command below:

```plain
wws runtimes install python latest
```

1. Run your worker with `wws`:

```bash
wws

⚙️ Preparing the project from: .
⚙️ Loading routes from: .
⏳ Loading workers from 1 routes...
✅ Workers loaded in 524.804167ms.
- http://127.0.0.1:8080/
=> ./index.py
🚀 Start serving requests at http://127.0.0.1:8080
```

1. Finally, open <http://127.0.0.1:8080> in your browser.

### Limitations

Currently, Wasm Workers Server only supports pure Python libraries like Beautiful Soup. Libraries that requires to compile native extensions are not available yet.
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved

## Examples

* [Basic](https://github.com/vmware-labs/wasm-workers-server/tree/main/examples/python-basic/)
Expand Down
44 changes: 44 additions & 0 deletions examples/python-libs/.wws.toml
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
version = 1

[[repositories]]
name = "wasmlabs"
url = "https://workers.wasmlabs.dev/repository/v1/index.toml"

[[repositories.runtimes]]
name = "python"
version = "3.11.1+20230217-1"
tags = [
"latest",
"3.11",
"3.11.1",
]
status = "active"
extensions = ["py"]
args = [
"--",
"/src/index.py",
]

[repositories.runtimes.binary]
url = "https://github.com/vmware-labs/webassembly-language-runtimes/releases/download/python%2F3.11.1%2B20230217-15dfbed/python-3.11.1.wasm"
filename = "python.wasm"

[repositories.runtimes.binary.checksum]
type = "sha256"
value = "66589b289f76bd716120f76f234e4dd663064ed5b6256c92d441d84e51d7585d"

[repositories.runtimes.polyfill]
url = "https://workers.wasmlabs.dev/repository/v1/files/python/3-1/poly.py"
filename = "poly.py"

[repositories.runtimes.polyfill.checksum]
type = "sha256"
value = "74d10132b0577a39e4ea30002d4605b7cdfb8f39abca327a45c8b313de7ea304"

[repositories.runtimes.wrapper]
url = "https://workers.wasmlabs.dev/repository/v1/files/python/3-1/wrapper.txt"
filename = "wrapper.txt"

[repositories.runtimes.wrapper.checksum]
type = "sha256"
value = "cf1edc5b1427180ec09d18f4d169580379f1b12001f30e330759f9a0f8745357"
36 changes: 36 additions & 0 deletions examples/python-libs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Python + libraries example

Run a Python worker that uses a Python library in Wasm Workers Server.

## Prerequisites

* Wasm Workers Server (wws):

```shell-session
curl -fsSL https://workers.wasmlabs.dev/install | bash
```

* Clone the repository:

```shell-session
git clone https://github.com/vmware-labs/wasm-workers-server.git &&
cd ./wasm-workers-server/examples/python-libs
```

* Install the Python libraries

```shell-session
pip3 install -r requirements.txt -t ./_libs
```

## Run

This example runs from the previously cloned repository (See [Prerequisites](#prerequisites)). Be sure you followed all the steps and you're in the `examples/python-libs` folder:
Angelmmiguel marked this conversation as resolved.
Show resolved Hide resolved

```shell-session
wws .
```

## Resources

* [Python documentation](https://workers.wasmlabs.dev/docs/languages/python)
2 changes: 2 additions & 0 deletions examples/python-libs/_libs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
!.gitignore
**/
25 changes: 25 additions & 0 deletions examples/python-libs/index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
from bs4 import BeautifulSoup

html_doc = """<!DOCTYPE html>
<head>
<title>Wasm Workers Server</title>
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta charset="UTF-8">
</head>
<body>
<main>
<h1>Hello from Wasm Workers Server</h1>
<p>
This page was generated by a Python file running in WebAssembly.
</p>
</main>
</body>
"""

def worker(req):
soup = BeautifulSoup(html_doc, 'html.parser')

res = Response(soup.get_text("</br></br>", True))
res.headers["x-generated-by"] = "wasm-workers-server"

return res
9 changes: 9 additions & 0 deletions examples/python-libs/index.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name = "libs"
version = "1"

[vars]
PYTHONPATH = "/opt/python/libs"

[[folders]]
from = "./_libs"
to = "/opt/python/libs"
1 change: 1 addition & 0 deletions examples/python-libs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
beautifulsoup4
Loading