-
Notifications
You must be signed in to change notification settings - Fork 257
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
wasi: allow users to configure a filesystem that can accept new files #390
Comments
I completely rewrote the description as the gap here is much smaller than I originally thought. While fs.FS doesn't specify extended APIs it is routine in practice to already support writes (ex os.File). |
Does that mean that it is possible to write new files from Wasm? I have tried using the |
@mmathys exactly. in fact this it is about time we got back to this one. if you don't mind, as it is usually interesting.. what's the use case you are working with that needs to open for writing? is it a log or config file? or are you sharing data? |
We are using files to keep state between function calls when writing functions in C. |
@ncruces noted in #740 that there's a corner case in the io.Reader docs hard to reason with when mapping to io.Reader based implementation of fdread
The "process the n > 0 bytes returned before considering the error err" ends up a bit dodgy as you have a choice to ignore that part or create expense by holding per-fd state to return the error when >n bytes are read. |
Looking at the issue and current implementation, I don't think there's anything missing from For instance, if you use But you don't have to use Or you can create an implementation that, instead or returning Or you don't create an actual Now, all this has subtleties, but it seems doable, and not something As an example, for my usage I wanted to lock my WASM even harder, so I created a file system that can only ever open a single file (not an entire directory). |
To fix this, I just replaced instances of f, err := c.fs.Open(fsOpenPath) by f, err := os.OpenFile(fsOpenPath, os.O_APPEND|os.O_CREATE|os.O_RDWR, 0644) and I get a writable FS. |
The first way we'll implement this is via a new ModuleConfig which allows you to pass a path instead of an |
https://github.com/tetratelabs/wazero/releases/tag/v1.0.0-pre.6 adds |
This is awesome! |
Great work! |
If someone is looking for it now and hits this issue and wonders, where it is; it's in github.com/tetratelabs/wazero/experimental/sys https://pkg.go.dev/github.com/tetratelabs/[email protected]/experimental/sys#FS Line 29 in 08998ed
|
See #2076 |
WASI defines portable system calls to manipulate files. We currently use fs.FS to map to many of these. read-only operations are implicit, but the status quo goes beyond this in the standard library.
For example, fs.FS can write existing files as implementations are free to go beyond the functionality defined on fs.File. Some of this is encouraged. Concretely, os.File goes beyond with mutations such as chmod and write. os.FS
Also, there are standard extensions to
fs.FS
includingfs.ReadDirFS
. The point is that in many cases, there is an existing mapping of a go interface to WASI functions, even if it isn't required byfs.FS
. By safe casts, you can get most functionality needed.However, there are gaps, notably creating new files. This issue is solved when the core WASI APIs requested by end users have a corresponding implementation. This is ideally done without introducing any wazero-specific interfaces. As such research is needed and validation is needed prior to introducing an interface not in Go's standard library.
Moreover, as the README suggests, completion of all WASI functions is a non-goal. In other words, if there is a tricky API, and hasn't been asked directly by end-users yet.. wait until that's the case, then solve it. WASI is completely changing its filesystem API so we can conserve energy by only focusing on what users need today.
Solving this issue is a high priority and 1.0 will be held back until it is solved. However, it is likely this will be solved way before 1.0, like in a week or two.
References:
fs.FS
in WASI:fs.FS
as Preopen parameter with implementation offd_prestat_get
#362 with some custom APIs to add write support to it. This was closed out because it coupled read-side with writes and the former is easier to implement. This issue was raised to carry that forward.The text was updated successfully, but these errors were encountered: