Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract Channel FD and self-mount #300

Closed
colinmarc opened this issue Oct 6, 2024 · 12 comments · Fixed by #304
Closed

Extract Channel FD and self-mount #300

colinmarc opened this issue Oct 6, 2024 · 12 comments · Fixed by #304

Comments

@colinmarc
Copy link
Contributor

Hi, I have a use case (containers) where I need to get the session FD and do the mounting myself. I took a look at changing the API to support this, but unfortunately, it seems like mnt::Mount and Session are pretty tied together at the moment.

Before I propose any significant API changes, do you have an idea how I could achieve this?

Thanks!

@colinmarc
Copy link
Contributor Author

I poked around a bit more. First, let me clarify roughly what I'd like to do:

  1. Create a Session with my FileSystem.
  2. Call AsFd::as_fd on the Session. Then, take that FD, and pass it into my own mount(2) call with fd=123.
  3. Call Session::run to proceed as normal.

Fundamentally, this seems like it would require splitting Session into two types: one which holds the /dev/fuse FD (the Channel), and one which knows the mountpoint and calls umount when it's dropped.

However, The fact that a session is also a mountpoint, as far as the API is concerned, seems to stem from libfuse. That's because fuse_session_new takes the mount args already. So while the above change would be fairly trivial with fuse_pure, it's not easy with libfuse.

I don't suppose you'd consider dropping support for libfuse? 😅

@colinmarc
Copy link
Contributor Author

Huh, there's also a (very wacky) libfuse3 way to do this: libfuse/libfuse#291

colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 6, 2024
This is a minor refactor of `Session`, with the aim of allowing callers
to create an unmounted session and mount it themselves (using the AsFd
implementation to get the FD).

One use case for this is when mounting inside containers, when you need
to call setns(2) before mounting.

Fixes cberner#300
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 6, 2024
This is a minor refactor of `Session`, with the aim of allowing callers
to create an unmounted session and mount it themselves (using the AsFd
implementation to get the FD).

One use case for this is when mounting inside containers, when you need
to call setns(2) before mounting.

Fixes cberner#300
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 7, 2024
This is a minor refactor of `Session`, with the aim of allowing callers
to create an unmounted session and mount it themselves (using the AsFd
implementation to get the FD).

One use case for this is when mounting inside containers, when you need
to call setns(2) before mounting.

Fixes cberner#300
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 7, 2024
This is a minor refactor of `Session`, with the aim of allowing callers
to create an unmounted session and mount it themselves (using the AsFd
implementation to get the FD).

One use case for this is when mounting inside containers, when you need
to call setns(2) before mounting.

Fixes cberner#300
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 7, 2024
This is a minor refactor of `Session`, with the aim of allowing callers
to create an unmounted session and mount it themselves (using the AsFd
implementation to get the FD).

One use case for this is when mounting inside containers, when you need
to call setns(2) before mounting.

Fixes cberner#300
@cberner
Copy link
Owner

cberner commented Oct 8, 2024

Can you tell me more about the use case? I'm trying to understand if this is common enough that it should be support in fuser

@colinmarc
Copy link
Contributor Author

colinmarc commented Oct 8, 2024

Can you tell me more about the use case? I'm trying to understand if this is common enough that it should be support in fuser

FUSE plays a special role in containers, where it can be used to simulate kernel-level stuff like sysfs or character devices. This is specifically permitted for unprivileged container runtimes by the kernel. However, various security safeguards mean that it requires a very specific dance:

  1. /dev/fuse has to be opened inside the container (inside the user namespace where the mount call happens)
  2. The mount (or fsconfig/move_mount) syscalls have to be done inside the container as well
  3. If the fuse daemon wants to run outside the container (e.g. in the runtime), the FD has to be passed back out to the supervisor process (using a socket or something)
  4. The channel can then be polled for requests normally (Session::run)

@colinmarc
Copy link
Contributor Author

One less-invasive way achieve this would be to offer a way to just iterate the Requests given a /dev/fuse FD. That would mean users wouldn't have access to the Filesystem abstraction, however.

@unexge
Copy link

unexge commented Oct 8, 2024

We also have a similar use-case in https://github.com/awslabs/mountpoint-s3-csi-driver. We want to run FUSE/fuser process inside an unprivileged Kubernetes Pod (so we can't do mount syscall). We already have a privileged Pod for doing the mount (i.e., opening /dev/fuse and performing mount syscall), and we want to pass obtained file descriptor to our unprivileged Pod where FUSE/fuser process lives.

colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 11, 2024
This is a minor refactor of `Session`, to make the API slightly more
flexible. Splitting `Mount` into a separate object allows callers to

 - Create a mount without a Session object
 - Create a session given an existing /dev/fuse FD

One use case for this is when mounting inside containers, when you need
to handle the session and mounting in separate processes.

Fixes cberner#300
@cberner
Copy link
Owner

cberner commented Oct 13, 2024

I see, thanks for the links! It sounds to me like the best thing to do is to add a method like Session::from_raw_fd. You can then open /dev/fuse and call mount() anyway that you want, pass the resulting fd to the correct process, and then call Session::from_raw_fd to create a fuser Session which can run and serve the fuse operations.

Does that sound like it will work for both of your use cases?

@colinmarc
Copy link
Contributor Author

colinmarc commented Oct 14, 2024

It sounds to me like the best thing to do is to add a method like Session::from_raw_fd.

Yes, that's what #301 does :)

Edit: and also #304, if you prefer less refactoring.

colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 14, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes cberner#300.
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 14, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes cberner#300.
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 14, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes cberner#300.
@cberner
Copy link
Owner

cberner commented Oct 14, 2024

Ah, great. I left some comments on #304. I'd rather not remove the libfuse support right now

colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 17, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes cberner#300.
colinmarc added a commit to colinmarc/fuser that referenced this issue Oct 19, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes cberner#300.
cberner pushed a commit that referenced this issue Oct 19, 2024
This is important for container runtimes, which need to do a special
namespace mount dance.

Fixes #300.
@unexge
Copy link

unexge commented Oct 21, 2024

Hey @cberner, is there any reason for not dropping support for libfuse? Are there any known limitations of pure-Rust mounting?

@cberner
Copy link
Owner

cberner commented Oct 21, 2024

Just unknown unknowns. libfuse is enabled in the default features, and I don't know what the limitations of the pure Rust implementations are. I think it might not work with some of the mac kernel extensions, but haven't tested them. If you're interested in researching and testing all the different platforms and configs, that'd be great!

@unexge
Copy link

unexge commented Oct 22, 2024

Thanks for the context @cberner! We'll be sharing if we find any limitations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants