Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot overlay mount an existing volume #24555

Open
AliveDevil opened this issue Nov 13, 2024 · 1 comment · May be fixed by #24565
Open

Cannot overlay mount an existing volume #24555

AliveDevil opened this issue Nov 13, 2024 · 1 comment · May be fixed by #24565
Assignees
@mheon
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@AliveDevil
Copy link

AliveDevil commented Nov 13, 2024 •
edited
Loading

Issue Description

When a container image has a volume VOLUME ["/data"], and that volume is -v /host:/container-volume:O,upperdir=/upper,workdir=/tmp/container, Podman will fail and not allow this overlay to exist.

Steps to reproduce the issue

Dockerfile: https://github.com/itzg/docker-minecraft-server/blob/bfc75cad21aaa0fc9d1d333ad16f355dc809edeb/Dockerfile#L61

Podman Create:

podman --debug create --replace --name MC-1.21.1-2024-11 -v "/srv/MC-1.21.1-2024-11/modpack:/data:O,upperdir=/srv/MC-1.21.1-2024-11/instance,workdir=/srv/MC-1.21.1-2024-11/tmp" ghcr.io/itzg/minecraft-server:java21-alpine

Describe the results you received

Error: two volumes found with destination /data: invalid argument

Describe the results you expected

No error, and the anonymous image is successfully mounted in the container, without creating unecessary local volumes.

podman info output

host:
  arch: amd64
  buildahVersion: 1.28.2
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2.1.6+ds1-1_amd64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: unknown'
  cpuUtilization:
    idlePercent: 99.87
    systemPercent: 0.01
    userPercent: 0.11
  cpus: 4
  distribution:
    codename: bookworm
    distribution: debian
    version: "12"
  eventLogger: journald
  hostname: minecraft-03
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.1.0-25-amd64
  linkmode: dynamic
  logDriver: journald
  memFree: 8856133632
  memTotal: 12504408064
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.8.1-1+deb12u1_amd64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/user/0/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: true
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_1.2.0-1_amd64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 1073737728
  swapTotal: 1073737728
  uptime: 1350h 7m 42.00s (Approximately 56.25 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 0
    stopped: 2
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 16733839360
  graphRootUsed: 5758803968
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 6
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.8
  Os: linux
  OsArch: linux/amd64
  Version: 4.3.1

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

No

Additional environment details

No response

Additional information

Upstream Latest Release

There are only the remote binaries available, which just call the existing APIs on containerd, no difference regardless.

Log 
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called create.PersistentPreRunE(podman --debug create --replace --name MC-1.21.1-2024-11 -v /srv/MC-1.21.1-2024-11/modpack:/data:O,upperdir=/srv/MC-1.21.1-2024-11/instance,workdir=/srv/MC-1.21.1-2024-11/tmp -e TYPE=NEOFORGE -e VERSION=1.21.1 ghcr.io/itzg/minecraft-server:java21-alpine) 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver                           
DEBU[0000] Using graph root /var/lib/containers/storage 
DEBU[0000] Using run root /run/containers/storage       
DEBU[0000] Using static dir /var/lib/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/libpod                    
DEBU[0000] Using volume path /var/lib/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is not being used 
DEBU[0000] Cached value indicated that native-diff is usable 
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false 
INFO[0000] [graphdriver] using prior storage driver: overlay 
DEBU[0000] Initializing event backend journald          
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument 
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
INFO[0000] Setting parallel job count to 13             
DEBU[0000] Pulling image ghcr.io/itzg/minecraft-server:java21-alpine (policy: missing) 
DEBU[0000] Looking up image "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Trying "ghcr.io/itzg/minecraft-server:java21-alpine" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386) 
DEBU[0000] exporting opaque data as blob "sha256:131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Looking up image "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Trying "ghcr.io/itzg/minecraft-server:java21-alpine" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386) 
DEBU[0000] exporting opaque data as blob "sha256:131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] User mount /srv/MC-1.21.1-2024-11/modpack:/data options [O upperdir=/srv/MC-1.21.1-2024-11/instance workdir=/srv/MC-1.21.1-2024-11/tmp] 
DEBU[0000] Error looking up container "MC-1.21.1-2024-11": no container with name or ID "MC-1.21.1-2024-11" found: no such container 
DEBU[0000] Looking up image "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Trying "ghcr.io/itzg/minecraft-server:java21-alpine" ... 
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage 
DEBU[0000] Found image "ghcr.io/itzg/minecraft-server:java21-alpine" as "ghcr.io/itzg/minecraft-server:java21-alpine" in local containers storage ([overlay@/var/lib/containers/storage+/run/containers/storage]@131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386) 
DEBU[0000] exporting opaque data as blob "sha256:131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Inspecting image 131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386 
DEBU[0000] exporting opaque data as blob "sha256:131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] exporting opaque data as blob "sha256:131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386" 
DEBU[0000] Inspecting image 131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386 
DEBU[0000] Inspecting image 131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386 
DEBU[0000] Inspecting image 131dd0a9849cf3996bffa3badb60a69bab9ae45dd7e31def472e0dc2e7e1f386 
DEBU[0000] Image has volume at "/data"                  
DEBU[0000] Adding anonymous image volume at "/data"     
DEBU[0000] using systemd mode: false                    
DEBU[0000] New container has a health check             
DEBU[0000] setting container name MC-1.21.1-2024-11     
DEBU[0000] No hostname set; container's hostname will default to runtime default 
DEBU[0000] Found apparmor_parser binary in /sbin/apparmor_parser 
DEBU[0000] Loading seccomp profile from "/usr/share/containers/seccomp.json" 
DEBU[0000] Successfully loaded 1 networks               
Error: two volumes found with destination /data: invalid argument

If possible, I'd like a fix for this to be backported to Debian 12 Podman.
@AliveDevil AliveDevil added the kind/bug Categorizes issue or PR as related to a bug. label Nov 13, 2024
@mheon mheon self-assigned this Nov 14, 2024
@mheon
Copy link
Member

mheon commented Nov 14, 2024

I'll take this one, I wrote the overriding logic that should catch this.

We don't handle Debian maintenance so I can't do backports once it's fixed, though, you'll have to file a bug with the maintainers about that.

mheon added a commit to mheon/libpod that referenced this issue Nov 14, 2024
@mheon
Overlay mounts supersede image volumes & volumes-from
4409108 
This matches the behavior of other volume and mount types. Image
volumes and volumes/mounts from the `--volumes-from` flag should
be overridden by actual user-specified named volumes and mounts,
but this was not true for overlay mounts. Fortunately, our
duplicate-mount detection logic still works, so we got a good
error message at least.

The fix is simple - extend our supersede logic, which currently
only works with named volumes and mounts, to also work with
overlay mounts.

Fixes containers#24555

Signed-off-by: Matt Heon <[email protected]>
@mheon mheon linked a pull request Nov 14, 2024 that will close this issue
Open
mheon added a commit to mheon/libpod that referenced this issue Nov 15, 2024
@mheon
Overlay mounts supersede image volumes & volumes-from
61b57b7 
This matches the behavior of other volume and mount types. Image
volumes and volumes/mounts from the `--volumes-from` flag should
be overridden by actual user-specified named volumes and mounts,
but this was not true for overlay mounts. Fortunately, our
duplicate-mount detection logic still works, so we got a good
error message at least.

The fix is simple - extend our supersede logic, which currently
only works with named volumes and mounts, to also work with
overlay mounts.

Fixes containers#24555

Signed-off-by: Matt Heon <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mheon mheon

Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Overlay mounts supersede image volumes & volumes-from mheon/libpod
2 participants
@AliveDevil @mheon