You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened?
EBS volumes get created and attached just fine but then are unavailable to kubernetes running under minikube.
AWS EC2 instances these days often don't have a /dev/xvdaa device for attaching EBS volumes but instead have a /dev/nvme1n1 device which gets attached and a symlink is then created: /dev/xvdaa -> /dev/nvme1n1
Under this circumstance, the MountVolume.MountDevice fails:
Events:
Type Reason Age From Message
Normal Scheduled 14m default-scheduler Successfully assigned default/postgres-deployment-7b6556544c-8ztdk to minikube
Normal SuccessfulAttachVolume 14m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-378c1c8f-500c-4bee-a16d-2ab9457a5511"
Warning FailedMount 2m31s (x14 over 14m) kubelet MountVolume.MountDevice failed for volume "pvc-378c1c8f-500c-4bee-a16d-2ab9457a5511" : rpc error: code = Internal desc = Failed to find device path /dev/xvdaa. no device path for device "/dev/xvdaa" volume "vol-0f1bd87aca480b924" found
Warning FailedMount 4s (x8 over 68s) kubelet MountVolume.MountDevice failed for volume "pvc-378c1c8f-500c-4bee-a16d-2ab9457a5511" : rpc error: code = Internal desc = Failed to find device path /dev/xvdaa. no device path for device "/dev/xvdaa" volume "vol-0f1bd87aca480b924" found
While investigating, I found that the minikube container itself does not seem to be aware of /dev/xvdaa:
[ec2-user@ip-172-31-31-77 ]$ minikube ssh
docker@minikube:$ ls -l /dev/xv* /dev/nvm*
ls: cannot access '/dev/xv*': No such file or directory
crw------- 1 root root 250, 0 Sep 20 20:30 /dev/nvme0
brw-rw---- 1 root disk 259, 0 Sep 20 20:30 /dev/nvme0n1
brw-rw---- 1 root disk 259, 1 Sep 20 20:30 /dev/nvme0n1p1
brw-rw---- 1 root disk 259, 2 Sep 20 20:30 /dev/nvme0n1p128
crw------- 1 root root 250, 1 Sep 20 20:30 /dev/nvme1
brw-rw---- 1 root disk 259, 3 Sep 20 20:30 /dev/nvme1n1
docker@minikube:~$
But the volume is definitely attached and AWS claims it's attached to /dev/xvdaa:
We will treat this as a feature request. Currently, the driver is not officially supported or previously tested by our team in minikube environments. It seems that there is a big opportunity to improve the resiliency of FindDevicePath here.
/kind bug
What happened?
EBS volumes get created and attached just fine but then are unavailable to kubernetes running under minikube.
AWS EC2 instances these days often don't have a /dev/xvdaa device for attaching EBS volumes but instead have a /dev/nvme1n1 device which gets attached and a symlink is then created: /dev/xvdaa -> /dev/nvme1n1
Under this circumstance, the MountVolume.MountDevice fails:
While investigating, I found that the minikube container itself does not seem to be aware of /dev/xvdaa:
But the volume is definitely attached and AWS claims it's attached to /dev/xvdaa:
[
What you expected to happen?
After the volume is attached, the device can be mounted even if the attachment Device name is a symlink.
How to reproduce it (as minimally and precisely as possible)?
pvc.yaml:
Anything else we need to know?:
Environment
kubectl version
):The text was updated successfully, but these errors were encountered: