cloud-init as backend for instance provisioning

[aavarghese]

Wondering in the context of similar tools, has skypilot looked into using clount-init as backend provisioner? Since it is a popular large project used across industries, we're curious about what are some of the contemplated pros/cons for using Skypilot vs cloud-init?

[Michaelvll]

Thanks for the question @aavarghese! Based on our best knowledge cloud-init is a tool for applying initial boot setup to a newly provisioned VM, instead of provisioning the VM.

We use cloud-init to apply the initial setup (e.g., private key setup, etc) on some of the supported clouds, such as AWS and Azure. Please check out the links below:

skypilot/sky/templates/aws-ray.yml.j2

Lines 86 to 115 in e6ee397

	# Use cloud init in UserData to set up the authorized_keys to get
	# around the number of keys limit and permission issues with
	# ec2.describe_key_pairs.
	# The | can ensure the line below are interpreted as a plain text,
	# without yaml specific syntax to be parsed, such as string that
	# contains `: `.
	# Note that sudo and shell need to be specified to ensure setup works.
	# Reference: https://cloudinit.readthedocs.io/en/latest/reference/modules.html#users-and-groups
	# The bootcmd is to disable automatic APT updates, to avoid the lock
	# when user call `apt install` on the node.
	# Reference: https://askubuntu.com/questions/1322292/how-do-i-turn-off-automatic-updates-completely-and-for-real
	UserData: |
	#cloud-config
	users:
	- name: skypilot:ssh_user
	shell: /bin/bash
	sudo: ALL=(ALL) NOPASSWD:ALL
	ssh_authorized_keys:
	- |-
	skypilot:ssh_public_key_content
	write_files:
	- path: /etc/apt/apt.conf.d/20auto-upgrades
	content: |
	APT::Periodic::Update-Package-Lists "0";
	APT::Periodic::Download-Upgradeable-Packages "0";
	APT::Periodic::AutocleanInterval "0";
	APT::Periodic::Unattended-Upgrade "0";
	- path: /etc/apt/apt.conf.d/10cloudinit-disable
	content: |
	APT::Periodic::Enable "0";

skypilot/sky/templates/azure-ray.yml.j2

Line 72 in e6ee397

cloudInitSetupCommands: {{cloud_init_setup_commands}}

Do you see any potential improvement for our usage of cloud-init?

[starpit]

hi @Michaelvll thanks for the help with this!

1. i see a gap in ibm cloud support?
2. cloud-init supports things like bootcmd and runcmd, which allow for e.g. launching the job as part of cloud-init, rather than requiring any extra layers on top. have you considered leveraging these cloud-init capabilities?

thanks,
nick