[Core] Config improvements

[colinjc]

Trying to capture some suggestions on how to improve the config file system to better work in large repos and organizations

1. Instead of the system only being configured in the file located at SKYPILOT_CONFIG, first look for a .skypilot.yaml dotfile in the pwd. The CLI should also accept a --config FILENAME option. They should be parsed and merged in the order of priority (flag, dotfile, user dir file).
2. All settings in the global config should be overridable in a task.yml
3. sky check should only look at clouds configured in the global config
4. Users should be able to specify the cloud target metadata in each task.yml. For example, a user should be able to set GCP project ID per task, instead of having to use the GOOGLE_PROJECT_ID env var

[Michaelvll]

Thanks for filing the issue @colinjc! Just for a record, we have the restriction for clouds to use (item 3.) merged in #3556, see docs here for allowed_clouds field: https://skypilot.readthedocs.io/en/latest/reference/config.html

We will keep working on the other items.

[github-actions]

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[github-actions]

This issue was closed because it has been stalled for 10 days with no activity.