[Serve] Make controller regions/ choose from replica resources

[euclidgame]

Fixes #3364 .

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
  • Unit test for the case where all resources are specified with clouds, regions and zones.
  • Unit test for the case where all resources are specified with clouds and regions, only part of them have zones.
  • Unit test for mixed cases: some resources have clouds, regions and zones; some don't have regions or zones.
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: conda deactivate; bash -i tests/backward_compatibility_tests.sh

[cblmemo]

Thanks for adding this feature @euclidgame ! This is awesome. The PR looks mostly good for me. Left some discussions ;)

[cblmemo]

This is a little bit confusing. Lets do explicit for loop instead?

[cblmemo]

Is it possible to left blank if enable any region? same apply for zone.

[euclidgame]

Yes, it is possible, but it requires special handling to determine whether an empty set indicates that any region is allowed, which means we should only add new regions if it’s the first time. Using a placeholder makes it easier. Maybe I can use None instead of the specific strings?

[cblmemo]

Thanks for adding this @euclidgame ! Looks good to me. Left some discussions ;)

[cblmemo]

Thanks for fixing this @euclidgame ! Left some discussions ;)

[cblmemo]

Could we revert this?

[cblmemo]

same, could we revert this?

[cblmemo]

lets add the expected output of this group as previous comments.

[cblmemo]

Previously, if the controller resources specified in the config yaml has a cloud, we directly returned it. however, after adding this, we might still want to respect the region & zones for replica resoruces? (e.g. config yaml only says cloud=aws, but replica resoruces further suggested some regions)

skypilot/sky/utils/controller_utils.py

Lines 508 to 509 in 9201def

	if controller_resources_to_use.cloud is not None:
	return {controller_resources_to_use}

[euclidgame]

I use controller_resources_to_use as a filter after requested_clouds_with_region_zone without the previous early return:

    filtered_clouds = ({controller_cloud} if controller_cloud else
                       requested_clouds_with_region_zone.keys())

    # Filter regions and zones and construct the result
    result = set()
    for cloud_name in filtered_clouds:
        regions = requested_clouds_with_region_zone.get(cloud_name, {None: {None}})

        # Filter regions if controller_resources_to_use.region is specified
        filtered_regions = ({controller_region}
                            if controller_region else regions.keys())

        for region in filtered_regions:
            zones = regions.get(region, {None})

            # Filter zones if controller_resources_to_use.zone is specified
            filtered_zones = ({controller_zone} if controller_zone else zones)

            # Create combinations of cloud, region, and zone
            for zone in filtered_zones:
                resource_copy = controller_resources_to_use.copy(
                    cloud=clouds.CLOUD_REGISTRY.from_str(cloud_name),
                    region=region,
                    zone=zone)
                result.add(resource_copy)

Might be tedious :-), let me know your thoughts.

[cblmemo]

We currently only allows single resource for controller, which might simplify it a little bit. Pls check:

skypilot/sky/utils/controller_utils.py

Lines 487 to 497 in c6ae536

	# TODO(tian): Support multiple resources for the controller. One blocker
	# here is the semantic if controller resources use `ordered` and we want
	# to override it with multiple cloud from task resources.
	if len(controller_resources) != 1:
	with ux_utils.print_exception_no_traceback():
	raise ValueError(
	CONTROLLER_RESOURCES_NOT_VALID_MESSAGE.format(
	controller_type=controller.value.controller_type,
	err=f'Expected exactly one resource, got '
	f'{len(controller_resources)} resources: '
	f'{controller_resources}').capitalize())

[euclidgame]

Thanks for the suggestions @cblmemo ! All fixed, please review :-)