Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: azure support #84

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ rules:
- infrastructure.cluster.x-k8s.io
resources:
- awsclusters
- azureclusters
- hetznerclusters
- kubevirtclusters
- nutanixclusters
Expand Down
4 changes: 3 additions & 1 deletion controllers/kamajicontrolplane_controller_cluster_patch.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ func (r *KamajiControlPlaneReconciler) patchCluster(ctx context.Context, cluster
switch cluster.Spec.InfrastructureRef.Kind {
case "AWSCluster":
return r.patchGenericCluster(ctx, cluster, endpoint, port, false)
case "AzureCluster":
return r.patchGenericCluster(ctx, cluster, endpoint, port, false)
case "HetznerCluster":
return r.patchGenericCluster(ctx, cluster, endpoint, port, false)
case "KubevirtCluster":
Expand Down Expand Up @@ -70,7 +72,7 @@ func (r *KamajiControlPlaneReconciler) checkOrPatchVSphereCluster(ctx context.Co
return nil
}

//+kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=kubevirtclusters;nutanixclusters;packetclusters;awsclusters;hetznerclusters,verbs=patch
//+kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=kubevirtclusters;nutanixclusters;packetclusters;awsclusters;hetznerclusters;azureclusters,verbs=patch
//+kubebuilder:rbac:groups=infrastructure.cluster.x-k8s.io,resources=kubevirtclusters/status;nutanixclusters/status;packetclusters/status,verbs=patch

func (r *KamajiControlPlaneReconciler) patchGenericCluster(ctx context.Context, cluster capiv1beta1.Cluster, endpoint string, port int64, patchStatus bool) error {
Expand Down
172 changes: 172 additions & 0 deletions docs/providers-azure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
# Kamaji and Azure

The Kamaji Control Plane provider was able to create an _Azure_ backed Kubernetes cluster by providing Kamaji Control Planes.

```
NAME READY SEVERITY REASON SINCE MESSAGE
Cluster/capi-quickstart True 31m
├─ClusterInfrastructure - AzureCluster/kamaji-quickstart-control-plane True 31m
├─ControlPlane - KamajiControlPlane/kamaji-azure-127
└─Workers
└─MachineDeployment/capi-quickstart-md-0 True 28s
└─3 Machines... True 12m See capi-quickstart-md-0-6848dccdffxn5j9b-cjgp5, capi-quickstart-md-0-6848dccdffxn5j9b-gk95g, ...
```

## Example manifests

The said cluster has been created with the following manifests.

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: capi-quickstart
namespace: default
spec:
clusterNetwork:
apiServerPort: 443
pods:
cidrBlocks:
- 10.244.0.0/16
controlPlaneRef:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KamajiControlPlane
name: kamaji-quickstart-control-plane
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureCluster
name: capi-quickstart
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureCluster
metadata:
name: capi-quickstart
namespace: default
spec:
identityRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureClusterIdentity
name: azure-identity
location: westeurope
controlPlaneEnabled: false
networkSpec:
subnets:
- name: node-subnet
role: node
vnet:
name: workload
resourceGroup: test-resource-group
subscriptionID: 00000000-0000-0000-0000-000000000000
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureClusterIdentity
metadata:
labels:
clusterctl.cluster.x-k8s.io/move-hierarchy: "true"
name: azure-identity
spec:
allowedNamespaces: {}
clientID: 00000000-0000-0000-0000-000000000000
clientSecret:
name: azure-client-secret
namespace: azure
tenantID: 00000000-0000-0000-0000-000000000000
type: ServicePrincipal
---
apiVersion: controlplane.cluster.x-k8s.io/v1alpha1
kind: KamajiControlPlane
metadata:
name: kamaji-quickstart-control-plane
namespace: default
spec:
apiServer:
extraArgs:
- --cloud-provider=external
controllerManager:
extraArgs:
- --cloud-provider=external
dataStoreName: default
addons:
coreDNS: { }
konnectivity: { }
kubeProxy: { }
kubelet:
cgroupfs: systemd
preferredAddressTypes:
- ExternalIP
- InternalIP
- Hostname
network:
serviceType: LoadBalancer
deployment:
replicas: 2
version: 1.30.0
---
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
name: capi-quickstart-md-0
namespace: default
spec:
clusterName: capi-quickstart
replicas: 2
selector:
matchLabels: null
template:
spec:
bootstrap:
configRef:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: capi-quickstart-md-0
clusterName: workload
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureMachineTemplate
name: capi-quickstart-md-0
version: v1.30.0
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureMachineTemplate
metadata:
name: capi-quickstart-md-0
namespace: default
spec:
template:
spec:
osDisk:
diskSizeGB: 128
osType: Linux
sshPublicKey: ""
vmSize: Standard_DS3_v2
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
name: capi-quickstart-md-0
namespace: default
spec:
template:
spec:
files:
- contentFrom:
secret:
key: worker-node-azure.json
name: workload-md-0-azure-json
owner: root:root
path: /etc/kubernetes/azure.json
permissions: "0644"
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
name: '{{ ds.meta_data["local_hostname"] }}'
preKubeadmCommands: []
```

## Technical considerations

The Cluster API Azure infrastructure provider supports starting from [v1.13.0](https://github.com/kubernetes-sigs/cluster-api-provider-azure).
The Azure Cluster API operator will create some additional network components for the control plane which are not needed but do not hinder the cluster from working.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly something I was curious about: from my understanding, the CAPZ will create a Load Balancer that would then be used for the AzureCluster Control Plane Endpoint.

That resource would be billed despite not being used, even if you delete it since the CAPZ infrastructure controller will recreate it back, isn't it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't try to remove the componenents. I can have a deeper look into it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started a PR on the azure operator

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lost this message, link to the CAPZ PR: kubernetes-sigs/cluster-api-provider-azure#4611


Once the cluster has been provisioned, you need to install the [Azure Cloud Controller Manager](https://github.com/kubernetes-sigs/cloud-provider-azure).