Skip to content

electrocucaracha/krd

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Kubernetes Reference Deployment

License GitHub Super-Linter Ruby Style Guide

visitors Scc Code Badge Scc COCOMO Badge

Summary

This project offers a reference for deploying a Kubernetes cluster. Its ansible playbooks allow to provision a deployment on Bare-metal or Virtual Machines.

The POD Descriptor File (PDF) is a YAML file which defines the setup used by KRD. The default PDF file can be overwritten to try different scenarios. There are different setups located in the samples folder which can be used.

Linux Distros Supported

Name Version
Ubuntu 18.04/20.04/22.04
openSUSE Leap
Fedora 37/38/39/40

Components

Name Description Source Status
Kubernetes Base Kubernetes deployment kubespray Done
Virtlet Allows to run VMs configure-virtlet.yml Tested
Kubevirt Allows to run VMs Tested
Virtink Allows to run VMs Tested
Multus Provides Multiple Network support in a pod Tested
NFD Node feature discovery nfd role Tested
Istio Service Mesh platform Tested
PMEM Persistent Memory CSI pmem role Implemented
QAT QuickAssist Technology Plugin qat_plugin role Tested
SR-IOV Single Root Input/Output Virtualization Plugin sriov_plugin role Implemented
Knative Platform for managing serverless workloads Tested
Kong Ingress Controller Tested
HAProxy Ingress Controller Tested
MetalLB Provides a network load-balancer implementation Tested
Falco Deploys a Falco services Tested
Rook Storage Operator Tested
OPA Gatekeeper Policy Controller Tested
Kyverno Kubernetes Native Policy Management Tested
Kubewarden Kubernetes Native Policy Management Tested
kube-monkey Netflix's Chaos Monkey for Kubernetes clusters Implemented
Nephio Management platform to on-board NFs at scale Implemented
LocalAI Allows to run LLMs locally Implemented
K8SGPT Scans K8s clusters for diagnosing issues Implemented
ArgoCD Declarative GitOps continuous delivery tool Implemented

Quick Deployment

The All-in-One bash script bash script offers the minimal unattended instructions required for deploying a KRD in a node. It can be externally consumed with one single command.

curl -fsSL http://bit.ly/KRDaio | bash

Installation

The Environment Setup section provides the steps required to install KRD dependencies and define the Cluster setup before its provisioning.

Provisioning

Override Default Cluster setup

Vagrant uses the default configuration to define the Virtual Machines used by the Kubernetes cluster, this configuration can be overwritten defining the config/pdf.yml file. The diagram tox environment generates the krd.png file to visualize the topology of the cluster.

tox -e diagram

Sample diagram

Install KRD actions

The KRD command script provides an easy way to install complementary Kubernetes projects to an existing cluster. Those projects are grouped in KRD actions and it's possible to get the current list of current supported actions executing the following instruction:

./krd_command.sh -h

The actions which require the execution of a distributed commands were implemented through the install_k8s_addons KRD action. This action uses the KRD_ADDONS_LIST environment variable to specify the Ansible playbook to be executed.

KRD_ADDONS_LIST=virtlet ./krd_command.sh -a install_k8s_addons

Note: Some KRD AddOns have a corresponding validation script in the tests folder.

Day-2 Operations

The functions defined in this project covers the lifecycle of a Kubernetes cluster. In other words, its possible to add more nodes, upgrade the existing deployment or remove the services. The following instruction shows how to upgrade the existing Kubernetes cluster to v1.18.10 using the Kubespray version v2.14.2:

KRD_KUBE_VERSION=v1.18.10 KRD_KUBESPRAY_VERSION=v2.14.2 ./krd_command.sh -a upgrade_k8s

Environment variables

Global

Name Default Description
KRD_DEBUG false Enables verbose execution
KRD_ANSIBLE_DEBUG false Enables ansible verbose execution
PKG_DEBUG false Enables cURL package installer verbose execution
KRD_KUBE_VERSION v1.30.4 Specifies the Kubernetes version to be upgraded
KRD_KUBESPRAY_VERSION v2.26.0 Specifies the Kubespray version to be used during the upgrade process
KRD_KUBESPRAY_REPO https://github.com/kubernetes-sigs/kubespray Specifies the Git repository to fetch the Kubespray's source code
KRD_ENABLE_TESTS false Enables the functional tests during the deployment process
KRD_HELM_CHART Specifies the Helm chart to be installed
KRD_FOLDER /opt/krd KRD source code destination folder
KRD_ACTIONS_LIST install_k8s Specifies a list of KRD actions to be installed during the All-in-One execution
KRD_ADDONS_LIST addons Specifies the Kubernetes AddOns to be installed by install_k8s_addons
KRD_HUGEPAGES_ENABLED true Enable/Disable HugeTLB Pages on cluster nodes
KRD_QAT_PLUGIN_MODE dpdk Specifies the deployment mode for Intel QuickAssist Plugin
KRD_KUBESPHERE_DEVOPS_ENABLED true Enable/Disable KubeSphere DevOps System
KRD_KUBESPHERE_METRICS_SERVER_ENABLED false Enable/Disable Horizontal Pod Autoscaler on KubeSphere
KRD_KUBESPHERE_SERVICEMESH_ENABLED false Enable/Disable KubeSphere Service Mesh
KRD_ENABLE_ISTIO_ADDONS false Enable/Disable Istio AddOns(Grafana, Kiali and Prometheus)
KRD_METALLB_ADDRESS_POOLS_LIST Specifies a list of L2 address pools for MetalLB configuration

Kubespray

Name Default Description
KRD_NETWORK_PLUGIN flannel Choose network plugin (calico, canal, cilium, contiv, flannel, weave)
KRD_FLANNEL_BACKEND_TYPE host-gw Type of flannel backend to use (vxlan, host-gw, udp)
KRD_CILIUM_TUNNEL_MODE disabled Encapsulation mode for communication between nodes (disabled, vxlan, geneve)
KRD_CILIUM_KUBE_PROXY_REPLACEMENT probe Specifies the type of kube-proxy replacement.
KRD_CILIUM_ENABLE_BPF_MASQUERADE true Enable/Disable native IP masquerade support in eBPF
KRD_CALICO_IPIP_MODE Never Configures Calico IP in IP encapsulation (Always, Never, CrossSubnet)
KRD_CALICO_VXLAN_MODE Never Configures Calico VXLAN encapsulation (Always, Never, CrossSubnet)
KRD_CALICO_NETWORK_BACKEND bird Configures Calico Network backend (bird, vxlan, none)
KRD_KUBE_PROXY_MODE ipvs Choose kube-proxy mode (iptables, ipvs)
KRD_DOWNLOAD_RUN_ONCE true Download images and binaries only once and then push them to the cluster nodes
KRD_DOWNLOAD_LOCALHOST true Make localhost the download delegate
KRD_MULTUS_ENABLED false Enable/Disable Kubernetes Multus CNI
KRD_CONTAINER_RUNTIME containerd Specifies the Container Runtime to be used for deploying kubernetes
KRD_DASHBOARD_ENABLED false Enable/Disable Kubernetes Dashboard
KRD_CERT_MANAGER_ENABLED true Enable/Disable Kubernetes Cert-Manager
KRD_INGRESS_NGINX_ENABLED true Enable/Disable NGINX Ingress Controller
KRD_KATA_CONTAINERS_ENABLED false Enable/Disable Kata Containers Runtime
KRD_CRUN_ENABLED false Enable/Disable crun
KRD_YOUKI_ENABLED false Enable/Disable youki
KRD_GVISOR_ENABLED false Enable/Disable gVisor
KRD_MANUAL_DNS_SERVER Set to use a custom cluster DNS
KRD_REGISTRY_MIRRORS_LIST Specifies a list of additional registry mirrors
KRD_INSECURE_REGISTRIES_LIST Specifies a list of insecure-registries (IP address or domain name)
KRD_LOCAL_VOLUME_PROVISIONER_ENABLED true Enable/Disable Local volume provisioner
KRD_DOCKER_VERSION latest Specifies the Docker version to be used for deploying Kubernetes
KRD_CONTAINERD_VERSION 1.7.11 Specifies the ContainerD version to be used for deploying Kubernetes
KRD_ENABLE_NODELOCALDNS true Enable/Disable NodeLocal DNSCache
KRD_NDOTS 1 Threshold for the number of dots which must appear in name resolution
KRD_RESOLVCONF_MODE none Specifies the DNS setup for non-k8s containers.
KRD_KUBE_PROXY_SCHEDULER sh Specifies the IPVS scheduling algorithm for allocating connections
KRD_METALLB_ENABLED false Enable/Disable MetalLB load-balancer
KRD_METALLB_ADDRESS_POOLS_LIST Specifies a lists of Layer 2 Address pools for MetalLB
KRD_KUBERNETES_AUDIT false Enable/Disable Auditing
KRD_KUBERNETES_AUDIT_WEBHOOK false Enable/Disable Audit Webhook
KRD_AUDIT_WEBHOOK_SERVER_URL Audit Webhook server URL
KRD_KUBELET_LOGFILES_MAX_NR 5 Maximum number of container log files that can be present for a container
KRD_KUBELET_LOGFILES_MAX_SIZE 10Mi Maximum size of the container log file before it is rotated
KRD_CONTAINER_CHECKPOINT_ENABLED false Enable/Disable Containers checkpoints creation

Contribution

This is an open project, several individuals contribute in different forms like coding, documenting, testing, spreading the word at events within others.

If you are interested in contributing, please refer to the code CONTRIBUTING guidelines.

Thanks to all the people who already contributed!

contributors

Visualization of the codebase