Connect KCI to a Kubernetes cluster

Kingsoft Cloud Container Instance (KCI) provides a serverless container service. KCI enables you to run containers in the cloud without having to purchase or manage the underlying servers, nor caring about how these containers are deployed or scheduled on the underlying physical resources.

Virtual Kubelet is an open source Kubernetes Kubelet implementation. Virtual nodes are implemented by using Virtual Kubelet. Virtual nodes allow KCI to serve as pods in a Kubernetes cluster. More precisely, KCI schedules and manages pod resources, and Kubernetes manages business loads as the business orchestration platform.

After KCI manages the infrastructure beneath pods, Kubernetes no longer needs to create or start pods or manage resource utilization of the underlying virtual machines. Instead, KCI ensures the availability of the resources that the pods need.

This topic describes how to connect KCI to a Kubernetes cluster by using virtual nodes.

Prerequisites

• You have created a Kubernetes cluster on Kingsoft Cloud. For more information, see Create a cluster.
• You have activated KCI.

Deploy virtual nodes

You can deploy virtual nodes in both Kubernetes clusters of Kingsoft Cloud Container Engine (KCE) and Kubernetes clusters built by yourself on Kingsoft Cloud.

Preparations

The following table describes the items that you must prepare before you deploy Virtual Kubelet.

Sample YAML file

The following YAML file is used to deploy virtual nodes in a Kubernetes cluster built by yourself. You can modify the parameters in the file as required.

vk-sa-rb.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: virtual-kubelet-sa
  namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: virtual-kubelet-rb
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: virtual-kubelet-sa
    namespace: kube-system

vk.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: virtual-kubelet
  namespace: kube-system
  labels:
    k8s-app: virtual-kubelet
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: virtual-kubelet
  template:
    metadata:
      name: virtual-kubelet
      labels:
        k8s-app: virtual-kubelet
    spec:
      serviceAccountName: virtual-kubelet-sa
      containers:
        - name: virtual-kubelet
          image: hub.kce.ksyun.com/ksyun/kci-virtual-kubelet:v1.6.0
          args:
            - --provider=kingsoftcloud
            # Specify the name of the virtual node. The name must conform to the naming rules for Kubernetes resources and be different from existing node names.
            - --nodename=${nodename_string}  
            # If the dnsPolicy field of the pod is set to ClusterFirst, specify the DNS address of the cluster. Multiple DNS addresses must be separated with commas (,).
            - --cluster-dns=${cluster-dns_string}
            # If the dnsPolicy field of the pod is set to Default, specify the DNS address of the virtual node. Multiple DNS addresses must be separated with commas (,).
            - --cluster-dns=${cluster-dns_string}
            # Specify the domain name of the cluster. Default value: cluster.local.
            - --cluster-domain=${cluster-domain_string}
            # Specify whether to delete the default taint generated when the virtual node is created. If the default taint is deleted, new pods can be scheduled to the node.
            - --disable-taint
            # Specify whether to use Leases for node heartbeats. We recommend that you use Leases for Kubernetes 1.13 and later.
            - --enable-node-lease   
          imagePullPolicy: IfNotPresent
          env:
            - name: VKUBELET_POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: KCI_ACCESS_KEY
              value: ${access_key}
            - name: KCI_SECRET_KEY
              value: ${secret_key}
            - name: KCI_REGION
              value: ${kci_region}  
            - name: KCI_CLUSTER_ID
              value: ${cluster_id} 
            - name: KCI_SUBNET_ID
              value: ${subnet_id}  
            - name: KCI_SECURITY_GROUP_IDS
              value: ${security_group_ids}  
            # Specify whether to enable kube-proxy for the virtual node. If kube-proxy is enabled, KCI pods scheduled to the virtual node are allowed to access ClusterIP services.
            - name: KCI_KUBE_PROXY
              value: ${true/false}   
            # If kube-proxy is enabled for a cluster built by yourself, you need to specify the API server address of the cluster in the format of IP address:Port number.
            - name: KCI_CLUSTER_SERVER
              value: ${cluster_server}
            # If the API server address is not specified and the cluster is not allowed to access the Internet, you can configure the following environment variable to call the Kingsoft Cloud API over the internal network:
            - name: KCI_REQUEST_INTERNAL

Deploy virtual nodes and check the status of the deployed virtual nodes.

# kubectl apply -f vk-sa-rb.yaml
# kubectl apply -f vk.yaml
# kubectl get node
NAME                STATUS   ROLES   AGE     VERSION
10.0.11.220         Ready    node    3d18h   v1.17.6
10.0.11.27          Ready    node    3d18h   v1.17.6
virtual-kubelet01   Ready    agent   55m     v1.17.6-vk-v1.6.0

Appendix: Parameter settings of Virtual Kubelet

The following table describes the parameters required for deploying Virtual Kubelet.

Note: KCI supports the dnsConfig field. You can also configure the DNS address for the pod by using the dnsPolicy and dnsConfig fields. If the DNS service is a ClusterIP service, enable kube-proxy for the pod to allow the pod to access ClusterIP services.

Schedule pods to virtual nodes

When KCI pods are created through virtual nodes in a Kubernetes cluster, you can define template annotations in the YAML file to bind pods to security groups and allocate resources. The following table describes the annotations that KCI supports.

Note:

1. To specify the specifications of a KCI pod, you need to set both the k8s.ksyun.com/kci-instance-cpu and k8s.ksyun.com/kci-instance-memory parameters. Values of the parameters must comply with the specifications supported by KCI. For more information, see Resource specifications.
2. If the specifications of a KCI pod are not specified, KCE automatically adjusts the pod specifications. For more information, see Specify KCI pod specifications.
3. The virtual-kubelet.io/provider:kingsoftcloud taint is generated by default when a virtual node is created. To schedule a KCI pod to the virtual node, you can configure a toleration for the pod or specify the node for scheduling.

Sample workload YAML file (specify the node for scheduling):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-kci
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations:
            "k8s.ksyun.com/kci-instance-cpu" : "4"
            "k8s.ksyun.com/kci-instance-memory" : "8"
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet01

Sample workload YAML file (configure a toleration):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-kci
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations:
            "k8s.ksyun.com/kci-instance-cpu" : "4"
            "k8s.ksyun.com/kci-instance-memory" : "8"
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      tolerations:
      - key: "virtual-kubelet.io/provider"
        operator: "Equal"
        value: "kingsoftcloud"
        effect: "NoSchedule"

Note: You must configure annotations in PodSpec rather than DeploymentSpec.

Verification

Log in to the KCE console. On the Cluster page, click the name of the cluster that you want to manage. On the page that appears, choose Workload > Deployment in the left navigation pane to view the status of resource deployment.

Deployment status:

Pod status:

Log in to the KCI console and view the creation status of KCI pods.

Limits

After Kubernetes is integrated with KCI, KCI runs on a virtual node instead of an actual Kubernetes node. As a result, the following Kubernetes-native features that are dependent on Kubernetes nodes, Kubelet, or kube-proxy are not supported:

• Running DaemonSet pods on virtual nodes is not supported.
• HostPath volumes are not supported.
• Running containers in privileged mode is not supported.
• Readiness probe and liveness probe are not supported.
• Running an init container is not supported.
• Services whose type is NodePort are not supported.
• Setting Pod to HostNetwork is not supported.