Skip to main content

Easily Manage your Application Shipment With Differentiated Configuration in Multi-Cluster

· 7 min read
Wei Duan

Under today's multi-cluster business scene, we often encounter these typical requirements: distribute to multiple specific clusters, specific group distributions according to business need, and differentiated configurations for multi-clusters.

KubeVela v1.3 iterates based on the previous multi-cluster function. This article will reveal how to use it to do swift multiple clustered deployment and management to address all your anxieties.

Before Starting​

  1. Prepare a Kubernetes cluster as the control plane of KubeVela.
  2. Make sure KubeVela v1.3 and KubeVela CLI v1.3.0 have been installed successfully.
  3. The list of Kubeconfig from sub clusters that you want to manage. We will take three clusters naming beijing-1, beijing-2 and us-west-1 as examples.
  4. Download and combine with Multi-Cluster-Demo to better understand how to use the KubeVela multi-cluster capabilities.

Distribute to Multiple Specified Clusters​

Distributing multiple specified clusters is the most basic multi-cluster management operation. In KubeVela, you will use a policy called topology to implement it. The cluster will be listed in the attribute clusters, an array.

First let's make sure switching kubeconfig to the control plane cluster, go with vela cluster join to include in the 3 clusters of Beijing-1, Beijing-2 and us-west-1:

➜   vela cluster join beijing-1.kubeconfig --name beijing-1
➜   vela cluster join beijing-2.kubeconfig --name beijing-2
➜   vela cluster join us-west-1.kubeconfig --name us-west-1
➜   vela cluster list
CLUSTER         TYPE            ENDPOINT                    ACCEPTED    LABELS
beijing-1       X509Certificate https://47.95.22.71:6443    true
beijing-2       X509Certificate https://47.93.117.83:6443   true
us-west-1       X509Certificate https://47.88.31.118:6443   true

Then open multi-cluster-demo, look into Basic.yaml:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: example-app
  namespace: default
spec:
  components:
    - name: hello-world-server
      type: webservice
      properties:
        image: oamdev/hello-world
        port: 8000
      traits:
        - type: scaler
          properties:
            replicas: 3
        - type: gateway
          properties:
            domain: testsvc-mc.example.com
            # classInSpec : true   If the sub clusters has Kubernetes versions below v1.20 installed, please add this field
            http:
              "/": 8000
  policies:
    - type: topology
      name: beijing-clusters
      properties:
        clusters: ["beijing-1","beijing-2"]

It can be seen that this app uses the component of type webservice and distributes 3 Deployments to beijing-1 and beijing-2 clusters through the topology policy.

Please note that the premise of successfully distributing resource into managed clusters is that it must contain the exactly same namespace as control plane did. Since each cluster has the default namespace by default, we won't be worry in this case. But suppose we change the namespace in basic.yaml to be multi-cluster, we will receive an error:

... 
 Status:        runningWorkflow

Workflow:

  mode: DAG
  finished: false
  Suspend: false
  Terminated: false
  Steps
  - id:9fierfkhsc
    name:deploy-beijing-clusters
    type:deploy
    phase:failed
    message:step deploy: step deploy: run step(provider=oam,do=components-apply): Found 1 errors. [(failed to apply component beijing-1-multi-cluster-0: HandleComponentsRevision: failed to create componentrevision beijing-1/multi-cluster/hello-world-server-v1: namespaces "multi-cluster" not found)]

Services:
...

In future versions of KubeVela, we plan to support a comprehensive Authentication System, more convenient and more securely to: create namespaces in managed cluster through the hub cluster in quick moves.

After creating the sub cluster's namespace, come back to the control plane cluster to create the application and ship out resources:

➜   vela up -f basic.yaml
Applying an application in vela K8s object format...
"patching object" name="example-app" resource="core.oam.dev/v1beta1, Kind=Application"
✅ App has been deployed 🚀🚀🚀
    Port forward: vela port-forward example-app
             SSH: vela exec example-app
         Logging: vela logs example-app
      App status: vela status example-app
  Service status: vela status example-app --svc hello-world-server

We use vela status <App Name> to view detailed infos about this app:

➜   vela status example-app
About:

  Name:         example-app
  Namespace:    default
  Created at:   2022-03-25 17:42:33 +0800 CST
  Status:       running

Workflow:

  mode: DAG
  finished: true
  Suspend: false
  Terminated: false
  Steps
  - id:wftf9d4exj
    name:deploy-beijing-clusters
    type:deploy
    phase:succeeded
    message:

Services:

  - Name: hello-world-server
    Cluster: beijing-1  Namespace: default
    Type: webservice
    Healthy Ready:3/3
    Traits:
      ✅ scaler      ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 60.205.222.30
  - Name: hello-world-server
    Cluster: beijing-2  Namespace: default
    Type: webservice
    Healthy Ready:3/3
    Traits:
      ✅ scaler      ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 182.92.222.128

Both the beijing-1 and beijing-2 have issued the corresponding resources, they also displayed external access IP addresses, and you can therefore make it public for your users.

Use Cluster Labels to Do Grouping​

In addition to the above basic need, we often encounter additional situations: cross-regional deployment to certain clusters, specify which cloud provider's cluster, etc. In order to achieve a similar goal, the labels feature can be used.

Here, suppose the us-west-1 cluster comes from AWS, we must additionally apply to the AWS cluster. You can use the vela cluster labels add command to tag the cluster. Of course, if there is more of AWS related clusters such as us-west-2, it will be handled as well after they were labeled:

➜  ~ vela cluster labels add us-west-1 provider=AWS
Successfully update labels for cluster us-west-1 (type: X509Certificate).
provider=AWS
➜  ~ vela cluster list
CLUSTER         TYPE            ENDPOINT                    ACCEPTED    LABELS
beijing-1       X509Certificate https://47.95.22.71:6443    true
beijing-2       X509Certificate https://47.93.117.83:6443   true
us-west-1       X509Certificate https://47.88.31.118:6443   true        provider=AWS

Next we update the basic.yaml to add an application policy topology-aws:

...
  policies:
    - type: topology
      name: beijing-clusters
      properties:
        clusters: ["beijing-1","beijing-2"]
    - type: topology
      name: topology-aws
      properties:
        clusterLabelSelector:
          provider: AWS

In order save your time, please deploy intermediate.yaml directly:

➜  ~ vela up -f intermediate.yaml

Review the status of the application again:

➜   vela status example-app

...

  - Name: hello-world-server
    Cluster: us-west-1  Namespace: default
    Type: webservice
    Healthy Ready:3/3
    Traits:
      ✅ scaler      ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 192.168.40.10

Differentiated Configuration​

Apart from above scenarios, we tend to have more application strategic needs, such as high availability of hoping to distribute 5 replicas. In this case, use the override policy:

...        
        clusterLabelSelector:
          provider: AWS
    -  type: override
       name: override-high-availability
       properties:
          components:
            - type: webservice
              traits:
              - type: scaler
                properties:
                  replicas: 5

At the same time, we hope that only AWS clusters can get high availability. Then we can expect KubeVela's workflow give us a hand. We use the following workflow: it aims to deploy this app by, first distributing to Beijing's clusters through the deploy-beijing policy, then distributing 5 copies to clusters which were labeled as AWS:

...                
                properties:
                  replicas: 5
  workflow:
    steps:
      - type: deploy
        name: deploy-beijing
        properties:
          policies: ["beijing-clusters"]
      - type: deploy
        name: deploy-aws
        properties:
          policies: ["override-high-availability","topology-aws"]

Then we attach the above policy and workflow to intermediate.yaml and make it to advanced.yaml:

...
  policies:
    - type: topology
      name: beijing-clusters
      properties:
        clusters: ["beijing-1","beijing-2"]
    - type: topology
      name: topology-aws
      properties:
        clusterLabelSelector:
          provider: AWS
    -  type: override
       name: override-high-availability
       properties:
          components:
            - type: webservice
              traits:
              - type: scaler
                properties:
                  replicas: 5
  workflow:
    steps:
      - type: deploy
        name: deploy-beijing
        properties:
          policies: ["beijing-clusters"]
      - type: deploy
        name: deploy-aws
        properties:
          policies: ["override-high-availability","topology-aws"]

Then deploy it, view the status of the application:

➜   vela up -f advanced.yaml
Applying an application in vela K8s object format...
"patching object" name="example-app" resource="core.oam.dev/v1beta1, Kind=Application"
✅ App has been deployed 🚀🚀🚀
    Port forward: vela port-forward example-app
             SSH: vela exec example-app
         Logging: vela logs example-app
      App status: vela status example-app
  Service status: vela status example-app --svc hello-world-serverapplication.core.oam.dev/podinfo-app configured
  
➜   vela status example-app

...

  - Name: hello-world-server
    Cluster: us-west-1  Namespace: default
    Type: webservice
    Healthy Ready:5/5
    Traits:
      ✅ scaler      ✅ gateway: Visiting URL: testsvc-mc.example.com, IP: 192.168.40.10

The above all are what we'd like to share with you for this time, thank you for reading and trying them out.

We invite you to explore KubeVela v1.3 for more to meet further complex requirements on business, such as dig deep in differentiated configurations to use override application policy to either override all resources on one type or only certain specific components.