gopaddle User Guide
  • 👋Welcome to gopaddle
  • Overview
    • 💡Getting Started
    • Register a Cluster
      • Register Rancher Prime - RKE2
      • Register K3S
      • Register MicroK8s
      • Register Kind
      • Register minikube
      • Register Kubeadm Cluster
      • Register AWS EKS Cluster
        • EKS Cluster with public or public/private access point
        • EKS Cluster with private access endpoint and a bastion host
        • Validate Cluster Connectivity
      • Register Azure AKS Cluster
      • Register Google GKE Cluster
      • Register Huawei Cloud Container Engine
    • Register GitHub Account
    • Register Jira Account
    • Register ChatGPT Assistant
    • 💻Kubernetes IDE
      • Filters
      • Editor
      • Flat vs Tree View
      • Developer Tools
    • 🙋AI Assistant
      • Chat with AI
      • Raise a Jira Ticket from Chat Window
      • Create Runbook from Chat Window
    • 📖Runbook Hub
      • Create Runbook Hub
      • Attach Runbook Hub to Cluster
      • Enhancing contents of Runbooks with AI
      • Detach Runbook Hub from Cluster
      • Syncing Runbook Hub with GitHub
      • Delete Runbook / Runbook Hub
    • ⏱️Installing Community Edition
      • MicroK8s Addon
        • On Ubuntu
        • On MacOS
      • Docker Desktop
      • SUSE Rancher Prime
      • Digital Ocean
      • Akamai Linode
      • Kind Cluster
      • Helm
      • Docker Compose
      • Accessing gopaddle UI
    • 📈Improving performance of resource discovery
    • Provision new Cluster
      • Register Cloud Account
        • AWS
          • Quickstart AWS Setup
          • IAM Access Policies
          • AWS Setup Script
        • Azure
          • Create Azure Application
          • Register Azure Cloud Authenticator
          • Register Azure Account
        • Google
      • Provision Clusters on Cloud
        • AWS EKS
          • AWS EKS Reference Architecture
          • Adding an AWS IAM Role (EKS Master / Node Pool)
          • Public EKS Cluster
          • All Private EKS Cluster (beta)
          • Creating a Node Pool
        • Azure AKS
          • Creating a Node Pool
          • Enable Public IP Node Access for Azure Deployments
          • VMSS Autoscaling Rules
        • Google GKE
          • Creating a Node Pool
  • 🔎Troubleshooting
    • Cluster Resource View Issues
      • Network Error ! ServerError: Response not successful: Received status code 503
      • Network Error ! TypeError: Failed to fetch
      • Network Error ! ServerParseError: Unexpected token 'j', "json: erro"... is not valid JSON
      • Updating Labels and Annotations does not get reflected in resources list
      • Filtered resources are not fully listed
    • Runbook Issues
      • Deleting a runbook from .gp.yaml does not detach annotation in resources
      • Deleting a Code Account from gopaddle UI does not detach annotation in resources
    • Jira Issues
      • Creating, Updating or Appending a Jira issue fails with error INVALID_INPUT
    • EKS Issues
    • Deployment Issues
    • Node Issues
    • Huawei Issues
Powered by GitBook
On this page
  • Increasing capacity of multipass VM in case of MacOS
  • Increasing capacity of kubeagent
  • Increase QPS and Burst limits of kubeagent
  1. Overview

Improving performance of resource discovery

PreviousAccessing gopaddle UINextProvision new Cluster

Last updated 1 year ago

As soon as a cluster is registered in gopaddle, gopaddle installs a few add-ons under gopaddle-servers namespace. The kubeagent add-on is responsible for discovering Kubernetes resources, tracking resource updates, and annotating the resources with runbook information. For large multi-node clusters, it is recommended to increase the capacity of the kubeagent to improve the performance of the Cluster Resource View and Runbook discovery.

Increasing capacity of multipass VM in case of MacOS

  1. Before increasing the capacity of the kubeagent, check the current memory capacity and consumption of the multipass microk8s vm on MacOS.

multipass info microk8s-vm
  1. To increase the capacity of the microk8s multipass VM on MacOS, please follow the prerequisites step 5 stated .

Increasing capacity of kubeagent

  1. Enable metrics-server to check the current consumption of the kubeagent pod.

microk8s enable metrics-server
  1. Get the current usage of the kubeagent pod.

kubectl top pod -n gopaddle-servers
  1. If the resource usage of the kubeagent exceeds the limits set for the deployment, increase the limits of the kubeagent deployment. The default requests and limits of the kubeagent deployment is as below:

        resources:
          limits:
            cpu: 1200m
            memory: 4G
          requests:
            cpu: 500m
            memory: 1G
  1. Update the requests and limits of the kubeagent, if necessary.

kubectl edit deploy/kubeagent -n gopaddle-servers
  1. Once the kubeagent deployment is updated, wait for the kubeagent to move to ready state

kubectl wait --for=condition=ready pod -l app=kubeagent -n gopaddle-servers  --timeout=15m
  1. In the gopaddle UI, Refresh the cluster View -> Resources to check the performance

Increase QPS and Burst limits of kubeagent

  1. Check the kubeagent logs and look for any throttling issues:

kubectl logs -l app=kubeagent -n gopaddle-servers --all-containers=true --tail=100

A sample throttling message would look like this:

Waited for 1.01761187s due to client-side throttling, not priority and fairness, request: GET:https://10.152.183.1:443/api/v1/nodes

  1. Increase the QPS and Burst parameters by editing the command line arguments of the container in the kubeagent deployment. The default QPS and Burst values are 100 and 200 respectively.

  2. Edit the kubeagent deployment.

kubectl edit deploy/kubeagent -n gopaddle-servers
  1. Update the command line arguments in the kubeagent specification.

containers:
      - args:
        - |-
          #!/bin/bash
          # echo "----------- start conatainer ------------"
          ./kubeagent --qps=200 --burst=400
        command:
        - /bin/sh
        - -c
  1. Once the kubeagent deployment is updated, wait for the kubeagent to move to ready state.

kubectl wait --for=condition=ready pod -l app=kubeagent -n gopaddle-servers  --timeout=15m
  1. In the gopaddle UI, Refresh the cluster View -> Resources to check the performance

here
📈
Page cover image