If you already have a managed Kubernetes cluster like Amazon EKS or Azure AKS or an on-premise Kubernetes cluster, you can register it in gopaddle for building and deploying applications.

📌 Supported Kubernetes versions: v1.20, v1.20, v1.22

📌 Currently gopaddle uses SSH tunnel via Bastion Host to connect to an all private Kubernetes access type. This requires the SSH port on Bastion Host to be opened to the public internet. In case of AWS EKS clusters, gopaddle is working on a Secure Systems Manager (SSM) session based tunnelling to connect to an all private EKS cluster. This is a work in progress and will be rolled out in gopaddle v2.4.6.

Registering an AWS EKS Cluster with gopaddle

Create an IAM User with least privileges. Let's call this User as the - gopaddle IAM User whose credentials will be used to register the EKS cluster with gopaddle.
1. Create an IAM policy with the least privileges as defined here;
2. Create an IAM User with programmatic API access and associate the IAM policy defined.
3. Grab the Access Key and the Secret Key of the IAM User. These credentails will be used to register the EKS Cluster in gopaddle.
Configure Kubernetes access control for the gopaddle IAM User
1. Connect to the Kubernetes API server in your local desktop environment and update the Kubernetes Role Based Access Control (RBAC) to allow the gopaddle IAM User to access the Kubernetes resources.
  1. Install AWS CLI in your local environment
  2. Install kubectl in your local environment
  3. Connect to your EKS Cluster using AWS CLI
    
    For an EKS Cluster with public or public/private access point
    Follow the instructions here to connect to your EKS cluster. For example, you can configure the AWS CLI to use the credentials of the IAM User who created the EKS Cluster and update the kubeconfig file in the local machine as below:
    
    aws configure
    aws eks update-kubeconfig --region <eks-region> --name <eks-cluster-name>
    
    Proceed to step 3.
    
    For an EKS Cluster with private access endpoint and a bastion host
    You can connect to the private Kubernetes access endpoint via Systems Manager Session.
    
    📌 Bastion Host must be within the same VPC as the EKS cluster and must have access to the Kubernetes API server endpoint.
    
    📌 INBOUND access to port 22 of the Bastion Host must be open.
    
    Create a Systems Manager Host Configuration for the bastion host. You can follow the AWS Systems Manager Quick Start guide to create a host configuration for the bastion host.
    For example, you can open this link in the browser. https://<eks-region>.console.aws.amazon.com/systems-manager/quick-setup?region=<eks-region>
    Choose Create to create a new Host Configuration
    
    Choose Host Management Option and choose Create;
    
    In the creation wizard, select the instances based on the tags or select the instance ID manually and choose Create.
    Once the Host Configuration is complete, proceed with the next steps.
    
    Install SSM Agent in the Bastion Host. For example, you can run these commands in an Ubuntu 18.04 based Bastion Host.
    ssh -i <pemfile> ubuntu@<bastionhost-public-ip>
    sudo apt update
    sudo snap install amazon-ssm-agent --classic
    sudo snap switch --channel=candidate amazon-ssm-agent
    sudo snap start amazon-ssm-agent
    sudo snap services amazon-ssm-agent
    
    You should be able to see the ssm agent is listed as below.
    
    Service Startup Current Notes
    amazon-ssm-agent.amazon-ssm-agent enabled active -
    
    Note: If the ssh to the bastion host fails with this error, make sure the port 22 is open in the security group for INBOUND access from global CIDR 0.0.0.0/0
    
    ssh: connect to host <bastionhost-public-ip> port 22: Operation timed out
    
    Install SSM plugin in your local machine
    Create an SSM IAM User with the SSM priviledges - AmazonSSMFullAccess and provide programatic access to the AWS APIs.
    Grab the SSM IAM User credentials.
    In your local machine, configure the SSM IAM User and the EKS region.
    
    aws configure
    Make sure the Access Key, Secret Key and the Region are added to the ~/.aws/config file.
    cat ~/.aws/config
    [default]
    aws_access_key_id=<SSMUser_ACCESS_KEY>
    aws_secret_access_key=<SSMUser_SECRET_KEY>
    region = <EKS_REGION>
    
    In your local machine, start the SSM port forwarding session.
    
    aws ssm start-session --target <instance ID of the bastion host> --document-name AWS-StartPortForwardingSessionToRemoteHost --parameters '{"host":["EKS API Server endpoint DNS Name"],"portNumber":["443"], "localPortNumber":["9444"]}'
    
    If you get the below error, make sure the EKS point is just a DNS Name and NOT a URL.
    
    SessionId: gp-ssm-user-0057180c6d1d1ec44 : lookup https://CC725FE9FA351D0C306F597C47491BE7.gr7.us-east-1.eks.amazonaws.com: no such host
    
    Form the Kubernetes Configuration file - kube.conf. Edit the template below and replace the configurations based on your EKS cluster
    
    ---
    apiVersion: v1
    clusters:
    - cluster:
    server: https://localhost:9444
    insecure-skip-tls-verify: true
    name: <EKS_CLUSTER_ARN>
    contexts:
    - context:
    cluster: <EKS_CLUSTER_ARN>
    user: cluster/<EKS_CLUSTER_NAME>
    name: <EKS_CLUSTER_ARN>
    current-context: <EKS_CLUSTER_ARN>
    kind: Config
    preferences: {}
    users:
    - name: cluster/<EKS_CLUSTER_NAME>
    user:
    exec:
    apiVersion: client.authentication.k8s.io/v1beta1
    args:
    - --region
    - <EKS_REGION>
    - eks
    - get-token
    - --cluster-name
    - <EKS_CLUSTER_NAME>
    command: /usr/local/bin/aws
    env:
    - name: AWS_ACCESS_KEY_ID
    value: <EKS_OWNER_ACCESS>
    - name: AWS_SECRET_ACCESS_KEY
    value: <EKS_OWNER_SECRET>
    
    📌 AWS does not accept localhost as a SAN IP (Subject Alternative Name) for the EKS Cluster. Because of this, the certificate authority data in the Kubernetes config file cannot be used when querying the Kubernetes API server via a tunneling proxy. We need to skip the TLS verification in this scenario.
    
    Set the Kubernetes configuration file.
    
    export KUBECONFIG=kube.conf
  4. Check cluster connectivity
    
    kubectl cluster-info
    Kubernetes control plane is running at https://localhost:9444
    CoreDNS is running at https://localhost:9444/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
    
    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
  1. If the above command fails with the below error make sure the INBOUND access to port 443 from the Bastion Host Security Group is open in the Additional Security Group of the EKS cluster.
    W1213 13:04:18.835859 17344 transport.go:243] Unable to cancel request for *exec.roundTripper
    I1213 13:04:18.835992 17344 versioner.go:56] Remote kubernetes server unreachable
    
    Edit the Additional Security Group in the EKS Network Configuration.
    
    Add the INBOUND Access to the Bastion Host Security Group

Create a ClusterRole and a Cluster RoleBinding

gopaddle IAM User needs these permissions to discover and create Kubernetes resources while building a container or deploying an application on this cluster.

kubectl apply -f https://gp-cloudformation-roles.s3.amazonaws.com/gp-external-eks-cluster-role.yaml
kubectl apply -f https://gp-cloudformation-roles.s3.amazonaws.com/gp-external-eks-cluster-rolebinding.yaml

Update the aws-auth configMap with the ClusterRole and the gopaddle IAM User.
```
kubectl edit cm aws-auth -n kube-system
```

Edit the contents to include the mapUsers section. Replace the AWS_ACCOUNT_ID with the EKS cluster Account ID and the GOPADDLE_IAM_USERNAME created in step (1).

apiVersion: v1
data:
...
  mapUsers: |
    - userarn: arn:aws:iam::<AWS_ACCOUNT_ID>:user/<GOPADDLE_IAM_USERNAME>
      username: <GOPADDLE_IAM_USERNAME>
      groups:
        - gopaddle
kind: ConfigMap

Register the cluster with gopaddle
1. In the gopaddle UI, navigate to the Infrastructure section in the left panel and select Cluster. Click on Add Cluster and select Register an external Cluster
  
  In the General Step, ensure the Cluster Name is same as the EKS cluster name in the AWS Console and choose the Cluster Provider as AWS.
2. In the Authentication Step, provide the gopaddle IAM User access credentials and the cluster access details.
  1. For an EKS cluster with public and public/private access endpoint
    Provide the Certificate Authority data, the gopaddle IAM User credentials created in step 1, the EKS cluster endpoint and the ARN details.
  2. For an EKS cluster with private access endpoint
    Skip TLS Verification and provide the gopaddle IAM User credentials created in step 1, the EKS cluster endpoint and the ARN details.
3. In the Bastion Host Step, either skip or configure the bastion host details based on the EKS cluster access type.
  1. For an EKS cluster with public and public/private access endpoint
    In the Bastion Host Step, skip the Use Bastion Host ? configuration and click on Finish.
  2. For an EKS cluster with private access endpoint
    Provide the bastion host public IP, SSH port, upload the bastion host PEM file and click on Finish.
(Optional) From the AWS Dashboard, open the firewall ports in the Cluster security group of the EKS cluster

Port	Destination	Purpose
32000	0.0.0.0/0 - If you are accessing the Grafana dashboard from a public network. <VPN CIDR Range> - If you are access the Grafana dashboard from within your VPN.	Grafana Dashboard

Registering a Google GKE Cluster

a) Install gcloud CLI in your local environment

b) Install jq. For instance, you can install jq in Ubuntu environments, as below:

apt update
apt install jq

c) Login to your Google Cloud account using gcloud. For more information check out - https://cloud.google.com/sdk/gcloud/reference/auth

gcloud auth login

d) Describe the GKE cluster and get the Cluster CA Certificate.

gcloud container clusters describe <clustername> --zone <cluster-zone> --project <project-name> --format json | jq '.masterAuth.clusterCaCertificate'

e) Generate a Kubeconfig file by replacing the values gathered in the previous step.

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <clusterCaCertificate>
    server: <server-end-point>
  name: gke_<project>_<zone>_<clustername>
contexts:
- context:
    cluster: gke_<project>_<zone>_<clustername>
    user: gke_<project>_<zone>_<clustername>
  name: gke_<project>_<zone>_<clustername>
current-context: gke_<project>_<zone>_<clustername>
kind: Config
preferences: {}
users:
- name: gke_<project>_<zone>_<clustername>
  user:
    auth-provider:
      name: gcp

f) In the gopaddle UI, click on the settings option in the top navigation panel and select Cloud.

g) Click on Create to create a new Cloud Account.

h) Choose Google as the cloud provider.

i) Provide the Service Account Email ID and upload the Service Account Key file in JSON format.

j) Once the Google Cloud Account is registered, navigate to the Infrastructure section in the left panel and select Cluster.

k) Click on Add Cluster and choose Register an external Cluster.

l) Provide the Google GKE Cluster name, cluster public endpoint and the port number.

m) Choose the Cloud provider as Google. Select the Cloud Account and the Google project ID and the Regions.

n) In the Authentication step, upload the Kubeconfig filegenerated in step (e).

0) Click on Finish, to register the cluster with gopaddle.

Registering an on-premise Kubernetes Cluster

Navigate to the Infrastructure option in the left panel and select Clusters
Click on Add Cluster and select Register an external Cluster
In the Cluster registration wizard, provide a Cluster Name, its IP address or Fully Qualified Domain Name (FQDN), and the Kubernetes proxy / master port.
Choose the Cluster Provider type.

In the Authentication Step, choose the type of the authentication to the Kubernetes API Server.

In case of Password based authentication provide the Username and Password of the master.

In case of Kube Config file based authentication, upload the Kubernetes Configuration file.

📌 If you have obtained the KubeConfig file from the Kubernetes master node, then make sure the server points to the remote IP and not localhost or 127.0.0.1.
📌 gopaddle supports only a secure HTTPS based server endpoints.
📌 Currently gopaddle does not support custom authentication plugins, except for the OIDC Provider plugin for AWS EKS clusters provisioned through gopaddle.

If you are connecting to the Kubernetes API server from the public internet, then skip the Bastion Host configuration. If you are accessing the Kubernetes API server via Bastion Host, provide the Bastion Host IP, SSH username, SSH port, Pem file.
Click on Finish, to register the cluster with gopaddle.

Firewall Access

gopaddle will connect with the Kubernetes cluster using the endpoint specified at the time of registration. Hence ensure that the INBOUND firewall ports are open for the kubernetes master IP and the port for INGRESS type for global CIDR (0.0.0.0/0).

In case of registering the cluster via Bastion Host, ensure that the SSH tunnel port is opened for the SSH server IP and port for INGRESS type for global CIDR (0.0.0.0/0).

Next - Adding a Docker Registry

Registering an AWS EKS Cluster with gopaddle

For an EKS Cluster with public or public/private access point

For an EKS Cluster with private access endpoint and a bastion host

Create a ClusterRole and a Cluster RoleBinding

Registering a Google GKE Cluster

Registering an on-premise Kubernetes Cluster

Firewall Access