This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

VMware vSphere

Preparing a VMware vSphere provider for EKS Anywhere

1: Requirements for EKS Anywhere on VMware vSphere
2: Preparing vSphere for EKS Anywhere
3: Customize OVAs: Ubuntu
4: Import OVAs
5: Custom DHCP Configuration
6:

1 - Requirements for EKS Anywhere on VMware vSphere

Preparing a VMware vSphere provider for EKS Anywhere

To run EKS Anywhere, you will need:

Prepare Administrative machine

Set up an Administrative machine as described in Install EKS Anywhere .

Prepare a VMware vSphere environment

To prepare a VMware vSphere environment to run EKS Anywhere, you need the following:

A vSphere 7+ environment running vCenter
Capacity to deploy 6-10 VMs
DHCP service running in vSphere environment in the primary VM network for your workload cluster
One network in vSphere to use for the cluster. EKS Anywhere clusters need access to vCenter through the network to enable self-managing and storage capabilities.
An OVA imported into vSphere and converted into a template for the workload VMs
User credentials to create VMs and attach networks, etc
One IP address routable from cluster but excluded from DHCP offering. This IP address is to be used as the Control Plane Endpoint IP

Below are some suggestions to ensure that this IP address is never handed out by your DHCP server.

You may need to contact your network engineer.
- Pick an IP address reachable from cluster subnet which is excluded from DHCP range OR
- Alter DHCP ranges to leave out an IP address(s) at the top and/or the bottom of the range OR
- Create an IP reservation for this IP on your DHCP server. This is usually accomplished by adding a dummy mapping of this IP address to a non-existent mac address.

Each VM will require:

2 vCPUs
8GB RAM
25GB Disk

The administrative machine and the target workload environment will need network access to:

vCenter endpoint (must be accessible to EKS Anywhere clusters)
public.ecr.aws
anywhere-assets.eks.amazonaws.com (to download the EKS Anywhere binaries, manifests and OVAs)
distro.eks.amazonaws.com (to download EKS Distro binaries and manifests)
d2glxqk2uabbnd.cloudfront.net (for EKS Anywhere and EKS Distro ECR container images)
api.ecr.us-west-2.amazonaws.com (for EKS Anywhere package authentication matching your region)
d5l0dvt14r5h8.cloudfront.net (for EKS Anywhere package ECR container images)
api.github.com (only if GitOps is enabled)

vSphere information needed before creating the cluster

You need to get the following information before creating the cluster:

Static IP Addresses: You will need one IP address for the management cluster control plane endpoint, and a separate one for the controlplane of each workload cluster you add.

Let’s say you are going to have the management cluster and two workload clusters. For those, you would need three IP addresses, one for each. All of those addresses will be configured the same way in the configuration file you will generate for each cluster.

A static IP address will be used for each control plane VM in your EKS Anywhere cluster. Choose IP addresses in your network range that do not conflict with other VMs and make sure they are excluded from your DHCP offering.

An IP address will be the value of the property controlPlaneConfiguration.endpoint.host in the config file of the management cluster. A separate IP address must be assigned for each workload cluster.
vSphere Datacenter Name: The vSphere datacenter to deploy the EKS Anywhere cluster on.
VM Network Name: The VM network to deploy your EKS Anywhere cluster on.
vCenter Server Domain Name: The vCenter server fully qualified domain name or IP address. If the server IP is used, the thumbprint must be set or insecure must be set to true.
thumbprint (required if insecure=false): The SHA1 thumbprint of the vCenter server certificate which is only required if you have a self-signed certificate for your vSphere endpoint.

There are several ways to obtain your vCenter thumbprint. If you have govc installed , you can run the following command in the Administrative machine terminal, and take a note of the output:
```
govc about.cert -thumbprint -k
```
template: The VM template to use for your EKS Anywhere cluster. This template was created when you imported the OVA file into vSphere.
datastore: The vSphere datastore to deploy your EKS Anywhere cluster on.
folder: The folder parameter in VSphereMachineConfig allows you to organize the VMs of an EKS Anywhere cluster. With this, each cluster can be organized as a folder in vSphere. You will have a separate folder for the management cluster and each cluster you are adding.
resourcePool: The vSphere Resource pools for your VMs in the EKS Anywhere cluster. If there is a resource pool: /<datacenter>/host/<resource-pool-name>/Resources

2 - Preparing vSphere for EKS Anywhere

Set up a vSphere cluster to prepare it for EKS Anywhere

Certain resources must be in place with appropriate user permissions to create an EKS Anywhere cluster using the vSphere provider.

Configuring Folder Resources

Create a VM folder:

For each user that needs to create workload clusters, have the vSphere administrator create a VM folder. That folder will host:

The VMs of the Control plane and Data plane nodes of each cluster.
A nested folder for the management cluster and another one for each workload cluster.
Each cluster VM in its own nested folder under this folder.

Follow these steps to create the user’s vSphere folder:

From vCenter, select the Menus/VM and Template tab.
Select either a datacenter or another folder as a parent object for the folder that you want to create.
Right-click the parent object and click New Folder.
Enter a name for the folder and click OK. For more details, see the vSphere Create a Folder documentation.

Configuring vSphere User, Group, and Roles

You need a vSphere user with the right privileges to let you create EKS Anywhere clusters on top of your vSphere cluster.

Configure via EKSA CLI

To configure a new user via CLI, you will need two things:

a set of vSphere admin credentials with the ability to create users and groups. If you do not have the rights to create new groups and users, you can invoke govc commands directly as outlined here.
a user.yaml file:

apiVersion: "eks-anywhere.amazon.com/v1"
kind: vSphereUser
spec:
  username: "eksa"                # optional, default eksa
  group: "MyExistingGroup"        # optional, default EKSAUsers
  globalRole: "MyGlobalRole"      # optional, default EKSAGlobalRole
  userRole: "MyUserRole"          # optional, default EKSAUserRole
  adminRole: "MyEKSAAdminRole"    # optional, default EKSACloudAdminRole
  datacenter: "MyDatacenter"
  vSphereDomain: "vsphere.local"  # this should be the domain used when you login, e.g. YourUsername@vsphere.local
  connection:
    server: "https://my-vsphere.internal.acme.com"
    insecure: false
  objects:
    networks:
      - !!str "/MyDatacenter/network/My Network"
    datastores:
      - !!str "/MyDatacenter/datastore/MyDatastore2"
    resourcePools:
      - !!str "/MyDatacenter/host/Cluster-03/MyResourcePool" # NOTE: see below if you do not want to use a resource pool
    folders:
      - !!str "/MyDatacenter/vm/OrgDirectory/MyVMs"
    templates:
      - !!str "/MyDatacenter/vm/Templates/MyTemplates"

NOTE: if you do not want to create a resource pool, you can instead specify the cluster directly as /MyDatacenter/host/Cluster-03 in user.yaml, where Cluster-03 is your cluster name. In your cluster spec, you will need to specify /MyDatacenter/host/Cluster-03/Resources for the resourcePool field.

Set the admin credentials as environment variables:

export EKSA_VSPHERE_USERNAME=<ADMIN_VSPHERE_USERNAME>
export EKSA_VSPHERE_PASSWORD=<ADMIN_VSPHERE_PASSWORD>

If the user does not already exist, you can create the user and all the specified group and role objects by running:

eksctl anywhere exp vsphere setup user -f user.yaml --password '<NewUserPassword>'

If the user or any of the group or role objects already exist, use the force flag instead to overwrite Group-Role-Object mappings for the group, roles, and objects specified in the user.yaml config file:

eksctl anywhere exp vsphere setup user -f user.yaml --force

Please note that there is one more manual step to configure global permissions here .

Configure via govc

If you do not have the rights to create a new user, you can still configure the necessary roles and permissions using the govc cli .

#! /bin/bash
# govc calls to configure a user with minimal permissions
set -x
set -e

EKSA_USER='<Username>@<UserDomain>'
USER_ROLE='EKSAUserRole'
GLOBAL_ROLE='EKSAGlobalRole'
ADMIN_ROLE='EKSACloudAdminRole'

FOLDER_VM='/YourDatacenter/vm/YourVMFolder'
FOLDER_TEMPLATES='/YourDatacenter/vm/Templates'

NETWORK='/YourDatacenter/network/YourNetwork'
DATASTORE='/YourDatacenter/datastore/YourDatastore'
RESOURCE_POOL='/YourDatacenter/host/Cluster-01/Resources/YourResourcePool'

govc role.create "$GLOBAL_ROLE" $(curl https://raw.githubusercontent.com/aws/eks-anywhere/main/pkg/config/static/globalPrivs.json | jq .[] | tr '\n' ' ' | tr -d '"')

govc role.create "$USER_ROLE" $(curl https://raw.githubusercontent.com/aws/eks-anywhere/main/pkg/config/static/eksUserPrivs.json | jq .[] | tr '\n' ' ' | tr -d '"')

govc role.create "$ADMIN_ROLE" $(curl https://raw.githubusercontent.com/aws/eks-anywhere/main/pkg/config/static/adminPrivs.json | jq .[] | tr '\n' ' ' | tr -d '"')

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$GLOBAL_ROLE" /

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$ADMIN_ROLE" "$FOLDER_VM"

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$ADMIN_ROLE" "$FOLDER_TEMPLATES"

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$USER_ROLE" "$NETWORK"

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$USER_ROLE" "$DATASTORE"

govc permissions.set -group=false -principal "$EKSA_USER"  -role "$USER_ROLE" "$RESOURCE_POOL"

Please note that there is one more manual step to configure global permissions here .

Configure via UI

Add a vCenter User

Ask your VSphere administrator to add a vCenter user that will be used for the provisioning of the EKS Anywhere cluster in VMware vSphere.

Log in with the vSphere Client to the vCenter Server.
Specify the user name and password for a member of the vCenter Single Sign-On Administrators group.
Navigate to the vCenter Single Sign-On user configuration UI.
- From the Home menu, select Administration.
- Under Single Sign On, click Users and Groups.
If vsphere.local is not the currently selected domain, select it from the drop-down menu. You cannot add users to other domains.
On the Users tab, click Add.
Enter a user name and password for the new user.
The maximum number of characters allowed for the user name is 300.
You cannot change the user name after you create a user. The password must meet the password policy requirements for the system.
Click Add.

For more details, see vSphere Add vCenter Single Sign-On Users documentation.

Create and define user roles

When you add a user for creating clusters, that user initially has no privileges to perform management operations. So you have to add this user to groups with the required permissions, or assign a role or roles with the required permission to this user.

Three roles are needed to be able to create the EKS Anywhere cluster:

Create a global custom role: For example, you could name this EKS Anywhere Global. Define it for the user on the vCenter domain level and its children objects. Create this role with the following privileges:

> Content Library
* Add library item
* Check in a template
* Check out a template
* Create local library
* Update files
> vSphere Tagging
* Assign or Unassign vSphere Tag
* Assign or Unassign vSphere Tag on Object
* Create vSphere Tag
* Create vSphere Tag Category
* Delete vSphere Tag
* Delete vSphere Tag Category
* Edit vSphere Tag
* Edit vSphere Tag Category
* Modify UsedBy Field For Category
* Modify UsedBy Field For Tag
> Sessions
* Validate session

Create a user custom role: The second role is also a custom role that you could call, for example, EKSAUserRole. Define this role with the following objects and children objects.

The pool resource level and its children objects. This resource pool that our EKS Anywhere VMs will be part of.
The storage object level and its children objects. This storage that will be used to store the cluster VMs.
The network VLAN object level and its children objects. This network that will host the cluster VMs.
The VM and Template folder level and its children objects.

Create this role with the following privileges:

> Content Library
* Add library item
* Check in a template
* Check out a template
* Create local library
> Datastore
* Allocate space
* Browse datastore
* Low level file operations
> Folder
* Create folder
> vSphere Tagging
* Assign or Unassign vSphere Tag
* Assign or Unassign vSphere Tag on Object
* Create vSphere Tag
* Create vSphere Tag Category
* Delete vSphere Tag
* Delete vSphere Tag Category
* Edit vSphere Tag
* Edit vSphere Tag Category
* Modify UsedBy Field For Category
* Modify UsedBy Field For Tag
> Network
* Assign network
> Resource
* Assign virtual machine to resource pool
> Scheduled task
* Create tasks
* Modify task
* Remove task
* Run task
> Profile-driven storage
* Profile-driven storage view
> Storage views
* View
> vApp
* Import
> Virtual machine
* Change Configuration
  - Add existing disk
  - Add new disk
  - Add or remove device
  - Advanced configuration
  - Change CPU count
  - Change Memory
  - Change Settings
  - Configure Raw device
  - Extend virtual disk
  - Modify device settings
  - Remove disk
* Edit Inventory
  - Create from existing
  - Create new
  - Remove
* Interaction
  - Power off
  - Power on
* Provisioning
  - Clone template
  - Clone virtual machine
  - Create template from virtual machine
  - Customize guest
  - Deploy template
  - Mark as template
  - Read customization specifications
* Snapshot management
  - Create snapshot
  - Remove snapshot
  - Revert to snapshot

Create a default Administrator role: The third role is the default system role Administrator that you define to the user on the folder level and its children objects (VMs and OVA templates) that was created by the VSphere admistrator for you.

To create a role and define privileges check Create a vCenter Server Custom Role and Defined Privileges pages.

Manually set Global Permissions role in Global Permissions UI

vSphere does not currently support a public API for setting global permissions. Because of this, you will need to manually assign the Global Role you created to your user or group in the Global Permissions UI.

Deploy an OVA Template

If the user creating the cluster has permission and network access to create and tag a template, you can skip these steps because EKS Anywhere will automatically download the OVA and create the template if it can. If the user does not have the permissions or network access to create and tag the template, follow this guide. The OVA contains the operating system (Ubuntu, Bottlerocket, or RHEL) for a specific EKS Distro Kubernetes release and EKS Anywhere version. The following example uses Ubuntu as the operating system, but a similar workflow would work for Bottlerocket or RHEL.

Steps to deploy the OVA

Go to the artifacts page and download or build the OVA template with the newest EKS Distro Kubernetes release to your computer.
Log in to the vCenter Server.
Right-click the folder you created above and select Deploy OVF Template. The Deploy OVF Template wizard opens.
On the Select an OVF template page, select the Local file option, specify the location of the OVA template you downloaded to your computer, and click Next.
On the Select a name and folder page, enter a unique name for the virtual machine or leave the default generated name, if you do not have other templates with the same name within your vCenter Server virtual machine folder. The default deployment location for the virtual machine is the inventory object where you started the wizard, which is the folder you created above. Click Next.
On the Select a compute resource page, select the resource pool where to run the deployed VM template, and click Next.
On the Review details page, verify the OVF or OVA template details and click Next.
On the Select storage page, select a datastore to store the deployed OVF or OVA template and click Next.
On the Select networks page, select a source network and map it to a destination network. Click Next.
On the Ready to complete page, review the page and click Finish. For details, see Deploy an OVF or OVA Template

To build your own Ubuntu OVA template check the Building your own Ubuntu OVA section in the following link .

To use the deployed OVA template to create the VMs for the EKS Anywhere cluster, you have to tag it with specific values for the os and eksdRelease keys. The value of the os key is the operating system of the deployed OVA template, which is ubuntu in our scenario. The value of the eksdRelease holds kubernetes and the EKS-D release used in the deployed OVA template. Check the following Customize OVAs page for more details.

Steps to tag the deployed OVA template:

Go to the artifacts page and take notes of the tags and values associated with the OVA template you deployed in the previous step.
In the vSphere Client, select Menu > Tags & Custom Attributes.
Select the Tags tab and click Tags.
Click New.
In the Create Tag dialog box, copy the os tag name associated with your OVA that you took notes of, which in our case is os:ubuntu and paste it as the name for the first tag required.
Specify the tag category os if it exist or create it if it does not exist.
Click Create.
Repeat steps 2-4.
In the Create Tag dialog box, copy the os tag name associated with your OVA that you took notes of, which in our case is eksdRelease:kubernetes-1-21-eks-8 and paste it as the name for the second tag required.
Specify the tag category eksdRelease if it exist or create it if it does not exist.
Click Create.
Navigate to the VM and Template tab.
Select the folder that was created.
Select deployed template and click Actions.
From the drop-down menu, select Tags and Custom Attributes > Assign Tag.
Select the tags we created from the list and confirm the operation.

3 - Customize OVAs: Ubuntu

Customizing Imported Ubuntu OVAs

There may be a need to make specific configuration changes on the imported ova template before using it to create/update EKS-A clusters.

Set up SSH Access for Imported OVA

SSH user and key need to be configured in order to allow SSH login to the VM template

Clone template to VM

Create an environment variable to hold the name of modified VM/template

export VM=<vm-name>

Clone the imported OVA template to create VM

govc vm.clone -on=false -vm=<full-path-to-imported-template> - folder=<full-path-to-folder-that-will-contain-the-VM> -ds=<datastore> $VM

Configure VM with cloud-init and the VMX GuestInfo datasource

Create a metadata.yaml file

instance-id: cloud-vm
local-hostname: cloud-vm
network:
  version: 2
  ethernets:
    nics:
      match:
        name: ens*
      dhcp4: yes

Create a userdata.yaml file

#cloud-config

users:
  - default
  - name: <username>
    primary_group: <username>
    sudo: ALL=(ALL) NOPASSWD:ALL
    groups: sudo, wheel
    ssh_import_id: None
    lock_passwd: true
    ssh_authorized_keys:
    - <user's ssh public key>

Export environment variable containing the cloud-init metadata and userdata

export METADATA=$(gzip -c9 <metadata.yaml | { base64 -w0 2>/dev/null || base64; }) \
       USERDATA=$(gzip -c9 <userdata.yaml | { base64 -w0 2>/dev/null || base64; })

Assign metadata and userdata to VM’s guestinfo

govc vm.change -vm "${VM}" \
  -e guestinfo.metadata="${METADATA}" \
  -e guestinfo.metadata.encoding="gzip+base64" \
  -e guestinfo.userdata="${USERDATA}" \
  -e guestinfo.userdata.encoding="gzip+base64"

Power the VM on

govc vm.power -on “$VM”

Customize the VM

Once the VM is powered on and fetches an IP address, ssh into the VM using your private key corresponding to the public key specified in userdata.yaml

ssh -i <private-key-file> username@<VM-IP>

At this point, you can make the desired configuration changes on the VM. The following sections describe some of the things you may want to do:

Add a Certificate Authority

Copy your CA certificate under /usr/local/share/ca-certificates and run sudo update-ca-certificates which will place the certificate under the /etc/ssl/certs directory.

Add Authentication Credentials for a Private Registry

If /etc/containerd/config.toml is not present initially, the default configuration can be generated by running the containerd config default > /etc/containerd/config.toml command. To configure a credential for a specific registry, create/modify the /etc/containerd/config.toml as follows:

# explicitly use v2 config format
version = 2

# The registry host has to be a domain name or IP. Port number is also
# needed if the default HTTPS or HTTP port is not used.
[plugins."io.containerd.grpc.v1.cri".registry.configs."registry1-host:port".auth]
  username = ""
  password = ""
  auth = ""
  identitytoken = ""
 # The registry host has to be a domain name or IP. Port number is also
 # needed if the default HTTPS or HTTP port is not used.
[plugins."io.containerd.grpc.v1.cri".registry.configs."registry2-host:port".auth]
  username = ""
  password = ""
  auth = ""
  identitytoken = ""

Restart containerd service with the sudo systemctl restart containerd command.

Convert VM to a Template

After you have customized the VM, you need to convert it to a template.

Cleanup the machine and power off the VM

This step is needed because of a known issue in Ubuntu which results in the clone VMs getting the same DHCP IP

sudo su
echo -n > /etc/machine-id
rm /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id
cloud-init clean -l --machine-id

Delete the hostname from file

/etc/hostname

Delete the networking config file

rm -rf /etc/netplan/50-cloud-init.yaml

Edit the cloud init config to turn preserve_hostname to false

vi /etc/cloud/cloud.cfg

Power the VM down

govc vm.power -off "$VM"

Take a snapshot of the VM

It is recommended to take a snapshot of the VM as it reduces the provisioning time for the machines and makes cluster creation faster.

If you do snapshot the VM, you will not be able to customize the disk size of your cluster VMs. If you prefer not to take a snapshot, skip this step.

govc snapshot.create -vm "$VM" root

Convert VM to template

govc vm.markastemplate $VM

Tag the template appropriately as described here

Use this customized template to create/upgrade EKS Anywhere clusters

4 - Import OVAs

Importing EKS Anywhere OVAs to vSphere

If you want to specify an OVA template, you will need to import OVA files into vSphere before you can use it in your EKS Anywhere cluster. This guide was written using VMware Cloud on AWS, but the VMware OVA import guide can be found here .

Note

If you don’t specify a template in the cluster spec file, EKS Anywhere will use the proper default one for the Kubernetes minor version and OS family you specified in the spec file. If the template doesn’t exist, it will import the appropriate OVA into vSphere and add the necessary tags.

The default OVA for a Kubernetes minor version + OS family will change over time, for example, when a new EKS Distro version is released. In that case, new clusters will use the new OVA (EKS Anywhere will import it automatically).

Warning

Do not power on the imported OVA directly as it can cause some undesired configurations on the OS template and affect cluster creation. If you want to explore or modify the OS, please follow the instructions to customize the OVA.

EKS Anywhere supports the following operating system families

Bottlerocket (default)
Ubuntu
RHEL

A list of OVAs for this release can be found on the artifacts page .

Using vCenter Web User Interface

Right click on your Datacenter, select Deploy OVF Template
Select an OVF template using URL or selecting a local OVF file and click on Next. If you are not able to select an OVF template using URL, download the file and use Local file option.

Note: If you are using Bottlerocket OVAs, please select local file option.
Select a folder where you want to deploy your OVF package (most of our OVF templates are under SDDC-Datacenter directory) and click on Next. You cannot have an OVF template with the same name in one directory. For workload VM templates, leave the Kubernetes version in the template name for reference. A workload VM template will support at least one prior Kubernetes major versions.
Select any compute resource to run (from cluster-1, 10.2.34.5, etc..) the deployed VM and click on Next
Review the details and click Next.
Accept the agreement and click Next.
Select the appropriate storage (e.g. “WorkloadDatastore“) and click Next.
Select destination network (e.g. “sddc-cgw-network-1”) and click Next.
Finish.
Snapshot the VM. Right click on the imported VM and select Snapshots -> Take Snapshot… (It is highly recommended that you snapshot the VM. This will reduce the time it takes to provision machines and cluster creation will be faster. If you prefer not to take snapshot, skip to step 13)
Name your template (e.g. “root”) and click Create.
Snapshots for the imported VM should now show up under the Snapshots tab for the VM.
Right click on the imported VM and select Template and Convert to Template

Steps to deploy a template using GOVC (CLI)

To deploy a template using govc, you must first ensure that you have GOVC installed . You need to set and export three environment variables to run govc GOVC_USERNAME, GOVC_PASSWORD and GOVC_URL.

Import the template to a content library in vCenter using URL or selecting a local OVA file

Using URL:

govc library.import -k -pull <library name> <URL for the OVA file>

Using a file from the local machine:

govc library.import <library name> <path to OVA file on local machine>

Deploy the template

govc library.deploy -pool <resource pool> -folder <folder location to deploy template> /<library name>/<template name> <name of new VM>

2a. If using Bottlerocket template for newer Kubernetes version than 1.21, resize disk 1 to 22G

govc vm.disk.change -vm <template name> -disk.label "Hard disk 1" -size 22G

2b. If using Bottlerocket template for Kubernetes version 1.21, resize disk 2 to 20G

govc vm.disk.change -vm <template name> -disk.label "Hard disk 2" -size 20G

Take a snapshot of the VM (It is highly recommended that you snapshot the VM. This will reduce the time it takes to provision machines and cluster creation will be faster. If you prefer not to take snapshot, skip this step)
```
govc snapshot.create -vm ubuntu-2004-kube-v1.25.6 root
```

Mark the new VM as a template

govc vm.markastemplate <name of new VM>

Important Additional Steps to Tag the OVA

Using vCenter UI

Tag to indicate OS family

Select the template that was newly created in the steps above and navigate to Summary -> Tags.
Click Assign -> Add Tag to create a new tag and attach it
Name the tag os:ubuntu or os:bottlerocket

Tag to indicate eksd release

Select the template that was newly created in the steps above and navigate to Summary -> Tags.
Click Assign -> Add Tag to create a new tag and attach it
Name the tag eksdRelease:{eksd release for the selected ova}, for example eksdRelease:kubernetes-1-25-eks-5 for the 1.25 ova. You can find the rest of eksd releases in the previous section . If it’s the first time you add an eksdRelease tag, you would need to create the category first. Click on “Create New Category” and name it eksdRelease.

Using govc

Tag to indicate OS family

Create tag category

govc tags.category.create -t VirtualMachine os

Create tags os:ubuntu and os:bottlerocket

govc tags.create -c os os:bottlerocket
govc tags.create -c os os:ubuntu

Attach newly created tag to the template

govc tags.attach os:bottlerocket <Template Path>
govc tags.attach os:ubuntu <Template Path>

Verify tag is attached to the template

govc tags.ls <Template Path>

Tag to indicate eksd release

Create tag category

govc tags.category.create -t VirtualMachine eksdRelease

Create the proper eksd release Tag, depending on your template. You can find the eksd releases in the previous section . For example eksdRelease:kubernetes-1-25-eks-5 for the 1.25 template.

govc tags.create -c eksdRelease eksdRelease:kubernetes-1-25-eks-5

Attach newly created tag to the template

govc tags.attach eksdRelease:kubernetes-1-25-eks-5 <Template Path>

Verify tag is attached to the template

govc tags.ls <Template Path>

Note

If the tags above are not applied as shown exactly, eks-a template validations will fail and CLI will abort

After you are done you can use the template for your workload cluster.

5 - Custom DHCP Configuration

Create a custom DHCP configuration for your vSphere deployment

If your vSphere deployment is not configured with DHCP, you may want to run your own DHCP server. It may be necessary to turn off DHCP snooping on your switch to get DHCP working across VM servers. If you are running your administration machine in vSphere, it would most likely be easiest to run the DHCP server on that machine. This example is for Ubuntu.

Install

Install DHCP server

sudo apt-get install isc-dhcp-server

Configure /etc/dhcp/dhcpd.conf

Update the ip address range, subnet, mask, etc to suite your configuration similar to this:

default-lease-time 600;
max-lease-time 7200;
 
ddns-update-style none;
 
authoritative;
 
subnet 10.8.105.0 netmask 255.255.255.0 {
range 10.8.105.9  10.8.105.41;
option subnet-mask 255.255.255.0;
option routers 10.8.105.1;
 option domain-name-servers 147.149.1.69;
}

Configure /etc/default/isc-dhcp-server

Add the main NIC device interface to this file, such as eth0 (this example uses ens160).

INTERFACESv4="ens160"

Restart DHCP

service isc-dhcp-server restart

Verify your configuration

This example assumes the ens160 interface:

tcpdump -ni ens160 port 67 -vvvv
 
tcpdump: listening on ens160, link-type EN10MB (Ethernet), capture size 262144 bytes
09:13:54.297704 IP (tos 0xc0, ttl 64, id 40258, offset 0, flags [DF], proto UDP (17), length 327)
    10.8.105.12.68 > 10.8.105.5.67: [udp sum ok] BOOTP/DHCP, Request from 00:50:56:90:56:cf, length 299, xid 0xf7a5aac5, secs 50310, Flags [none] (0x0000)
          Client-IP 10.8.105.12
          Client-Ethernet-Address 00:50:56:90:56:cf
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Request
            Client-ID Option 61, length 19: hardware-type 255, 2d:1a:a1:33:00:02:00:00:ab:11:f2:c8:ef:ba:aa:5a:2f:33
            Parameter-Request Option 55, length 11:
              Subnet-Mask, Default-Gateway, Hostname, Domain-Name
              Domain-Name-Server, MTU, Static-Route, Classless-Static-Route
              Option 119, NTP, Option 120
            MSZ Option 57, length 2: 576
            Hostname Option 12, length 15: "prod-etcd-m8ctd"
            END Option 255, length 0
09:13:54.299762 IP (tos 0x0, ttl 64, id 56218, offset 0, flags [DF], proto UDP (17), length 328)
    10.8.105.5.67 > 10.8.105.12.68: [bad udp cksum 0xe766 -> 0x502f!] BOOTP/DHCP, Reply, length 300, xid 0xf7a5aac5, secs 50310, Flags [none] (0x0000)
          Client-IP 10.8.105.12
          Your-IP 10.8.105.12
          Server-IP 10.8.105.5
          Client-Ethernet-Address 00:50:56:90:56:cf
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: ACK
            Server-ID Option 54, length 4: 10.8.105.5
            Lease-Time Option 51, length 4: 600
            Subnet-Mask Option 1, length 4: 255.255.255.0
            Default-Gateway Option 3, length 4: 10.8.105.1
            Domain-Name-Server Option 6, length 4: 147.149.1.69
            END Option 255, length 0
            PAD Option 0, length 0, occurs 26

6 -

vCenter endpoint (must be accessible to EKS Anywhere clusters)
public.ecr.aws
anywhere-assets.eks.amazonaws.com (to download the EKS Anywhere binaries, manifests and OVAs)
distro.eks.amazonaws.com (to download EKS Distro binaries and manifests)
d2glxqk2uabbnd.cloudfront.net (for EKS Anywhere and EKS Distro ECR container images)
api.ecr.us-west-2.amazonaws.com (for EKS Anywhere package authentication matching your region)
d5l0dvt14r5h8.cloudfront.net (for EKS Anywhere package ECR container images)
api.github.com (only if GitOps is enabled)