This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Packages

List of EKS Anywhere curated packages

Curated package list

Name Description Versions GitHub
ADOT ADOT Collector is an AWS distribution of the OpenTelemetry Collector, which provides a vendor-agnostic solution to receive, process and export telemetry data. v0.25.0 https://github.com/aws-observability/aws-otel-collector
Cert-manager Cert-manager is a certificate manager for Kubernetes clusters. v1.9.1 https://github.com/cert-manager/cert-manager
Cluster Autoscaler Cluster Autoscaler is a component that automatically adjusts the size of a Kubernetes Cluster so that all pods have a place to run and there are no unneeded nodes. v9.21.0 https://github.com/kubernetes/autoscaler
Emissary Ingress Emissary Ingress is an open source Ingress supporting API Gateway + Layer 7 load balancer built on Envoy Proxy. v3.3.0 https://github.com/emissary-ingress/emissary/
Harbor Harbor is an open source trusted cloud native registry project that stores, signs, and scans content. v2.7.1
v2.5.1 		https://github.com/goharbor/harbor
https://github.com/goharbor/harbor-helm
MetalLB MetalLB is a virtual IP provider for services of type LoadBalancer supporting ARP and BGP. v0.13.7 https://github.com/metallb/metallb/
Metrics Server Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines. v3.8.2 https://github.com/kubernetes-sigs/metrics-server
Prometheus Prometheus is an open-source systems monitoring and alerting toolkit that collects and stores metrics as time series data. v2.41.0 https://github.com/prometheus/prometheus

1 - Packages configuration

Full EKS Anywhere configuration reference for curated packages.

This is a generic template with detailed descriptions below for reference. To generate your own package configuration, follow instructions from Package Management section and modify it using descriptions below.

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: PackageBundleController
metadata:
  name: eksa-packages-bundle-controller
  namespace: eksa-packages
spec:
  activeBundle: v1-21-83
  defaultImageRegistry: 783794618700.dkr.ecr.us-west-2.amazonaws.com
  defaultRegistry: public.ecr.aws/eks-anywhere
  privateRegistry: ""
  upgradeCheckInterval: 24h0m0s

---
apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: PackageBundle
metadata:
  name: package-bundle
  namespace: eksa-packages
spec:
  packages:
    - name: hello-eks-anywhere
      source:
        repository: hello-eks-anywhere
        versions:
          - digest: sha256:c31242a2f94a58017409df163debc01430de65ded6bdfc5496c29d6a6cbc0d94
            images:
              - digest: sha256:26e3f2f9aa546fee833218ece3fe7561971fd905cef40f685fd1b5b09c6fb71d
                repository: hello-eks-anywhere
            name: 0.1.1-083e68edbbc62ca0228a5669e89e4d3da99ff73b
            schema: H4sIAJc5EW...

---
apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: my-hello-eks-anywhere
  namespace: eksa-packages
spec:
  config: |
        title: "My Hello"
  packageName: hello-eks-anywhere
  targetNamespace: eksa-packages

API Reference

Packages:

packages.eks.amazonaws.com/v1alpha1

Resource Types:

PackageBundleController

↩ Parent

PackageBundleController is the Schema for the packagebundlecontroller API.

Name Type Description Required
apiVersion string packages.eks.amazonaws.com/v1alpha1 true
kind string PackageBundleController true
metadata object Refer to the Kubernetes API documentation for the fields of the `metadata` field. true
spec object PackageBundleControllerSpec defines the desired state of PackageBundleController.
 false
status object PackageBundleControllerStatus defines the observed state of PackageBundleController.
 false

PackageBundleController.spec

↩ Parent

PackageBundleControllerSpec defines the desired state of PackageBundleController.

Name Type Description Required
activeBundle string ActiveBundle is name of the bundle from which packages should be sourced.
 false
bundleRepository string Repository portion of an OCI address to the bundle

Default: eks-anywhere-packages-bundles
 false
createNamespace boolean Allow target namespace creation by the controller

Default: false
 false
defaultImageRegistry string DefaultImageRegistry for pulling images

Default: 783794618700.dkr.ecr.us-west-2.amazonaws.com
 false
defaultRegistry string DefaultRegistry for pulling helm charts and the bundle

Default: public.ecr.aws/eks-anywhere
 false
logLevel integer LogLevel controls the verbosity of logging in the controller.

Format: int32
 false
privateRegistry string PrivateRegistry is the registry being used for all images, charts and bundles
 false
upgradeCheckInterval string UpgradeCheckInterval is the time between upgrade checks. The format is that of time's ParseDuration.

Default: 24h
 false
upgradeCheckShortInterval string UpgradeCheckShortInterval time between upgrade checks if there is a problem. The format is that of time's ParseDuration.

Default: 1h
 false

PackageBundleController.status

↩ Parent

PackageBundleControllerStatus defines the observed state of PackageBundleController.

Name Type Description Required
detail string Detail of the state.
 false
spec object Spec previous settings
 false
state enum State of the bundle controller.

Enum: ignored, active, disconnected, upgrade available
 false

PackageBundleController.status.spec

↩ Parent

Spec previous settings

Name Type Description Required
activeBundle string ActiveBundle is name of the bundle from which packages should be sourced.
 false
bundleRepository string Repository portion of an OCI address to the bundle

Default: eks-anywhere-packages-bundles
 false
createNamespace boolean Allow target namespace creation by the controller

Default: false
 false
defaultImageRegistry string DefaultImageRegistry for pulling images

Default: 783794618700.dkr.ecr.us-west-2.amazonaws.com
 false
defaultRegistry string DefaultRegistry for pulling helm charts and the bundle

Default: public.ecr.aws/eks-anywhere
 false
logLevel integer LogLevel controls the verbosity of logging in the controller.

Format: int32
 false
privateRegistry string PrivateRegistry is the registry being used for all images, charts and bundles
 false
upgradeCheckInterval string UpgradeCheckInterval is the time between upgrade checks. The format is that of time's ParseDuration.

Default: 24h
 false
upgradeCheckShortInterval string UpgradeCheckShortInterval time between upgrade checks if there is a problem. The format is that of time's ParseDuration.

Default: 1h
 false

PackageBundle

↩ Parent

PackageBundle is the Schema for the packagebundle API.

Name Type Description Required
apiVersion string packages.eks.amazonaws.com/v1alpha1 true
kind string PackageBundle true
metadata object Refer to the Kubernetes API documentation for the fields of the `metadata` field. true
spec object PackageBundleSpec defines the desired state of PackageBundle.
 false
status object PackageBundleStatus defines the observed state of PackageBundle.
 false

PackageBundle.spec

↩ Parent

PackageBundleSpec defines the desired state of PackageBundle.

Name Type Description Required
packages []object Packages supported by this bundle.
 true
minControllerVersion string Minimum required packages controller version
 false

PackageBundle.spec.packages[index]

↩ Parent

BundlePackage specifies a package within a bundle.

Name Type Description Required
source object Source location for the package (probably a helm chart).
 true
name string Name of the package.
 false
workloadonly boolean WorkloadOnly specifies if the package should be installed only on the workload cluster
 false

PackageBundle.spec.packages[index].source

↩ Parent

Source location for the package (probably a helm chart).

Name Type Description Required
repository string Repository within the Registry where the package is found.
 true
versions []object Versions of the package supported by this bundle.
 true
registry string Registry in which the package is found.
 false

PackageBundle.spec.packages[index].source.versions[index]

↩ Parent

SourceVersion describes a version of a package within a repository.

Name Type Description Required
digest string Digest is a checksum value identifying the version of the package and its contents.
 true
name string Name is a human-friendly description of the version, e.g. "v1.0".
 true
dependencies []string Dependencies to be installed before the package
 false
images []object Images is a list of images used by this version of the package.
 false
schema string Schema is a base64 encoded, gzipped json schema used to validate package configurations.
 false

PackageBundle.spec.packages[index].source.versions[index].images[index]

↩ Parent

VersionImages is an image used by a version of a package.

Name Type Description Required
digest string Digest is a checksum value identifying the version of the package and its contents.
 true
repository string Repository within the Registry where the package is found.
 true

PackageBundle.status

↩ Parent

PackageBundleStatus defines the observed state of PackageBundle.

Name Type Description Required
state enum PackageBundleStateEnum defines the observed state of PackageBundle.

Enum: available, ignored, invalid, controller upgrade required
 true
spec object PackageBundleSpec defines the desired state of PackageBundle.
 false

PackageBundle.status.spec

↩ Parent

PackageBundleSpec defines the desired state of PackageBundle.

Name Type Description Required
packages []object Packages supported by this bundle.
 true
minControllerVersion string Minimum required packages controller version
 false

PackageBundle.status.spec.packages[index]

↩ Parent

BundlePackage specifies a package within a bundle.

Name Type Description Required
source object Source location for the package (probably a helm chart).
 true
name string Name of the package.
 false
workloadonly boolean WorkloadOnly specifies if the package should be installed only on the workload cluster
 false

PackageBundle.status.spec.packages[index].source

↩ Parent

Source location for the package (probably a helm chart).

Name Type Description Required
repository string Repository within the Registry where the package is found.
 true
versions []object Versions of the package supported by this bundle.
 true
registry string Registry in which the package is found.
 false

PackageBundle.status.spec.packages[index].source.versions[index]

↩ Parent

SourceVersion describes a version of a package within a repository.

Name Type Description Required
digest string Digest is a checksum value identifying the version of the package and its contents.
 true
name string Name is a human-friendly description of the version, e.g. "v1.0".
 true
dependencies []string Dependencies to be installed before the package
 false
images []object Images is a list of images used by this version of the package.
 false
schema string Schema is a base64 encoded, gzipped json schema used to validate package configurations.
 false

PackageBundle.status.spec.packages[index].source.versions[index].images[index]

↩ Parent

VersionImages is an image used by a version of a package.

Name Type Description Required
digest string Digest is a checksum value identifying the version of the package and its contents.
 true
repository string Repository within the Registry where the package is found.
 true

Package

↩ Parent

Package is the Schema for the package API.

Name Type Description Required
apiVersion string packages.eks.amazonaws.com/v1alpha1 true
kind string Package true
metadata object Refer to the Kubernetes API documentation for the fields of the `metadata` field. true
spec object PackageSpec defines the desired state of an package.
 false
status object PackageStatus defines the observed state of Package.
 false

Package.spec

↩ Parent

PackageSpec defines the desired state of an package.

Name Type Description Required
packageName string PackageName is the name of the package as specified in the bundle.
 true
config string Config for the package.
 false
packageVersion string PackageVersion is a human-friendly version name or sha256 checksum for the package, as specified in the bundle.
 false
targetNamespace string TargetNamespace defines where package resources will be deployed.
 false

Package.status

↩ Parent

PackageStatus defines the observed state of Package.

Name Type Description Required
currentVersion string Version currently installed.
 true
source object Source associated with the installation.
 true
detail string Detail of the state.
 false
spec object Spec previous settings
 false
state enum State of the installation.

Enum: initializing, installing, installing dependencies, installed, updating, uninstalling, unknown
 false
targetVersion string Version to be installed.
 false
upgradesAvailable []object UpgradesAvailable indicates upgraded versions in the bundle.
 false

Package.status.source

↩ Parent

Source associated with the installation.

Name Type Description Required
digest string Digest is a checksum value identifying the version of the package and its contents.
 true
registry string Registry in which the package is found.
 true
repository string Repository within the Registry where the package is found.
 true
version string Versions of the package supported.
 true

Package.status.spec

↩ Parent

Spec previous settings

Name Type Description Required
packageName string PackageName is the name of the package as specified in the bundle.
 true
config string Config for the package.
 false
packageVersion string PackageVersion is a human-friendly version name or sha256 checksum for the package, as specified in the bundle.
 false
targetNamespace string TargetNamespace defines where package resources will be deployed.
 false

Package.status.upgradesAvailable[index]

↩ Parent

PackageAvailableUpgrade details the package’s available upgrade versions.

Name Type Description Required
tag string Tag is a specific version number or sha256 checksum for the package upgrade.
 true
version string Version is a human-friendly version name for the package upgrade.
 true

2 - Configuration Best Practice

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

3 - ADOT Configuration

OpenTelemetry Collector provides a vendor-agnostic solution to receive, process and export telemetry data. It removes the need to run, operate, and maintain multiple agents/collectors. ADOT Collector is an AWS-supported distribution of the OpenTelemetry Collector.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for ADOT

3.1 - v0.21.1

Configuring ADOT in EKS Anywhere package spec

Example

We included a sample configuration below for reference. For in-depth examples and use cases, please refer to ADOT workshop. 

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: my-adot
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: adot
  targetNamespace: observability
  config: | 
    mode: daemonset

Configurable parameters and default values under spec.config

Parameter Description Default
General
hostNetwork Indicates if the pod should run in the host networking namespace. false
image.pullPolicy Specifies image pull policy: IfNotPresent, Always, Never. "IfNotPresent"
mode Specifies Collector deployment options: daemonset, deployment, or statefulset. "daemonset"
ports.[*].containerPort Specifies containerPort used. See footnote 1
ports.[*].enabled Indicates if a port is enabled. See footnote 1
ports.[*].hostPort Specifies hostPort used. See footnote 1
ports.[*].protocol Specifies protocol used. See footnote 1
ports.[*].servicePort Specifies servicePort used. See footnote 1
resources.limits.cpu Specifies CPU resource limits for containers. 1
resources.limits.memory Specifies memory resource limits for containers. "2Gi"
Config
config.config Specifies Collector receiver, processor, exporter, and extensions configurations. Refer to aws-otel-collector for full details. Note EKS Anywhere ADOT package version matches the exact aws-otel-collector version. See footnote 2
config.config.receiver Specifies how data gets in the Collector. Receivers can be either push or pull based, and support one or more data source. See footnote 2
config.config.processor Specifies how processors are run on data between the stage of being received and being exported. Processors are optional though some are recommended. See footnote 2
config.config.exporters Specifies how data gets sent to backends/destinations. Exporters can be either push or pull based, and support one or more data source. See footnote 2
config.config.extensions Specifies tasks that do not involve processing telemetry data. Examples of extensions include health monitoring, service discovery, and data forwarding. Extensions are optional. See footnote 2
config.config.service Specifies what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it is not enabled. See footnote 2
Deployment mode only
replicaCount Specifies replicaCount for pods. 1
service.type Specifies service types: ClusterIP, NodePort, LoadBalancer, ExternalName. "ClusterIP"

  1. The default ports enables otlp and otlp-http. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    ports:
      otlp:
        enabled: true
        containerPort: 4317
        servicePort: 4317
        hostPort: 4317
        protocol: TCP
      otlp-http:
        enabled: true
        containerPort: 4318
        servicePort: 4318
        hostPort: 4318
        protocol: TCP
     ↩︎

  2. The default config.config deploys an ADOT Collector with the metrics pipeline, which includes otlp and prometheus receiver, and logging exporter. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    config:
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317
            http:
              endpoint: 0.0.0.0:4318
        prometheus:
          config:
            scrape_configs:
              - job_name: opentelemetry-collector
                scrape_interval: 10s
                static_configs:
                  - targets:
                      - ${MY_POD_IP}:8888
      processors:
        batch: {}
        memory_limiter: null
      exporters:
        logging:
          loglevel: info
      extensions:
        health_check: {}
        memory_ballast: {}
      service:
        telemetry:
          metrics:
            address: 0.0.0.0:8888
        extensions:
          - health_check
          - memory_ballast
        pipelines:
          metrics:
            exporters:
              - logging
            processors:
              - memory_limiter
              - batch
            receivers:
              - otlp
              - prometheus
     ↩︎

3.2 - v0.23.0

Configuring ADOT in EKS Anywhere package spec

Example

We included a sample configuration below for reference. For in-depth examples and use cases, please refer to ADOT workshop. 

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: my-adot
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: adot
  targetNamespace: observability
  config: | 
    mode: daemonset

Configurable parameters and default values under spec.config

Parameter Description Default
General
hostNetwork Indicates if the pod should run in the host networking namespace. false
image.pullPolicy Specifies image pull policy: IfNotPresent, Always, Never. "IfNotPresent"
mode Specifies Collector deployment options: daemonset, deployment, or statefulset. "daemonset"
ports.[*].containerPort Specifies containerPort used. See footnote 1
ports.[*].enabled Indicates if a port is enabled. See footnote 1
ports.[*].hostPort Specifies hostPort used. See footnote 1
ports.[*].protocol Specifies protocol used. See footnote 1
ports.[*].servicePort Specifies servicePort used. See footnote 1
resources.limits.cpu Specifies CPU resource limits for containers. 1
resources.limits.memory Specifies memory resource limits for containers. "2Gi"
Config
config.config Specifies Collector receiver, processor, exporter, and extensions configurations. Refer to aws-otel-collector for full details. Note EKS Anywhere ADOT package version matches the exact aws-otel-collector version. See footnote 2
config.config.receiver Specifies how data gets in the Collector. Receivers can be either push or pull based, and support one or more data source. See footnote 2
config.config.processor Specifies how processors are run on data between the stage of being received and being exported. Processors are optional though some are recommended. See footnote 2
config.config.exporters Specifies how data gets sent to backends/destinations. Exporters can be either push or pull based, and support one or more data source. See footnote 2
config.config.extensions Specifies tasks that do not involve processing telemetry data. Examples of extensions include health monitoring, service discovery, and data forwarding. Extensions are optional. See footnote 2
config.config.service Specifies what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it is not enabled. See footnote 2
Deployment mode only
replicaCount Specifies replicaCount for pods. 1
service.type Specifies service types: ClusterIP, NodePort, LoadBalancer, ExternalName. "ClusterIP"

  1. The default ports enables otlp and otlp-http. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    ports:
      otlp:
        enabled: true
        containerPort: 4317
        servicePort: 4317
        hostPort: 4317
        protocol: TCP
      otlp-http:
        enabled: true
        containerPort: 4318
        servicePort: 4318
        hostPort: 4318
        protocol: TCP
     ↩︎

  2. The default config.config deploys an ADOT Collector with the metrics pipeline, which includes otlp and prometheus receiver, and logging exporter. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    config:
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317
            http:
              endpoint: 0.0.0.0:4318
        prometheus:
          config:
            scrape_configs:
              - job_name: opentelemetry-collector
                scrape_interval: 10s
                static_configs:
                  - targets:
                      - ${MY_POD_IP}:8888
      processors:
        batch: {}
        memory_limiter: null
      exporters:
        logging:
          loglevel: info
      extensions:
        health_check: {}
        memory_ballast: {}
      service:
        telemetry:
          metrics:
            address: 0.0.0.0:8888
        extensions:
          - health_check
          - memory_ballast
        pipelines:
          metrics:
            exporters:
              - logging
            processors:
              - memory_limiter
              - batch
            receivers:
              - otlp
              - prometheus
     ↩︎

3.3 - v0.25.0

Configuring ADOT in EKS Anywhere package spec

Example

We included a sample configuration below for reference. For in-depth examples and use cases, please refer to ADOT workshop. 

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: my-adot
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: adot
  targetNamespace: observability
  config: | 
    mode: daemonset

Configurable parameters and default values under spec.config

Parameter Description Default
General
hostNetwork Indicates if the pod should run in the host networking namespace. false
image.pullPolicy Specifies image pull policy: IfNotPresent, Always, Never. "IfNotPresent"
mode Specifies Collector deployment options: daemonset, deployment, or statefulset. "daemonset"
ports.[*].containerPort Specifies containerPort used. See footnote 1
ports.[*].enabled Indicates if a port is enabled. See footnote 1
ports.[*].hostPort Specifies hostPort used. See footnote 1
ports.[*].protocol Specifies protocol used. See footnote 1
ports.[*].servicePort Specifies servicePort used. See footnote 1
resources.limits.cpu Specifies CPU resource limits for containers. 1
resources.limits.memory Specifies memory resource limits for containers. "2Gi"
Config
config.config Specifies Collector receiver, processor, exporter, and extensions configurations. Refer to aws-otel-collector for full details. Note EKS Anywhere ADOT package version matches the exact aws-otel-collector version. See footnote 2
config.config.receiver Specifies how data gets in the Collector. Receivers can be either push or pull based, and support one or more data source. See footnote 2
config.config.processor Specifies how processors are run on data between the stage of being received and being exported. Processors are optional though some are recommended. See footnote 2
config.config.exporters Specifies how data gets sent to backends/destinations. Exporters can be either push or pull based, and support one or more data source. See footnote 2
config.config.extensions Specifies tasks that do not involve processing telemetry data. Examples of extensions include health monitoring, service discovery, and data forwarding. Extensions are optional. See footnote 2
config.config.service Specifies what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it is not enabled. See footnote 2
Deployment mode only
replicaCount Specifies replicaCount for pods. 1
service.type Specifies service types: ClusterIP, NodePort, LoadBalancer, ExternalName. "ClusterIP"

  1. The default ports enables otlp and otlp-http. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    ports:
      otlp:
        enabled: true
        containerPort: 4317
        servicePort: 4317
        hostPort: 4317
        protocol: TCP
      otlp-http:
        enabled: true
        containerPort: 4318
        servicePort: 4318
        hostPort: 4318
        protocol: TCP
     ↩︎

  2. The default config.config deploys an ADOT Collector with the metrics pipeline, which includes otlp and prometheus receiver, and logging exporter. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    config:
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317
            http:
              endpoint: 0.0.0.0:4318
        prometheus:
          config:
            scrape_configs:
              - job_name: opentelemetry-collector
                scrape_interval: 10s
                static_configs:
                  - targets:
                      - ${MY_POD_IP}:8888
      processors:
        batch: {}
        memory_limiter: null
      exporters:
        logging:
          loglevel: info
      extensions:
        health_check: {}
        memory_ballast: {}
      service:
        telemetry:
          metrics:
            address: 0.0.0.0:8888
        extensions:
          - health_check
          - memory_ballast
        pipelines:
          metrics:
            exporters:
              - logging
            processors:
              - memory_limiter
              - batch
            receivers:
              - otlp
              - prometheus
     ↩︎

4 - Cert-Manager Configuration

The cert-manager package adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing and using those certificates.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for Cert-Manager

4.1 - v1.9.1

Configuring Cert-Manager in EKS Anywhere package spec

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: my-cert-manager
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: cert-manager
  config: | 
    global:
        logLevel: 4

The following table lists the configurable parameters of the cert-manager package spec and the default values.

Parameter Description Default
General
namespace The namespace to use for installing cert-manager package cert-manager
imagePullPolicy The image pull policy IfNotPresent
global
global.logLevel The log level: integer from 0-6 2
Webhook
webhook.timeoutSeconds The time in seconds to wait for the webhook to connect with the kube-api server 0

5 - Cluster Autoscaler Configuration

Cluster Autoscaler is a component that automatically adjusts the size of a Kubernetes Cluster so that all pods have a place to run and there are no unneeded nodes.

Configuration options for Cluster Autoscaler

5.1 - v9.21.0

Configuring Cluster Autoscaler in EKS Anywhere package spec

Parameter Description Default
General
cloudProvider Cluster Autoscaler cloud provider. This should always be clusterapi.
Example:
cloudProvider: “clusterapi”
 “clusterapi”
autoDiscovery.clusterName Name of the kubernetes cluster this autoscaler package should autoscale.
Example:
autoDiscovery.clusterName: “mgmt-cluster”
 false
clusterAPIMode Where Cluster Autoscaler should look for a kubeconfig to communicate with the cluster it will manage. See https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/clusterapi/README.md#connecting-cluster-autoscaler-to-cluster-api-management-and-workload-clusters
Example:
clusterAPIMode: “incluster-kubeconfig”
 “incluster-incluster”
clusterAPICloudConfigPath Path to kubeconfig for connecting to Cluster API Management Cluster, only used if clusterAPIMode=kubeconfig-kubeconfig or incluster-kubeconfig
Example:
clusterAPICloudConfigPath: “/etc/kubernetes/value”
 “/etc/kubernetes/mgmt-kubeconfig”
extraVolumeSecrets Additional volumes to mount from Secrets.
Example:
extraVolumeSecrets: {}
 {}

6 - Emissary Configuration

Emissary Ingress is an open-source Kubernetes-native API Gateway + Layer 7 load balancer + Kubernetes Ingress built on Envoy Proxy.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for Emissary

6.1 - v3.0.0

Configuring Emissary Ingress in EKS Anywhere package spec

Parameter Description Default
General
hostNetwork Whether Emissary will use the host network, useful for on-premise setup .
Example:
hostNetwork: false
 false
createDefaultListeners Whether Emissary should be created with default listeners, HTTP on port 8080 and HTTPS on port 8443.
Example:
createDefaultListeners: false
 false
replicaCount Replica count for Emissary to deploy.
Example:
replicaCount: 2
 2
daemonSet Whether to create Emissary as a Daemonset instead of a deployment
Example:
daemonSet: false
 false

6.2 - v3.3.0

Emissary version 0.3.3 has decoupled the CRD portion of the package, and now supports installing multiple instances of the emissary package in the same cluster.

Configuring Emissary Ingress in EKS Anywhere package spec

Parameter Description Default
General
hostNetwork Whether Emissary will use the host network, useful for on-premise setup .
Example:
hostNetwork: false
 false
createDefaultListeners Whether Emissary should be created with default listeners, HTTP on port 8080 and HTTPS on port 8443.
Example:
createDefaultListeners: false
 false
replicaCount Replica count for Emissary to deploy.
Example:
replicaCount: 2
 2
daemonSet Whether to create Emissary as a Daemonset instead of a deployment
Example:
daemonSet: false
 false

7 - Harbor configuration

Harbor is an open source trusted cloud native registry project that stores, signs, and scans content. Harbor extends the open source Docker Distribution by adding the functionalities usually required by users such as security, identity and management. Having a registry closer to the build and run environment can improve the image transfer efficiency. Harbor supports replication of images between registries, and also offers advanced security features such as user management, access control and activity auditing.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for Harbor

7.1 - v2.5.0

Trivy, Notary and Chartmuseum are not supported at this moment.

Configuring Harbor in EKS Anywhere package spec

The following table lists the configurable parameters of the Harbor package spec and the default values.

Parameter Description Default
General
externalURL The external URL for Harbor core service https://127.0.0.1:30003
imagePullPolicy The image pull policy IfNotPresent
logLevel The log level: debug, info, warning, error or fatal info
harborAdminPassword The initial password of the Harbor admin account. Change it from the portal after launching Harbor Harbor12345
secretKey The key used for encryption. Must be a string of 16 chars ""
Expose
expose.type How to expose the service: nodePort or loadBalancer, other values will be ignored and the creation of the service will be skipped. nodePort
expose.tls.enabled Enable TLS or not. true
expose.tls.certSource The source of the TLS certificate. Set as auto, secret or none and fill the information in the corresponding section: 1) auto: generate the TLS certificate automatically 2) secret: read the TLS certificate from the specified secret. The TLS certificate can be generated manually or by cert manager 3) none: configure no TLS certificate. secret
expose.tls.auto.commonName The common name used to generate the certificate. It’s necessary when expose.tls.certSource is set to auto
expose.tls.secret.secretName The name of the secret which contains keys named: tls.crt - the certificate; tls.key - the private key harbor-tls-secret
expose.nodePort.name The name of the NodePort service harbor
expose.nodePort.ports.http.port The service port Harbor listens on when serving HTTP 80
expose.nodePort.ports.http.nodePort The node port Harbor listens on when serving HTTP 30002
expose.nodePort.ports.https.port The service port Harbor listens on when serving HTTPS 443
expose.nodePort.ports.https.nodePort The node port Harbor listens on when serving HTTPS 30003
expose.loadBalancer.name The name of the service harbor
expose.loadBalancer.IP The IP address of the loadBalancer. It only works when the loadBalancer supports assigning an IP address ""
expose.loadBalancer.ports.httpPort The service port Harbor listens on when serving HTTP 80
expose.loadBalancer.ports.httpsPort The service port Harbor listens on when serving HTTPS 30002
expose.loadBalancer.annotations The annotations attached to the loadBalancer service {}
expose.loadBalancer.sourceRanges List of IP address ranges to assign to loadBalancerSourceRanges []
Internal TLS
internalTLS.enabled Enable TLS for the components (core, jobservice, portal, and registry) true
Persistence
persistence.resourcePolicy Setting it to keep to avoid removing PVCs during a helm delete operation. Leaving it empty will delete PVCs after the chart is deleted. Does not affect PVCs created for internal database and redis components. keep
persistence.persistentVolumeClaim.registry.size The size of the volume 5Gi
persistence.persistentVolumeClaim.registry.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.jobservice.size The size of the volume 1Gi
persistence.persistentVolumeClaim.jobservice.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.database.size The size of the volume. If an external database is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.database.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external database is used, the setting will be ignored ""
persistence.persistentVolumeClaim.redis.size The size of the volume. If an external Redis is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.redis.storageClass Specify the storageClass used to provision the volumem, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external Redis is used, the setting will be ignored ""
Registry
registry.relativeurls If true, the registry returns relative URLs in Location headers. The client is responsible for resolving the correct URL. Needed if harbor is behind a reverse proxy false

7.2 - v2.5.1

Notary and Chartmuseum are not supported at this moment.

Configuring Harbor in EKS Anywhere package spec

The following table lists the configurable parameters of the Harbor package spec and the default values.

Parameter Description Default
General
externalURL The external URL for Harbor core service https://127.0.0.1:30003
imagePullPolicy The image pull policy IfNotPresent
logLevel The log level: debug, info, warning, error or fatal info
harborAdminPassword The initial password of the Harbor admin account. Change it from the portal after launching Harbor Harbor12345
secretKey The key used for encryption. Must be a string of 16 chars ""
Expose
expose.type How to expose the service: nodePort or loadBalancer, other values will be ignored and the creation of the service will be skipped. nodePort
expose.tls.enabled Enable TLS or not. true
expose.tls.certSource The source of the TLS certificate. Set as auto, secret or none and fill the information in the corresponding section: 1) auto: generate the TLS certificate automatically 2) secret: read the TLS certificate from the specified secret. The TLS certificate can be generated manually or by cert manager 3) none: configure no TLS certificate. secret
expose.tls.auto.commonName The common name used to generate the certificate. It’s necessary when expose.tls.certSource is set to auto
expose.tls.secret.secretName The name of the secret which contains keys named: tls.crt - the certificate; tls.key - the private key harbor-tls-secret
expose.nodePort.name The name of the NodePort service harbor
expose.nodePort.ports.http.port The service port Harbor listens on when serving HTTP 80
expose.nodePort.ports.http.nodePort The node port Harbor listens on when serving HTTP 30002
expose.nodePort.ports.https.port The service port Harbor listens on when serving HTTPS 443
expose.nodePort.ports.https.nodePort The node port Harbor listens on when serving HTTPS 30003
expose.loadBalancer.name The name of the service harbor
expose.loadBalancer.IP The IP address of the loadBalancer. It only works when loadBalancer supports assigning an IP address ""
expose.loadBalancer.ports.httpPort The service port Harbor listens on when serving HTTP 80
expose.loadBalancer.ports.httpsPort The service port Harbor listens on when serving HTTPS 30002
expose.loadBalancer.annotations The annotations attached to the loadBalancer service {}
expose.loadBalancer.sourceRanges List of IP address ranges to assign to loadBalancerSourceRanges []
Internal TLS
internalTLS.enabled Enable TLS for the components (core, jobservice, portal, and registry) true
Persistence
persistence.resourcePolicy Setting it to keep to avoid removing PVCs during a helm delete operation. Leaving it empty will delete PVCs after the chart is deleted. Does not affect PVCs created for internal database and redis components. keep
persistence.persistentVolumeClaim.registry.size The size of the volume 5Gi
persistence.persistentVolumeClaim.registry.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.jobservice.size The size of the volume 1Gi
persistence.persistentVolumeClaim.jobservice.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.database.size The size of the volume. If an external database is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.database.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external database is used, the setting will be ignored ""
persistence.persistentVolumeClaim.redis.size The size of the volume. If an external Redis is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.redis.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external Redis is used, the setting will be ignored ""
persistence.persistentVolumeClaim.trivy.size The size of the volume 5Gi
persistence.persistentVolumeClaim.trivy.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
Trivy
trivy.enabled The flag to enable Trivy scanner true
trivy.vulnType Comma-separated list of vulnerability types. Possible values os and library. os,library
trivy.severity Comma-separated list of severities to be checked UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
trivy.skipUpdate The flag to disable Trivy DB downloads from GitHub false
trivy.offlineScan The flag prevents Trivy from sending API requests to identify dependencies. false
Registry
registry.relativeurls If true, the registry returns relative URLs in Location headers. The client is responsible for resolving the correct URL. Needed if harbor is behind a reverse proxy false

7.3 - v2.7.1

Notary and Chartmuseum are not supported at this moment.

Configuring Harbor in EKS Anywhere package spec

The following table lists the configurable parameters of the Harbor package spec and the default values.

Parameter Description Default
General
externalURL The external URL for Harbor core service https://127.0.0.1:30003
imagePullPolicy The image pull policy IfNotPresent
logLevel The log level: debug, info, warning, error or fatal info
harborAdminPassword The initial password of the Harbor admin account. Change it from the portal after launching Harbor Harbor12345
secretKey The key used for encryption. Must be a string of 16 chars ""
Expose
expose.type How to expose the service: nodePort or loadBalancer, other values will be ignored and the creation of the service will be skipped. nodePort
expose.tls.enabled Enable TLS or not. true
expose.tls.certSource The source of the TLS certificate. Set as auto, secret or none and fill the information in the corresponding section: 1) auto: generate the TLS certificate automatically 2) secret: read the TLS certificate from the specified secret. The TLS certificate can be generated manually or by cert manager 3) none: configure no TLS certificate. secret
expose.tls.auto.commonName The common name used to generate the certificate. It’s necessary when expose.tls.certSource is set to auto
expose.tls.secret.secretName The name of the secret which contains keys named: tls.crt - the certificate; tls.key - the private key harbor-tls-secret
expose.nodePort.name The name of the NodePort service harbor
expose.nodePort.ports.http.port The service port Harbor listens on when serving HTTP 80
expose.nodePort.ports.http.nodePort The node port Harbor listens on when serving HTTP 30002
expose.nodePort.ports.https.port The service port Harbor listens on when serving HTTPS 443
expose.nodePort.ports.https.nodePort The node port Harbor listens on when serving HTTPS 30003
expose.loadBalancer.name The name of the service harbor
expose.loadBalancer.IP The IP address of the loadBalancer. It only works when loadBalancer supports assigning an IP address ""
expose.loadBalancer.ports.httpPort The service port Harbor listens on when serving HTTP 80
expose.loadBalancer.ports.httpsPort The service port Harbor listens on when serving HTTPS 30002
expose.loadBalancer.annotations The annotations attached to the loadBalancer service {}
expose.loadBalancer.sourceRanges List of IP address ranges to assign to loadBalancerSourceRanges []
Internal TLS
internalTLS.enabled Enable TLS for the components (core, jobservice, portal, and registry) true
Persistence
persistence.resourcePolicy Setting it to keep to avoid removing PVCs during a helm delete operation. Leaving it empty will delete PVCs after the chart is deleted. Does not affect PVCs created for internal database and redis components. keep
persistence.persistentVolumeClaim.registry.size The size of the volume 5Gi
persistence.persistentVolumeClaim.registry.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.jobservice.jobLog.size The size of the volume 1Gi
persistence.persistentVolumeClaim.jobservice.jobLog.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
persistence.persistentVolumeClaim.database.size The size of the volume. If an external database is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.database.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external database is used, the setting will be ignored ""
persistence.persistentVolumeClaim.redis.size The size of the volume. If an external Redis is used, the setting will be ignored 1Gi
persistence.persistentVolumeClaim.redis.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning. If an external Redis is used, the setting will be ignored ""
persistence.persistentVolumeClaim.trivy.size The size of the volume 5Gi
persistence.persistentVolumeClaim.trivy.storageClass Specify the storageClass used to provision the volume, or the default StorageClass will be used (the default). Set it to - to disable dynamic provisioning ""
Trivy
trivy.enabled The flag to enable Trivy scanner true
trivy.vulnType Comma-separated list of vulnerability types. Possible values os and library. os,library
trivy.severity Comma-separated list of severities to be checked UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
trivy.skipUpdate The flag to disable Trivy DB downloads from GitHub false
trivy.offlineScan The flag prevents Trivy from sending API requests to identify dependencies. false
Registry
registry.relativeurls If true, the registry returns relative URLs in Location headers. The client is responsible for resolving the correct URL. Needed if harbor is behind a reverse proxy false

8 - MetalLB Configuration

MetalLB is a load-balancer implementation for on-premises Kubernetes clusters, using standard routing protocols.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for MetalLB

8.1 - v0.12.1

FRRouting is currently not supported for MetalLB.

Configuring MetalLB in EKS Anywhere package spec

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: mylb
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: metallb
  targetNamespace: metallb-system
  config: |
    IPAddressPools:
      - name: default
        addresses:
          - 10.220.0.93/32
          - 10.220.0.94/32
          - 10.220.0.95/32
      - name: bgp
        addresses:
          - 10.220.0.97-10.220.0.99
    L2Advertisements:
      - IPAddressPools:
          - default
    BGPAdvertisements:
      - IPAddressPools:
          - bgp 
    BGPPeers:
      - myASN: 123
        peerASN: 55001
        peerAddress: 1.2.3.4
        keepaliveTime: 30s
Parameter Description Default
IPAddressPools[] A list of IPAddressPool. None
L2Advertisements[] A list of L2Advertisement. None
BGPAdvertisements[] A list of BGPAdvertisement. None
BGPPeers[] A list of BGPPeer. None
— — —
IPAddressPool A list of IP address ranges over which MetalLB has authority. You can list multiple ranges in a single pool and they will all share the same settings. Each range can be either a CIDR prefix, or an explicit start-end range of IPs.
name Name for the address pool. None
addresses[] A list of string representing CIRD or IP ranges. None
autoAssign AutoAssign flag used to prevent MetalLB from automatic allocation for a pool. true
— — —
L2Advertisement L2Advertisement allows MetalLB to advertise the LoadBalancer IPs provided by the selected pools via L2.
IPAddressPools[] The list of IPAddressPools to advertise via this advertisement, selected by name. None
— — —
BGPAdvertisement BGPAdvertisement allows MetalLB to advertise the IPs coming from the selected IPAddressPools via BGP, setting the parameters of the BGP Advertisement.
aggregationLength The aggregation-length advertisement option lets you “roll up” the /32s into a larger prefix. Defaults to 32. Works for IPv4 addresses. 32
aggregationLengthV6 The aggregation-length advertisement option lets you “roll up” the /128s into a larger prefix. Defaults to 128. Works for IPv6 addresses. 128
communities[] The BGP communities to be associated with the announcement. Each item can be a community of the form 1234:1234 or the name of an alias defined in the Community CRD. None
IPAddressPools[] The list of IPAddressPools to advertise via this advertisement, selected by name. None
localPref The BGP LOCAL_PREF attribute which is used by BGP best path algorithm, Path with higher localpref is preferred over one with lower localpref. None
— — —
BGPPeer Peers for the BGP protocol.
bfdProfile The name of the BFD Profile to be used for the BFD session associated to the BGP session. If not set, the BFD session won’t be set up. None
holdTime Requested BGP hold time, per RFC4271. None
keepaliveTime Requested BGP keepalive time, per RFC4271. None
myASN AS number to use for the local end of the session. None
password Authentication password for routers enforcing TCP MD5 authenticated sessions. None
peerASN AS number to expect from the remote end of the session. None
peerAddress Address to dial when establishing the session. None
peerPort Port to dial when establishing the session. 179
routerID BGP router ID to advertise to the peer. None
sourceAddress Source address to use when establishing the session. None

8.2 - v0.13.5

FRRouting is currently not supported for MetalLB.

Configuring MetalLB in EKS Anywhere package spec

Starting at v0.13.5, keys within each config section start with lowercase. For example:

L2Advertisements:
    - IPAddressPools:
        - default

Becomes:

L2Advertisements:
    - ipAddressPools:
        - default

Top-level section names remain capitalized as they represent CRDs:

config: |
    IPAddressPools:
    ...

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: mylb
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: metallb
  targetNamespace: metallb-system
  config: |
    IPAddressPools:
      - name: default
        addresses:
          - 10.220.0.93/32
          - 10.220.0.94/32
          - 10.220.0.95/32
      - name: bgp
        addresses:
          - 10.220.0.97-10.220.0.99
    L2Advertisements:
      - ipAddressPools:
          - default
    BGPAdvertisements:
      - ipAddressPools:
          - bgp 
        autoAssign: false
    BGPPeers:
      - myASN: 123
        peerASN: 55001
        peerAddress: 1.2.3.4
        keepaliveTime: 30s
Parameter Description Default Required
IPAddressPools[] A list of ip address pools. See IPAddressPool. None False
L2Advertisements[] A list of Layer 2 advertisements. See L2Advertisement. None False
BGPAdvertisements[] A list of BGP advertisements. See BGPAdvertisement. None False
BGPPeers[] A list of BGP peers. See BGPPeer. None False
— — — —
IPAddressPool A list of IP address ranges over which MetalLB has authority. You can list multiple ranges in a single pool and they will all share the same settings. Each range can be either a CIDR prefix, or an explicit start-end range of IPs.
name Name for the address pool. None True
addresses[] A list of string representing CIRD or IP ranges. None True
autoAssign AutoAssign flag used to prevent MetalLB from automatic allocation for a pool. true False
— — — —
L2Advertisement L2Advertisement allows MetalLB to advertise the LoadBalancer IPs provided by the selected pools via L2.
ipAddressPools[] The list of IPAddressPool names to advertise. None True
name Name for the L2Advertisement. None False
— — — —
BGPAdvertisement BGPAdvertisement allows MetalLB to advertise the IPs coming from the selected ipAddressPools via BGP, setting the parameters of the BGP Advertisement.
aggregationLength The aggregation-length advertisement option lets you “roll up” the /32s into a larger prefix. Defaults to 32. Works for IPv4 addresses. 32 False
aggregationLengthV6 The aggregation-length advertisement option lets you “roll up” the /128s into a larger prefix. Defaults to 128. Works for IPv6 addresses. 128 False
communities[] The BGP communities to be associated with the announcement. Each item can be a community of the form 1234:1234 or the name of an alias defined in the Community CRD. None False
ipAddressPools[] The list of IPAddressPool names to be advertised via BGP. None True
localPref The BGP LOCAL_PREF attribute which is used by BGP best path algorithm, Path with higher localpref is preferred over one with lower localpref. None False
peers[] List of peer names. Limits the bgppeer to advertise the ips of the selected pools to. When empty, the loadbalancer IP is announced to all the BGPPeers configured. None False
— — — —
BGPPeer Peers for the BGP protocol.
holdTime Requested BGP hold time, per RFC4271. None False
keepaliveTime Requested BGP keepalive time, per RFC4271. None False
myASN AS number to use for the local end of the session. None True
password Authentication password for routers enforcing TCP MD5 authenticated sessions. None False
peerASN AS number to expect from the remote end of the session. None True
peerAddress Address to dial when establishing the session. None True
peerPort Port to dial when establishing the session. 179 False
routerID BGP router ID to advertise to the peer. None False
sourceAddress Source address to use when establishing the session. None False

8.3 - v0.13.7

FRRouting is currently not supported for MetalLB.

Configuring MetalLB in EKS Anywhere package spec

Starting at v0.13.5, keys within each config section start with lowercase. See v0.13.5 for details.

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: mylb
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: metallb
  targetNamespace: metallb-system
  config: |
    IPAddressPools:
      - name: default
        addresses:
          - 10.220.0.93/32
          - 10.220.0.94/32
          - 10.220.0.95/32
      - name: bgp
        addresses:
          - 10.220.0.97-10.220.0.99
    L2Advertisements:
      - ipAddressPools:
          - default
    BGPAdvertisements:
      - ipAddressPools:
          - bgp 
        autoAssign: false
    BGPPeers:
      - myASN: 123
        peerASN: 55001
        peerAddress: 1.2.3.4
        keepaliveTime: 30s
Parameter Description Default Required
IPAddressPools[] A list of ip address pools. See IPAddressPool. None False
L2Advertisements[] A list of Layer 2 advertisements. See L2Advertisement. None False
BGPAdvertisements[] A list of BGP advertisements. See BGPAdvertisement. None False
BGPPeers[] A list of BGP peers. See BGPPeer. None False
— — — —
IPAddressPool A list of IP address ranges over which MetalLB has authority. You can list multiple ranges in a single pool and they will all share the same settings. Each range can be either a CIDR prefix, or an explicit start-end range of IPs.
name Name for the address pool. None True
addresses[] A list of string representing CIRD or IP ranges. None True
autoAssign AutoAssign flag used to prevent MetalLB from automatic allocation for a pool. true False
— — — —
L2Advertisement L2Advertisement allows MetalLB to advertise the LoadBalancer IPs provided by the selected pools via L2.
ipAddressPools[] The list of IPAddressPool names to advertise. None True
name Name for the L2Advertisement. None False
— — — —
BGPAdvertisement BGPAdvertisement allows MetalLB to advertise the IPs coming from the selected ipAddressPools via BGP, setting the parameters of the BGP Advertisement.
aggregationLength The aggregation-length advertisement option lets you “roll up” the /32s into a larger prefix. Defaults to 32. Works for IPv4 addresses. 32 False
aggregationLengthV6 The aggregation-length advertisement option lets you “roll up” the /128s into a larger prefix. Defaults to 128. Works for IPv6 addresses. 128 False
communities[] The BGP communities to be associated with the announcement. Each item can be a community of the form 1234:1234 or the name of an alias defined in the Community CRD. None False
ipAddressPools[] The list of IPAddressPool names to be advertised via BGP. None True
localPref The BGP LOCAL_PREF attribute which is used by BGP best path algorithm, Path with higher localpref is preferred over one with lower localpref. None False
peers[] List of peer names. Limits the bgppeer to advertise the ips of the selected pools to. When empty, the loadbalancer IP is announced to all the BGPPeers configured. None False
— — — —
BGPPeer Peers for the BGP protocol.
holdTime Requested BGP hold time, per RFC4271. None False
keepaliveTime Requested BGP keepalive time, per RFC4271. None False
myASN AS number to use for the local end of the session. None True
password Authentication password for routers enforcing TCP MD5 authenticated sessions. None False
peerASN AS number to expect from the remote end of the session. None True
peerAddress Address to dial when establishing the session. None True
peerPort Port to dial when establishing the session. 179 False
routerID BGP router ID to advertise to the peer. None False
sourceAddress Source address to use when establishing the session. None False
password Authentication password for routers enforcing TCP MD5 authenticated sessions. None False
passwordSecret passwordSecret is a reference to the authentication secret for BGP Peer. The secret must be of type ‘kubernetes.io/basic-auth’ and the password stored under the “password” key. Example: 
passwordSecret:
  name: mySecret
  namespace: metallb-system
 None False

9 - Metrics Server Configuration

Metrics Server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

Configuration options for Metrics Server

9.1 - v3.8.2

Configuring Metrics Server in EKS Anywhere package spec

Parameter Description Default
General
args Additional args to provide to metrics-server
Example:
cloudProvider: ["–kubelet-insecure-tls"]
[]

10 - Prometheus Configuration

Prometheus is an open-source systems monitoring and alerting toolkit. It collects and stores metrics as time series data.

Best Practice

Any package configuration options listed under Reference/Packages should be modified through package yaml files (with kind: Package) through command eksctl anywhere apply package -f packageFileName. Modifying objects outside of package yaml files may lead to unpredictable behaviors.

For automatic namespace (targetNamespace) creation, see createNamespace field: PackagebundleController.spec

Configuration options for Prometheus

10.1 - v2.39.1

Configuring Prometheus in EKS Anywhere package spec

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: generated-prometheus
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: prometheus
  targetNamespace: observability
  config: |
    server:
      replicaCount: 2
      statefulSet:
        enabled: true

Configurable parameters and default values under spec.config

Parameter Description Default
General
rbac.create Specifies if clusterRole / role and clusterRoleBinding / roleBinding will be created for prometheus-server and node-exporter true
sourceRegistry Specifies image source registry for prometheus-server and node-exporter "783794618700.dkr.ecr.us-west-2.amazonaws.com"
Node-Exporter
nodeExporter.enabled Indicates if node-exporter is enabled true
nodeExporter.hostNetwork Indicates if node-exporter shares the host network namespace true
nodeExporter.hostPID Indicates if node-exporter shares the host process ID namespace true
nodeExporter.image.pullPolicy Specifies node-exporter image pull policy: IfNotPresent, Always, Never "IfNotPresent"
nodeExporter.image.repository Specifies node-exporter image repository "prometheus/node-exporter"
nodeExporter.resources Specifies resource requests and limits of the node-exporter container. Refer to the Kubernetes API documentation ResourceRequirements field for more details {}
nodeExporter.service Specifies how to expose node-exporter as a network service See footnote 1
nodeExporter.tolerations Specifies node tolerations for node-exporter scheduling to nodes with taints. Refer to the Kubernetes API documentation toleration field for more details. See footnote 2
serviceAccounts.nodeExporter.annotations Specifies node-exporter service account annotations {}
serviceAccounts.nodeExporter.create Indicates if node-exporter service account will be created true
serviceAccounts.nodeExporter.name Specifies node-exporter service account name ""
Prometheus-Server
server.enabled Indicates if prometheus-server is enabled true
server.global.evaluation_interval Specifies how frequently the prometheus-server rules are evaluated "1m"
server.global.scrape_interval Specifies how frequently prometheus-server will scrape targets "1m"
server.global.scrape_timeout Specifies how long until a prometheus-server scrape request times out "10s"
server.image.pullPolicy Specifies prometheus-server image pull policy: IfNotPresent, Always, Never "IfNotPresent"
server.image.repository Specifies prometheus-server image repository "prometheus/prometheus"
server.name Specifies prometheus-server container name "server"
server.persistentVolume.accessModes Specifies prometheus-server data Persistent Volume access modes "ReadWriteOnce"
server.persistentVolume.enabled Indicates if prometheus-server will create/use a Persistent Volume Claim true
server.persistentVolume.existingClaim Specifies prometheus-server data Persistent Volume existing claim name. It requires server.persistentVolume.enabled: true. If defined, PVC must be created manually before volume will be bound ""
server.persistentVolume.size Specifies prometheus-server data Persistent Volume size "8Gi"
server.remoteRead Specifies prometheus-server remote read configs. Refer to Prometheus docs remote_read for more details []
server.remoteWrite Specifies prometheus-server remote write configs. Refer to Prometheus docs remote_write for more details []
server.replicaCount Specifies the replicaCount for prometheus-server deployment / statefulSet. Note: server.statefulSet.enabled should be set to true if server.replicaCount is greater than 1 1
server.resources Specifies resource requests and limits of the prometheus-server container. Refer to the Kubernetes API documentation ResourceRequirements field for more details {}
server.retention Specifies prometheus-server data retention period "15d"
server.service Specifies how to expose prometheus-server as a network service See footnote 3
server.statefulSet.enabled Indicates if prometheus-server is deployed as a statefulSet. If set to false, prometheus-server will be deployed as a deployment false
serverFiles.“prometheus.yml”.scrape_configs Specifies a set of targets and parameters for prometheus-server describing how to scrape them. Refer to Prometheus docs scrape_config for more details See footnote 4
serviceAccounts.server.annotations Specifies prometheus-server service account annotations {}
serviceAccounts.server.create Indicates if prometheus-server service account will be created true
serviceAccounts.server.name Specifies prometheus-server service account name ""

  1. Node-exporter service is exposed as a clusterIP with port: 9100 (controlled by nodeExporter.service.servicePort below) and targetPort: 9100 (controlled by nodeExporter.service.hostPort below) by default. Note the annotation prometheus.io/scrape: "true" is mandatory in order for node-exporter to be discovered by prometheus-server as a scrape target. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    nodeExporter:
      service:
        annotations:
          prometheus.io/scrape: "true"
        hostPort: 9100
        servicePort: 9100
        type: ClusterIP
     ↩︎

  2. Node-exporter pods have the following toleration by default, which allows daemonSet to be scheduled on control plane node.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    nodeExporter:
      tolerations:
        # For K8 version prior to 1.24
        - key: "node-role.kubernetes.io/master"
          operator: "Exists"
          effect: "NoSchedule"
        # For K8 version 1.24+
        - key: "node-role.kubernetes.io/control-plane"
          operator: "Exists"
          effect: "NoSchedule"
     ↩︎

  3. Prometheus-server service is exposed as a clusterIP with port: 9090 (controlled by server.service.servicePort below) and targetPort: 9090 (not overridable) by default. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    server:
      service:
        enabled: true
        servicePort: 9090
        type: ClusterIP
     ↩︎

  4. Prometheus-server by default has the following scrape configs.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: | 
    serverFiles:
      prometheus.yml:
        scrape_configs:
          - job_name: prometheus
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            static_configs:
            - targets:
              - localhost:9090
          - job_name: kubernetes-apiservers
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
              separator: ;
              regex: default;kubernetes;https
              replacement: $1
              action: keep
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-nodes
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - separator: ;
              regex: __meta_kubernetes_node_label_(.+)
              replacement: $1
              action: labelmap
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: kubernetes.default.svc:443
              action: replace
            - source_labels: [__meta_kubernetes_node_name]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: /api/v1/nodes/$1/proxy/metrics
              action: replace
            kubernetes_sd_configs:
            - role: node
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-nodes-cadvisor
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - separator: ;
              regex: __meta_kubernetes_node_label_(.+)
              replacement: $1
              action: labelmap
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: kubernetes.default.svc:443
              action: replace
            - source_labels: [__meta_kubernetes_node_name]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
              action: replace
            kubernetes_sd_configs:
            - role: node
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-service-endpoints
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: drop
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_node_name]
              separator: ;
              regex: (.*)
              target_label: node
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-service-endpoints-slow
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 5m
            scrape_timeout: 30s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_node_name]
              separator: ;
              regex: (.*)
              target_label: node
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: prometheus-pushgateway
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
              separator: ;
              regex: pushgateway
              replacement: $1
              action: keep
            kubernetes_sd_configs:
            - role: service
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-services
            honor_labels: true
            honor_timestamps: true
            params:
              module:
              - http_2xx
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /probe
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__address__]
              separator: ;
              regex: (.*)
              target_label: __param_target
              replacement: $1
              action: replace
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: blackbox
              action: replace
            - source_labels: [__param_target]
              separator: ;
              regex: (.*)
              target_label: instance
              replacement: $1
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: service
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-pods
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: drop
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_pod_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_name]
              separator: ;
              regex: (.*)
              target_label: pod
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_phase]
              separator: ;
              regex: Pending|Succeeded|Failed|Completed
              replacement: $1
              action: drop
            kubernetes_sd_configs:
            - role: pod
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-pods-slow
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 5m
            scrape_timeout: 30s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_pod_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_name]
              separator: ;
              regex: (.*)
              target_label: pod
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_phase]
              separator: ;
              regex: Pending|Succeeded|Failed|Completed
              replacement: $1
              action: drop
            kubernetes_sd_configs:
            - role: pod
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
     ↩︎

10.2 - v2.41.1

Configuring Prometheus in EKS Anywhere package spec

Example

apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  name: generated-prometheus
  namespace: eksa-packages-<cluster-name>
spec:
  packageName: prometheus
  targetNamespace: observability
  config: |
    server:
      replicaCount: 2
      statefulSet:
        enabled: true

Configurable parameters and default values under spec.config

Parameter Description Default
General
rbac.create Specifies if clusterRole / role and clusterRoleBinding / roleBinding will be created for prometheus-server and node-exporter true
sourceRegistry Specifies image source registry for prometheus-server and node-exporter "783794618700.dkr.ecr.us-west-2.amazonaws.com"
Node-Exporter
nodeExporter.enabled Indicates if node-exporter is enabled true
nodeExporter.hostNetwork Indicates if node-exporter shares the host network namespace true
nodeExporter.hostPID Indicates if node-exporter shares the host process ID namespace true
nodeExporter.image.pullPolicy Specifies node-exporter image pull policy: IfNotPresent, Always, Never "IfNotPresent"
nodeExporter.image.repository Specifies node-exporter image repository "prometheus/node-exporter"
nodeExporter.resources Specifies resource requests and limits of the node-exporter container. Refer to the Kubernetes API documentation ResourceRequirements field for more details {}
nodeExporter.service Specifies how to expose node-exporter as a network service See footnote 1
nodeExporter.tolerations Specifies node tolerations for node-exporter scheduling to nodes with taints. Refer to the Kubernetes API documentation toleration field for more details. See footnote 2
serviceAccounts.nodeExporter.annotations Specifies node-exporter service account annotations {}
serviceAccounts.nodeExporter.create Indicates if node-exporter service account will be created true
serviceAccounts.nodeExporter.name Specifies node-exporter service account name ""
Prometheus-Server
server.enabled Indicates if prometheus-server is enabled true
server.global.evaluation_interval Specifies how frequently the prometheus-server rules are evaluated "1m"
server.global.scrape_interval Specifies how frequently prometheus-server will scrape targets "1m"
server.global.scrape_timeout Specifies how long until a prometheus-server scrape request times out "10s"
server.image.pullPolicy Specifies prometheus-server image pull policy: IfNotPresent, Always, Never "IfNotPresent"
server.image.repository Specifies prometheus-server image repository "prometheus/prometheus"
server.name Specifies prometheus-server container name "server"
server.persistentVolume.accessModes Specifies prometheus-server data Persistent Volume access modes "ReadWriteOnce"
server.persistentVolume.enabled Indicates if prometheus-server will create/use a Persistent Volume Claim true
server.persistentVolume.existingClaim Specifies prometheus-server data Persistent Volume existing claim name. It requires server.persistentVolume.enabled: true. If defined, PVC must be created manually before volume will be bound ""
server.persistentVolume.size Specifies prometheus-server data Persistent Volume size "8Gi"
server.remoteRead Specifies prometheus-server remote read configs. Refer to Prometheus docs remote_read for more details []
server.remoteWrite Specifies prometheus-server remote write configs. Refer to Prometheus docs remote_write for more details []
server.replicaCount Specifies the replicaCount for prometheus-server deployment / statefulSet. Note: server.statefulSet.enabled should be set to true if server.replicaCount is greater than 1 1
server.resources Specifies resource requests and limits of the prometheus-server container. Refer to the Kubernetes API documentation ResourceRequirements field for more details {}
server.retention Specifies prometheus-server data retention period "15d"
server.service Specifies how to expose prometheus-server as a network service See footnote 3
server.statefulSet.enabled Indicates if prometheus-server is deployed as a statefulSet. If set to false, prometheus-server will be deployed as a deployment false
serverFiles.“prometheus.yml”.scrape_configs Specifies a set of targets and parameters for prometheus-server describing how to scrape them. Refer to Prometheus docs scrape_config for more details See footnote 4
serviceAccounts.server.annotations Specifies prometheus-server service account annotations {}
serviceAccounts.server.create Indicates if prometheus-server service account will be created true
serviceAccounts.server.name Specifies prometheus-server service account name ""

  1. Node-exporter service is exposed as a clusterIP with port: 9100 (controlled by nodeExporter.service.servicePort below) and targetPort: 9100 (controlled by nodeExporter.service.hostPort below) by default. Note the annotation prometheus.io/scrape: "true" is mandatory in order for node-exporter to be discovered by prometheus-server as a scrape target. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    nodeExporter:
      service:
        annotations:
          prometheus.io/scrape: "true"
        hostPort: 9100
        servicePort: 9100
        type: ClusterIP
     ↩︎

  2. Node-exporter pods have the following toleration by default, which allows daemonSet to be scheduled on control plane node.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    nodeExporter:
      tolerations:
        # For K8 version prior to 1.24
        - key: "node-role.kubernetes.io/master"
          operator: "Exists"
          effect: "NoSchedule"
        # For K8 version 1.24+
        - key: "node-role.kubernetes.io/control-plane"
          operator: "Exists"
          effect: "NoSchedule"
     ↩︎

  3. Prometheus-server service is exposed as a clusterIP with port: 9090 (controlled by server.service.servicePort below) and targetPort: 9090 (not overridable) by default. See below specification for details.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: |
    server:
      service:
        enabled: true
        servicePort: 9090
        type: ClusterIP
     ↩︎

  4. Prometheus-server by default has the following scrape configs.

    apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
...
spec:
  config: | 
    serverFiles:
      prometheus.yml:
        scrape_configs:
          - job_name: prometheus
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            static_configs:
            - targets:
              - localhost:9090
          - job_name: kubernetes-apiservers
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
              separator: ;
              regex: default;kubernetes;https
              replacement: $1
              action: keep
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-nodes
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - separator: ;
              regex: __meta_kubernetes_node_label_(.+)
              replacement: $1
              action: labelmap
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: kubernetes.default.svc:443
              action: replace
            - source_labels: [__meta_kubernetes_node_name]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: /api/v1/nodes/$1/proxy/metrics
              action: replace
            kubernetes_sd_configs:
            - role: node
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-nodes-cadvisor
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: https
            authorization:
              type: Bearer
              credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token
            tls_config:
              ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              insecure_skip_verify: false
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - separator: ;
              regex: __meta_kubernetes_node_label_(.+)
              replacement: $1
              action: labelmap
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: kubernetes.default.svc:443
              action: replace
            - source_labels: [__meta_kubernetes_node_name]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor
              action: replace
            kubernetes_sd_configs:
            - role: node
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-service-endpoints
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: drop
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_node_name]
              separator: ;
              regex: (.*)
              target_label: node
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-service-endpoints-slow
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 5m
            scrape_timeout: 30s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_node_name]
              separator: ;
              regex: (.*)
              target_label: node
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: endpoints
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: prometheus-pushgateway
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
              separator: ;
              regex: pushgateway
              replacement: $1
              action: keep
            kubernetes_sd_configs:
            - role: service
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-services
            honor_labels: true
            honor_timestamps: true
            params:
              module:
              - http_2xx
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /probe
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__address__]
              separator: ;
              regex: (.*)
              target_label: __param_target
              replacement: $1
              action: replace
            - separator: ;
              regex: (.*)
              target_label: __address__
              replacement: blackbox
              action: replace
            - source_labels: [__param_target]
              separator: ;
              regex: (.*)
              target_label: instance
              replacement: $1
              action: replace
            - separator: ;
              regex: __meta_kubernetes_service_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_service_name]
              separator: ;
              regex: (.*)
              target_label: service
              replacement: $1
              action: replace
            kubernetes_sd_configs:
            - role: service
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-pods
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 1m
            scrape_timeout: 10s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: drop
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_pod_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_name]
              separator: ;
              regex: (.*)
              target_label: pod
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_phase]
              separator: ;
              regex: Pending|Succeeded|Failed|Completed
              replacement: $1
              action: drop
            kubernetes_sd_configs:
            - role: pod
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
          - job_name: kubernetes-pods-slow
            honor_labels: true
            honor_timestamps: true
            scrape_interval: 5m
            scrape_timeout: 30s
            metrics_path: /metrics
            scheme: http
            follow_redirects: true
            enable_http2: true
            relabel_configs:
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_slow]
              separator: ;
              regex: "true"
              replacement: $1
              action: keep
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scheme]
              separator: ;
              regex: (https?)
              target_label: __scheme__
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
              separator: ;
              regex: (.+)
              target_label: __metrics_path__
              replacement: $1
              action: replace
            - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
              separator: ;
              regex: (.+?)(?::\d+)?;(\d+)
              target_label: __address__
              replacement: $1:$2
              action: replace
            - separator: ;
              regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
              replacement: __param_$1
              action: labelmap
            - separator: ;
              regex: __meta_kubernetes_pod_label_(.+)
              replacement: $1
              action: labelmap
            - source_labels: [__meta_kubernetes_namespace]
              separator: ;
              regex: (.*)
              target_label: namespace
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_name]
              separator: ;
              regex: (.*)
              target_label: pod
              replacement: $1
              action: replace
            - source_labels: [__meta_kubernetes_pod_phase]
              separator: ;
              regex: Pending|Succeeded|Failed|Completed
              replacement: $1
              action: drop
            kubernetes_sd_configs:
            - role: pod
              kubeconfig_file: ""
              follow_redirects: true
              enable_http2: true
     ↩︎