ARTIFACTORY: How to deploy Artifactory HA in different availability zones in Kubernetes

In this article, we will go over deploying Artifactory’s Helm chart with multiple replicas, on zones that are in different AZ’s (Availability zones). 

Using EKS with two nodes in eu-central-1 for this example, we have two nodes, one in AZ eu-central-1a, and the other is in AZ eu-central-1b. Deploying two pods in both nodes comes down to using simple Affinity and Toleration configurations. Before we continue, it’s important to note that Artifactory’s Helm chart comes configured with PodAntiAffinity by default, meaning that it will deploy your pods on separate nodes by default. However, it is quite random, and we do need to make sure that they are on our specific eu-central-1 nodes with the different AZ’s.

Let’s begin with the first option:

Tolerations

Utilizing tolerations and the default PodAntiAffinity that comes with the chart, we can only use tolerations to force the chart to only run on our eu-central-1 nodes. Tolerations depend on node taints, so for our first step, we will need to taint each node with its specific label. 

For node 1a, I will be applying a taint named  “dedicated=node-eu-a:NoSchedule”:

kubectl taint nodes ip-10-0-0-5.eu-central-1.compute.internal dedicated=node-eu-a:NoSchedule

For node 1b, I will be applying a taint named “dedicated=node-eu-b:NoSchedule”:

kubectl taint nodes ip-10-0-1-168.eu-central-1.compute.internal dedicated=node-eu-b:NoSchedule

Now in our values.yaml, we can apply a Toleration rule where our Pods will only be able to schedule on both of our nodes, based on the taints that we gave them. 

For example:

artifactory:
  replicatCount: 2
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "node-eu-a" 
    effect: "NoSchedule"
  - key: "dedicated"
    operator: "Equal"
    value: "node-eu-b" 
    effect: "NoSchedule"

With the above configurations, we only limit our pods to schedule specifically on both of our nodes based on the taints that we gave, and with the flexibility of the default PodAntiAffinity, each pod will deploy on a separate node. 

If you are using the bundled Nginx and Postgresql that comes with the chart, and you want it on one of the nodes, we will have to also configure it’s tolerations as well:

artifactory:
  replicaCount: 2
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "node-eu-a" 
    effect: "NoSchedule"
  - key: "dedicated"
    operator: "Equal"
    value: "node-eu-b" 
    effect: "NoSchedule"
postgresql:
  primary:
    tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "node-eu-a" 
      effect: "NoSchedule"
nginx:
  tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "node-eu-b" 
    effect: "NoSchedule"

 
The above example will force Postgres to deploy on 1a, and Nginx on 1b, but you can configure it however you like.

NodeAffinity and custom PodAntiAffinity:

Other than tolerations, we can also override the default PodAntiAffiinity with a custom one to split the pods, and use a secondary NodeAffinity configuration, to force our pods to only run on both of our nodes. Same concept, but utilizing NodeAffinity instead of Tolerations. 

To make this work, we will first have to apply the same label (Not a taint) to both of our nodes, so then we can use NodeAffinity to force our pods to only be scheduled on our nodes:

kubectl label nodes ip-10-0-0-5.eu-central-1.compute.internal app=artifactory

kubectl label nodes ip-10-0-1-168.eu-central-1.compute.internal app=artifactory

With the above commands, we labeled both of our nodes with app=artifactory. Now we can use NodeAffinity to force our pods to be scheduled based on the label, and PodAntiAffinity to make sure that they are separated:

artifactory:
  replicaCount: 2
  affinity:
    nodeAffinity: 
     requiredDuringSchedulingIgnoredDuringExecution: 
       nodeSelectorTerms: 
       - matchExpressions: 
         - key: app
           operator: In 
           values: 
           - artifactory
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - artifactory
        topologyKey: "kubernetes.io/hostname"

Note that we need to provide our own custom PodAntiAffinity, because when using the artifactory.affinity key, it completely overrides the default one that comes with the chart. You can also use preferredDuringSchedulingIgnoredDuringExecution, Instead of requiredDuringSchedulingIgnoredDuringExecution, which will give the pods the option to be scheduled on the other node, in case their node went down. Furthermore, you can use any topologyKey you would like, as long as the label exists on both nodes.

The same will have to be added to the bundled Nginx and Postgresql, as with the Tolerations example.

Finally, this is an example with 2 nodes, and 2 pods, but you can use this as a base standard with a higher number of nodes and pods.