haha!

That was hilarious!

From what I understood and tried, it goes by batches. So if you come with a lot of pods to schedule at once it will fit them all in a big instance, if you come with a few pods at once it will take a smaller instance. According to resource requests it will choose a bunch of satisfying instance types and ask aws for the cheapest. Test done by scaling a basic nginx deployment to different numbers: With 30 replicas, karpenter spins up a t3a.small eks node. With 100 replicas, karpenter spins up a c4a.4xlarge eks node. Keep in mind that there's also the node pod capacity that can induce new nodes creation even if there are available resources (I was surprised it needed a new node for nginx pods with all the available CPU & mem, but turns out the m6g.medium instances I was using have an 8 pod limit capacity)

It will find the cheapest available instance type based on the workload requirements and batch size. With cluster rebalancing it'll even calculate what has changed and make sure the cluster is running with optimal utilization and low cost

Yes, if you don’t limit the scope of nodes Karpenter or CAS are controlling you will have race conditions. You can do that by specifying node groups for CAS and provisioner labels for Karpenter

Yep! HPA creates new pods. The scheduler will try to place them. If it fails Karpenter will create new nodes.

For some workloads a single point of failure is acceptable. With Karpenter it’s up to the application to decide

Not for now.

The node expiration does it indirectly: when an underused node is terminated by karpenter upon expiration, the pods to reschedule will use an existing node if available capacity, if not it will recreate a node accordingly.

Node consolidation is out now kzfaq.infoxX3aBgpY3B4?feature=share

Roughly, yes. There’s a lot of assumptions made with node groups and more control (and requirements) talking to nodes

I remember reading that article but don’t remember the company, sorry

@@JustinGarrison no worries, thank you for your response, do you happen to remember any useful docs that shed light on how to set a provisioner to handle your baseline workloads with on demand instances and provision spot instances for what exceeds that?

Yes, it works with any Kubernetes cluster running in AWS (not just EKS)

@@JustinGarrison Thanks for the info.

The provisioners can set limits in multiple dimensions. You can set caps at instance types, total number of CPU/memory and other options. It's possible to set up provisioners per namespace (if that's how dev teams are separated) and restrict what instances teams can deploy

Everyone learns differently and have you ever seen Bill Nye, Alton Brown, or Patch Adams?