I'm using minikube, starting it with
minikube start --memory 8192
For 8Gb of RAM for the node. I'm allocating pods with the resource constraints
resources:
limits:
memory: 256Mi
requests:
memory: 256Mi
So 256Mb of RAM for each node which would give me, I would assume, 32 pods until 8Gb memory limit has been reached but the problem is that whenever I reach the 8th pod to be deployed, the 9th will never run because it's constantly OOMKilled.
For context, each pod is a Java application with a frolvlad/alpine-oraclejdk8:slim Docker container ran with -Xmx512m -Xms128m (even if JVM was indeed using the full 512Mb instead of 256Mb I would still be far from the 16 pod limit to hit the 8Gb cap).
What am I missing here? Why are pods being OOMKilled with apparently so much free allocatable memory left?
Thanks in advance
You must understand the way requests and limits work.
Requests are the requirements for the amount of allocateable resources requred on the node for a pod to get scheduled on it. These will not cause OOMs, they will cause pod not to get scheduled.
Limits on the other side, are hard limits for given pod. The pod will be capped at this level. So, even if you have 16GB RAM free, but have a 256MiB limit on it, as soon as your pod reaches this level, it will experience an OOM kill.
If you want, you can define only requests. Then, your pods will be able to grow to full node capacity, without being capped.
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/