I'm 100% aware of the pitfalls of the JVM and advances/strides that have been done in the JVM world for it to understand the ergonomics/resources of a container. Also, yes I know that Java by default tries to allocate 1/4 the RAM available in the environment it's running...
So I had some thoughts and questions. I have made a 50/50 rule. If my application needs 1GB of Xmx, then I create 2GB container which gives another 1GB of RAM for the JVM overhead and any container/swap stuff (though not sure how swap really works within a container).
So I was thinking if my application needs 6GB of Xmx do I really need to create 12GB container or can I get away with 7GB or 8GB container? How much headroom do we need to give inside the container when it comes to RAM?
If your container is dedicated to the JVM, then you don't need to use a percentage.
You need to know how much java heap you need -- in this case 6GB -- and how much everything else needs, which it looks like you've already proven is less than 2GB.
Then just add them up. An 8GB container should be fine in this case. Note that stack and heap memory are separate, though, so if you will need a large number of threads running simultaneously, don't forget to add 1MB of stack space for each one (or whatever you set with -Xss)
ALSO, I really recommend setting the minimum and maximum Java heap size to the SAME value -- 6GB in this case. Then you're guaranteed that there will no surprises when Java tries to grow the heap.