As far as I know reference assignment is atomic in a 64 bit JVM. Now, I assume the jvm doesn't use atomic pointers internally to model this, since otherwise there would be no need for Atomic References. So my questions are:
Is atomic reference assignment in the "specs" of java/Scala and guaranteed to happen or is it just a happy coincidence that it is that way most times ?
Is atomic reference assignment implied for any language that compiles to the JVM's bytecode (e.g. clojure, Groovy, JRuby, JPython...etc) ?
How can reference assignment be atomic without using an atomic pointer internally ?
First of all, reference assignment is atomic because the specification says so. Besides that, there is no obstacle for JVM implementors to fulfill this constraint, as 64 Bit references are usually only used on 64 Bit architectures, where atomic 64 Bit assignment comes for free.
Your main confusion stems from the assumption that the additional “Atomic References” feature means exactly that, due to its name. But the AtomicReference class offers far more, as it encapsulates a volatile reference, which has stronger memory visibility guarantees in a multi-threaded execution.
Having an atomic reference update does not necessarily imply that a thread reading the reference will also see consistent values regarding the fields of the object reachable through that reference. All it guarantees is that you will read either the null reference or a valid reference to an existing object that actually was stored by some thread. If you want more guarantees, you need constructs like synchronization, volatile references, or an AtomicReference.
AtomicReference also offers atomic update operations like compareAndSet or getAndSet. These are not possible with ordinary reference variables.