Direct Memory Overview
- It is not part of the virtual machine runtime data area, nor is it a memory area defined in the Java Virtual Machine Specification.
- Direct memory is the memory range outside the Java heap that is directly applied to the system.
- Originated from NIO, operate Native memory through DirectByteBuffer stored in the heap
- Usually, the speed of accessing direct memory will be due to the Java heap, that is, high read and write performance.
- Because of performance considerations, direct memory may be considered for occasions with frequent reads and writes.
- Java's NIO library allows Java programs to use direct memory for data buffers
Direct memory footprint code example
/**
* 查看直接内存的占用与释放
*/
public class BufferTest {
private static final int BUFFER = 1024 * 1024 * 1024; // 1GB
public static void main(String[] args) {
// 直接分配本地内存空间
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(BUFFER);
System.out.println("直接内存分配完毕，请求指示！");
Scanner scanner = new Scanner(System.in);
scanner.next();
System.out.println("直接内存开始释放！");
byteBuffer = null;
System.gc();
scanner.next();
}
}
After running, check the PID according to the jps command, and you can see the specific memory usage in the task manager
> jps
20192 BufferTest
You can see that it takes up more than 1G of memory space, just enter a string of text
You can see that the memory usage has dropped to 18.3M
Reasons to use direct memory
To read and write files, you need to interact with the disk, and you need to switch from user mode to kernel too. When the kernel is too large, the operation of memory as shown in the figure below is required. Using IO, two copies of memory are required to store duplicate data, which is inefficient.
When using NIO, as shown in the figure below, the direct buffer area divided by the operating system can be directly accessed by Java code, and there is only one copy. NIO is suitable for reading and writing operations on large files.
Code example:
/**
* io与nio例子
*/
public class BufferTest1 {
private static final String FILE = "D:\\迅雷下载\\新建文件夹\\XXX.mkv";
private static final int _10Mb = 1024 * 1024 * 10;
public static void main(String[] args) {
long sum = 0;
String src = FILE;
for (int i = 0; i < 3; i++) {
String dest = "D:\\迅雷下载\\新建文件夹\\XXX" + i + ".mkv";
sum += directBudder(src, dest); // 总花费时间为：31345
// sum += io(src, dest); // 总花费时间为：62520
}
System.out.println("总花费时间为：" + sum);
}
private static long directBudder(String src, String dest) {
long start = System.currentTimeMillis();
FileChannel inChannel = null;
FileChannel outChannel = null;
try {
inChannel = new FileInputStream(src).getChannel();
outChannel = new FileOutputStream(dest).getChannel();
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(_10Mb);
while (inChannel.read(byteBuffer) != -1) {
byteBuffer.flip(); // 修改为读数据模式
outChannel.write(byteBuffer);
byteBuffer.clear(); // 清空
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (inChannel != null) {
try {
inChannel.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (outChannel != null) {
try {
outChannel.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
long end = System.currentTimeMillis();
return end - start;
}
private static long io(String src, String dest) {
long start = System.currentTimeMillis();
FileInputStream fis = null;
FileOutputStream fos = null;
try {
fis = new FileInputStream(src);
fos = new FileOutputStream(dest);
byte[] buffer = new byte[_10Mb];
while (true) {
int len = fis.read(buffer);
if (len == -1) {
break;
}
fos.write(buffer, 0, len);
}
} catch (IOException e) {
e.printStackTrace();
}
long end = System.currentTimeMillis();
return end - start;
}
}
Conclusion: In the case of a video size of 1.5G, io takes 62520, nio takes 31345, and nio is faster
Features of direct memory:
- May also cause OutOfMemoryError exception
- Since the direct memory is outside the Java heap, its size will not be directly limited by the maximum heap size specified by -Xmx, but the system memory is limited, and the Java heap and direct memory are still limited by the maximum size that the operating system can give. Memory.
- shortcoming:
- Allocation and recovery costs are high
- Not subject to JVM memory recovery management
- Direct memory size can be set by -XX:MaxDirectMemorySize
- If not specified, the default is consistent with the maximum value of the heap -Xmx parameter.
Direct memory OOM code example
/**
* 本地内存OOM OutOfMemoryError: Direct buffer memory
*/
public class BufferTest2 {
private static final int BUFFER = 1024 * 1024 * 20; // 20MB
public static void main(String[] args) {
ArrayList<ByteBuffer> list = new ArrayList<>();
int count = 0;
try {
while (true) {
// 使用直接内存
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(BUFFER);
list.add(byteBuffer);
count++;
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}finally {
System.out.println(count);
}
}
}
operation result:
362
Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at BufferTest2.main(BufferTest2.java:19)
UnSafe OutOfMemoryError code example
/**
* -Xmx20m -XX:MaxDirectMemorySize=10m
*/
public class MaxDirectMemorySizeTest {
private static final long _1MB = 1024 * 1024;
public static void main(String[] args) throws IllegalAccessException {
Field unsafeField = Unsafe.class.getDeclaredFields()[0];
unsafeField.setAccessible(true);
Unsafe unsafe = (Unsafe) unsafeField.get(null);
while (true) {
unsafe.allocateMemory(_1MB);
}
}
}
operation result:
Exception in thread "main" java.lang.OutOfMemoryError
at sun.misc.Unsafe.allocateMemory(Native Method)
at MaxDirectMemorySizeTest.main(MaxDirectMemorySizeTest.java:17)
Conclusion: direct memory will also OOM
Simple understanding:
java process memory = java heap + native memory