Table of contents
1. FileUtils.copyFile
1. Starting from an example
Generally, during development, files are copied through the file tool class, so how about its performance? How is it achieved? Let’s analyze the following FileUtils.copyFile today.
private static void copyFileByUtils() {
String srcFilePath = "H:\\CSDN\\JWFS.rar";// 文件大小 68.8 MB
String destFilePath = "H:\\CSDN\\netty\\nio\\JWFS.rar";
long start = System.currentTimeMillis();
try {
FileUtils.copyFile(new File(srcFilePath),new File(destFilePath));
} catch (IOException e) {
e.printStackTrace();
}
long end = System.currentTimeMillis();
System.out.println("copyFileByUtils 消耗："+(end-start)+"毫秒");
}
2. It's still pretty fast, let's explore the source code...
private static void doCopyFile(File srcFile, File destFile, boolean preserveFileDate) throws IOException {
if (destFile.exists() && destFile.isDirectory()) {
throw new IOException("Destination '" + destFile + "' exists but is a directory");
} else {
FileInputStream fis = null;
FileOutputStream fos = null;
FileChannel input = null;
FileChannel output = null;
try {
fis = new FileInputStream(srcFile);
fos = new FileOutputStream(destFile);
input = fis.getChannel();
output = fos.getChannel();
long size = input.size();
long pos = 0L;
for(long count = 0L; pos < size; pos += output.transferFrom(input, pos, count)) {
count = size - pos > 31457280L ? 31457280L : size - pos;
}
} finally {
IOUtils.closeQuietly(output);
IOUtils.closeQuietly(fos);
IOUtils.closeQuietly(input);
IOUtils.closeQuietly(fis);
}
if (srcFile.length() != destFile.length()) {
throw new IOException("Failed to copy full contents from '" + srcFile + "' to '" + destFile + "'");
} else {
if (preserveFileDate) {
destFile.setLastModified(srcFile.lastModified());
}
}
}
}
Found a rare vocabulary, FileChannel, study it.
Two, FileChannel
This way it is “在非直接缓冲区中，通过Channel实现文件的复制”.
1. Read operation
- Read the disk file into the memory of the kernel address space provided by the operating system OS, copy it for the first time, and switch the OS context to the kernel mode;
- Copy the contents of the file in the kernel address space memory to the memory in the user address space provided by the JVM, the second copy, the OS context switches to the user mode;
2. Write operation
- Copy the contents of the file in the JVM memory of the user address space to the memory in the kernel address space provided by the OS. For the first copy, the OS context switches to the kernel mode;
- Write the content of the file in the memory in the kernel address space to the disk file, copy it for the second time, and after the writing operation is completed, the OS context finally switches to the user mode;
The memory controlled by the JVM is called the heap memory. Generally, the memory operated by Java belongs to the heap memory. The heap memory is uniformly managed by the JVM. According to the above flow chart, it can be found that a file read and write needs to go through 4 times of copy and 4 times. Secondary user control and kernel space context switching.
3. Code example
package com.guor.demo.io;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
public class CopyFileTest {
private static void copyFileByChannel() {
FileInputStream fileInputStream = null;
FileOutputStream fileOutputStream = null;
FileChannel intChannel = null;
FileChannel outChannel = null;
String srcFilePath = "H:\\CSDN\\JWFS.rar";// 文件大小 68.8 MB
String destFilePath = "H:\\CSDN\\netty\\nio\\JWFS.rar";
long start = System.currentTimeMillis();
try {
fileInputStream = new FileInputStream(srcFilePath);
fileOutputStream = new FileOutputStream(destFilePath);
// 获取通道
intChannel = fileInputStream.getChannel();
outChannel = fileOutputStream.getChannel();
// 创建非直接缓冲区
ByteBuffer buffer = ByteBuffer.allocate(1024);
while (intChannel.read(buffer)!=-1){
buffer.flip();
outChannel.write(buffer);
buffer.clear();
}
long end = System.currentTimeMillis();
System.out.println("copyFileByChannel 消耗："+(end-start)+"毫秒");
}catch (Exception e){
System.out.println(e);
}finally {
if(outChannel!=null){
try {
outChannel.close();
} catch (IOException e) {
}
}
if(intChannel!=null){
try {
intChannel.close();
} catch (IOException e) {
}
}
if(fileOutputStream!=null){
try {
fileOutputStream.close();
} catch (IOException e) {
}
}
if(fileInputStream!=null){
try {
fileInputStream.close();
} catch (IOException e) {
}
}
}
}
public static void main(String[] args) {
copyFileByChannel();
}
}
4. Console output
It feels that there is still a gap with the speed of FileUtils.copyFile.
3. How to reduce the number of copy and context switching?
1. Why can't we abandon the step of kernel space and read directly to user space?
Because there is a GC in the JVM, the GC will clean up useless objects from time to time and compress the file area. If a file is being copied in the JVM at a certain moment, the location of the file in the JVM may change due to the GC’s compression operation , causing an exception in the program. Therefore, in order to ensure that the position of the file in the memory does not change, it can only be put into the memory of the OS.
2. How to reduce the number of copy and context switching?
Using a direct buffer, you can point to a piece of memory in the OS (called a physical mapping file) through an address variable in the JVM, and then you can directly use the memory in the OS through the JVM.
A new way is introduced below “在直接缓冲区中，通过Channel实现文件的复制”.
In this way, the data assignment operation is performed in the kernel space, and the number of direct copies between the user space and the kernel space is 0, that is, zero copy.
3. Code example
private static void copyFileByMapped() {
String srcFilePath = "H:\\CSDN\\JWFS.rar";// 文件大小 68.8 MB
String destFilePath = "H:\\CSDN\\netty\\nio\\JWFS.rar";
long start = System.currentTimeMillis();
FileChannel inChannel = null;
FileChannel outChannel = null;
try {
// 文件的输入通道
inChannel = FileChannel.open(Paths.get(srcFilePath), StandardOpenOption.READ);
// 文件的输出通道
outChannel = FileChannel.open(Paths.get(destFilePath), StandardOpenOption.WRITE, StandardOpenOption.READ, StandardOpenOption.CREATE);
// 输入通道和输出通道之间的内存映射文件，内存映射文件处于堆外内存
MappedByteBuffer inMappedBuffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
MappedByteBuffer outMappedBuffer = outChannel.map(FileChannel.MapMode.READ_WRITE, 0, inChannel.size());
// 直接对内存映射文件进行读写
byte[] bytes = new byte[inMappedBuffer.limit()];
inMappedBuffer.get(bytes);
outMappedBuffer.put(bytes);
long end = System.currentTimeMillis();
System.out.println("copyFileByMapped 消耗："+(end-start)+"毫秒");
}catch (Exception e){
System.out.println(e);
}finally {
try {
inChannel.close();
outChannel.close();
} catch (IOException e) {
}
}
}
4. Console output
Direct buffer copy files are 10 times faster than non-direct buffer copy files.
Nezha Boutique Series Articles
Summary of Java learning route, brick movers attack Java architects
Summary of 100,000 words and 208 Java classic interview questions (with answers)
