1. Second transmission
Second transmission means no transmission, and the implementation logic is to check whether the file already exists in the database or cache. If there is, just get it directly from the existing file (return the file address). To judge whether it is the same file here, the information digest algorithm
is used . For details, please refer to: An article to understand the current commonly used encryption technology system. Information digest algorithms are often used to ensure the integrity of information and prevent information from being tampered with during transmission (it is impossible to determine whether the information has been monitored). The most commonly used is the MD5 algorithm.
2. Partial upload
Uploading in pieces is equivalent to splitting a large file into many small parts, just like the hls file, piece by piece, and after the upload is completed, they are combined to form a complete file.
It is generally used together with breakpoint resume.
3. Resuming from breakpoints
During the multipart upload process, if the upload is interrupted due to session failure, network problems, or system problems, upload again after recovery, instead of restarting from the beginning, it is a resumable upload.
4. RandomAccessFile class
It supports free jumping to any position of the file for reading and writing . The role of the RandomAccessFile class is obvious, and it can support fragmented uploads and breakpoint resumes.
Part of its source code is as follows:
public RandomAccessFile(File file, String mode)
throws FileNotFoundException
{
String name = (file != null ? file.getPath() : null);
int imode = -1;
if (mode.equals("r"))
imode = O_RDONLY;
else if (mode.startsWith("rw")) {
imode = O_RDWR;
rw = true;
if (mode.length() > 2) {
if (mode.equals("rws"))
imode |= O_SYNC;
else if (mode.equals("rwd"))
imode |= O_DSYNC;
else
imode = -1;
}
}
......
fd = new FileDescriptor();
fd.attach(this);
path = name;
open(name, imode);
}
r : read-only mode.
rw: read-write mode
rws: Relative to rw, rws synchronously updates the modification of "file content" or "metadata" to the external storage device.
rwd : Relative to rw, rwd synchronously updates the modification of the "content of the file" to the external storage device.
File content refers to the data actually stored in the file, and metadata is used to describe file attributes such as file size information, creation and modification times.
Code example:
public static void main(String[] args) throws IOException {
RandomAccessFile randomAccessFile = new RandomAccessFile(new File("input.txt"), "rw");
System.out.println("读取之前的偏移量：" + randomAccessFile.getFilePointer() +
",当前读取到的字符" + (char) randomAccessFile.read() +
"，读取之后的偏移量：" + randomAccessFile.getFilePointer());
randomAccessFile.seek(4);
// 从偏移量 4 的位置开始往后写入字节数据
randomAccessFile.write(new byte[]{
'H', 'I', 'J', 'K','L','M'});
System.out.println("读取之前的偏移量：" + randomAccessFile.getFilePointer() +
",当前读取到的字符" + (char) randomAccessFile.read() +
"，读取之后的偏移量：" + randomAccessFile.getFilePointer());
}
The write() method is overwriting. For the input.txt file, the initial value is ABCDE, and the value after execution is ABCDHIJKLM. The execution results are as follows:
读取之前的偏移量：0,当前读取到的字符A，读取之后的偏移量：1
读取之前的偏移量：10,当前读取到的字符，读取之后的偏移量：10
5. Realize fragment upload + breakpoint resume
(In the case of concurrency, define the flag bit to confirm the completion of each fragment upload and notify the merge.) 2. The core code for implementing RandomAccessFile on the backend to realize the merge is as follows:
RandomAccessFile raf = new RandomAccessFile(file,"rw");
raf.seek(keyIndex);//seek（int n）从n处处理
byte[] bytes = fileValue.getBytes();//提区文件的字节流
raf.write(bytes);//上面的seek方法已经把指针放在这里从这直接写入