This article comes from the blog Vander, CSDN blog, if you need to reprint, please indicate the source, respect the original thank you
Blog address: http://blog.csdn.net/l540675759/article/details/62111148
导读:
1.http关于下载的相关字段.
2.Android多线程下载的原理.
3.Android上面进行Http请求下载的需要解决的一些问题.
- 1
- 2
- 3
- 4
1. http some fields about download
(1)Transfer-Encoding
(2)content-length
(3)range
- 1
- 2
- 3
Transfer-Encoding
Transfer-Encoding is an HTTP header field, which literally means transfer encoding .
Transfer-Encoding is used to change the message format.In HTTP1.1, Transfer-Encoding only defines one transfer encoding: chunked.
Transfer-Encoding:chunked
- 1
And Transfer-Encoding: chunked means that the length of the output content cannot be determined. Normal static pages, pictures and the like are basically not used, but some interfaces like dynamic will use this field, because the size cannot be determined. It can be transmitted in chunks.
However, in Android's multi-threaded download, when the Transfer-Endcoding keyword appears, the length of the entity is often unable to determine the length of the entity, so the content-length keyword is not available in the response header , so multi-threaded download cannot be performed.
content-length
Indicates the size of the entity body, in bytes, but when Transfer-Encoding:chunck appears, this keyword generally does not appear.
content-length:2030
- 1
Range
For a range request that only needs to obtain part of the resource, including the header field Range can inform the server of the specified range of resources.The above example indicates that the resource from the 5001th byte to the 10000th byte is requested .
Note that the following situation is requesting data from 5001 bytes to the maximum length of the file.
Range:5001-
- 1
The server that receives the request with the Range header field will return the corresponding status code of 206 Partial Content after processing the request.
When the scope request cannot be processed, a response with a status code of 200 OK and all resources will be returned.
Range:bytes = 5001-10000
- 1
2. The principle of Android multi-threaded download
1.通过创建线程池来维护多线程,注意当前线程数量超过corePoolSize数量时,会形成阻塞.
2.根据content-length来确定各个子线程的分段请求区域.(Range),
正常如果在做多线程下载的时候,服务器会把数据大小和url都从请求中返回直接一次请求就可以.
3.使用RandomAccessFile在分段区域位置读写数据,实现文件的拼接.
- 1
- 2
- 3
- 4
- 5
- 6
Multi-threaded key implementation code
(1) Code to create thread pool
private static final ThreadPoolExecutor sThreadPool =
new ThreadPoolExecutor(
MAX_THREAD,
MAX_THREAD,
60,
TimeUnit.MILLISECONDS,
new SynchronousQueue<Runnable>(), new ThreadFactory() {
private AtomicInteger mInteger = new AtomicInteger(1);
@Override
public Thread newThread(Runnable r) {
Thread thread = new Thread(r, "download thread # " + mInteger.getAndIncrement());
return thread;
}
});
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
(2) When you get the data that needs to be downloaded by multiple threads, determine the segment request area of each sub-thread according to the content-length
private void processDownload(String url, long length, DownloadCallback callback) {
//100 2 50 0-49 50-99
long threadDownloadSize = length / CORE_THREAD;
for (int i = 0; i < CORE_THREAD; i++) {
long startSize = i * threadDownloadSize;
long endSize = 0;
if (i == CORE_THREAD - 1) {
endSize = length - 1;
} else {
endSize = (i + 1) * threadDownloadSize - 1;
}
Log.d("线程详细数据", "第" + i + "个线程 " + "startSize : " + startSize + "endSize : " + endSize);
sThreadPool.execute(new DownloadRunnable(startSize, endSize, url, callback));
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
(3) Use RandomAccessFile for IO operations, because RandomAccessFile supports operations to read and write data
at any location. Note that each time you use RandomAccessFile for IO operations, you need to specify the location of the operation file
//mStart---->从什么位置开始写入文件
randomAccessFile.seek(mStart);
- 1
- 2
Core code
File file = FileStorageManager.getInstance().getFileByName(mUrl);
try {
//rwd读写模式
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rwd");
//对文件制定偏移位置.
randomAccessFile.seek(mStart);
byte[] buffer = new byte[1024 * 500];
int len;
InputStream inStream = response.body().byteStream();
while ((len = inStream.read(buffer, 0, buffer.length)) != -1) {
randomAccessFile.write(buffer, 0, len);
}
mCallBack.success(file);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
3.Some problems that need to be solved for Http request download on Android.
- File storage location
- Whether the file is damaged
- Is the file size sufficient
- Update of the progress bar
- Data preservation
File storage location:
In Android, files are generally stored in the SD card, but some mobile phones do not support it. All will make a judgment if it does not support it and store it in the cache directory:
Add read and write permissions before saving:
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
- 1
- 2
- 3
Core code
/**
* 生成文件,并储存
*
* @param url
* @return
*/
public File getFileByName(String url) {
File parent;
//判断SD卡储存是否可用
if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
parent = mContext.getExternalCacheDir();
} else {
//不可用就存入缓存目录
parent = mContext.getCacheDir();
}
String fileName = MD5Utils.generateCode(url);
File file = new File(parent, fileName);
if (!file.exists()) {
try {
file.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}
return file;
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
Whether the file is damaged
Generally speaking, the method to check whether a file is damaged is to compare whether the MD5 value of all the contents of the file is the same to determine whether the file is damaged or changed. At this time, the server will have a good file's MD5 for comparison after downloading.
Is the storage space sufficient?
It is necessary to check whether the storage space is sufficient before storing the file. Code reference is provided here:
Get SD card free space
void readSDCard() {
String state = Environment.getExternalStorageState();
if(Environment.MEDIA_MOUNTED.equals(state)) {
File sdcardDir = Environment.getExternalStorageDirectory();
StatFs sf = new StatFs(sdcardDir.getPath());
long blockSize = sf.getBlockSize();
long blockCount = sf.getBlockCount();
long availCount = sf.getAvailableBlocks();
Log.d("", "block大小:"+ blockSize+",block数目:"+ blockCount+",总大小:"+blockSize*blockCount/1024+"KB");
Log.d("", "可用的block数目：:"+ availCount+",剩余空间:"+ availCount*blockSize/1024+"KB");
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
Get storage space inside the system
void readSystem() {
File root = Environment.getRootDirectory();
StatFs sf = new StatFs(root.getPath());
long blockSize = sf.getBlockSize();
long blockCount = sf.getBlockCount();
long availCount = sf.getAvailableBlocks();
Log.d("", "block大小:"+ blockSize+",block数目:"+ blockCount+",总大小:"+blockSize*blockCount/1024+"KB");
Log.d("", "可用的block数目：:"+ availCount+",可用大小:"+ availCount*blockSize/1024+"KB");
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
What StatFs gets are all in block
这里我解释一下block的概念：
1.硬件上的"block size", 应该是"sector size"，Linux的扇区大小是512byte.
2.有文件系统的分区的"block size", 是"block size"，大小不一，可以用工具查看.
3.没有文件系统的分区的"block size"，也叫“block size”，大小指的是1024 byte.
4.Kernel buffer cache 的"block size", 就是"block size"，大部分PC是1024.
5.磁盘分区的"cylinder size"，用fdisk可以查看。
我们这里的"block size"是第二种情况，一般SD卡都是fat32的文件系统，"block size"是4096byte.
- 1
- 2
- 3
- 4
- 5
- 6
- 7
Update download progress
This block mainly obtains the sum of the current downloads of each thread and compares it with the total length to get the percentage of the download progress. There are many ways to update the progress, such as Handler and broadcast.
Data preservation
Use RandomAccessFile to realize segmented data and insert it at any time. The splicing of files is completed.
Reference article:
HTTP 协议中的 Transfer-Encoding
https://imququ.com/post/transfer-encoding-header-in-http.html
Android http协议实现文件下载
http://www.cnblogs.com/duanxz/p/3514781.html
Java IO的RandomAccessFile的使用
http://blog.csdn.net/czplplp_900725/article/details/37809579
Java RandomAccessFile用法
http://blog.csdn.net/akon_vm/article/details/7429245
Android：StatFs类 获取系统/sdcard存储空间信息
http://www.cnblogs.com/zhangs1986/p/3251171.html
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
<link href="https://csdnimg.cn/release/phoenix/mdeditor/markdown_views-8cccb36679.css" rel="stylesheet">
</div>
This article comes from the blog Vander, CSDN blog, if you need to reprint, please indicate the source, respect the original thank you