Research and Improvement of Differential Upgrading Algorithm
The file difference algorithm finds the difference by comparing the fragments of different files, and outputs the difference description as a difference file.
The difference algorithm is the key technology of the difference upgrade, therefore, the performance of the difference algorithm determines the performance of the whole upgrade system.
Research on Difference Algorithm
Differential update is also called incremental encoding, and the differential byte upgrade file only includes the description of the byte difference between firmware versions, so the formed file is much smaller than the firmware version file.
Software that can be upgraded through wireless firmware includes system, driver, etc.
The byte difference algorithm reduces the cost of data transmission by reducing the size of the upgrade package.
Therefore, how to compress the generated difference files to the minimum becomes the core aspect of competition among various software technology providers.
A large part of the file difference algorithm is based on the LCS (Longest Common Subsequence) problem, which aims to find out the fragments of lovesickness for difference description and byte replacement between binary files.
Among the differential algorithms, the Bsdiff and Xdetla algorithms are currently the most widely used. In UNIX systems and some modern software such as Chrome, the incremental description of files based on the Bsdiff algorithm is used, and it is widely used in mobile Internet cloud storage. Field, mobile phone operating system update.
Bsdiff
Bsdiff is a differential algorithm based on the LCS problem, which uses an approximate matching algorithm to compare files.
Bsdiff adopts the suffix dictionary sorting algorithm, so that the whole matching process can be performed according to binary search, so as to achieve O(log N) search time. The specific process is as follows:
- Enter the old version file OLD
- Get file suffix array
- Suffix array lexicographically sorted array I
- Recursive binary search for similar fragments of new version files.
Suffix sorting uses the qsufsort algorithm, for example, the old version file is "Bsdifff", the qsufsort algorithm will generate a complete suffix array of "f", "ff", "iff", "diff", "sdiff", "Bsdiff".
Then use quicksort to sort the array lexicographically, namely "Bsdiff", "fidd", "f", "ff", "iff", "sdiff". The array I generated by this algorithm is the starting position of the lexicographical array in the old file. In this example, the array is I={0,2,5,4,3,1}.
And Bsdiff uses the array generated by qsufsort to match similar bytes between files.
When the old and new versions match, the new version file pointer is Scan, and the old version file pointer is Pos. Every time Scan moves forward, it calls recursive binary search to find the position with the longest matching bytes of the old and new version files and returns the matching length len . The Completely matched area is a completely matched area, which means that the old and new version files in this area are exactly the same.
The key code of the recursive binary search algorithm search is as follows, where newsize is the byte length of the new version file, oldsize is the byte length of the old version file, scan is the file pointer in the new version file, pos is the file pointer in the old version file, x is the position of the currently searched data in the array I, st represents the start position of the binary interval, and en represents the end position of the binary interval.
In the Bsdiff algorithm, the key step is to find the approximate matching area. This approximate matching area "sticks" to the left and right of the complete matching area. The existence of the approximate matching area is to reduce the size of the Patch difference file.