Front-end resources to achieve an idea incremental update

"Incremental update" simply the need to re-load the resource at the time of the updated version, only need to load some small diff information, and then merged into the current resource, similar to git merge results.

 

"Suppose there is a large ultra-Web project, after JS source code compression than 10MB (actual how could such a big thing =. =), After each update will require the user to re-load the JS is unacceptable, then how to solve this problem from an engineering point of view? "

Beginning immediately thought of several good solutions, such as:

1. Extraction module infrequently updated on the basis of a long-term caching;
2. If such a React Vue2.0 or support server-side rendering of the frame, then uses the server side rendering method of loading and then gradually block of JS;
3. If Hybrid development, you can consider using local resources to load, similar to the "offline package" idea (before encountered this stuff every day when Tencent internship).

Later, under the guidance of the interviewer think of an "incremental update" solution, it simply is not need to reload the resource when the updated version, only need to load some small diff information, and then merged into the current resources, git merge similar effect.

1, the UE using LocalStorage or other storage schemes, store a copy of the original code + timestamp:

{ 
 
    timeStamp: "20161026xxxxxx", 
 
    data: "aaabbbccc" 
 
}  

2, every time resource loading time stamp to send this to the server;

3, the server receives recognition from the timestamp in the client version and the latest version to do a diff, diff return information for both:

diff("aaabbbccc", "aaagggccc"); 
 
// Suppose we say so diff information:
 
// [3, "-3", "+ggg", 3]  

After 4, the client receives the diff information, local resources and time stamp to update to the latest, achieve incremental updates:

mergeDiff("aaabbbccc", [3, "-3", "+ggg", 3]); 
 
//=> "aaagggccc"  

practice

Here again the realization of the core idea of ​​this program is, simply put, it is to achieve diff and mergeDiff two functions.

Today found a good diff algorithm:

GitHub – kpdecker/jsdiff: A javascript text differencing implementation.

We just call it diffChars method to compare differences between two strings:

was oldStr = 'aaabbbccc' ;
 
was newStr = 'aaagggccc' ;
 
JsDiff.diffChars(oldStr, newStr); 
 
//=> 
 
//[ { count: 3, value: 'aaa' }, 
 
//  { count: 3, added: undefined, removed: true, value: 'bbb' }, 
 
//  { count: 3, added: true, removed: undefined, value: 'ggg' }, 
 
//  { count: 3, value: 'ccc' } ]  

Diff above information a little bit redundant, we can customize a more compact representation method to speed up transmission speed:

[3, "-3", "+ggg", 3] 

Integer representing the number of characters unchanged, "-" represents the number of characters in a string beginning is removed, the "+" at the beginning of the string representing the newly added characters. So we can write a minimizeDiffInfo function:

function minimizeDiffInfo(originalInfo){ 
 
    var result = originalInfo.map(info => { 
 
        if(info.added){ 
 
            return '+' + info.value; 
 
        } 
 
        if(info.removed){ 
 
            return '-' + info.count; 
 
        } 
 
        return info.count; 
 
    }); 
 
    return JSON.stringify(result); 
 
} 
 
  
 
was diffInfo = [
 
    { count: 3, value: 'aaa' }, 
 
    { count: 3, added: undefined, removed: true, value: 'bbb' }, 
 
    { count: 3, added: true, removed: undefined, value: 'ggg' }, 
 
    { count: 3, value: 'ccc' } 
 
]; 
 
minimizeDiffInfo(diffInfo); 
 
// => '[. 3, "-3", "+ GGG",. 3]'   
user information after termination diff be streamlined, the latest generation of resources:

mergeDiff('aaabbbccc', '[3, "-3", "+ggg", 3]'); 
 
//=> 'aaagggccc' 
 
  
 
function mergeDiff(oldString, diffInfo){ 
 
    was newstring = '' ;
 
    var diffInfo = JSON.parse (diffInfo);
 
    There are p = 0 ;
 
    for(var i = 0; i < diffInfo.length; i++){ 
 
        was info = diffInfo [i];
 
        if(typeof(info) == 'number'){ 
 
            newString += oldString.slice(p, p + info); 
 
            p + = info;
 
            continue; 
 
        } 
 
        if(typeof(info) == 'string'){ 
 
            if(info[0] === '+'){ 
 
                var addedString = info.slice(1, info.length); 
 
                newString += addedString; 
 
            } 
 
            if(info[0] === '-'){ 
 
                var removedCount = parseInt(info.slice(1, info.length)); 
 
                p += removedCount; 
 
            } 
 
        } 
 
    } 
 
    return newString; 
 
}  

actual effect

Are interested can directly run this:

GitHub – starkwang/Incremental

Create-react-app use this little tool to quickly generate an React project, just change a couple of lines of code, then compare the two versions look after the old and new build:

var JsDiff = require('diff'); 
 
var fs = require('fs'); 
 
  
 
var newFile = fs.readFileSync('a.js', 'utf-8'); 
 
var oldFile = fs.readFileSync('b.js', 'utf-8'); 
 
console.log('New File Length: ', newFile.length); 
 
console.log('Old File Length: ', oldFile.length); 
 
  
 
was diffInfo  
 
= getDiffInfo(JsDiff.diffChars(oldFile, newFile)); 
 
console.log('diffInfo Length: ', diffInfo.length); 
 
console.log(diffInfo); 
 
  
 
was Result = mergeDiff (oldFile, diffInfo);
 
console.log(result === newFile); 

Here are the results:

 

After the code can be seen 21w build multiple characters (212KB), and diff information is quite short, only 151 characters, compared to reload the new version, it reduced 1,000 times (of course, I only changed two or three here lines of code, little is natural).

Some did not relate to the issue

Just above the core ideas realized again, what the actual project there are more to consider:

1, the server for every request can not be re-evaluated once diff, so be sure to do the diff cache of information;

2, the UE implementation persistent storage, such as loved LocalStorage, Indexed DB, Web SQL, or using the interface provided by the native app;

3, fault tolerance, client and server-side consistency proof to achieve mandatory refresh.