How to display the early stage of the big data era

With the vigorous development of web technology, front-end display and interaction are becoming more and more complex, and a large amount of data is generated during user access and operation. As a result, front-end data analysis has also become particularly important. Of course, for webmasters, you can use various existing service platforms such as Baidu Statistics. However, if the existing statistical platforms cannot meet your needs, you want to develop your own customized data statistics platform, or you are A pure geek who wants to understand the hidden technology behind it, or if you are interested in front-end statistics, this article can satisfy your curiosity. The following is a step-by-step description of what data the front-end has, how to collect the front-end data, and how to display the results of data statistics.

What?

There are actually a lot of front-end data, ranging from PV, UV, and advertisement clicks that are generally concerned by the public, to the client's network environment and login status, to browser and operating system information, and finally to page performance and JS exceptions. collected at the front end. There is a lot of data and it is very complicated. If it is not well classified, it will definitely lead to confusion in statistics, and it is not conducive to the organization of statistical codes. The following is a classification of several common data requirements:

1. Access

Access data is based on each user's Open the target page on the browser for statistics. It is based on PV as the granularity of statistics. One PV only counts the access data once. Access data can be regarded as the most basic and most extensive statistics, and many indicators can be counted. Some of the more common indicators are listed below:
PV/UV: the most basic PV (number of page visits) , UV (number of unique visitors)
• Page source: the referrer of the page, which can locate the entry of the page
• Operating system: understand the user's OS status, help analyze the characteristics of the user group, especially the mobile terminal, the distribution of iOS and Android is even more It makes sense
. Browser: The proportion of various browsers can be counted, and it can provide reference value for investigations such as whether it is compatible with IE6 and the application of new technologies (HTML5, CSS3, etc.).
Resolution: Provide reference for page design, especially is responsive design
•Login rate: Baidu has also begun to attach importance to logins, and logged-in users have higher analytical value, and it is very important to guide users to log in.
• Geographical distribution: The distribution of access users in geographic locations allows operations, activities, etc. to be carried out for different regions.
Network Type: wifi/3G/2G, to decide whether the product needs to adapt to different network environments.
Access period: to grasp the distribution of user access time, guide peaks and valleys, and save bandwidth
. Duration of stay: to determine whether the page content is attractive, It is more meaningful for pages that need to be read for a long time.
Reach depth: similar to the stay time, such as Baidu Baike, the page arrival depth when users browse directly reflects the quality of the entry.

2. Performance The DOM structure of the

page is more and more complex, but it needs to The pursuit of user experience puts forward higher requirements on the performance of the page. The performance monitoring data is mainly used to measure the smoothness of the page, and there are also some main indicators:
• White screen time: The time occupied by the user from opening the page until something is displayed on the page is the white screen time
. Screen time: the time it takes for all the content on the first screen of the user's browser to be displayed
• User actionable time: The user can perform normal clicks, input, etc.
• Total page download time: All resources on the page are loaded and presented. Time spent, that is, the time of page onload.
Custom time point: For developers, it is completely possible to customize some time points, such as: the time when a component init is completed, the time when an important module is loaded, etc.

Here It only explains the meaning of these indicators, and the specific judgment and statistical methods will be introduced in detail in subsequent articles.

3. Click

Among all the operations of the user, click should be the most important behavior, including: the click of the mouse on the pc side, and the touch of the finger on the mobile side. Each click of a user is an appeal. There is actually a lot of information that can be mined from the click data. The following only lists the indicators we are currently concerned about:
•Total page clicks
•Per capita clicks: For navigational pages, This indicator is very important
• Outgoing url: Similarly, for navigating web pages, you can directly understand the whereabouts of web page traffic
• Click time: All the user's click behaviors, the distribution in time, reflects the user's click operation habits . The
first time Click time: the same as above, but only the first click of the user is counted. If the time is too long, does it mean that the page is very stuck and the user cannot click for a long time?
•Click heat map: According to the location of the user's click, we can draw the click heat map of the entire page, and we can intuitively understand the hot spots of the page.

4. Exceptions The exceptions

here refer to JS exceptions, and the user's browser reports JS This will greatly reduce the user experience. For today's browser models and versions flying all over the sky, it is inevitable that NB programmers will miss the gun. Of course, QA can cover most of the bugs, but it will also There are some bugs appearing online. There are only two ways to capture exceptions in JS: window.onerror and try/catch. How we do this will be described in detail in subsequent articles. Here we only list what information generally needs to be collected when an exception is captured (mainly Used to debug exceptions):
• Exception message: This is the most important basis for identifying an exception, such as: 'e.src' is empty or not an object
• JS file name
• The line where the exception is located
• The browser where the exception occurred
• The stack Information: When necessary, the stack information of the function call is required, but note that the stack information may be relatively large and needs to be intercepted

5. Others

In addition to the 4 types of basic data statistics requirements mentioned above, of course, we can also define some other statistical requirements according to the actual situation, such as the user's browser's support for canvas, and another special one - user rounds The number of times the map is turned and the data statistics needs are met by the front-end, and the results of each statistic reflect the value of the front-end data.

How to collect?

On the front end, data can be obtained by injecting JS scripts, using some JS APIs (such as !!window.localStorage to check whether the browser supports localStorage) or listening to some events (such as click, window.onerror, onload, etc.). After capturing these data, the data needs to be sent back to the server, usually by accessing a fixed url and using the data as the query string of the url, such as: http://www.baidu.com/u.gif?data1 =hello&data2=hi.

In the process of practice, we extracted a set of framework alog for front-end statistics, which is convenient for developers to write their own statistics scripts. For the specific usage and API, see github. Let's use alog to briefly explain how to collect front-end data:

For example: you need to count the PV of the page, by the way, add the page source (refer)


// Load alog, alog is a
void function that supports asynchronous (e,t,n, a,o,i,m){
e.alogObjectName=o,e[o]=e[o]||function(){(e[o].q=e[o].q||[]).push(arguments)},e[o ].l=e[o].l||+new Date,i=t.createElement(n),i.asyn=1,i.src=a,m=t.getElementsByTagName(n)[0],m .parentNode.insertBefore(i,m)
}(window,document,"script","http://uxrp.github.io/alog/dist/alog.min.js","alog");

// define a Statistics module pv
alog('define', 'pv', function(){
   var pvTracker = alog.tracker('pv');
   pvTracker.set('ref', document.referrer); // set ref parameter
   return pvTracker ;
});

// Create an instance of pv statistics module
alog('pv.create', {
    postUrl: 'http://localhost/u.gif' // Specify the url address for uploading data
});

// Upload data
alog('pv.send', "pageview"); // Indicate that pageview


deploys the above code on the page, and the browser will send the following http request:
http://localhost/u.gif?t=pageview&ref=yourRefer

Another example: JS exception collection requires event monitoring
// load alog
void function(e,t,n,a,o,i,m){
e .alogObjectName=o,e[o]=e[o]||function(){(e[o].q=e[o].q||[]).push(arguments)},e[o] .l=e[o].l||+new Date,i=t.createElement(n),i.asyn=1,i.src=a,m=t.getElementsByTagName(n)[0],m. parentNode.insertBefore(i,m)
}(window,document,"script","http://uxrp.github.io/alog/dist/alog.min.js","alog");

// define a statistic Module err
alog('define', 'err', function(){
   var errTracker = alog.tracker('err');
   window.onerror = function(message, file, line) { //Monitor window.onerror
        errTracker.send ('err', {msg:message, js:file, ln:line});
    };
   return errTracker;
});

// Create an instance of err statistics module
alog('err.create', {
    postUrl: 'http://localhost/u.gif'
});

At this time, as long as the JS in the page is abnormal, it will send the following How to display the HTTP request
http://localhost/u.gif?t=err&msg=errMessage&js=jsFileName&ln=errLine After collecting

the data, after a series of data processing, summarizing and other operations, we need to use vivid charts to present the data,

Allow users (product decision makers, developers, etc.) to easily and quickly understand the data. We recommend using Baidu's open source javascript chart library ECharts. Here are some common data display methods:

The proportion of browsers: The login situation of
browser
users: The geographical distribution of login users:

Location Sometimes you need to see the fluctuation of multiple days, such as the fluctuation of the proportion of browsers over multiple days. Sometimes you may need to use some tables to display the situation stack : feature feature2 Summary The front-end data has a lot of analytical value. It is the real feedback of online users, which directly reflects the user experience of the product. According to the steps described in this article, you can build your own front-end data platform.