Some time ago, a friend asked me a problem that their company encountered. They said that the back-end did not implement the paging function for some reason, so 20,000 pieces of data were returned at one time, and the front-end used the select component to display it in the user interface. I listened After that, I immediately understood his confusion. If you use hard-coded methods to directly render 20,000 pieces of data into the select, it will definitely be stuck. Later, he said that it needs to support search, which is also realized by the front-end, and I immediately became interested. The plan that came to mind was as follows:
Lazy loading + paging (front-end maintenance of lazy loaded data distribution and paging)
Use virtual scrolling technology (currently React's antd4.0 already supports virtual scrolling long select lists)
Lazy loading and paging are generally used to optimize long lists, similar to the paging function of tables.The specific idea is that the user only loads the data that can be seen each time, and then loads the next page of data when scrolling to the bottom.
Virtual scrolling technology can also be used to optimize long lists.The core idea is to only render the number of lists in the visible area at a time.When scrolling, elements are dynamically added and the entire scrolling content is supported through the top padding.The realization of the idea is also very simple.
Through the above analysis, the problem of my friend can be solved, but the most aspiring front-end engineer, the author carefully sorted it out, and abstracted a practical problem based on the first solution:
How to render big data list and support search function?
The author will explore the value of this problem by simulating the implementation schemes of front-end engineers of different ranks. I hope to inspire everyone and learn to really think deeply.
text
The author will analyze the above issues through the technical perspective of programmers with different experience, and then we will start our performance.
Before starting the code, let's make basic preparations. The author uses nodejs to build a data server to provide basic data requests. The core code is as follows:
app.use(async (ctx, next) => {
if(ctx.url === '/api/getMock') {
let list = []
// 生成指定个数的随机字符串
function genrateRandomWords(n) {
let words = 'abcdefghijklmnopqrstuvwxyz你是好的嗯气短前端后端设计产品网但考虑到付款啦分手快乐的分类开发商的李开复封疆大吏师德师风吉林省附近',
len = words.length,
ret = ''
for(let i=0; i< n; i++) {
ret += words[Math.floor(Math.random() * len)]
}
return ret
}
// 生成10万条数据的list
for(let i = 0; i< 100000; i++) {
list.push({
name: `xu_0${i}`,
title: genrateRandomWords(12),
text: `我是第${i}项目, 赶快????吧~~`,
tid: `xx_${i}`
})
}
ctx.body = {
state: 200,
data: list
}
}
await next()
})
复制代码
The above author is a basic mock data server implemented by koa, so that we can simulate the real back-end environment to start our front-end development (of course, you can also manually generate 100,000 data directly in the front-end). Among them, the genrateRandomWords method is used Generate a specified number of strings, which is widely used in mock data technology, and interested friends can learn about it. The following front-end code author uniformly uses react to achieve (the same is true for vue).
Junior Engineer's Plan
The hard-coded scheme of requesting data directly from the backend and rendering to the page is as follows:
The code might look like this:
Request backend data:
fetch(`${SERVER_URL}/api/getMock`).then(res => res.json()).then(res => {
if(res.state) {
data = res.data
setList(data)
}
})
复制代码
Render the page
{
list.map((item, i) => {
return <div className={styles.item} key={item.tid}>
<div className={styles.tit}>{item.title} <span className={styles.label}>{item.name}</span></div>
<div>{item.text}</div>
</div>
})
}
复制代码
Search data
const handleSearch = (v) => {
let searchData = data.filter((item, i) => {
return item.title.indexOf(v) > -1
})
setList(searchData)
}
复制代码
In this way, the basic requirements can be achieved, but there is an obvious disadvantage, that is, the data is rendered to the page at one time, and the large amount of data will cause the page performance to be greatly reduced, causing the page to freeze.
Intermediate engineer's plan
As a front-end development engineer with certain experience, you must have an understanding of page performance, so you will be familiar with anti-shake functions and throttling functions, and have used solutions such as lazy loading and paging. Next, let's take a look at intermediate engineers The scheme:
Through the optimization of this process, the code is basically available, and the specific implementation scheme is introduced below:
Lazy loading + paging solutionThe implementation of lazy loading is mainly by monitoring the scrolling of the window. When a placeholder element is visible, the next data is loaded. The principle is as follows:
Here we listen to the scroll event of the window and use getBoundingClientRect on the poll element to obtain the distance of the poll element relative to the visual window, so as to implement a lazy loading solution by ourselves.
In the process of scrolling, we also need to pay attention to one problem. When the user scrolls back, there is actually no need to do any processing, so we need to add a one-way lock. The specific code is as follows:
function scrollAndLoading() {
if(window.scrollY > prevY) { // 判断用户是否向下滚动
prevY = window.scrollY
if(poll.current.getBoundingClientRect().top <= window.innerHeight) {
// 请求下一页数据
}
}
}
useEffect(() => {
// something code
const getData = debounce(scrollAndLoading, 300)
window.addEventListener('scroll', getData, false)
return () => {
window.removeEventListener('scroll', getData, false)
}
}, [])
复制代码
Among them, prevY stores the last scroll distance of the window, and only updates its value when scrolling down and the scroll height is greater than the last time.
As for the logic of paging, native javascript is also very simple to implement paging. We define several dimensions:
Current page number of curPage
pageSize The number of pages displayed
data The amount of data passed in
With these conditions, our basic paging function can be completed. The core code of front-end paging is as follows:
let data = [];
let curPage = 1;
let pageSize = 16;
let prevY = 0;
// other code...
function scrollAndLoading() {
if(window.scrollY > prevY) { // 判断用户是否向下滚动
prevY = window.scrollY
if(poll.current.getBoundingClientRect().top <= window.innerHeight) {
curPage++
setList(searchData.slice(0, pageSize * curPage))
}
}
}
复制代码
Anti-shake function implementation Because the anti-shake function is relatively simple, here is a simple anti-shake function code:
function debounce(fn, time) {
return function(args) {
let that = this
clearTimeout(fn.tid)
fn.tid = setTimeout(() => {
fn.call(that, args)
}, time);
}
}
复制代码
The search function code is as follows:
const handleSearch = (v) => {
curPage = 1;
prevY = 0;
searchData = data.filter((item, i) => {
// 采用正则来做匹配, 后期支持前端模糊搜索
let reg = new RegExp(v, 'gi')
return reg.test(item.title)
})
setList(searchData.slice(0, pageSize * curPage))
}
复制代码
It needs to be implemented in combination with paging, so in order not to affect the source data, we use temporary data searchData to store it. The effect is as follows:
After searching:
Both before and after the search, lazy loading is used, so there is no need to worry about the performance bottleneck caused by the large amount of data~
Senior engineer's plan
As a programmer with a long battlefield, we should consider more elegant implementation methods, such as componentization, algorithm optimization, multi-threading, etc., such as the big data rendering in our problem, we can also use virtual long lists. It is more elegant and concise to solve our needs. As for the implementation of the virtual long list, I have already pointed it at the beginning, and I will not introduce it in detail here. For a larger amount of data, such as 1 million (although it will not be encountered in actual development) Brain scene), what should we do?
At the first point, we can use the js buffer to process 1 million pieces of data in fragments. The idea code is as follows:
function multistep(steps,args,callback){
var tasks = steps.concat();
setTimeout(function(){
var task = tasks.shift();
task.apply(null, args || []); //调用Apply参数必须是数组
if(tasks.length > 0){
setTimeout(arguments.callee, 25);
}else{
callback();
}
},25);
}
复制代码
In this way, we can compare the js process blocking problem caused by a large number of calculations. For more performance optimization solutions, please refer to the author's previous article:
We can also use web workers to move in the logic that requires a lot of calculations on the front end to ensure the fast response of the js main process, let the web worker thread calculate in the background, and notify the main process through the communication mechanism of the web worker after the calculation is completed. For example, fuzzy search, we can further optimize the search algorithm, such as dichotomy, etc., so these are issues that senior engineers should consider. But we must distinguish the scene and find a more cost-effective solution.
At last
If you want to learn more front-end skills, actual combat and learning routes, welcome to join our technical group on the public account "Interesting Front-end" to learn and discuss together, and explore the boundaries of the front-end together.