Talking about the nodejs middle layer

Preface

nodejsThe emergence of the front-end industry has brought unlimited possibilities to the front-end industry, and many students who were only responsible for client-side development have slowly begun to contact and use server-side technology.

Although nodejsbrought a lot of benefits, but it also has its own limitations. And those old traditional programming languages compared as JAVA, PHP. nodejsAnd can not be a substitute for them, but also in the future can be estimated, it is difficult to shake The status of those old programming languages.

At present, nodejsthere are mainly the following application scenarios.

• Front-end engineering, for example rollup, webpackexploration in the direction of engineering
• nodejsmiddle layer
• Client integration nodejs, such aselectron
• Some less complex applications on the market choose to nodejsbe the back-end programming language

This article mainly talks about nodejssome practices as an intermediate layer, check the figure below.

In the traditional development model, the browser Serverdirectly communicates with the layer, and the addition of the middle layer means that an additional layer is added between the browser and Serverthe layer.

It turns out that the client Serversends the request directly , and the Serverlayer receives the request and returns the result to the browser after calculation processing.

Now the browser sends the request to node层, node层and then Server层initiates the request after a round of processing . Server层After the processing is completed, the response result is returned to node层, and node层finally the data is returned to the browser.

Because node层of the emergence, Server层you can only focus on the business itself, instead of paying attention to the special requirements of the front-end field.

node层serverData can be obtained from the layer, and then converted into UIa data format that meets the front-end requirements through the calculation and integration of the data. In addition, if the entire application adopts a microservice architecture, there Server层will be many servers that manage individual business modules, which node层are well adapted The architecture of microservices, it can initiate requests to multiple servers to obtain data from different modules and then integrate and transform it and send it to the front end.

The following focuses on nodejspart of the practice as an intermediate layer.

Proxy forwarding

Proxy forwarding has many extensive applications in practice. The browser first sends the request to node服务器, and after the request is received, it node服务器can do some processing on the request, such as changing the original path, changing the information in the request header, and then changing the modified The request is sent to the remote real server.

The remote server calculates the response result and then returns it node服务器, and node服务器it can still selectively process the response and then return it to the browser.

Proxy forwarding can solve the cross-domain problems often encountered in the daily development of the front-end, and it also shields the details of the remote real server, so that the browser only node服务器communicates with. The following is a simple practice.

const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');

const app = express();//创建应用
 
app.use("/api",createProxyMiddleware( //设置代理转发
  { 
     target: 'http://www.xxx.com', //举例随便写的地址
     changeOrigin: true,
     pathRewrite: function (path) { 
       return path.replace('/api', '/server/api');
     }
  })
);

app.use("*",(req,res)=>{  //不是以'/api'开头的路由全部返回"hello world"
  res.send("hello world");
})

app.listen(3000);

http-proxy-middlewareIt is a third-party dependency package, which can be very convenient to set up proxy forwarding and needs to be npminstalled.

If the current access path starts /apiat the beginning, then the request will be http-proxy-middlewareintercepted. Observe http-proxy-middlewarethe parameters configured inside.

• targetRepresents the address of the remote real server.
• changeOriginSet to true, which means forwarding the request to the targetaddress.
• pathRewriteIt is to do some processing on the request path and /apiconvert it into /server/api.

The meaning of the above case is very obvious, if the current browser visits http://localhost:3000/api/list. Because this path /apistarts with it, it will be intercepted, which triggers the pathRewritefunction to modify the access path. The final access path becomes http://www.xxx.com/server/api/list, and then a request to this path will be initiated, and after the response is obtained Then return to the browser.

Interface aggregation

The interface forwarding described above is rarely used separately in practice. If it is only for forwarding data, it is better to use the nginxconfiguration directly , and the forwarding will be done.

If both interface aggregation and interface forwarding are required, then it is the preferred way to solve it from the code level.

What does interface aggregation mean? Suppose that a company now has two sales systems, one is online e-commerce platform sales, and the other is offline physical stores. They are operated by different teams and maintain different data systems.

If the current request is only to query the information of a certain product on the e-commerce platform, you only need to forward the interface to the e-commerce platform system. Similarly, if you only query the sales performance of an offline physical store on a certain day, you can directly forward the request Query the offline data system, and then return the response data. The plug-in described above http-proxy-middlewaresupports the configuration of multiple proxy paths, and the detailed documentation can be queried.

Now there is such a demand, the goal is to query the comparison of the online and offline sales data of a certain product this week. Then at this time, you need node层to send a request to two remote servers to obtain online sales data and offline sales data respectively. After the two parts of data are aggregated and processed, they are returned to the front end. The simple practice is as follows.

const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');

const app = express();//创建应用

//伪代码
app.get("/getSaleInfo",async (req,res)=>{ 
   const online_data =  await getOnline(); //获取线上数据
   const offline_data = await getOffline(); //获取线下数据
   res.send(dataHanlder(online_data,offline_data)); //对数据处理后返回给前端
})

proxyHanlder(app);//伪代码,将代理转发的逻辑封装起来

app.use("*",(req,res)=>{
  res.send("hello world");
})

app.listen(3000);

/getSaleInfoIt represents a custom route that aggregates two pieces of data. If there are more requirements for aggregating data, this piece of logic must be separately encapsulated in the routing module for management, and it must be written in front of the proxy forwarding.

This ensures that the interface that needs to be forwarded is handed over to the logical processing of forwarding, and the interface that needs to personalize the data is written separately for routing operation data.

Data cache

Caching plays an insignificant role in improving system performance and reducing database pressure. Generally, commonly used caching software is redisthat it can be understood as a database with data stored in memory. Since the data is stored in memory, the read and write speed is very fast and can Respond extremely quickly to user requests.

Cached data in node层deployment redismanagement can improve overall application performance. However, it is not recommended to store all data in redisit, and only those data that do not change frequently should be cached.

For example, the information data of a product, the browser initiates a request for a product, and wants to view the details of the product. The request reaches the nodelayer for the first time , and it redisis empty at this time. Then nodestart the request serverlayer to get the response result, and then return the response result Before giving it to the browser, the access path of the request is used as the keyvalue, and the response result is valuestored redisin it. In this way, when the same request is sent later, first check rediswhether the data of the request is cached, and if it is cached, return the data directly , If there is no cache, go to the request serverlayer and go through the above process again.

redisYou can also set the expiration time and clearing of the cached data, which can be based on specific business operations. The simple practice is as follows.

const express = require('express');

const app = express();//创建应用

//伪代码
app.use("*",(req,res,next)=>{
   const path = req.originalUrl; //获取访问路径
   if(redisClient.getItem(path)){ //查看redis中有没有缓存该条接口的数据
   	 res.send(redisClient.getItem(path)); // 返回缓存数据
   }else{
     next(); //不执行任何操作,直接放行		
   }
})


aggregate(app); //伪代码,将接口聚合的逻辑封装起来

proxyHanlder(app);//伪代码,将代理转发的逻辑封装起来

app.use("*",(req,res)=>{
  res.send("hello world");
})

app.listen(3000);

Interface current limit

nodeThe middle layer can restrict the uncontrolled access of the front-end. For example, some malicious scripts loop through the interface. Dozens of visits per second increase the load of the server.

redisCan help us achieve this functionality. The first user to access, parse out this request of ipaddresses, ipas the keyvalue valueis set to 0save to redisthe.

The user visits the second time, takes out the corresponding one ipfound , and then increments it . If the same person repeats a large number of visits, it will increase to a very large number in a short period of time. We can obtain this number each time to determine whether the terminal exceeds the set value. If the expected standard is exceeded, the request will be rejected. The simple practice is as follows.redisvalue1value

const express = require('express');

const app = express();//创建应用

//伪代码
app.use("*",(req,res,next)=>{

  const ip = req.ip;

  let num = 0;

  if(redisClient.getItem(ip)){ //是否缓存了当前的ip字段
    num = redisClient.incr(ip); //每访问一下,计数加1
  }else{
    redisClient.setItem(ip,0);
    redisClient.setExpireTime(5); //设置过期时间为5秒,5秒后再获取该ip为空
  }

  if(num > 20){ 
    res.send("非法访问");
  }else{
    next();//放行
  }

})

cacheData(app)//伪代码.缓存接口数据

aggregate(app); //伪代码,将接口聚合的逻辑封装起来

proxyHanlder(app);//伪代码,将代理转发的逻辑封装起来

app.use("*",(req,res)=>{
  res.send("hello world");
})

app.listen(3000);

Set up a layer of current-limiting middleware in front of the application, and judge whether the end has cached each time the access comes. There is definitely no cache for the first access, so set the current ipcorresponding value to 0and add the expiration time to 5seconds. The next time is the same. Of users will valueautomatically increase when they visit again 1.

The final effect is that access is denied if the 5interface is called more than 20once in a second.

Log operation

The system has no logs, which is equivalent to people without eyes. The logs can help us find and analyze the errors in the online system. In addition, the log data can also be used for statistical calculations to draw certain conclusions and trends.

nodeThe layer can take on the function of managing logs, taking the interface access log as an example. Create a new log folder in the system, and each time there is a request for access, first parse the requested path, current access time, and carry parameters and terminal data information Then create a txtfile in the log folder to store the log situation of the day, combine the above data and the response result of the request into a record and insert it into the txtfile. The next time you visit, continue with the above process to txtadd the access log to the file. Like the proxy forwarding described above , The plug-in http-proxy-middlewaresupports configuring how to return the response result, then the request and response can be obtained at the same time in the corresponding event function hook, and these two pieces of data can be stored in the log.

There are also many configuration strategies that can be formulated here. You can choose one log text per day, or if the traffic is huge, you can also choose one log text per hour, depending on the actual situation.

In addition, as time goes on, the file content of the log folder will become more and more. This requires writing linuxoperating system timing tasks to migrate and back up these log data.

The simple practice of log operation is as follows.

//伪代码
app.use("/getList",async (req,res)=>{
  const list = await getProductList(); //获取商品数据
  const { 访问时间,访问路径,参数 } = req;
  logger.log('info',`${访问时间}-${访问路径和参数}:${list}`);//将数据存储到日志文件中 
  res.send(list);//将结果返回给客户端
})

end

The middle layer can also do many other things, such as monitoring, authentication, and server-side rendering (ssr). This part is because the content is relatively large and can be divided into chapters, and there are also a large number of articles on how to practice on the Internet, which can be searched and learned.

In fact, all the functions mentioned above can be done by other programming languages, which has also become a concern for many people to question whether an additional layer of architecture needs to be added.

Adding a nodejsmiddle layer is definitely good news for front-end students. Because it allows the front-end to take on more work tasks and makes the front-end business more important. In addition, the back-end only needs to pay attention to its own business, and the front-end continues to do what it is good at On the whole, it can improve development efficiency.

However, from a macro perspective, adding an additional layer to the architecture will inevitably cause a loss in the performance of the entire application.In addition, it will increase the operation and maintenance costs at the deployment and test levels.

At present, front-end and back-end separation has become the mainstream development model. Many types of applications require seo support and first-screen loading speed, so server-side rendering is indispensable. Front-end projects are currently mostly used reactor vueframework development, if you use nodejsserver-side rendering Task, then you can ensure that a set of code can do both client-side rendering and server-side rendering, and these tasks can be done by front-end programmers independently. Server-side rendering technology is very important, and a section will be explained later.

In summary, the nodejsmost valuable function of the middle layer is server-side rendering and interface data aggregation. If the number of enterprise applications is small, the business is simple and has not been scaled, it is not recommended to add the middle layer, which will make simple things complicated.