Cross-domain: The strange "person" that back-end engineers are most familiar with

Abstract: Cross-domain is both familiar and unfamiliar to back-end engineers.

This article is shared from Huawei Cloud Community " The Cross-Domain Journey of Back-End Veteran Drivers ", author: Brother Yong java actual combat sharing.

Cross-domain is both familiar and unfamiliar to back-end engineers.

In the past two months, I participated in the incubation of an educational product as an architect, and had an unforgettable cross-domain journey .

Writing this article, I want to share my experience and thinking on the knowledge point of cross-domain, hoping to inspire everyone.

1 Meet cross-domain

The product has multiple terminals: institution terminal, bureau terminal, parent terminal, etc. Each end has an independent domain name, some are accessed on a PC, some are accessed through a WeChat official account, and some are displayed in H5 after scanning the code.

The interface domain name called by the access layer uniformly uses api.training.comthis independent domain name, and Nginx is used to configure request forwarding.

Usually, the cross-domain we refer to refers to: CORS .

CORS is a W3C standard, the full name is "Cross-origin resource sharing", it requires browsers and servers to support him at the same time, allowing browsers to send XMLHttpRequest requests to cross-origin servers, thus overcoming AJAX can only be the same origin Limitations of use.

So how to define homology? Let's first look at the address of a typical website:

The same origin means that the protocol, domain name, and port number are exactly the same .

The following table gives an http://www.training.com/dir/page.html example of comparing with the source of the URL:

When a user accesses the application ( http://admin.training.com ) through a browser, the domain name of the calling interface is not the domain name of the same origin ( http://api.training.com ), which is an obvious cross-domain scenario.

2 Detailed explanation of CORS

The cross-origin resource sharing standard adds a new set of HTTP header fields, allowing the server to declare which origin sites have permission to access which resources through the browser.

The specification requires that for those HTTP request methods that may have side effects on server data (especially HTTP requests other than GET, or POST requests with certain MIME types), the browser must first use the OPTIONS method to initiate a preflight request (preflight request ), so as to know whether the server allows the cross-domain request.

After the server confirms the permission, the actual HTTP request is initiated. In the return of the preflight request, the server can also notify the client whether to carry identity credentials (including Cookies and HTTP authentication related data).

2.1 Simple request

When the request meets the following conditions at the same time , the CORS verification mechanism will use the simple request, otherwise the CORS verification mechanism will use the preflight request.

1. Use one of GET, POST, HEAD methods;

2. Only the following security header fields are used, and other header fields are not allowed to be artificially set;

- Accept
- Accept-Language
- Content-Language
- Content-Type is limited to one of three types: text/plain, multipart/form-data, application/x-www-form-urlencoded:
- HTML header header field: DPR, Download, Save-Data, Viewport-Width, WIdth

3. Any XMLHttpRequestUpload object in the request does not have any event listeners registered; the XMLHttpRequestUpload object can be accessed using the XMLHttpRequest.upload property;

4. The ReadableStream object is not used in the request.

In the simple request mode, the browser directly sends a cross-domain request, and carries the Origin header in the request header, indicating that this is a cross-domain request. After receiving the request, the server will return the verification result through the Access-Control-Allow-Origin and Access-Control-Allow-Methods response headers according to its own cross-domain rules.

The response carries the cross-domain header Access-Control-Allow-Origin. The simplest access control can be done using Origin and Access-Control-Allow-Origin. In this example, the Access-Control-Allow-Origin: * returned by the server indicates that the resource can be accessed by any foreign domain. If the server only allows access from http://admin.training.com , the content of the header field is as follows:

Access-Control-Allow-Origin: http://admin.training.com

Now, except for http://admin.training.com , no other domain can access this resource.

2.2 Preflight Request

When the browser finds that the request sent by the page is not a simple request, it will not immediately execute the corresponding request code, but will trigger the pre-request mode. The pre-request mode will first send a preflight request (pre-authentication request). The preflight request is an OPTION request, which is used to ask the server to be accessed across domains whether to allow pages under the current domain name to send cross-domain requests. The real HTTP request can only be sent after obtaining the cross-domain authorization of the server.

The OPTIONS request header will contain the following headers:

After the server receives the OPTIONS request, it sets the header to communicate with the browser to determine whether to allow the request.

If the preflight request is verified, the browser will send the real cross-domain request.

3 Backend configuration

I have tried two methods for the backend configuration, and after two months of testing, they can all run very stably.

Nginx configuration recommended by MND;
SpringBoot comes with CorsFilter configuration.

▍Nginx configuration recommended by MND

Nginx configuration is equivalent to configuration at the request forwarding layer.

location / {
 if ($request_method = 'OPTIONS') {
 add_header 'Access-Control-Allow-Origin' '*';
 add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
 #
 # Custom headers and headers various browsers *should* be OK with but aren't
 #
 add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
 #
 # Tell client that this pre-flight info is valid for 20 days
 #
 add_header 'Access-Control-Max-Age' 1728000;
 add_header 'Content-Type' 'text/plain; charset=utf-8';
 add_header 'Content-Length' 0;
 return 204;
 }
 if ($request_method = 'POST') {
 add_header 'Access-Control-Allow-Origin' '*' always;
 add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
 add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range' always;
 add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
 }
 if ($request_method = 'GET') {
 add_header 'Access-Control-Allow-Origin' '*' always;
 add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
 add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range' always;
 add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
 }
}

When configuring the Access-Control-Allow-Headers attribute, because the custom header contains signatures and tokens, the number is large. For simplicity and convenience, I configure Access-Control-Allow-Headers to *.

There is no abnormality under Chrome and firefox, but the following error is reported under IE11:

The request header content-type is not present in the Access-Control-Allow-Headers list.

It turns out that IE11 requires that the values of Access-Control-Allow-Headers returned by the preflight request must be separated by commas.

▍SpringBoot comes with CorsFilter

First, the basic framework has the following cross-domain configuration by default.

public void addCorsMappings(CorsRegistry registry) {
 registry.addMapping("/**")
 .allowedOrigins("*")
 .allowedMethods("POST", "GET", "PUT", "OPTIONS", "DELETE")
 .allowCredentials(true)
 .allowedHeaders("*")
 .maxAge(3600);
}

But after the deployment is complete, a CORS exception is still reported when entering:

From the nginx and tomcat logs, only one OPTION request is received. There is an interceptor ActionInterceptor in the springboot application, which obtains the token from the header, calls the user service to query the user information, and puts it into the request. When the token data is not obtained, the data in JSON format will be returned to the front end.

But from the phenomenon, CorsMapping did not take effect.

why? In fact, it is still the concept of execution order. The figure below shows the execution order of filters, interceptors, and controllers.

The DispatchServlet.doDispatch() method is the core entry method of SpringMVC.

// Determine handler for the current request.
mappedHandler = getHandler(processedRequest);
if (!mappedHandler.applyPreHandle(processedRequest, response)) {
 return;
}
// Actually invoke the handler.
mv = ha.handle(processedRequest, response, mappedHandler.getHandler());

So where is CorsMapping initialized? After debugging, it is located in AbstractHandlerMapping .

protected HandlerExecutionChain getCorsHandlerExecutionChain(HttpServletRequest request,
HandlerExecutionChain chain, CorsConfiguration config) {
if (CorsUtils.isPreFlightRequest(request)) {
HandlerInterceptor[] interceptors = chain.getInterceptors();
chain = new HandlerExecutionChain(new PreFlightHandler(config), interceptors);
}
else {
chain.addInterceptor(new CorsInterceptor(config));
 }
return chain;
}

There is a preflight judgment in the code, which is processed through PreFlightHandler.handleRequest(), but it is behind the normal business interceptor.

The final choice of CorsFilter is mainly based on two reasons:

The execution order of the filter has the highest priority;
By debugging the source code of CorsFilter, it is found that the source code has many details.

private CorsConfiguration corsConfig() {
 CorsConfiguration corsConfiguration = new CorsConfiguration();
 corsConfiguration.addAllowedOrigin("*");
 corsConfiguration.addAllowedHeader("*");
 corsConfiguration.addAllowedMethod("*");
 corsConfiguration.setAllowCredentials(true);
 corsConfiguration.setMaxAge(3600L);
 return corsConfiguration;
}
@Bean
public CorsFilter corsFilter() {
 UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
 source.registerCorsConfiguration("/**", corsConfig());
 return new CorsFilter(source);
}

In the following code, when allowHeader is a wildcard *, CorsFilter will concatenate Access-Control-Request-Headers with commas when setting Access-Control-Allow-Headers, so as to avoid the problem of IE11 response headers.

public List<String> checkHeaders(@Nullable List<String> requestHeaders) {
 if (requestHeaders == null) {
 return null;
 }
 if (requestHeaders.isEmpty()) {
 return Collections.emptyList();
 }
 if (ObjectUtils.isEmpty(this.allowedHeaders)) {
 return null;
 }
 boolean allowAnyHeader = this.allowedHeaders.contains(ALL);
   List<String> result = new ArrayList<>(requestHeaders.size());
 for (String requestHeader : requestHeaders) {
 if (StringUtils.hasText(requestHeader)) {
 requestHeader = requestHeader.trim();
 if (allowAnyHeader) {
 result.add(requestHeader);
 }
 else {
 for (String allowedHeader : this.allowedHeaders) {
 if (requestHeader.equalsIgnoreCase(allowedHeader)) {
 result.add(requestHeader);
 break;
 }
 }
 }
 }
 }
 return (result.isEmpty() ? null : result);
}

The execution effect of the browser is as follows:

4 preflight response codes: 200 vs 204

After the backend configuration was completed, a friend in the team asked me: "Brother Yong, is the response code returned by the pre-check request 200 or 204?". This question really stuck with me.

The pre-check response code of our API gateway is 200, and the pre-check response code of CorsFilter is also 200.

The example preflight response codes given by MDN are all 204.

https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

I can only take the Google method, and I suddenly found that the developers of the famous API gateway Kong also had some discussions on this issue.

The preflight response code recommended by MDN was 200, so Kong also synchronized with MDN to 200;
The page was updated since then. See its contents on Sept 30th, 2018: https://web.archive.org/web/20180930031917/https ://developer.mozilla.org/en-US/docs/Glossary/Preflight_request
Later, MDN changed the response code to 204, so the developers of Kong debated whether to keep in sync with MDN.
The core point of the debate is whether there is an urgent need . The 200 response code works just fine, and seems to be forever. And replace it with 204, not sure if there is a hidden problem.
At the end of the day, framework developers still rely on the underlying implementation of the browser. On this issue, there is not enough authoritative information to support framework developers, and various knowledge points are scattered in every corner of the Internet, full of incomplete details and partial solutions, which make framework developers very confused.

Finally, the preflight response code in Kong's source code is still 200, which is not in sync with MDN.

I carefully checked the major mainstream websites, and 95% of the pre-check response codes are 200. After more than two months of testing, Nginx configures the pre-check response code 204, and there is no problem in the mainstream browsers Chrome, Firefox, and IE11.

Therefore, 200 works everywhere, and 204 is also very well supported in the current mainstream browsers.

5 Chrome: Non-secure private network

I thought the cross-domain problem was solved in this way. Unexpectedly, there was still a small episode.

The product director needs to give a presentation to the client, and I am responsible for setting up the presentation environment. Applying for a domain name, preparing the Alibaba Cloud server, packaging the application, and deploying it all went smoothly.

However, when accessing the demo environment on the company's intranet, there is a page that keeps reporting CORS errors, and the error content is similar to the following figure:

The cross domain error type is: InsecurePrivateNetwork.

This is completely different from the cross-domain error I encountered before, and I panicked. Immediately Google, it turns out that this is a new feature after chrome is updated to 94, you can manually turn off this feature.

Open the tab page chrome://flags/#block-insecure-private-network-requests
Set its Block insecure private network requests to Disabled, and then restart it, which is equivalent to disabling this function.

But this is treating the symptoms rather than the root cause. What's a little strange is that when we are not accessing the demo environment on the company's intranet, the demo environment is completely normal, and the wrong page can also be accessed normally.

Looking carefully at the official documents, CORS-RFC1918 points out that the following three requests will be affected.

Public network access to private network;
Public network access to local devices;
Private network access to local devices.

In this way, I located the problem on the wrong third-party interface address. Many products of the company rely on this interface service. When accessing on the company intranet, the domain name mapping address is similar: 172.16.xx.xx.

And this ip happens to be the private network specified in rfc1918.

10.0.0.0     -  10.255.255.255 (10/8 prefix)
172.16.0.0   -  172.31.255.255 (172.16/12 prefix)
192.168.0.0  - 192.168.255.255 (192.168/16 prefix)

When the intranet accesses this page through Chrome, non-secure private network interception will be triggered.

How to solve it? The official plan is divided into two steps:

The private network can only be accessed through Https;
In the future, add specific preflight headers, such as: Access-Control-Request-Private-Network, etc.

Of course there are some temporary methods:

Turn off this feature of Chrome;
Switch to another browser such as Firefox;
Turn off the network intranet and turn on the mobile phone hotspot;
Modify the local host to bind the external network ip.

Based on the official solution, the production environment completely uses Https, and there is no such cross-domain problem in company intranet access.

6 review

API Gateway is very suitable for the architecture of the current product. At the beginning of the architecture design, multiple terminals of the system will call our API gateway. The API gateway can be deployed in SAAS and privatized, has a separate domain name, and provides a complete signature algorithm. Considering the launch time node, team members' familiarity with the API gateway, and the time and cost of deploying multiple environments, in order to deliver as soon as possible, from the architectural level, I made some balances and compromises.

The interface domain name called by the access layer uniformly uses api.training.comthis independent domain name, and Nginx is used to configure request forwarding. At the same time, the front-end leader and I have unified the front-end and back-end protocols, keeping them consistent with our API gateway, and making pre-preparations for the subsequent switch back to the API gateway.

The API gateway can do authentication, current limiting, gray scale, etc., and can configure CORS at the same time. The internal server does not need to pay special attention to cross-domain issues.

At the same time, in the process of solving cross-domain problems, my mentality has also changed. From the initial contempt, to gradually sinking my heart, understanding the principle of CORS step by step, and distinguishing the advantages and disadvantages of different solutions, things gradually went smoothly. I also observed: "Some project teams have given feedback on the problem of Chrome's non-secure private network and provided solutions. For technical managers, they must pay attention to the feedback problems in the project, do a good job of sorting out and analyzing them, and sort out the plan .so that when similar problems arise, it will be organized."

Click to follow and learn about Huawei Cloud's fresh technologies for the first time~