In the daily development process, remote debugging of local code has always been the most ideal development state. This article introduces a lightweight and simple gadget "localhost" developed within Jingdong Group, from multi-angle solution thinking, to introduction of principles, to the final solution implementation, to identify and solve problems during the development stage.

background

origin

A long time ago, I participated in a technology sharing conference. At that time, a local cloud native development solution in a remote cluster was shared. After listening to it, I was very interested in the part that can run services remotely while developing locally. , as a front-end development itself, it is true that every time some problems are verified, it cannot be debugged in the real environment during the local development stage. Local testing, packaging, release, and verification are required. When dealing with some compatibility issues, this process It may have to be repeated many times, which will waste a lot of time. At the same time, as a developer of cyber cloud testing, we often encounter front-end development students who need to debug compatibility issues with different devices on the cloud testing platform, and some even need to borrow equipment offline. The purpose of the cloud testing platform is to solve all mobile problems online, and development problems are also problems.

Pain points

Imagine a scenario where a test student reported a problem with a white screen opening on a mobile phone page. The mobile phone did reproduce this problem on the cloud testing platform. You searched for some possible solutions, then modified the release verification, and found that it did not work. , what to do at this time.

Or the back-end students have completed the development in advance and sent the test environment to the front-end students to debug the interface in real time. They need to constantly modify fields or logic or debug problems. What should I do at this time?

Or low-code development students, after writing the material components, want to see the effect on the actual device, but they have to find the device separately to see it, or wait for the test classmates to report bugs, and time is wasted in this process. What should they do?

I think development students have more or less experienced these scenarios. When jointly debugging or debugging some problems, there is always no way to save and view the code immediately after modifying it locally. As a tool platform, the idea of how to solve this pain point and better integrate with the platform has always been in my mind.

think

From the time I listened to the sharing session, I was already thinking about what solutions could be used to achieve this goal. At that time, the sharing mentioned the processing of local code for cloud native development. What I didn’t expect was that their solution was very straightforward. Directly "move" the local code to the remote, and synchronize the code to the remote development container when saving locally in real time. In order to solve the time consumption of HTTP protocol transmission, a set of protocols is customized for code synchronization. Of course, this The plan was really overkill for the scenario I envisioned, but it was really bold and careful and gave me a lot of ideas.

Later, based on the idea of code synchronization, I imagined the cloud IDE solution. The purpose of code synchronization itself is to hope that local code can be run remotely. Then it can be developed directly through the cloud IDE and run the service directly. It works like this The service is also remote and can be accessed directly. There is no problem with this idea, but according to everyone's current development habits, cloud IDE is not a long-term solution. Moreover, for this idea, developing a cloud IDE and making dumplings for a mouthful of vinegar is okay, but it is not necessary ∠( ᐛ 』∠)＿.

Later, I chatted with some colleagues about front-end development issues and came up with this idea, which gave me a new idea. Through the whistle proxy and webpack plug-in, the webpack plug-in is packaged in listening mode and uploads the compiled code to the remote server in real time. , the whistle agent is used to solve some problems during debugging. Of course, this solution was developed by other students in the company a long time ago and is called Carefree, but it is no longer maintained, which is a pity. This solution can only solve the needs of front-end development students, and we have some separate processing of proxy servers on the cloud testing platform, which is not very suitable.

After a period of time, I imagined the final form of this requirement. The previous solutions were very heavy. As a user, I might be wandering between being difficult to use and uncomfortable (I don’t mean to say that the previous solutions were bad). , just not suitable for this requirement...), I think it is simple and lightweight enough, and it can be used right out of the box without any environment preparation work. The simplest way is not to work hard on the code, but to turn to the service. As long as the requests and responses on the service are well maintained and the local can interact, I can achieve what I want. At this time, my thoughts gathered in one direction-intranet penetration.

There are many applications for intranet penetration on the market, which involves another issue, security. If intranet penetration is abused, it is likely that isolated intranet services will be mapped to the external network and be attacked, resulting in losses. Even if I sell it, I won’t be able to pay for it... Moreover, private use of intranet penetration itself is difficult to control and trace, which poses a huge security risk.

I struggled for a long time. Since I am not allowed to use it secretly, I can use it openly. I can implement an application myself, as long as I limit the functions and security boundaries (I have already reported it to the security department, don’t worry). And if you implement it yourself, you can satisfy all your ideas, it is lightweight and simple, it can be combined with the cloud testing platform, etc. There is nothing more exciting than realizing your own ideas. So there is the following content.

what is localhost

introduce

localhost is a lightweight and simple intranet penetration tool written in Go language. This tool supports the mapping of web services. It will map services started in localhost to remote servers. When accessing remote corresponding services, the actual request will be sent to the local service using the tunnels established between local and remote to achieve access. Local service purposes. Why use Go to write? Compared with other languages, Go is easy to compile and there are no additional running environment requirements after compilation. Local cross-compilation can obtain executable files for various systems, which is very user-friendly.

use

The help document is here: localhost help document , you can download the tool in the help document or in the cloud real machine usage page of the cyber cloud testing platform .

As mentioned above, localhost is lightweight and simple, and its usage is very simple. It is an independent executable file with only two startup parameters. The startup method is as follows:

# windows
localhost.exe -token xxxxxxxx -port xxxx

# mac
localhost -token xxxxxxxx -port xxxx

The parameter -token is the token used at that time. It can be applied on the cloud real machine usage page of the cyber cloud testing platform. It is valid for half an hour each time.

The -port parameter is the port of the currently started service locally.

When designing the client part, in order to lower the user threshold as much as possible, the parameters are deliberately designed to be as few as possible. The port parameter is essential, so why does the design need a token parameter? The original intention of this parameter It is to identify the user's identity, similar to the key of the ssh protocol, so that the behavior can be traced. Later, the token is used to control the usage behavior, such as the validity period of a single use, etc. I have excluded other parameters without affecting functionality and safety. I hope it is simple enough to use.

principle

The overall tool is divided into three parts: server, client, and web. The server and client are the main parts of service mapping, including logic such as channel establishment and channel forwarding. The web side provides some interactive interfaces and forwarding processing by route aggregation agents.

core logic

The core part of localhost is intranet penetration, which is jointly implemented by the server and the client. As we all know, locally started services can only be accessed locally or within the LAN. Remote devices cannot access local services, but local devices can access remote services. They communicate through the tunnel established when accessing the remote locally, so that the remote device can access the local service. purpose served.

The overall flow chart is as follows:

First, in the server, the web-side service is started, which is the scheduling part of the underlying service in the overall service. It provides some web interfaces called by the client and reverse proxy when user requests. The web-side service is a long-term Running on the remote server, what this part does is explained in detail below.

The client will be distributed to internal students for downloading and use. When running the client, the client calls the web service, starts the corresponding server, and passes in the relevant parameter information and returns it to the client.

When the server side is started, it will start monitoring two port services, one is to monitor the port for user requests, and the other is to monitor the tunnel connection port on the client side. When the client actively initiates a connection with the remote server, it uses the current connection Open the communication tunnel. When a user request comes in, the request message is forwarded to the local client through the established communication tunnel.

After the client side is started and the call to the web-side service returns successfully, the client side gets the relevant information and connects to the corresponding server side. After the tunnel is successfully established, it waits for the request information forwarded by the server side. When a request is heard, the client will actively connect to the local service on the specified port and forward the request message to the current local service. At the same time, the response message will also be returned through the existing link. Finally, Return to the original requester.

In this way, you can achieve the purpose of remote access to local services and penetrate the service through the transfer server. This method does not require users to pay attention to the intermediate process. They only need to start the provided client service. It is simple to use and basically useless. Cost, does not affect the code.

route aggregation reverse proxy

During the use process, some students may have such a question. After the client is started, the link to display the access will be returned. Why can you directly access your own local services by accessing this link? When the client is started, the interface service of the web side will be called. At this time, the current user identity will be recorded, and when the server is started, a mapping relationship will be formed between the started server port and the current user ERP, and a mapping relationship will be established based on the current ERP. Rules for a reverse proxy. Among the services started on the web side, there is a service that monitors the current request. When the web side service is accessed from anywhere, the accessed route will be monitored and judged. If the ERP information of the existing connection can be matched, the current accessed will be The request message is forwarded to the corresponding server service, and then the server forwards the request to the corresponding client.

The flow chart is as follows:

In addition, after accessing through the aggregated ERP route for the first time, the currently accessed IP will be recorded, and the mapping relationship between IP and ERP will also be mapped. In this way, in subsequent visits, the route does not need to carry ERP information, and it can also be used Map to the current local service, which can solve the problem of access paths to some static resources, and can also be used to solve the problem of history routing access.

future

history routing processing

When the routing of the front-end project adopts the history mode, the history routing access will be incorrect in the above-mentioned ERP aggregation routing. Although it can be accessed by removing the ERP after accessing the ERP routing for the first time, in general it will still happen. This has caused some trouble. The follow-up plan is to use pan-domain name resolution to remove the routing processing, resolve the user ID through the domain name, and return the routing to the user's original appearance. This will not cause additional trouble and understanding costs.

Support https

The current localhost tool still uses the http protocol in the overall process of interaction. Currently, it can only be used by removing the https configuration of the local project. However, after communicating with some students, we found that https requests are necessary in some projects, so we will use it in the future. It will support https. It is envisaged that as long as the request is https during access, the https of the local project will be accessed by default, without additional explicit configuration or declaration.

Finish

The road to remote local code has gone through many directions and generated many ideas throughout the process. Finally, the tool localhost was born. In the future, we will combine more scenarios and continue to improve this tool. I hope it can help everyone in the future. In the process of work, it is faster and more economical.

Author: JD Retail Xie Tian

Source: JD Cloud Developer Community Please indicate the source when reprinting

localhost tool: the remote road to local code | JD Cloud technical team