Recently, the reverse proxy of the Go language has been used a lot. In fact, I wrote a TCP-level proxy about two years ago, and I also used the Go language at that time. The difference is that I only tried to use it occasionally. Recently, it is because work needs to be used. Compared with the proxy at the TCP level, the HTTP proxy is more troublesome to implement. If we are just a simple reverse proxy, OK, that's fine, it's almost the same as a Host replacement. However, many times when we are a reverse proxy, the requirements are more diverse. For example, we may want to make some changes to the response content of the proxy, or we may wish to be a reverse proxy for HA.
The Go language provides a very convenient reverse proxy component in its own built-in library. Using the built-in httputil component, we can develop a simple reverse proxy very quickly, as we will see later. However, once we meet the diverse needs we mentioned earlier, we cannot simply write code, we have to understand some components, and then personalize the components according to our needs to meet our needs. Below, I will look at the built-in reverse proxy library of the Go language from simple to complex.
Built-in reverse proxy
According to the official documentation of the Go language: Package httputil , we can write such a simple piece of code, which is copied from the official documentation of the Go language:
Here I explain this code:
- Line 2-5 : Created a simple server that responds to some URLs, the content is the string inside
- Line 7-12 : This is the real code to create a reverse proxy, here a ReverProxy entity of a reverse proxy is created
- Line 14-24 : These are the reverse proxy that imitates a client request, and then get the output response
Execute this code and you should see the result:
this call was relayed by the reverse proxy
Director
It seems that the reverse proxy is very simple. It has been completed in 6 lines of code, and it also includes error handling. OK, it's time to do something interesting. Let's be a reverse proxy for baidu.com. Check out the effect. Why is the agent baidu.com, because it can carry out a lot of operations, play it by yourself:
A very simple idea is to proxy like this, and then we try to run the code and we will find the root! Book! Row! Do not! Pass! So what to do? At this time, you need to think about the possible problems in the middle, or you should use some tools to analyze the difference between the middle links. However, I am not going to find out how to find this problem here. I will tell you that the problem is that the HTTP request header is wrong, so we should change the request header of the request, so the code should be modified as follows:
A key point here is Line 11. If you compare this code of mine with the code of defaultDirector that comes with Go language , you will also find that there is actually an extra line of Line 11 , but this line and its key, many The Server may block requests that the Host does not serve for itself due to various considerations, and we, as a proxy, need to make up for this.
OK, run this code, and then we can successfully access baidu.com through localhost:9090 , which seems to have been successful.
ModifyResponse
Sometimes, my proxy may be accessed by others, so due to copyright and other factors, I will do some small means, such as the response header here:
This Server: BWS/1.1 is not very good, I think it is necessary to change to my own server name: Server: ProjectZoo , so, there is something else to do, how to modify this response header, Go language also provides us with Well, that's ModifyResponse , come on, try it:
On the way to this problem I added a few lines of code, and then revisited to see:
Everything was as I expected and it worked! This is actually an example of simply modifying the response, which can be very powerful, but there is no need to expand it. It should be emphasized that if you need to modify the response body, don't forget to update the Content-Length of the Header at the same time , otherwise, this is a pit left for yourself.
Transport
In the previous two operations, we have done some operations before and after the reverse proxy of HTTP requests. However, the unpleasant part is that we process them in two places, which lacks some uniformity. So is there a way to do these things in one place? Obviously, the Go language provides us with this one place, which is Transport. The official documentation describes Transport as follows:
// The transport used to perform proxy requests.
This seems a bit too simple, let's take a look at how the RoundTripper interface is described:
RoundTrip executes a single HTTP transaction, returning
a Response for the provided Request.RoundTrip should not attempt to interpret the response. In
particular, RoundTrip must return err == nil if it obtained
a response, regardless of the response's HTTP status code.
A non-nil err should be reserved for failure to obtain a
response. Similarly, RoundTrip should not attempt to
handle higher-level protocol details such as redirects,
authentication, or cookies.RoundTrip should not modify the request, except for
consuming and closing the Request's Body. RoundTrip may
read fields of the request in a separate goroutine. Callers
should not mutate the request until the Response's Body has
been closed.RoundTrip must always close the body, including on errors,
but depending on the implementation may do so in a separate
goroutine even after RoundTrip returns. This means that
callers wanting to reuse the body for subsequent requests
must arrange to wait for the Close call before doing so.
The Request's URL and Header fields must be initialized.
It can be seen that the function of Transport is still very simple. Although Request and Response can be modified, it is not expected to be modified here in terms of rationality. The more demand is to modify Response. In this case, let's take a look at how to implement Transport, let's take a DefaultTransportlook :
In fact, there is nothing in DefaultTransportitself . It is nothing more than setting some default parameters to build http.Transportthis structure. There is a pit here. Line 2Proxy：ProxyFromEnviroment will use the system default proxy. If you set the environment variable HTTP_PROXY , then this proxy will be used by default. , remember to remember, this is a pit!
summary
DefaultTransportI won't dig further here , because most of the usual needs can be met here, and then it actually involves the core functions of the HTTP module in the Go language, which will be discussed in later articles. In this article, although I only talked about three components, these three small components are enough. Moreover, in this article, I also focus on describing several places where pits may appear, which are all worthwhile. Let's pay attention, because when using ReverseProxy, the problems generally encountered are inseparable from these.