socks5 proxy protocol based on netty

socks5 proxy protocol based on netty

socks5 protocol

Introduction

The socks5 protocol is a standard proxy protocol that works on the four layers of the network. In theory, it can proxy any application layer protocol. Protocol standard RFC1928, user/password authentication standard RFC1929. The Chinese version of the agreement can be found here

Existing open source implementations

The most used is the open source implementation of ss5 under linux. The installation and configuration of ss5 are as follows:

./configure --with-debug --with-epollio --with-gssapi
make
make install
chomd +x /etc/init.d/ss5

Modify /etc/init.d/ss5, add ports, logs and other configurations:

export SS5_SOCKS_PORT=1081
export SS5_CONFIG_FILE=/tmp/ss5.conf
export SS5_PASSWORD_FILE=/tmp/ss5.passwd
export SS5_LOG_FILE=/tmp/ss5.log
export SS5_PROFILE_PATH=/tmp

Modify /etc/sysconfig/ss5 to configure the server startup user:

SS5_OPTS=" -u root"

Modify /etc/opt/ss5/ss5.conf to configure the authentication method:

permit -        0.0.0.0/0       -       0.0.0.0/0       -       -       -       -       -    
#不使用用户认证,
auth 0.0.0.0/0 – –
#使用用户名/密码认证,
auth 0.0.0.0/0 – u

Modify /etc/opt/ss5/ss5.passwd and configure the user password:

user passord

Start the ss5 service:

service ss5 start

Why do it yourself

You need to record proxy access logs more accurately, and you need your own special authentication method. ss5 is developed based on c. I am general in c language, and it is still a little laborious to change it. However, the socks5 protocol developed based on java is not particularly mature. I finally decided to develop it myself with netty4.1.

Basic concepts of netty

NIO 和 BIO

BIO blocking IO, NIO non-blocking IO. NIO is based on an event notification mechanism, which can handle multiple IOs at the same time in a single-threaded situation. Traditional BIO needs to allocate a separate thread for each IO request, and the allocation of threads needs to occupy memory, and the allocation cost is very high. Therefore, the concurrency of traditional BIO can be up to thousands per second (related to server configuration), while NIO can support tens of thousands of concurrency per second.

NIO is not without its shortcomings. The asynchronous characteristics of NIO have higher programming requirements. Compared with the traditional development of BIO, it is more difficult to control, and the development and debugging are also more difficult.

Introduction to netty

In java development, when it comes to NIO, one must think of netty. Netty is a NIO development framework, which can reduce the difficulty of NIO application development. Developers do not need to pay attention to communication connections, and can focus on encoding the received information. Converted to objects for business logic processing. The netty framework is a typical application of the pipe filter pattern. Netty3.x and 4.x versions have major changes, this article mainly uses netty4.1.

EventLoopGroup和EventLoop

As mentioned earlier, NIO can support multiple IOs in a single-threaded model. In fact, there is not one thread, but a limited number of threads. The EventLoopGroup here is the thread pool, and the EventLoop is the thread. The number of threads of EventLoopGroup is related to the number of CPU cores, and can also be customized during initialization.

The server side of NIO applications usually has more than one thread pool. Generally, a boss thread pool and a worker thread pool are established. The boss thread pool is used to receive client connections, and the received connections are handed over to the worker thread pool for monitoring. A worker thread can monitor multiple io.

//boss线程池，这里定义线程数量为2
EventLoopGroup boss = new NioEventLoopGroup(2);
//worker线程池，线程数量结合cpu核心计算出来Runtime.getRuntime().availableProcessors() * 2
EventLoopGroup worker = new NioEventLoopGroup();

ServerBootstrap和Bootstrap

Bootstrap is a type constructor of builder mode, ServerBootstrap is used for server-side construction, and Bootstrap is used for client-side construction. Mainly configure thread pool, ChannelFactory, basic parameters, ChannelHandler linked list initialization, etc. ServerBootstrap calls bind to listen to the port, and Bootstrap calls connect to connect to the server.

ChannelFuture is often encountered in netty. Since all operations of netty are asynchronous, bind, connet, close, etc. will not return to the interface immediately, but are called back through the listening method. Add a listener method by calling ChannelFuture.addListener(). There are several built-in listening methods CLOSE, CLOSE_ON_FAILURE, FIRE_EXCEPTION_ON_FAILURE in ChannelFutureListener.

Channel，ChannelPipeline，ChannelHandler，ChannelHandlerContext

A Channel is assigned a ChannelPipeline, each ChannelPipeline is a linked list of multiple ChannelHandlers, and each ChannelHandler corresponds to a ChannelHandlerContext. Take a look at the image below to understand: 

• The Channel is responsible for the underlying communication. The Channel will be bound to a worker thread. After the Channel receives the Event, it will be passed to the ChannelPipeline, and the Event will flow in the ChannelHandler linked list. 

• ChannelPipeline is responsible for message delivery, which is a doubly linked list composed of ChannelHandlers, which are divided into upstream message ChannelHandler and downstream message ChannelHandler according to message type. Therefore, there are two sub-interfaces, ChannelInboundHandler only cares about upstream messages, and ChannelOutboundHandler only cares about downstream messages. If both upstream and downstream messages need a relationship, ChannelDuplexHandler can be inherited. ChannelHandler passes the upstream message to the next Handler by calling fireChannelRead(), and forms the downstream message by calling write(). After calling write(), the upstream message will not be passed to the next Handler.

• ChannelHandler is a unit for processing messages and is the core of netty development. Most developers are writing various Handlers. Netty has built-in handlers for many protocols, but unfortunately there are not many documents. As mentioned above, ChannelHandler can be divided into ChannelInboundHandler and ChannelOutboundHandler according to the message type. According to the function, it is divided into encoding and decoding Handler (Encoding Encoder/Decoding Decoder) and message processing Handler. Usually, the upstream message needs to be decoded and converted into an object, and then the object is passed to the subsequent business processing Handler. After the business is processed, the downlink object is encoded and converted into ByteBuf and passed to the Channel to the network layer.

ByteBuf

Understanding ByteBuf has many benefits for message encoding and decoding. ByteBuf is the core object of Channel for network communication. ByteBuf is like a buffer pool, and ByteBuf is logically a byte container. The data in ByteBuf is divided into three parts by two pointers, as shown in the following figure: 

• The data in front of the reader index is the data that has been read, and these data can be thrown away
• Starting from the reader index, the data before the writer index is readable data
• Start from the writer index, which is the writable area

Implement socks5 protocol using netty

The codec of the socks5 protocol has been built in netty, we only need to implement the logic processing Handler, the following are several main codecs:

• Socks5InitialRequestDecoder is responsible for negotiating the authentication method between the server and the client, returns the authentication method supported by the server, and decodes the message into a DefaultSocks5InitialRequest object
• Socks5PasswordAuthRequestDecoder is a protocol that uses the authentication method of username and password. It returns whether the authentication is passed and decodes the message into a DefaultSocks5PasswordAuthRequest object.
• Socks5CommandRequestDecoder is responsible for the connection establishment of the target server, returns whether the establishment is successful, and decodes the message into a DefaultSocks5CommandRequest object
• Socks5ServerEncoder is responsible for encoding all downstream objects and converting them to ByteBuf

The encoding and decoding work does not need to be developed by ourselves, and the rest is relatively simple, we only need to deal with Socks5Request at all levels. The structure of the ChannelHandler chain is as follows:

Socks5ServerEncoder
Socks5InitialRequestDecoder
Socks5InitialRequestHandler(实现自己的处理逻辑)
Socks5PasswordAuthRequestDecoder
Socks5PasswordAuthRequestHandler(实现自己的处理逻辑)
Socks5CommandRequestDecoder
Socks5CommandRequestHandler(实现自己的处理逻辑)

By implementing 3 ChannelHandlers, we can easily get a socks5 protocol. All the code can be found on github, the project name is socks5-netty .