Author: ShawnYan Original source: https://tidb.net/blog/3d57f54d
Introduction to TiProxy
TiProxy is an open source, lightweight TiDB database proxy based on the Apache 2.0 protocol. It is written in the Go language and supports the MySQL protocol. TiProxy supports load balancing, receiving requests from applications and then sending them to the TiDB cluster. Supports automatic failover. When the backend TiDB Server fails, the connection can be automatically transferred to other nodes to improve application availability.
TiProxy compilation
TiProxy is a lightweight component, and the compilation steps are very simple. The compilation environment is the same as the previous "TiDB source code compilation - TiUP" .
The compilation steps are as follows:
git clone https://github.com/shawn0915/tiproxy --depth=1
cd tiproxy
go mod download -x
go mod tidy -v
export VERSION='v0.1.1-ShawnYan'
make
Log output:
$ make
go build -gcflags '' -ldflags ' -X github.com/pingcap/TiProxy/pkg/util/versioninfo.TiProxyVersion=v0.1.1-ShawnYan -X github.com/pingcap/TiProxy/pkg/util/versioninfo.TiProxyGitHash=b944a8fe77a56bd39a7de3bf17d7deb1da039494' -tags '' -o ./bin/tiproxy ./cmd/tiproxy
go build -gcflags '' -ldflags ' -X github.com/pingcap/TiProxy/pkg/util/versioninfo.TiProxyVersion=v0.1.1-ShawnYan -X github.com/pingcap/TiProxy/pkg/util/versioninfo.TiProxyGitHash=b944a8fe77a56bd39a7de3bf17d7deb1da039494' -tags '' -o ./bin/tiproxyctl ./cmd/tiproxyctl
Check version number:
$ ./bin/tiproxy -v
./bin/tiproxy version v0.1.1-ShawnYan, commit 81f4897fcf859154255adc86f875eda3f7e5c8a0
$ ./bin/tiproxyctl -v
./bin/tiproxyctl version test, commit test commit
The first problem encountered here is that the variable tiproxyctlis not recognized VERSION. The reason is that there is hard coding in the code. ( Hard code in tiproxyctl#340 )
After modification, the version number is obtained normally.
$ ./bin/tiproxy -v
./bin/tiproxy version v0.1.1-ShawnYan, commit b944a8fe77a56bd39a7de3bf17d7deb1da039494-dirty
$ ./bin/tiproxyctl -v
./bin/tiproxyctl version v0.1.1-ShawnYan, commit b944a8fe77a56bd39a7de3bf17d7deb1da039494-dirty
Try TiProxy
TiProxy's log format supports TiDB native format, jsonformat and consoleformat. Next, start the TiProxy service. The log adopts jsonformat and DEBUGlevel.
./bin/tiproxy --config ./conf/proxy.toml --log_encoder console --log_level debug
After tiproxy starts, try to connect:
mysql --comments --host 127.0.0.1 --port 6000 -u root
It is worth noting that the TiDB server and client use non-encrypted connections by default [1] , and TiDB will not generate a TLS certificate by default when it starts [2] , but TLS authentication is enabled by default when tiproxy starts. To simplify the test, Choose to modify the default configuration items and do not require tls configuration.
vi conf/proxy.toml
[proxy]
require-backend-tls = false
Otherwise, the connection will fail and an error will be reported:
$ mysql --comments --host 127.0.0.1 --port 6000 -u root
ERROR 1105 (HY000): Unknown error%!(EXTRA string=TiProxy fails to connect to TiDB, please check network)
And the console print log:
2023/08/23 12:00:27.645 +09:00 INFO main.proxy proxy/proxy.go:159 new connection {"connID": 0, "client_addr": "127.0.0.1:42944"}
2023/08/23 12:00:27.645 +09:00 DEBUG main.proxy.conn.be.authenticator backend/authenticator.go:135 frontend send capabilities unsupported by proxy {"connID": 0, "client_addr": "127.0.0.1:42944", "common": "CLIENT_LONG_PASSWORD|CLIENT_LONG_FLAG|CLIENT_LOCAL_FILES|CLIENT_PROTOCOL_41|CLIENT_INTERACTIVE|CLIENT_TRANSACTIONS|CLIENT_SECURE_CONNECTION|CLIENT_MULTI_STATEMENTS|CLIENT_MULTI_RESULTS|CLIENT_PLUGIN_AUTH|CLIENT_CONNECT_ATTS|CLIENT_PLUGIN_AUTH_LENENC_CLIENT_DATA|CLIENT_DEPRECATE_EOF", "frontend": "CLIENT_PS_MULTI_RESULTS|CLIENT_CAN_HANDLE_EXPIRED_PASSWORDS|CLIENT_SESSION_TRACK|CLIENT_QUERY_ATTRIBUTES|MULTI_FACTOR_AUTHENTICATION", "proxy": "CLIENT_FOUND_ROWS|CLIENT_CONNECT_WITH_DB|CLIENT_ODBC|CLIENT_RESERVED"}
2023/08/23 12:00:27.646 +09:00 INFO main.proxy.conn.be backend/backend_conn_mgr.go:218 connected to backend {"connID": 0, "client_addr": "127.0.0.1:42944", "ns": "default", "backend_addr": "127.0.0.1:4000"}
2023/08/23 12:00:27.646 +09:00 DEBUG main.proxy.conn.be.authenticator backend/authenticator.go:200 backend does not support capabilities from proxy {"connID": 0, "client_addr": "127.0.0.1:42944", "common": "CLIENT_LONG_PASSWORD|CLIENT_FOUND_ROWS|CLIENT_LONG_FLAG|CLIENT_CONNECT_WITH_DB|CLIENT_LOCAL_FILES|CLIENT_PROTOCOL_41|CLIENT_INTERACTIVE|CLIENT_TRANSACTIONS|CLIENT_SECURE_CONNECTION|CLIENT_MULTI_STATEMENTS|CLIENT_MULTI_RESULTS|CLIENT_PLUGIN_AUTH|CLIENT_CONNECT_ATTS|CLIENT_DEPRECATE_EOF", "proxy": "CLIENT_ODBC|CLIENT_RESERVED|CLIENT_PLUGIN_AUTH_LENENC_CLIENT_DATA", "backend": "CLIENT_SSL"}
2023/08/23 12:00:27.646 +09:00 INFO main.proxy.conn client/client_conn.go:61 new connection failed {"connID": 0, "client_addr": "127.0.0.1:42944", "proxy-protocol": false, "backend_addr": "127.0.0.1:4000", "quit source": "proxy error", "error": "require TLS config on TiProxy when require-backend-tls=true"}
Start TiProxy using TiUP
The next version of TiUP should include this feature. Thanks to the contribution of @xhebox, we can use tiup to call the tiproxy component. Here is a firsthand experience of how to use tiup playground to start tidb and tiproxy. Of course, tiup cluster is also supported.
Since the tiup containing the code for this function has not yet been released, you need to compile tiup yourself. For the compilation steps, please refer to "TiDB Source Code Compilation - TiUP" .
The compiled tiup version information is as follows:
$ tiup --version
1.12.5 tiup
Go Version: go1.21.0
Git Ref: master
GitHash: a9580bd
The tiproxy component has not yet been released to the official mirror library, so if you want to use tiup to schedule the tiproxy component, you need to first upload the tiproxy package to the currently used tiup mirror library. The local mirror library is used here. Use the following command to upload:
tiup mirror publish tiproxy v0.1.1 ./tiproxy.tar.gz tiproxy --desc tiproxy
tiup mirror publish tiproxy nightly ./tiproxy.tar.gz tiproxy --desc tiproxy
For specific operation steps, please refer to "TiUP: A must-have tool for TiDBAer" . Note that you need to submit nightlythe version at the same time, otherwise an error message will be prompted.
Error: Playground bootstrapping failed: component tiproxy doesn't have nightly version on platform linux/amd64
After the above steps are completed, you can see that tiup-playgroundtiproxy related options have been added.
$ ./tiup-playground --version
tiup version 1.12.5 tiup
Go Version: go1.21.0
Git Ref: master
GitHash: a9580bd
$ ./tiup-playground --help | grep tiproxy
--tiproxy int TiProxy instance number
--tiproxy.binpath string TiProxy instance binary path
--tiproxy.config string TiProxy instance configuration file
--tiproxy.host host Playground TiProxy host. If not provided, TiProxy will still use host flag as its host
--tiproxy.port int Playground TiProxy port. If not provided, TiProxy will use 6000 as its port
--tiproxy.timeout int TiProxy max wait time in seconds for starting, 0 means no limit (default 60)
Start the demo as follows:
tiup playground 7.0 --tag 7.0 --without-monitor --tiflash 0 --tiproxy 1
Connect tiproxy via port 6000:
At this point, tiproxy has been started successfully and successfully connected to the backend tidb cluster through tiproxy. Due to limited space, the function demonstration and test content of tiproxy will be documented separately.
Expectation
1. Source code repo names are lowercase
The current repo name is [pingcap/TiProxy]. It is recommended to change it to lowercase pingcap/tiproxyand unify the naming method. After all, it is pingcap/tidbnot [pingcap/TiDB], it is tikv/tikvnot [tikv/TiKV], it is tikv/pdnot [tikv/PD].
2. tiproxy port is not 4000
I believe everyone has reached a consensus that port 4000 is the default port of TiDB Server, so it is really not recommended to change the port of tidb server to other ports and the port of tiproxy to 4000 when tiproxy and tidb server are started at the same time. There are related posts in the AskTUG forum: Chat Post | Database port problem
3. tiproxy can be started independently
Students who have an in-depth understanding of TiDB know that the three core components of TiDB, TiDB Server / TiKV / PD, can be started independently. I hope TiProxy can also do this, and can be started independently without relying on PD, instead of "lying flat" when it cannot find PD, and constantly retrying to connect to PD without time limit.
[2023/08/22 11:47:46.122 +09:00] [WARN] [main.infosync.etcdcli] [[email protected]/retry_interceptor.go:62] [retrying of unary invoker failed] [target=etcd-endpoints://0xc000802380/127.0.0.1:2379] [attempt=0] [error="rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:2379: connect: connection refused\""]
[2023/08/22 11:47:46.122 +09:00] [INFO] [main.infosync.etcdcli] [[email protected]/client.go:210] [Auto sync endpoints failed.] [error="context deadline exceeded"]
When tiproxy fails to connect to pd, an error will be reported when connecting to port 6000 through the client:
$ mysql -uroot -hlocalhost -P6000
ERROR 1105 (HY000): Unknown error%!(EXTRA string=No available TiDB instances, please check TiDB cluster)
4. Package version upgrade
It is recommended to upgrade the package version, such as [ go 1.19 ] to [ go 1.21 ], and align it with the core components. Please refer to the post: [ Announcing upgrade to Go 1.21 ].
There are other packages, such as:
github.com/go-mysql-org/go-mysql v1.6.0
github.com/pingcap/tidb v1.1.0-beta.0.20230103132820-3ccff46aa3bc
github.com/pingcap/tidb/parser v0.0.0-20230103132820-3ccff46aa3bc
Complaint Issue: upgrade go-sql-driver/mysql version to v1.7.1#2246 .
5. Monitoring panel
TiProxy has provided some APIs to manage it, but we still hope to have a minimalist Web page for visualization purposes. After all, HAProxy has it http://localhost:9999/haproxy-status, and so does ProxySQL http://localhost:6080. Even TiDB Lightning has it http://localhost:8289, see the documentation: TiDB Lightning Web Interface .
6. Operating system support
Since tiproxy has not been uploaded to tiup mirror, you can only download tiproxy from github repo asset or compile it yourself if you want to test tiproxy . When testing the package on github, it was found that it cannot run on CentOS 7. For related discussions, see: tiproxy cannot be used . This is just a point, tiproxy compatibility testing should also be done.
7. Compilation parameters
It is recommended to add make cleanoptions, or the make file can be improved. makeIt would be great if it could be done with one command like tiup. ( support make clean option#343 )
In addition, I hope that environment variables will GO111MODULE="on"also be written into the make file.
8. Error log
It is recommended to refine the error code, otherwise it will be really easy for users UNKNOWN.
ERROR 1105 (HY000): Unknown error%!(EXTRA string=TiProxy fails to connect to TiDB, please check network)
I would like to adjust the alarm level of some logs. For example, the query error is now DEBUG, it can be changed to WARN, or it can be output to a log file separately.
2023/08/22 23:51:33.138 +08:00 DEBUG main.proxy.conn.be backend/backend_conn_mgr.go:283 got a mysql error {"connID": 0, "client_addr": "127.0.0.1:41942", "ns": "default", "error": "ERROR 1105 (HY000): conflict hypo index name hypo_a", "cmd": "Query"}
2023/08/22 23:59:08.745 +08:00 DEBUG main.proxy.conn.be backend/backend_conn_mgr.go:283 got a mysql error {"connID": 0, "client_addr": "127.0.0.1:41942", "ns": "default", "error": "ERROR 8108 (HY000): Unsupported type *ast.DropProcedureStmt", "cmd": "Query"}
In addition, after turning on the DEBUG log, there is a large amount of main.infosync.etcdclilog output, which is printed every 3 seconds. It is recommended to optimize it.
2023/08/22 23:59:02.799 +08:00 DEBUG main.infosync.etcdcli [email protected]/retry_interceptor.go:53 retrying of unary invoker {"target": "etcd-endpoints://0xc000980000/127.0.0.1:2379", "attempt": 0}
Summarize
It is said that the cloud version of TiProxy has been used in TiDB Cloud. It is expected that TiProxy will be GA soon and eventually reach a level that can replace HAProxy or ProxySQL.
Attached:
End.
The author of the open source framework NanUI switched to selling steel, and the project was suspended. The first free list in the Apple App Store is the pornographic software TypeScript. It has just become popular, why do the big guys start to abandon it? TIOBE October list: Java has the biggest decline, C# is approaching Java Rust 1.73.0 Released A man was encouraged by his AI girlfriend to assassinate the Queen of England and was sentenced to nine years in prison Qt 6.6 officially released Reuters: RISC-V technology becomes the key to the Sino-US technology war New battlefield RISC-V: Not controlled by any single company or country, Lenovo plans to launch Android PC