My docker essay 42: TiDB database deployment

This article introduces the containerized deployment of TiDB database.

origin

Some time ago, localization replacement was put on the agenda, and the relevant department (this is really a certain department) intervened as a leader. Therefore, corresponding things need to be initiated, but there seems to be no movement recently. Because of identity issues, I had to do what I could in private.

In terms of database, some colleagues have already connected to Dameng. At present, they just installed Dameng database and added a technical group, and then there is nothing more. For some reasons, I didn't research this in depth. I happened to see that TiDB is highly compatible with MySQL, is open source on github, and is written in golang. I naturally liked it, so I took some time to get in touch with it.

Since it is domestically produced, the operating system must also be domestically produced. Of course it is still a Linux operating system, and the hardware chip must be arm64 (aarch64). The strange thing is that after searching for information on the Internet, there is very little practice on the arm platform, and the docker image has not been updated. Even if the image is pulled on the arm64 platform system, it is still x86 architecture, or it cannot run. I had no choice but to start from scratch.

Generally, TiDB is deployed in cluster mode. As a starting point, non-cluster mode is used, which is a simple replacement of MySQL.

The contents introduced in this article include: downloading the TiDB source code, compiling, packaging the image, and program testing.

First introduction to TiDB

This section covers some basics.

• TiDB project address: https://github.com/pingcap. Documentation Center: https://docs.pingcap.com/zh/tidb/stable.

• TiDB is highly compatible with the MySQL 5.7 protocol, MySQL 5.7 common functions and syntax.

• TiDB has 2 versions, Long-Term Support Releases (LTS) and Development Milestone Releases (DMR). The former is released approximately every six months, while the latter is released approximately every two months.

• The maximum identifier length is 64 characters. Such as database name, table name, field name, etc.

• The maximum limit of column data for a single table is 1017, which can be adjusted to a maximum of 4096.

Environmental preparation

Since it needs to be compiled, docker and golang need to be installed. The versions used in this article are as follows:

# docker -v
Docker version 20.10.3, build 48d30b5
# go version
go version go1.20.2 linux/arm64

golang environment variable configuration:

export PATH=$PATH:/usr/local/go/bin
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
export GOPROXY=https://goproxy.io,direct

Server deployment

Download image (failed)

Download the official image:

# docker pull pingcap/pd-arm64
Using default tag: latest
Error response from daemon: manifest for pingcap/pd-arm64:latest not found: manifest unknown: manifest unknown

Note: The author did not find an officially available image.

Source code compilation

Download the source code, the version used in this article: tidb-server v6.5.2 . The download file is tidb-6.5.2.tar.gz.

Upload the compressed package to the server, decompress it, enter the project directory, and download the dependent packages:

# go mod vendor

Compile:

# make server
fatal: 不是一个 git 仓库（或者任何父目录）：.git
...
CGO_ENABLED=1 GO111MODULE=on go build  -tags codes  -ldflags '-X "github.com/pingcap/tidb/parser/mysql.TiDBReleaseVersion=" -X "github.com/pingcap/tidb/util/versioninfo.TiDBBuildTS=2023-05-28 07:42:53" -X "github.com/pingcap/tidb/util/versioninfo.TiDBGitHash=" -X "github.com/pingcap/tidb/util/versioninfo.TiDBGitBranch=" -X "github.com/pingcap/tidb/util/versioninfo.TiDBEdition=Community" ' -o bin/tidb-server ./tidb-server
Build TiDB Server successfully!

Note: Possibly for internationalization purposes, the compiled timestamp written uses UTC time.

tidb-server use

After successful compilation, binthere will be a service program in the directory tidb-server. Please refer -hto the usage help.

./bin/tidb-server -h

Simple options, listed below:

-P 指定端口，默认4000
-config 指定配置文件
-log-file 指定日志文件，可带目录，不存在则自动创建
-path 数据存储目录，默认为/tmp/tidb

Note: The configuration file can be found at https://kgithub.com/pingcap/tidb-docker-compose/blob/master/config/tidb.toml

Make a mirror

Mainly refer to the official Dockerfile. Since it is compiled manually, you can directly copy the final binary file. The complete content is as follows:

# cat Dockerfile-arm 
FROM rockylinux:9-minimal

COPY tidb-server /tidb-server

WORKDIR /
EXPOSE 4000 
ENTRYPOINT ["/tidb-server"]

Container startup script

docker-compose.yamldocument:

# cat docker-compose.yaml 
version: '3.8'

services:
  tidb:
    image: "arm64v8/tidb:6.5.2" 
    container_name: tidb
    hostname: tidb
    restart: always
    #command: "sh ./myrun-arm.sh"
    command:
      - --config=/tidb.toml
      - --log-file=/logs/tidb.log
    volumes:
      - ./config/tidb.toml:/tidb.toml:ro
      - ./logs:/logs
      - ./tidb_data:/data/tidb
      - /usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static
    environment:
      - AA=aa
    ports:
      - "4000:4000"
      - "10080:10080"
    networks:
      - my-net

networks:
  my-net:
    name: "my-tidb"
    driver: bridge

File description:

• Use commandoptions that specify program startup,
• Mount the configuration file, log directory, and data directory to the host machine.
• Because it was tested on an x86 service, qemu-aarch64-staticthe program is mounted.
• Export 2 ports, currently only 4000 is used.

start up:

docker-compose up -d

Client use

The client uses the mysql command line, Navicat and other graphical tools, and can be connected and used normally.

Mysql test

Start the container.

You can also run the database service program directly as a test without adding parameters, as follows:

./bin/tidb-server

Create user and change root password

By default, tidbno password is set. You can connect to the database and add it yourself. Example:

mysql -h 127.0.0.1 -P 4000 
use mysql;

# 创建latelee用户
CREATE USER 'latelee'@'%' IDENTIFIED BY '123456';
GRANT ALL PRIVILEGES ON *.* TO 'latelee'@'%' WITH GRANT OPTION;

# 修改root密码
ALTER USER 'root' IDENTIFIED BY '123456';

summary

Due to limited capabilities, this article will only tidbuse it as an ordinary database program. However, clusters may not be studied until later because the top priority is to replace them mysql.

When the mysql container starts, you can specify account information or initialize mysql scripts. There will be time in the future to refer to the official documentation and migrate this mechanism to tidb.

attached

Error handling

This section lists some error message records.

TiDB does not fully support all character sets of the mysql database. The test found the following error:

ERROR 1115 (42000) at line 227 in file: 'mysql/sql/mytest.sql': Unknown character set: 'utf8mb3'
ERROR 1146 (42S02) at line 245 in file: 'mysql/sql/mytest.sql': Table 'testdb.userinfo' doesn't exist

Solution:

将 utf8mb3 改为 utf8mb4，utf8mb3_general_ci 改为 utf8mb4_general_ci