Deploy single-node OceanBase locally using OBD offline

Use OBD offline to deploy a single-copy single-node community version of OceanBase locally

One: Check the hardware and software requirements

Introduction to software resources:

The OceanBase database is a distributed cluster product that requires at least three machines in the production environment, and a stand-alone version can be deployed in the learning environment. The following table lists the installation packages applicable to different operating system versions:

operating system	Installation package download address
Anlios 7、CentOS 7、RedHat 7、Ubuntu 20.X、Debian 9.X	https://open.oceanbase.com/softwareCenter/community
Anlios 8、CentOS 8、RedHat 8、Debian 10	https://mirrors.aliyun.com/oceanbase/community/stable/el/8/x86_64/

Hardware resource preparation:

project	describe
system	Anolis OS version 8.X (kernel Linux version 3.10.0 and above) -- recommended Red Hat Enterprise Linux Server version 7.X, version 8.X (kernel Linux version 3.10.0 and above) -- recommended CentOS Linux 7.X version, version 8.X (kernel Linux version 3.10.0 and above) – recommended
CPU	The minimum requirement for enterprise users is 16 cores, and 32 cores and above are recommended. The minimum requirement for individual users is 2 cores, and 8 cores and above are recommended. For performance test scenarios, 24 cores and above are recommended. For production environments, 32 cores and above are recommended.
Memory	The minimum requirement for enterprise users is 64G, and 256G and above is recommended. The minimum requirement for individual users is 8G for a simple trial. The minimum requirement for long-term use is no less than 32G, and 64G and above are recommended. The memory below 64G is recommended to use OBD to deploy clusters. It is not recommended to use OCP to deploy small-scale cluster performance tests. Scenarios recommend 128G and above 256G and above are recommended for production environments
disk type	SSD is recommended
disk storage space	4 times or more the size of memory
File system	EXT4 or XFS, when the data exceeds 16T, use XFS
network card	Gigabit Internet and above

Two: Environment and Configuration Check

Before deploying an OceanBase cluster, you need to check the system environment and configuration.

View resources:

If you are an enterprise user, it is recommended that all machines use the same hardware and software configuration when deploying an OceanBase cluster. If you are deploying a cluster version of the OceanBase database, you can go to the official website to view the relevant environment configuration inspection. OceanBase database supports operating systems such as Red Hat Enterprise Linux Server 7.X, CentOS 7.X and Anolis OS 8.X.

Three: Get the installation package and install it

Download link: https://open.oceanbase.com/softwareCenter/community

1. Deploying a single machine and a single node only requires the following three packages: ob-deploy-1.2.1-9.el7.x86_64.rpm, oceanbase-ce-3.1.2-10000392021123010.el7.x86_64.rpm, oceanbase-ce-3.1.2 -10000392021123010.el7.x86_64.rpm (Note: The version number can be downloaded according to your needs)

2. Install obd:

   rpm -ivh ob-deploy-1.2.1-9.el7.x86_64.rpm
   

3. Set environment variables:

   source /etc/profile.d/obd.sh
   

4. Build local sources:

   obd mirror clone /home/frank/rpm/*.rpm
   

5. Remove remote source:

   rm -rf ~/.obd/mirror/remote/
   

6. view local source

obd mirror list local

7. Create mini-local-example.yamla configuration file, obtain it: https://github.com/oceanbase/obdeploy/blob/master/example/mini-local-example.yaml

   The content is as follows:

oceanbase-ce:
  servers:
    # Please don't use hostname, only IP can be supported
    - 127.0.0.1
  global:
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /root/observer	
    # The directory for data storage. The default value is $home_path/store.
    # data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    # redo_dir: /redo
    # Please set devname as the network adaptor's name whose ip is  in the setting of severs.
    # if set severs as "127.0.0.1", please set devname as "lo"
    # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
    devname: lo
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
    zone: zone1
    cluster_id: 1
    # please set memory limit to a suitable value which is matching resource.
    memory_limit: 8G # The maximum running memory for an observer
    system_memory: 4G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
    stack_size: 512K
    cpu_count: 16
    cache_wash_threshold: 1G
    __min_full_resource_pool_memory: 268435456
    workers_per_cpu_quota: 10
    schema_history_expire_time: 1d
    # The value of net_thread_count had better be same as cpu's core number.
    net_thread_count: 4
    sys_bkgd_migration_retry_num: 3
    minor_freeze_times: 10
    enable_separate_sys_clog: 0
    enable_merge_by_turn: FALSE
    datafile_disk_percentage: 20 # The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.
    syslog_level: INFO # System log level. The default value is INFO.
    enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
    enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
    max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
    # root_password: # root user password, can be empty

Precautions:

1. servers: only ip can be used, not hostname (hostname)
2. cpu_count: The number of CPUs can be greater than the number of physical CPUs.
3. home_pathIt is the working directory of the OceanBase database. The OceanBase database starts under this directory. You need to create it in advance.

8. Deploy observers

obd cluster deploy myob -c mini-local-example.yaml

9. View deployment results

obd cluster display myob
obd cluster list

10. Parameter setting (this step can be omitted, the yum configuration file has been loaded)

echo fs.aio-max-nr = 131072 >> /etc/sysctl.conf
sysctl -p
ulimit -n
ulimit -n 65535
ulimit -n

10. Start the cluster

obd cluster start myob

11. Connect to the database:

Installation: obclient connection tool (note: libobclient needs to be installed)

Install libobclient:

sudo rpm -ivh libobclient-2.0.0-2.el7.x86_64.rpm;

Install obclient:

sudo rpm -ivh obclient-2.0.0-2.el7.x86_64.rpm;

12. Use the system user command to connect:

obclient -h127.0.0.1 -P2881 -uroot
mysql -h127.0.0.1 -P2881 -uroot@sys -p -c -A oceanbase

Four: The process of creating a new tenant

View allocatable resources (zones):

select zone,cpu_total,cpu_assigned,round(mem_total/1024/1024/1024,2) MEM_TOTAL_GB,round(mem_assigned/1024/1024/1024,2) MEM_ASSIGNED_GB FROM __ALL_VIRTUAL_SERVER_STAT;

Query all unit specification (unit) configurations (resource pool configurations):

select name,max_cpu,min_cpu,max_memory,min_memory,max_iops,max_disk_size,max_session_num from __all_unit_config;

Note: mysql -h127.0.0.1 -P2881 -uroot@sys -p -c -A oceanbase connection can query

1. Create a resource configuration (unit)

CREATE RESOURCE UNIT uc1 MAX_CPU 8,MIN_CPU 8, MAX_MEMORY '20G', MIN_MEMORY '20G', MAX_IOPS 128, MIN_IOPS 128, MAX_DISK_SIZE '2T', MAX_SESSION_NUM 64;

Modify resource configuration:

ALTER RESOURCE UNIT uc1 MAX_CPU 8, MIN_CPU 8, MAX_MEMORY '16G', MIN_MEMORY '16G';
ALTER RESOURCE UNIT uc1 max_session_num 2000;

2. Create a resource pool

create resource pool p1 unit='uc1' , unit_num=1;   p1：资源池名称

Note: All zones use the same resource unit specification (keep the name consistent with step 1)

or:

CREATE RESOURCE POOL rp1 UNIT 'uc1', UNIT_NUM 2, ZONE_LIST ('zone1', 'zone2');

Delete resource pool:

DROP RESOURCE POOL uc1 ;

3. The problem of creating a resource pool: machine resource 'zone1' is not enough to hold a new unit (you will encounter it)

Problem description: The remaining available resources on zone1 are insufficient

Solution:

1. View the memory_limit parameter

show parameters like 'memory_limit';

2. Use alter sytem to modify the memory_limit parameter

alter system set memory_limit='12G';  (参数放大)

Description: The size can be set according to the server configuration

4. Create a tenant (TENANT) and associate it with this resource pool.

CREATE TENANT test_tenant resource_pool_list=('p1');                     test_tenant : 租户名称

或者直接创建资源池创建租户：CREATE RESOURCE POOL pool1 UNIT 'uc1', UNIT_NUM 1, ZONE_LIST ('zone1');

Create a new tenant reference:

create tenant if not exists obmysql resource_pool_list=('p1','p2'), primary_zone='RANDOM',comment 'mysql tenant/instance', charset='utf8mb4'  set ob_tcp_invited_nodes='%' ;
或
CREATE TENANT IF NOT EXISTS test_tenant CHARSET='utf8mb4',ZONE_LIST=('zone1','zone2','zone3'), PRIMARY_ZONE='zone1;zone2,zone3', RESOURCE_POOL_LIST=('pool1') SET ob_tcp_invited_nodes='%' ;

illustrate:

1）参数ob_tcp_invited_nodes='%' 是指允许所有客户端访问。对于 MySQL 模式，可选字符集有：utf8mb4, binary, gbk, gb18030。缺省为utf8mb4。
2）Zone List和primary_zone中的列表写法，使用逗号，则逗号两侧优先级相同，使用分号，则分号左侧优先级高于右侧。此处PRIMARY_ZONE='zone1;zone2,zone3'则表示优先将主副本分配至zone1中，减少跨Zone及跨服务器的操作。

5. Create other user admin in the tenant

create user if not exists admin@'%' identified by 'admin';

Five: New users connect to the new tenant database

1. The user grants permission to remotely connect to the mysql database

grant all on scoot.* to admin@'%';

2. Use the newly created user admin to log in to the test_tenant tenant

obclient -h 127.0.0.1 -P2881 -uadmin@test_tenant -p -c -A       用户名：admin@test_tenant

Tenant test_tenant password is empty to log in:

obclient -h 127.0.0.1 -P2881 -uroot@test_tenant -c -A oceanbase     

3. There is a problem: ERROR 1227 (42501): Access denied

solve:

1. Display the permissions of the connection:

SHOW VARIABLES LIKE 'ob_tcp_invited_nodes';

2. Set tenant whitelist

SET GLOBAL ob_tcp_invited_nodes='%';


新租户白名单设置：
ALTER TENANT csmysql SET VARIABLES ob_tcp_invited_nodes=’%’;
alter tenant 租户名 set variables ob_tcp_invited_nodes='%';  alter tenant test_tenant set variables ob_tcp_invited_nodes='%';

4. Modify user admin permissions

Grant user-level full privileges:

GRANT ALL ON *.* TO admin;

(operating under the login of the new tenant)

5. Modify user password

 ALTER USER admin IDENTIFIED BY 'password';

Example: The example of changing the password of user sqluser01 to * 1 is as follows: ALTER USER sqluser01 IDENTIFIED BY '* 1 ';

Six: Related command operations:

Create and drop databases:

create database scoot;  drop database scoot;

Query cluster name:

show parameters like 'cluster';

Query the tenants in the database:

 select * from gv$tenant;

Start the cluster:

obd cluster start myob

Stop the cluster:

obd cluster stop myob

Destroy the cluster:

obd cluster destroy myob

Version query:

SELECT version();

Query the tenant's user and password:

select user_name,passwd from __all_user;

Show the user's running threads:

show processlist 

Turn off the firewall:

systemctl stop firewalld

Boot without starting the firewall:

systemctl disable firewalld

View firewall status:

systemctl status firewalld

And view the current server list:

select zone, svr_ip, svr_port, with_rootserver, status from __all_server order by zone, with_rootserver desc;

System user connection: obclient -h 127.0.0.1 -uroot@sys -p -c -A oceanbase

Seven: Notes on basic SQL operations

修改语句：
OB目前的DDL修改列类型（兼容类型向上修改） 例如： int 改 bigint、varchar 改 text、blob 改 long blob 等，
修改列类型（不兼容类型修改） 例如： int 改 varchar、char 改 varchar 等。