手把手地教你搭建Oracle Sharding
Sharding架构是数据库层面的一种分片技术，可以使分过区的数据分布在各不相同的独立数据库里。Sharding是Oracle Database 12c Release 2的新特性，它能为适合于 Sharding技术的OLTP应用提供线性扩展和完全错误隔离的能力，可以将 Sharding简单地理解为Oracle 表分区技术的扩展。
本文将向您展示如何从头一步一步搭建Sharded Database的过程，展示过程中还会穿插讲解一些相关的概念。
本例将搭建如下 Sharded Database (SDB).
· 一共3台Host，即 SDB1, SDB2, SDB3。 均已安装Oracle Linux 6.7。
· 在HostSDB1上安装Shard Director和Shard Catalog。
· 在HostSDB2和HostSDB3上各安装一个Shard。
下面是对Oracle Sharding 主要部件的简要解释：
Sharded Database(SDB) – 是一个逻辑上的Oracle Database,它由多个物理上互相独立的Oracle Databases (Shards) 组成，Shards之间不共享任何软件和硬件，即Share Nothing.
Shards – 是一个独立的数据库。
ShardCatalog - 它也是一个Oracle Database，主要用于Shard的自动部署、集中管理以及跨Shard的查询。
ShardDirectors - 跟据Sharding Key来提供到Shard的路由。OracleSharding中可以设置多个位于不同Host的Shard Director。
Oracle Sharding 支持如下三种Sharding方式：
System-ManagedSharding – 这种Sharding方式不需要用户指定数据到Shard的mapping关系（可以简单理解为数据按consistenthash之后再分布到各个Shard），本例中将使用这种Sharding 方式。
CompositeSharding –这种Sharding方式需要用户指定数据到Shard的mapping关系（这里将不做详细讲解）。
UsingSubpartitions with Sharding – Oracle Sharding是基于表分区技术的， 因此 Sharding可以支持所有的subpartition方法。
下面开始搭建环境。
一． 安装软件：
OracleDatabase 12c Release 2
OracleDatabase 12c Release 2 Global Service Manager (GSM/GDS)
1. 在所有节点上安装Oracle Database 12c Release 2。(注：只安装软件，不创建DB)
主机hosts文件写上本机和各个shard node的IP解析
1.1 安装前准备。
yum ×××tall oracle-database-server-12cR2-pre×××tall-1.0-1.el6.x86_64
yum ×××tall gcc-c++.x86_64
echo 'oracle:oracle' | chpasswd
mkdir /u01
chown -R oracle:o×××tall /u01
su - oracle
./runInstaller
1.2 开始安装，点击Next。
1.3 选择Installdatabase software only，点击Next。
1.4 选择Single×××tance database ×××tallation，点击Next。
1.5 选择EnterpriseEdition，点击Next。
1.6 接下来的所有步骤都使用默认值。点击Next。
1.7 在Summary页面，选择SaveResponse File， 将用于其它节点的安装。
1.8 安装过程中，应该按要求新开一个终端，用root执行脚本。
1.9 在另外两个节点准备环境并静默安装Oracle Database 12c Release 2。
yum ×××tall oracle-database-server-12cR2-pre×××tall-1.0-1.el6.x86_64
yum ×××tall gcc-c++.x86_64
echo 'oracle:oracle' | chpasswd
mkdir /u01
chown -R oracle:o×××tall /u01
su -oracle
./runInstaller -silent -responseFile /home/oracle/db.rsp
2. 在Shard Director 所在节点安装Oracle Database 12c Release 2 Global Service Manager (GSM/GDS)， 本例中即HOST SDB1。安装过程与安装OracleDatabase 12c Release 2类似，均使用默认值。
二． 创建ShardCatalog Database。本例中将会在HostSDB1上创建。
1. 准备环境并启动DBCA。
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_1
mkdir /u01/app/oracle/oradata
mkdir /u01/app/oracle/fast_recovery_area
$ORACLE_HOME/bin/dbca
2. 选择Create a database，点击Next。
3. 选择Advanced configuration，点击Next。
4. 选择Oracle Single Instance database, 选择General Purpose orTransaction Process模板，点击Next。
5. 输入Global Database name和SID，请不要选中Create asContainer database，点击Next。
6. 选择File System，选择Oracle-Managed Files(OMF)，点击Next。
7. 选择Specify Fast Recovery Area ，Enable archiving，点击Next。
8. 接下来的页使用默认值，并跳过Data Vault 选项页。然后选择Use Automatic Shared Memory Management。
9. 选择Use Unicode(AL32UTF8)，点击Next。
10. 不要选择Configure EnterpriseManager(EM) database express，点击Next。
11. 本例所有的用户都用同样密码 : oracle。
12. 选择Create database。
13. 点击Finish。
三． 设置OracleSharding Manage和路由层。本例中在HostSDB1上设置。
1. 设置catalog database 环境变量并启动监听。
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_1
export ORACLE_SID=shard
export PATH=$ORACLE_HOME/bin:$PATH
lsnrctl start
2. 赋角色和权限。GSMCATUSER是12c内置的一个用户，shard director 用这个用户连到catalog database。
sqlplus / as sysdba
alter user gsmcatuser account unlock;
alter user gsmcatuser identified by oracle;
create user mysdbadmin identified by oracle;
grant connect,create session,gsmadmin_role to mysdbadmin;
grant inherit privileges on user sys to GSMADMIN_INTERNAL;
exec DBMS_SCHEDULER.SET_AGENT_REGISTRATION_PASS('oracle');
3. 进入到GDSCTL命令行，创建shard catalog。注：GDSCTL是一个命令行工具，用于管理和配置Global Data Services framework。
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/gsmhome_1
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
gdsctl
create shardcatalog -database 10.10.9.30:1521:shard -chunks 12 -user mysdbadmin/oracle -sdb shard -region region1,region2
4. 创建并启动shard director。并设置操作系统安全认证。
add gsm -gsm sharddirector1 -listener 1522 -pwd oracle -catalog 10.10.9.30:1521:shard -region region1
start gsm -gsm sharddirector1
add credential -credential region1_cred -osaccount oracle -ospassword oracle
exit
5. 连接到每一个Shard Hosts(本例中为HOST SDB2和HOST SDB3)，注册Scheduler agents, 并创建好oradata和fast_recovery_area文件夹。
ssh oracle@sdb2
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_1
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
schagent -start
schagent -status
echo oracle | schagent -registerdatabase 10.10.9.30 8080
mkdir /u01/app/oracle/oradata
mkdir /u01/app/oracle/fast_recovery_area
ssh oracle@sdb3
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/dbhome_1
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
schagent -start
schagent -status
echo oracle | schagent -registerdatabase 10.10.9.30 8080
mkdir /u01/app/oracle/oradata
mkdir /u01/app/oracle/fast_recovery_area
四． 开始布署SharedDatabase。本例将布署System-ManagedSDB。
1. 准备。
export ORACLE_BASE=/u01/app/oracle
export ORACLE_HOME=/u01/app/oracle/product/12.2.0/gsmhome_1
export PATH=$PATH:$HOME/bin:$ORACLE_HOME/bin
gdsctl
set gsm -gsm sharddirector1
connect mysdbadmin/oracle
add shardgroup -shardgroup primary_shardgroup -deploy_as primary -region region1
create shard -shardgroup primary_shardgroup -destination sdb2 -credential region1_cred -sys_password oracle
create shard -shardgroup primary_shardgroup -destination sdb3 -credential region1_cred -sys_password oracle
config shard
2. 布署。
deploy
TIPs:
deploy命令会调用远程每一个节点上的dbca去静默安装sharded database。我们可以通过dbca的日志文件去监控安装进度。
deploy 还会在catalogdatabase 上提交一些job来完成相关事务。我们可以查询dba_scheduler_jobs 来监控进度。
另外，GSM日志可以用于deploy过程的监控和诊断。GSM日志的位置可以通过如下命令查到。
GDSCTL>status gsm
3. 验证安装是否成功。
Sharded Database (SDB) 安装布署到此完成。我们看到，SDB的安装布署非常容易，几乎所有的管理配置都是通过GDSCTL的几条简单命令完成的。另外，Oracle Sharding还高度整合了Oracle Data Guard：如果你想布署standby database，可以通过GDSCTL的一两条命令来定义，Oracle Sharding 会自动帮你布署好standbys。
建立service：
GDSCTL>add service -service oltp_rw_srvc -role primary
GDSCTL>
GDSCTL>config service
Name Network name Pool Started Preferred all
---- ------------ ---- ------- -------------
oltp_rw_srvc oltp_rw_srvc.shardcat.oradbcl shardcat No Yes
oud
GDSCTL>
GDSCTL>start service -service oltp_rw_srvc
GDSCTL>
GDSCTL>status service
Service "oltp_rw_srvc.shardcat.oradbcloud" has 2 ×××tance(s). Affinity: ANYWHERE
Instance "shardcat%1", name: "sh1", db: "sh1", region: "region1", status: ready.
Instance "shardcat%11", name: "sh2", db: "sh2", region: "region1", status: ready.
GDSCTL>
（2016-05-14更新：其实这个service，用于adg的主备切换后，这个service漂移到备库上。）
<ORA-28040>所有节点上均修改该文件 sqlnet.ora
SQLNET.ALLOWED_LOGON_VERSION_SERVER =8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT =8
利用应用用户登录，创建sharded table和duplicated table
[oracle12c@sdb1 ~]$ db_env
[oracle12c@sdb1 ~]$ sqlplus "/ as sysdba"
郑州不孕不育医院：http://jbk.39.net/yiyuanzaixian/zztjyy//
alter session enable shard ddl;
create user app_schema identified by oracle;
grant all privileges to app_schema;
grant gsmadmin_role to app_schema;
grant select_catalog_role to app_schema;
grant connect, resource to app_schema;
grant dba to app_schema;
grant execute on dbms_crypto to app_schema;
利用应用用户登录，创建sharded table和duplicated table
conn app_schema/oracle
alter session enable shard ddl;
CREATE TABLESPACE SET TSP_SET_1 using template (datafile size 100m extent management local segment space management auto );
CREATE TABLESPACE products_tsp datafile size 100m extent management local uniform size 1m;
-- Create sharded table family
CREATE SHARDED TABLE Customers
(
CustId VARCHAR2(60) NOT NULL,
FirstName VARCHAR2(60),
LastName VARCHAR2(60),
Class VARCHAR2(10),
Geo VARCHAR2(8),
CustProfile VARCHAR2(4000),
Passwd RAW(60),
CONSTRAINT pk_customers PRIMARY KEY (CustId),
CONSTRAINT json_customers CHECK (CustProfile IS JSON)
) TABLESPACE SET TSP_SET_1
PARTITION BY CONSISTENT HASH (CustId) PARTITIONS AUTO;
CREATE SHARDED TABLE Orders
(
OrderId INTEGER NOT NULL,
CustId VARCHAR2(60) NOT NULL,
OrderDate TIMESTAMP NOT NULL,
SumTotal NUMBER(19,4),
Status CHAR(4),
constraint pk_orders primary key (CustId, OrderId),
constraint fk_orders_parent foreign key (CustId)
references Customers on delete cascade
) partition by reference (fk_orders_parent);
CREATE SEQUENCE Orders_Seq;
CREATE SHARDED TABLE LineItems
(
OrderId INTEGER NOT NULL,
CustId VARCHAR2(60) NOT NULL,
ProductId INTEGER NOT NULL,
Price NUMBER(19,4),
Qty NUMBER,
constraint pk_items primary key (CustId, OrderId, ProductId),
constraint fk_items_parent foreign key (CustId, OrderId)
references Orders on delete cascade
) partition by reference (fk_items_parent);
-- duplicated table
CREATE DUPLICATED TABLE Products
(
ProductId INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
Name VARCHAR2(128),
DescrUri VARCHAR2(128),
LastPrice NUMBER(19,4)
) TABLESPACE products_tsp;
在shardcat检查:
select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files order by tablespace_name;
TABLESPACE_NAME MB
------------------------------ ----------
PRODUCTS_TSP 100
SYSAUX 690
SYSTEM 880
TSP_SET_1 100
UNDOTBS1 410
USERS 5
select table_name, partition_name, tablespace_name from dba_tab_partitions where tablespace_name like 'C%TSP_SET_1' order by tablespace_name;
select table_name, partition_name, tablespace_name from dba_tab_partitions where tablespace_name like '%SET%';
col TABLE_NAME for a20
col PARTITION_NAME for a20
col TABLESPACE_NAME for a20
/
TABLE_NAME PARTITION_NAME TABLESPACE_NAME
-------------------- -------------------- --------------------
CUSTOMERS CUSTOMERS_P1 TSP_SET_1
ORDERS CUSTOMERS_P1 TSP_SET_1
LINEITEMS CUSTOMERS_P1 TSP_SET_1
select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files;
TABLESPACE_NAME MB
-------------------- ----------
SYSTEM 880
SYSAUX 690
UNDOTBS1 410
USERS 5
TSP_SET_1 100
PRODUCTS_TSP 100
SQL> l
1* select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files
SQL> /
TABLESPACE_NAME MB
-------------------- ----------
SYSTEM 880
SYSAUX 690
UNDOTBS1 410
USERS 5
TSP_SET_1 100
PRODUCTS_TSP 100
6 rows selected.
SQL>
SQL>
SQL>
SQL>
SQL>
SQL> select a.name Shard, count( b.chunk_number) Number_of_Chunks from
2 gsmadmin_internal.database a, gsmadmin_internal.chunk_loc b where
3 a.database_num=b.database_num group by a.name;
SHARD NUMBER_OF_CHUNKS
------------------------------ ----------------
sh1 6
sh2 6
SQL>
在on shard node 1上可以检查:
[oracle12c@sdb2 trace]$ export ORACLE_SID=sh1
[oracle12c@sdb2 trace]$ sqlplus "/ as sysdba"
SQL*Plus: Release 12.2.0.0.2 Beta on Mon May 9 23:51:44 2016
Copyright (c) 1982, 2015, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.0.2 - 64bit Beta
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
SQL> set pages 1000
SQL> select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files order by
2 tablespace_name;
TABLESPACE_NAME MB
------------------------------ ----------
C001TSP_SET_1 100
C002TSP_SET_1 100
C003TSP_SET_1 100
C004TSP_SET_1 100
C005TSP_SET_1 100
C006TSP_SET_1 100
PRODUCTS_TSP 100
SYSAUX 650
SYSTEM 890
SYS_SHARD_TS 100
TSP_SET_1 100
UNDOTBS1 110
USERS 5
13 rows selected.
SQL>
SQL> col TABLE_NAME for a30
SQL> col PARTITION_NAME for a30
SQL> col TABLESPACE_NAME for a30
SQL>
SQL> select table_name, partition_name, tablespace_name from dba_tab_partitions
2 where tablespace_name like 'C%TSP_SET_1' order by tablespace_name;
TABLE_NAME PARTITION_NAME TABLESPACE_NAME
------------------------------ ------------------------------ ------------------------------
LINEITEMS CUSTOMERS_P1 C001TSP_SET_1
CUSTOMERS CUSTOMERS_P1 C001TSP_SET_1
ORDERS CUSTOMERS_P1 C001TSP_SET_1
CUSTOMERS CUSTOMERS_P2 C002TSP_SET_1
ORDERS CUSTOMERS_P2 C002TSP_SET_1
LINEITEMS CUSTOMERS_P2 C002TSP_SET_1
CUSTOMERS CUSTOMERS_P3 C003TSP_SET_1
LINEITEMS CUSTOMERS_P3 C003TSP_SET_1
ORDERS CUSTOMERS_P3 C003TSP_SET_1
LINEITEMS CUSTOMERS_P4 C004TSP_SET_1
CUSTOMERS CUSTOMERS_P4 C004TSP_SET_1
ORDERS CUSTOMERS_P4 C004TSP_SET_1
CUSTOMERS CUSTOMERS_P5 C005TSP_SET_1
ORDERS CUSTOMERS_P5 C005TSP_SET_1
LINEITEMS CUSTOMERS_P5 C005TSP_SET_1
CUSTOMERS CUSTOMERS_P6 C006TSP_SET_1
ORDERS CUSTOMERS_P6 C006TSP_SET_1
LINEITEMS CUSTOMERS_P6 C006TSP_SET_1
18 rows selected.
###########################################
在on shard node 2上可以检查:
[oracle12c@sdb3 trace]$ export ORACLE_SID=sh2
[oracle12c@sdb3 trace]$
[oracle12c@sdb3 trace]$
[oracle12c@sdb3 trace]$ sqlplus "/ as sysdba"
SQL*Plus: Release 12.2.0.0.2 Beta on Mon May 9 23:52:06 2016
Copyright (c) 1982, 2015, Oracle. All rights reserved.
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.0.2 - 64bit Beta
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
SQL> set pages 1000
SQL> select TABLESPACE_NAME, BYTES/1024/1024 MB from sys.dba_data_files order by
2 tablespace_name;
TABLESPACE_NAME MB
------------------------------ ----------
C007TSP_SET_1 100
C008TSP_SET_1 100
C009TSP_SET_1 100
C00ATSP_SET_1 100
C00BTSP_SET_1 100
C00CTSP_SET_1 100
PRODUCTS_TSP 100
SYSAUX 650
SYSTEM 890
SYS_SHARD_TS 100
TSP_SET_1 100
UNDOTBS1 115
USERS 5
13 rows selected.
SQL>
SQL>
SQL> l
1 select table_name, partition_name, tablespace_name from dba_tab_partitions
2* where tablespace_name like 'C%TSP_SET_1' order by tablespace_name
SQL> /
TABLE_NAME PARTITION_NAME TABLESPACE_NAME
------------------------------ ------------------------------ ------------------------------
ORDERS CUSTOMERS_P7 C007TSP_SET_1
LINEITEMS CUSTOMERS_P7 C007TSP_SET_1
CUSTOMERS CUSTOMERS_P7 C007TSP_SET_1
ORDERS CUSTOMERS_P8 C008TSP_SET_1
CUSTOMERS CUSTOMERS_P8 C008TSP_SET_1
LINEITEMS CUSTOMERS_P8 C008TSP_SET_1
LINEITEMS CUSTOMERS_P9 C009TSP_SET_1
ORDERS CUSTOMERS_P9 C009TSP_SET_1
CUSTOMERS CUSTOMERS_P9 C009TSP_SET_1
LINEITEMS CUSTOMERS_P10 C00ATSP_SET_1
ORDERS CUSTOMERS_P10 C00ATSP_SET_1
CUSTOMERS CUSTOMERS_P10 C00ATSP_SET_1
ORDERS CUSTOMERS_P11 C00BTSP_SET_1
LINEITEMS CUSTOMERS_P11 C00BTSP_SET_1
CUSTOMERS CUSTOMERS_P11 C00BTSP_SET_1
LINEITEMS CUSTOMERS_P12 C00CTSP_SET_1
CUSTOMERS CUSTOMERS_P12 C00CTSP_SET_1
ORDERS CUSTOMERS_P12