Data pieces
related concepts
Sub-table sub-library
fragments: a server data exists in the database. Split particular way. Dispersed stored in multiple database services, it achieves the effect of a single server load.
Vertical segmentation
longitudinal segmentation: by business type
What is vertically divided?
• longitudinal segmentation
- the single table, split into multiple tables, and dispersed to different database (host).
- a database composed of a plurality of tables, each table corresponding to different services may be classified according to the service table, which is divided
cloth to a different database (host), to achieve special purposes, so that different libraries (host) share different business.
Server
Taobaodb a server
registry server b
store table c server
product table d server
sales table e
inventory table f
Split Horizon
What is the level of segmentation?
• horizontal segmentation
- in accordance with the table in a field of some kind of rule, the written record to the table across multiple libraries (host). -
Simply put, according to the data line segmentation, certain rows in the stored table to the specified database (host) in the
horizontal segmentation - fragmentation specified field. Take the remainder 0123
Mycat java-based distributed database system middleware - to provide solutions for distributed storage environment, high concurrency
1 for large amounts of data stored in
2 Oracle SQLServer MongoDB support MySQL
. 3 separate read and write support
4 serving data slice
5 to achieve a high service database available
6alibabacobar open source
Fragmentation rules
1 enumeration \
2 fragment fixed
range stipulated 3
4 modulo
5 date column
6 Wild modulo
7ASCII seeking to touch wildcard
8 string into
9 programming designated
10 hash consistency
mycat service offers 10 fragmentation rules.
--1-by-enumeration method intfile Sharding
- 2 fixed slice Rule1
- agreed scope. 3 Long-Sharding-Auto
-. 4-mod Long Method MOD
-. 5 date column-by-partition method DATE Sharding
-. 6 wild modulo pattern-by-Sharding
- 7ASCII wild symbol modulo-by-prefixpattern Sharding
-. 8-by-program specifies the substring Sharding
- split hash string parsing. 9-by-StringHash Sharding
- 10 consistency hash sharding-by-murmur
.mycat work project?
When Mycat receive a SQL
- SQL will first resolve the lookup table involved, then look at the definition of this table
- if partakers sheet rule is to get the value in SQL fragmented field and match slicing function, to give the corresponding fragment list QL
- then these fragments are sent to the SQL execution to the last collection and processing results of all the data pieces returned, and output to the client
Deployment environment
1-pack
yum the install Java-1.8.0--Y OpenJDK
Which Java
Java -version
the tar-Server--xf myCat the RELEASE-1.6-20,161,028,204,710-linux.tar.gz
Music Videos myCat / usr / local / - and where tomcat-pack as
ls / usr / local /
2 Profile
Bin - mycat command
Catlet - extended function software
Conf - profile .xml mycat profile
(1) Server.xml username \ password \ logic library name (Schema Information)
(2) the Schema.xml configuration data slice
( 3) Rule.xml fragmentation rules
(4) * txt file function calls.
JAR package Lib -mycat use of
logs - logs and start running log
Wrapper - boot logging
Mycat.log - after being given the implementation of content
(1) server .xml (Fixed default) configuration mycat Service
<user>
(2) the configuration data slice
the Schema.xml
CP //usr/local/mycat/conf/schema.xml / the root /
vim /usr/local/mycat/conf/schema.xml
sed -i ‘56,77d’ /usr/local/mycat/conf/schema.xml
[root@mycat ~]# sed -i ‘16,18d’ /usr/local/mycat/conf/schema.xml
[root@mycat ~]# sed -i ‘36,39d’ /usr/local/mycat/conf/schema.xml
Vim
- defined fragment information
of all virtual table eventually add dn3
specify the hostname of the database
data node
Specify the host name ip address (username and password)
select user()
(3) Rule.xml
fragmentation rules
3 configuration database server
(1) create a database for storing data \
53: 54 Db1: the DB2 55: db3
(2) connected to a user authorized mycat admin
4 Start mycat Services
Cd / usr / local / mycat / bin
/ usr / local / mycat / bin / mycat Start | Status | dump | STOP |
Ss -antulp | grep: 8066 port number myCat
PS -C the Java
PID TTY the TIME CMD
? The Java 1586 00:00:47
5 client link
connecting
two selection table
3 Table building
4 into the recording
Mysql -u -p -H -P8066
the Show Databases;
the Show Tables
Asc HotNews
Two fragmentation rules
enumeration method (sharding-by-intfile)
seeking Magic (mod-long)
1
enumeration method
field value must be chosen within the range recited
Schama.xml
Rule.xml
<function name = "the hash-int" - enumeration algorithm
class = "io.mycat.route.function.PartitionByFileMap">
Partition-the hash-int.txt - modification algorithm rules
Vim /usr/local/mycat/conf/partion-hash-int.txt
cat partition-hash-int.txt
10000=0
10010=1
10020=2
(1) The sharding-by-intfile build table
(2) Use testdb
mysql> create table employee (
-> id int primary key auto_increment,
-> sharding_id int,
-> name char(15) ,tal char(11)
-> );
51 desc db1.employee;
52 desc db2.employee
53desc db3.employee
Insert field
Employee INTO INSERT (sharding_id, name, TAL) values (10020, "lucyi", "264 362")
53.54.55
verify the effect of fragmentation
Select * from db1.employee
Select * from db2.employee
Select * from db3.employee
Method mod-long modulo
modulo result storing the digital data in accordance with the set field values
schema.xml
Client connection 56
to create the table Inserts:
Create Table HotNews (ID int (100), Comment char (200 is), worker char (100));
INSERT INTO HotNews (ID, Comment, worker) values (. 7, "lonux", "haha"), (52 is, "Linux",
"xixi")); 53 is 54 is 55 to verify
the Select * from db1.hotnews
the Select * from db2.hotnews
the Select * from db3.hotnewsmod-Long "
＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊
Do not Fragment
schema.xml
create table company ( id int(10) primary key, name char(20), addir char(200), epnum int )
insert into company(id,name,addir,epnum) values(3,“lc”,“huawei”,1040);
＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊
Plus 1 new library
Server.xml
Plus 2 new table
Schema.xml
3 heavy
/usr/local/mycat/bin/mycat stop
/usr/local/mycat/bin/mycat start
４客户端测试
reate table user(name char(100), id int(10), class char(200) );
insert into user(id ,name , class) values(10,“jiji”,“nsd1904”);
select * from user;
| name | id | class |
| jiji | 10 | nsd1904 |
| xxix | 10 | nsd1905 |
| xigua | 10 | nsd1903 |
53测试
select * from db1.user;
| name | id | class |
| xxix | 30 | nsd1905 |
±-----±-----±--------+
54 Test
SELECT * from db2.user;
| name | ID | class |
± ± ----- ------ ± -------- +
| jiji | 10 | nsd1904 |
data pieces
related to concept
Sub-table sub-library
fragments: a server data exists in the database. Split particular way. Dispersed stored in multiple database services, it achieves the effect of a single server load.
Vertical segmentation
longitudinal segmentation: by business type
What is vertically divided?
• longitudinal segmentation
- the single table, split into multiple tables, and dispersed to different database (host).
- a database composed of a plurality of tables, each table corresponding to different services may be classified according to the service table, which is divided
cloth to a different database (host), to achieve special purposes, so that different libraries (host) share different business.
?
Server
Taobaodb a server
registry b servers
store tables c server
product table d server
sales table e
inventory table f
Split Horizon
What is the level of segmentation?
• horizontal segmentation
- in accordance with the table in a field of some kind of rule, the written record to the table across multiple libraries (host). -
Simply put, according to the data line segmentation, certain rows in the stored table to the specified database (host) in the
horizontal segmentation - fragmentation specified field. Take the remainder 0123
?
MyCat java-based distributed database system middleware - providing distributed storage solution for highly concurrent environments
1 for large amounts of data stored in
2 sqlserver MongoDB support the Oracle MySQL
3 separate read and write support
4 provides service data slice
5 achieve high availability database services
6alibabacobar open source
Fragmentation rules
1 enumeration \
2 fragment fixed
range stipulated 3
4 modulo
5 date column
6 Wild modulo
7ASCII seeking to touch wildcard
8 string into
9 programming designated
10 hash consistency
mycat service offers 10 fragmentation rules.
--1-by-enumeration method intfile Sharding
- 2 fixed slice Rule1
- agreed scope. 3 Long-Sharding-Auto
-. 4-mod Long Method MOD
-. 5 date column-by-partition method DATE Sharding
-. 6 wild modulo pattern-by-Sharding
- 7ASCII wild symbol modulo-by-prefixpattern Sharding
-. 8-by-program specifies the substring Sharding
- split hash string parsing. 9-by-StringHash Sharding
- 10 consistency hash sharding-by-murmur
.mycat work project?
When Mycat receive a SQL
- SQL will first resolve the lookup table involved, then look at the definition of this table
- if partakers sheet rule is to get the value in SQL fragmented field and match slicing function, to give the corresponding fragment list QL
- then these fragments are sent to the SQL execution to the last collection and processing results of all the data pieces returned, and output to the client
Deployment environment
1-pack
yum the install Java-1.8.0--Y OpenJDK
Which Java
Java -version
the tar-Server--xf myCat the RELEASE-1.6-20,161,028,204,710-linux.tar.gz
Music Videos myCat / usr / local / - and where tomcat-pack as
ls / usr / local /
2 Profiles
Bin - mycat command
Catlet - software extensions
Conf - profile .xml mycat profile
Server.xml username \ password \ logical library name (Information Schema)
Schema.xml configuration data fragmentation
Rule.xml fragmentation rules
* .txt file function calls
jar package Lib -mycat use of
logs - logs and start running log
Wrapper - boot logging
Mycat.log - after being given the implementation of content
server .xml (Fixed default) configuration mycat Service
<user>
(2) the configuration data slice
the Schema.xml
CP //usr/local/mycat/conf/schema.xml / the root /
vim /usr/local/mycat/conf/schema.xml
sed -i ‘56,77d’ /usr/local/mycat/conf/schema.xml
[root@mycat ~]# sed -i ‘16,18d’ /usr/local/mycat/conf/schema.xml
[root@mycat ~]# sed -i ‘36,39d’ /usr/local/mycat/conf/schema.xml
Vim
- defined fragment information
of all virtual table eventually add dn3
specify the hostname of the database
data node
Specify the host name ip address (username and password)
select user()
Rule.xml
fragmentation rules
3 configuration database server
Create a database to store data \
53: 54 Db1: the DB2 55: db3
Authorized users connecting mycat admin
4 Start mycat Services
Cd / usr / local / mycat / bin
/ usr / local / mycat / bin / mycat Start | Status | dump | STOP |
Ss -antulp | grep: 8066 port number myCat
PS -C the Java
PID TTY the TIME CMD
? The Java 1586 00:00:47
5 client link
connecting
two selection table
3 Table building
4 into the recording
Mysql -u -p -H -P8066
the Show Databases;
the Show Tables
Asc HotNews
Two fragmentation rules
enumeration method (sharding-by-intfile)
seeking Magic (mod-long)
1
enumeration method
field value must be chosen within the range recited
Schama.xml
Rule.xml
<function name = "the hash-int" - enumeration algorithm
class = "io.mycat.route.function.PartitionByFileMap">
Partition-the hash-int.txt - modification algorithm rules
Vim /usr/local/mycat/conf/partion-hash-int.txt
cat partition-hash-int.txt
10000=0
10010=1
10020=2
According sharding-by-intfile built table
Use testdb
mysql> create table employee (
-> id int primary key auto_increment,
-> sharding_id int,
-> name char(15) ,tal char(11)
-> );
51 desc db1.employee;
52 desc db2.employee
53desc db3.employee
Insert field
Employee INTO INSERT (sharding_id, name, TAL) values (10020, "lucyi", "264 362")
53.54.55
verify the effect of fragmentation
Select * from db1.employee
Select * from db2.employee
Select * from db3.employee
Method mod-long modulo
modulo result storing the digital data in accordance with the set field values
schema.xml
Client connection 56
to create the table Inserts:
Create Table HotNews (ID int (100), Comment char (200 is), worker char (100));
INSERT INTO HotNews (ID, Comment, worker) values (. 7, "lonux", "haha"), (52 is, "Linux",
"xixi")); 53 is 54 is 55 to verify
the Select * from db1.hotnews
the Select * from db2.hotnews
the Select * from db3.hotnewsmod-Long "
＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊
Do not Fragment
schema.xml
create table company ( id int(10) primary key, name char(20), addir char(200), epnum int )
insert into company(id,name,addir,epnum) values(3,“lc”,“huawei”,1040);
＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊＊
Plus 1 new library
Server.xml
Plus 2 new table
Schema.xml
3 heavy
/usr/local/mycat/bin/mycat stop
/usr/local/mycat/bin/mycat start
４客户端测试
reate table user(name char(100), id int(10), class char(200) );
insert into user(id ,name , class) values(10,“jiji”,“nsd1904”);
select * from user;
| name | id | class |
| jiji | 10 | nsd1904 |
| xxix | 10 | nsd1905 |
| xigua | 10 | nsd1903 |
53测试
select * from db1.user;
| name | id | class |
| xxix | 30 | nsd1905 |
±-----±-----±--------+
５４测试
select * from db2.user;
| name | id | class |
±------±-----±--------+
| jiji | 10 | nsd1904 |