Distributed Database-Course Summary

Reference resources

http://172.16.16.164:8000/courses/81 The latest experiment, the first 5 chapters, understand it, and you can complete the operation of the database.

HBase
http://172.16.16.164:8000/courses/81/assignments/709?module_item_id=3779

MongoDB参考：
http://172.16.16.164:8000/courses/81/assignments/711?module_item_id=3781
http://172.16.16.164:8000/courses/81/assignments/712?module_item_id=3782

Database model characteristics: Column family document key-value pair graph
Database characteristics: Home page of the official website
Comparison with relational database: Logical structure ( Database table row cell) Operation statement comparison (the content can be basically compared)
Database scenario:
Database statement operation: database operation data table operation data operation in the table ( CRUD index advanced query)
Principle of database: principle of hbase, mongodb storage engine B+ tree
Database programming: java and python operations The database can be connected and completed CRUD can complete appropriate advanced queries
Comprehensive case: everyone can complete it

HBase column family database

Introduction

Hbase-Hadoop Database is a highly available, high-performance, column-oriented, scalable, real-time read and write distributed database.
Large: A table can have hundreds of millions of rows and millions of columns.
Scalable: Nodes can be added or removed based on load.
Column-oriented: Compared with row databases, space utilization is high.
Sparse: NULL columns do not occupy storage space, so the table can be designed to be very sparse.
High availability: HDFS-based multi-copy mechanism, WAL (Write-Ahead-Log) pre-writing mechanism, Replication mechanism

Comparison with relational databases

Storage comparison

MySQL	HBase
databasedb	namespace
table table	table
Column field	Column family + column ID
OK	rowkey row
cell cell	rowkey+column family+column ID+version

Operation statement

table level statement

kenmyo
mysql

CREATE TABLE exam_result (
 id INT,
 name VARCHAR(20),
 chinese DECIMAL(3,1),
 math DECIMAL(3,1),
 english DECIMAL(3,1)
);

hbase

#创建一张名为Student的表，包含基本信息（baseinfo）、学校信息（schoolinfo）两个列簇
create 'student','haseinfo','schoolinfo'

删取表
mysql

drop table user;

hbase

#删除表前需要先禁用表
disable 'student'
#删除表
drop 'student'

table data statement

Increase

put 'student', '1','baseinfo:name','tom'
put 'student', '1','baseinfo:birthday','1990-01-09'
put 'student', '1','baseinfo:age','29'
put 'student', '1','schoolinfo:name','Havard'
put 'student', '1','schoolinfo:localtion','Boston'

put 'student', '2','baseinfo:name','jack'
put 'student', '2','baseinfo:birthday','1998-08-22'
put 'student', '2','baseinfo:age','21'
put 'student', '2','schoolinfo:name','yale'
put 'student', '2','schoolinfo:localtion','New Haven'

put 'student', '3','baseinfo:name','maike'
put 'student', '3','baseinfo:birthday','1995-01-22'
put 'student', '3','baseinfo:age','24'
put 'student', '3','schoolinfo:name','yale'
put 'student', '3','schoolinfo:localtion','New Haven'

put 'student', '4','baseinfo:name','maike-jack'

Get information about the specified row, column family, and column in the specified row

# 获取指定行中所有列的数据信息
get 'student','3'

# 获取指定行中指定列族下所有列的数据信息
get 'student','3','baseInfo'

# 获取指定行中指定列的数据信息
get 'student','3','baseinfo:name'

Delete the specified row and column in the specified row

# 删除指定行
delete 'student','3'

# 删除指定行中指定列的数据
delete 'student','3','baseinfo:name'

get query

# 获取指定行中所有列的数据信息
get 'student','3'
# 获取指定行中指定列族下所有列的数据信息
get 'student','3','baseinfo'
# 获取指定行中指定列的数据信息
get 'student','3','baseinfo:name'

scan query

#查询整表数据
scan 'student'
#查询指定列簇的数据
scan 'student', {
    
    COLUMN=>'baseinfo'}
# 查询指定列的数据
scan 'student', {
    
    COLUMNS=> 'baseinfo:birthday'}
# 查看指定列两个版本的数据(3.3中我们设置了3个版本)
scan 'student', {
    
    COLUMNS=> 'baseinfo:birthday',VERSIONS=>2}
# 查看前3条数据
scan 'student',{
    
    LIMIT=>3}

Database principles

Database architecture
Insert image description here

region positioning
data writing
data reading
WAL mechanism
minor merge-store
major merge-store
region split
region merge
Region load balancing

storage structure

Insert image description here

Architecture

MongoDB document database

Introduction

MongoDB is a document-based NoSQL database. Data is stored in MongoDB in the form of documents (corresponding to records in relational databases). Documents are actually JSON strings. The advantage of using JSON is that it is very intuitive. Through a series of Key- Value key-value pairs represent data, which is in line with our reading habits.

There is good support for JSON in Java and Python. After the data is read from MongoDB, it can be used directly without conversion; it supports rich data structures, and Value can be ordinary integers, strings, arrays, and nested The advantage of using nested subdocuments is that you can get the data you need with just one simple query in MongoDB.
Insert image description here