Four classification NoSQL database
KV key:
memcache+redis
Document database (bson format more):
MongoDB
MongoDB is a distributed file storage based database. Written by C ++ language. Designed to provide scalable, high-performance data storage solution for WEB applications.
MongoDB is a product with function between relational databases and non-relational databases, non-relational database functions among the richest and most like a relational database.
Column-store database
Cassandra, HBase ----- Distributed File System
Figure relational database
Neo4J, InfoGrid ----- social networking, recommendation systems. Focus on building relationship map
Distributed Database Theory
In a distributed database principle CAP CAP + BASE
What are the traditional ACID
Relational database to follow the rules ACID
transactions in English is transaction, and the real world of trading is very similar, it has the following four characteristics:
1, A (Atomicity) atomicity
atomicity is easy to understand, that is to say in the affairs of all the operations done either all or nothing, a successful transaction is a transaction where the conditions of all the operations are successful, as long as there is a failed operation the entire transaction will fail, need to roll back. Such as bank transfer, account transfer from A to B account 100 yuan, is divided into two steps: 1) taking account A 100; 2) deposit $ 100 to $ B account. These two steps are completed either together or do not finish together, if only to complete the first step, the second step fails, the money will somehow less than 100 yuan.
2, C (Consistency) Consistency
Consistency is also relatively easy to understand, that has been in the database to a consistent state, run the transaction will not change the original database consistency constraints.
3, Independence I (Isolation)
so-called independence means that do not affect each other concurrent transactions, data to be accessed if a transaction is being modified by another transaction, as long as the other uncommitted transactions, data it visits it is not affected by uncommitted transactions. For example, there is a current transaction is transferred from account A to account B 100 yuan, in the case of this transaction has not been completed, if B at this time to check their accounts, can not see the newly added $ 100
4, D (Durability) Persistence
Persistence means that once the transaction commits, modify it made will be permanently stored on the database, it will not be lost even if downtime occurs.
CAP
C: Consistency (strong consistency)
A: Availability (availability)
P: Partition tolerance (fault tolerance partition)
3 into the CAP 2
CAP theory means that in a distributed storage system, can only achieve the above two points.
And because the current network hardware there will be delays and packet loss problems, so
Partition tolerance that we must need to achieve.
So we can only make trade-offs between consistency and availability, NoSQL systems can not simultaneously guarantee three points.
================================================== ================================================== ===================
C: strong consistency A: high availability P: distributed tolerant
CA traditional Oracle database
AP majority of site architecture choices
CP Redis、Mongodb
Note: The distributed architecture of the time must make a choice.
Take a balance between consistency and availability. Most of the extra web applications, in fact, does not require strong consistency.
So sacrifice in exchange for C P, which is the direction of distributed database product
=================================== ================================================== ==================================
choices consistency and availability
For web2.0 site, many key features of relational databases are often useless
Database transaction consistency demands
a lot of web real-time system does not require strict database transaction, requires low consistency of read, write consistency on some occasions not high. It allows eventual consistency.
Read and write real-time database of real-time demand
for relational databases, data immediately after inserting a query, be sure to read out this data, but for many web applications, does not require such a high real-time, For example, after send a message after a few seconds or even ten seconds, my subscribers only see this dynamic is perfectly acceptable.
Complex SQL queries, especially multi-table queries related needs of
any large amount of data web systems are very taboo multiple large table associated with the query, and analyze complex data type of report query, especially the SNS type of site, from the perspective of demand and product design, to avoid the generation of this situation. More often only a single primary key table queries, and simple single-table query paging conditions, SQL functions are greatly weakened.
Figure classic CAP
CAP core theory is: a distributed system can not simultaneously satisfy consistency, availability, fault tolerance, and partition these three requirements, can only meet two good while.
Therefore, in accordance with the principles of the CAP into a NoSQL database to meet the CA principle, to meet the principles of CP and AP meet the principles of three categories:
CA - single point cluster, meet consistency, system availability, scalability, usually on less powerful.
CP - meet consistency, will tolerate partition systems, performance is usually not particularly high.
AP - meet availability, partition tolerance of the system, usually some possible low consistency requirements.
BASE
BASE availability relational database is to solve the problems caused by the strong consistency due to lower solutions proposed.
BASE is actually the following three terms abbreviations:
Basic available (Basically Available)
soft state (Soft state)
final consistency (Eventually consistent)
The idea is that in exchange for improved scalability and overall system performance by allowing the system to relax for a moment of data consistency requirements. Why do they say, the reason is that large systems are often due to geographical distribution and very high performance requirements, impossible to use distributed transactions to complete these indicators, in order to obtain these indicators, we have to adopt another way to accomplish here BASE way is to solve this problem
+ Introduction to distributed cluster
Distributed Systems
Distributed system (distributed system)
connection (local or wide area network) composed of a plurality of computers and the software components to communicate over a computer network. The system is based on distributed network software systems. It is precisely because of the nature of the software, the distributed system is highly cohesive and transparency. Therefore, the difference between the network and distributed system that more high-level software (especially the operating system), rather than hardware. Distributed systems can be applied on different platforms such as: Pc, workstations, LANs and WANs and so on.
Simply put:
1 Distributed: Different multiple servers deployed on top of different service modules (engineering), between them by between Rpc / Rmi communication and calls, provide external services and collaboration within the group.
Cluster 2: Different multiple servers deployed above the same service module, a unified dispatch by distributed scheduling software, to provide services and access.
Redis
Outline
Redis: REmote DIctionary Server (remote dictionary server), is completely free open source, written in C, and comply with the BSD license, is a high-performance (key / value) distributed in-memory database, based on memory to run, and persistence support the NoSQL database, is one of the most popular NoSql current database, also known as a data structure server.
Brief introduction
1, the support Redis data persistence, data in memory can be kept on disk, restart time can be loaded again using
2, Redis only support simple key-value data types, while also providing a storage list, set, zset, hash data structures, etc.
3, Redis supports the backup data, i.e., data backup master-slave mode,
Features
1, memory storage, and persistence: redis support asynchronous writes the data in memory on the hard disk, and do not affect service
2, taking the latest N pieces of data operations, such as: the latest 10 can comment on the List ID set inside Redis
3, such a simulation similar to the HttpSession need to set an expiration time function
4, publish, subscribe messaging system
5, timers, counters
Feature
Single process
1, single-process model to deal with the client's request. In response to reading and writing events is through packaging epoll function to do. Redis actual processing speed totally dependent on the efficiency of the primary process. epoll is the Linux kernel to handle large quantities of epoll file descriptor has been improved, enhanced version of Linux is the next IO interface multiplexer select / poll, it can significantly improve the program has only a small amount of active cases in a large number of concurrent connections the system CPU utilization.
2, the default database 16, the array-like table from scratch, using the initial default number bank zero
3, select a command to switch the database
4, dbsize view the current number of key database
5, flushdb: clear the current library
6, Flushall; pass to kill all libraries
7, Redis indexes are zero
8, the default port is 6379
9, unified password management, password 16 banks are the same, or are OK or not is also connected on a
Redis data types
1, string (string)
redis string is the most basic type, you can be understood as type of exactly the same with Memcached, a key corresponding to a value.
The string type is binary safe. Redis meaning of string can contain any data. Such as jpg image or serialized object.
Redis basic string data type is a type, a redis string value can be up to 512M
2, hash (hash, similar to java in the Map)
Hash (hash)
Redis hash is a collection of key-value pairs.
Redis hash field is a string type and the value of the mapping table, hash is particularly suited for storing objects.
Java-like inside the Map <String, Object>
3, list (list)
List (List)
Redis list is a simple list of strings, sort insertion order. You can add a head guide elements of the list (on the left) or tails (to the right).
The bottom floor is actually a linked list
4, set (collection)
Set (collection)
Redis is a string Set the type of unordered collection. It is achieved through HashTable implementation.
5, zset (sorted set: an ordered collection)
zset (sorted set: an ordered collection)
Redis zset and is set as a collection of string type elements, and does not allow duplicate members.
The difference is that a double score will be associated with each type of element.
It is to redis from small to large order of collection of member passing score. zset member is unique, but the score (score) it can be repeated.
Command Reference
Common Commands
Redis 键(key)
1、 keys *
2, exists key name ----- determine whether there is a key
eg：exists name
3, move key db ----- there is no current library, has been removed
eg: move name 2 ----- No. 2 to remove the name library
4, expire key seconds: set the expiration time for the given key
eg: expire age 10 ----- survive 10 seconds
5, ttl key to see how many seconds expired, -1 means never expires, -2 is expired
eg:ttl name
6, type key to see what type your key
eg:type name
Redis string (String)
1、set/get/del/append/strlen
2, Incr / decr / incrby / decrby, it must be in order to add or subtract numbers
3, getrange / setrange
getrange: Gets a value within the range of the specified range, similar to the relationship between ...... and from zero to minus one indicates the entire
SetRange designated section within a range of values is provided, setrange key value format is a specific value
4、 setex(set with expire)键秒值/setnx(set if not exist)
setex: Sets the key expiration time, dynamic setting.
setex keys seconds of the true value
setnx: Only settings key in the key does not exist.
5、 mset/mget/msetnx
mset: simultaneously providing one or more key-value pair.
mget: Get all (one or more) of values of the given key.
msetnx: simultaneously providing one or more key-value pairs, if and only if all the given key does not exist.
6, getset (to get re-set)
Redis list (List)
1、 lpush/rpush/lrange
2、 lpop/rpop
3, lindex, obtained according to the indexing subscript element (top to bottom)
Get a list of elements lindex key index by index
4 len
5, lrem key value deletion of the N
Remove * left to right from the two values v1 equal elements, the number of the returned value is actually removed
* LREM list3 0 value indicates delete all the given value. Zero is all values
6, ltrim key index End index start, after the cut-off value range is assigned to the specified key
ltrim: intercepting element specified index range, starting index key format ltrim list end index
7, rpoplpush purpose of the list of the source list
Removes the last element of the list, and adds the element to another list and return
8、 lset key index value
9、 linsert key before/after 值1 值2
Add specific value before and after the value of an existing list
Performance Summary:
It is a string list, left, right can be inserted to add;
if the key does not exist, create a new list;
if the key already exists, the new content;
if the value of the whole is removed, the corresponding button will disappear.
List operation both head and tail are high efficiency, but if the intermediate element is operated, the efficiency is very bleak.
Redis collection (Set)
1、 sadd/smembers/sismember
2、 scard
Get the number of elements inside the collection
3、 srem key value
Delete elements in the collection
4、 srandmember key
An integer (a few random number)
5、 spop key
Random pop
6、 smove key1 key2
In the key1 value in a role where key1 is assigned a value key2
7, difference: sdiff
8, the intersection: sinter
9, and sets: sunion
Redis hash (Hash)
The concept: KV mode unchanged, but V is a key-value pair
1、 hset/hget/hmset/hmget/hgetall/hdel
2, choose
3、 hexists key
A value of the key in the key inside
4, HKEY / whale
5、 hincrby/hincrbyfloat
6、 hsetnx
Redis ordered set Zset (sorted set)
The concept: on the basis of the set, plus a score value. Before the set is k1 v1 v2 v3, now zset is k1 score1 v1 score2 v2
1、 zadd/zrange
2, zrangebyscore key ----- a, withscores; b, (not included; c, limit how much effect the step returns limit (limit subscript start step)
Begin end score score
3、 zrem key
A score value corresponding to the value, is to remove elements
4, zcard / zcount key score interval / zrank key values value, is to obtain the value of the subscript / zscore key corresponding values, obtained score
5, zrevrank key values value, reverse effect is obtained index values
6, zrevrange
7, zrevrangebyscore key end of the score began to score
