NoSQL non-relational database
NoSQL concept
Not Only SQL: Not only SQL, it refers to non-relational databases, it is a useful supplement to relational databases. The final data is still stored in a relational database. Non-relational databases mainly improve the query speed of the database, and are generally used as data caches.
Non-relational database
A non-relational database is not strictly a database. It should be a collection of data structured storage methods, which can be documents or key-value pairs.
advantage
- Flexible format: The format of the stored data can be key, value, document, image, etc. It is flexible to use and has a wide range of application scenarios, while relational databases only support basic types.
- Fast speed: Nosql can use hard disk or RAM as a carrier, while relational databases can only use hard disk;
- Low cost: The nosql database is easy to deploy and is basically open source software.
Disadvantage
- Does not provide sql support, the learning and use cost is high;
- The data structure is relatively complex and inconvenient for complex queries.
Why use NOSQL
The specific performance is the solution to the following three high problems:
High Performance-High concurrent access to the database
At the same time, there are concurrent access by a large number of users. It often reaches tens of thousands of read and write requests per second. Relational databases can barely handle tens of thousands of SQL queries, but hard disk IO can no longer handle tens of thousands of SQL write data requests.
-
For example, Tmall’s Double 11, from 0 am to 2 am, has reached tens of millions of visits per second.
-
During the 12306 Spring Festival, users kept checking whether there were remaining tickets when they went home to buy train tickets during the New Year.
Huge Storage-Huge data storage
The amount of data in the database is extremely large, and massive amounts of data are generated in the database tables every day.
Similar to QQ, WeChat, and Weibo, users generate massive user dynamics every day, generating tens of millions of records every day. For relational databases, performing SQL queries in a table with hundreds of millions of records is extremely inefficient and even intolerable.
High Scalability && High Availability- High scalability and high availability requirements
Expansion and upgrade of relational databases are troublesome. For many websites that need to provide 24-hour uninterrupted service, it is very painful to upgrade and expand the database system, which often requires downtime for maintenance and data migration.
Non-relational databases can be expanded by constantly adding server nodes, without the need to maintain the original database.
Redis catalog file
| Directory or file | effect |
|---|---|
| redis-benchmark.exe | A tool command for performance testing |
| redis-check-aof.exe | AOF file inspection and repair tool (AOF is one of its file storage formats) |
| redis-check-dump.exe | RDB file inspection and modification tool (RDB is one of its file storage formats) |
| redis-cli.exe | Client launcher |
| redis-server.exe | Server-side startup program (will not start automatically, by default it must be started manually each time) |
| redis.window.conf | Server configuration file |
string type operation command
5 data types of Redis
Redis is an advanced key-value storage system, where value supports five data types, which refers to the type of its value, and the key can be considered as a string type. Redis is not written in Java, it is written in C language.
| The data type of the value | Description |
|---|---|
| string type | String |
| list type | List: Elements can be repeated, and the elements are indexed and ordered |
| set type | Collection: The elements are not repeatable, the elements have no index number, and no order |
| hash type | Value consists of multiple key-value pairs |
| zset type | Collection: elements are not repeatable, each element has an index number, and a score value, which can be sorted according to the score |
String type string
Stored in binary in Redis, there is no encoding and decoding process.
Whether it is a string, integer, or floating point type, it will be written as a string.
The maximum data length that the string type value can hold in Redis is 512M, which is the most commonly used data type in the future.
Common commands
| command | Features |
|---|---|
| set key | Store the key and value of the string type, if the key does not exist, add it, if it exists, modify it |
| setnx key value | If the key does not exist, add it. If it does not exist, do nothing, and will not overwrite the previous key and value |
| get key | Get value by key |
| del key | Delete key and value by key |
List type operation commands
Overview
In Redis, the List type is a linked list of strings sorted in the order of insertion. Like a normal linked list in a data structure, we can add new elements to its left and right.
When inserting, if the key does not exist, Redis will create a new linked list for the key.
If all elements in the linked list are removed, then the key will also be deleted from the database.
The maximum number of elements that can be contained in the list is 4G (4.1 billion)
Common commands
| command | behavior |
|---|---|
| lpush key element element | Add 1 or more elements from the left |
| rpush key element element | Add 1 or more elements from the right |
| lpop key | Delete the leftmost element and return |
| rpop key | Delete the rightmost element and return |
| lrange key starts and ends | Find the elements in the specified index range and return, each element has 2 index numbers, index numbers from left to right: 0~length-1 index numbers from right to left: -1~-length If you want to get all the elements in the entire list, How to write the index number range? 0~-1 |
| llen key | Get how many elements are in the list |
Command demonstration
A Redis server can include multiple databases, and the client can only connect to a certain database in Redis, just like creating multiple databases in a mysql server, and the client specifies which database to connect to when connecting.
There are 16 databases numbered db0-db15 in Redis. We cannot create new databases, nor can we delete databases. There is no table structure in the database, and the client connects to the 0th database by default. But how many databases can be set through the configuration file.
Common methods of Jedis class
-
Each method is the command name in redis, and the parameters of the method are the parameters of the command.
-
Each Jedis object is similar to the Connection object in JDBC. Obtaining a Jedis object is essentially obtaining a connection object.
| Connect and close | Features |
|---|---|
| new Jedis(String host, int port) | Create a connection object parameter 1: host name parameter 2: port number 6379 |
| void close() | Close the connection |
| Methods of operating on string | Description |
|---|---|
| set(String key,String value) | Add string type keys and values |
| String get(String key) | Get value by key |
| del(String … keys) | Delete one or more keys and values |
| Methods of operating the list | Description |
|---|---|
| lpush(String key,String…values) | Add 1 or more elements from the left |
| List<String> lrange(String key,long start,long end) | 获取一个范围内所有的元素 |
代码
package com.itheima;
import redis.clients.jedis.Jedis;
/**
* Jedis的基本使用
*/
public class Demo1Base {
public static void main(String[] args) {
//1.创建Jedis连接对象
Jedis jedis = new Jedis("localhost", 6379);
//2.向服务器添加1个字符串类型的键和值
jedis.set("book","人鬼情喂鸟");
//3.从服务器中通过键获取值
String book = jedis.get("book");
//4.关闭连接
jedis.close();
//5.打印输出到控制台
System.out.println(book);
}
}
Jedis连接池的使用
jedis连接资源的创建与销毁是很消耗程序性能，所以jedis为我们提供了jedis的连接池技术，jedis连接池在创建时初始化一些连接对象存储到连接池中，使用jedis连接资源时不需要自己创建jedis对象，而是从连接池中获取一个资源进行redis的操作。使用完毕后，不需要销毁该jedis连接资源，而是将该资源归还给连接池，供其他请求使用。
Jedis连接池API
用于创建连接池的配置信息
| JedisPoolConfig配置类 | 功能说明 |
|---|---|
| JedisPoolConfig() | 构造方法，创建一个配置对象 |
| void setMaxTotal() | 连接池中最大连接数 |
| void setMaxWaitMillis() | 设置最长等待时间，单位是毫秒 |
| JedisPool连接池类 | 说明 |
|---|---|
| JedisPool(配置对象,服务器名,端口号) | 构造方法，创建连接池的类 参数1：上面的配置对象 参数2：服务器名 参数3：端口号 |
| Jedis getResource() | 从连接池中获取一个创建好的连接对象，返回Jedis对象 |
JedisPool的基本使用
需求：
使用连接池优化jedis操作，从连接池中得到一个创建好的Jeids对象，并且使用这个Jedis对象。向Redis数据库写入一个set集合，并且取出集合。打印到控制台，并且查看数据库中信息。
开发步骤
- 创建连接池配置对象，设置最大连接数10，设置用户最大等待时间2000毫秒
- 通过配置对象做为参数，创建连接池对象
- 从连接池里面获取jedis连接对象，执行redis命令。
- 执行redis命令写入list集合
- 执行redis命令读取list集合
- 输出读取的数据
- 关闭连接对象(通常连接池不关闭)
执行代码
package com.itheima;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.util.List;
/**
* Jedis连接池的基本使用
*/
public class Demo2Pool {
public static void main(String[] args) {
//1.创建连接池的配置对象
JedisPoolConfig config = new JedisPoolConfig();
//2.设置连接池的参数
config.setMaxTotal(10); //最大连接数
config.setMaxWaitMillis(2000); //最长等待时间为2秒钟
//3.创建连接池，使用上面配置对象
JedisPool pool = new JedisPool(config,"localhost", 6379);
//4.从连接池中获取连接对象
Jedis jedis = pool.getResource();
//5.使用连接对象
jedis.lpush("students", "孙悟空", "猪八戒", "白骨精");
List<String> students = jedis.lrange("students", 0, -1);
System.out.println(students);
//6.关闭连接对象
jedis.close();
}
}
小结
| JedisPool连接池类 | 作用 |
|---|---|
| JedisPool(配置对象,服务器名,端口号) | 创建连接池 参数1：配置对象 参数2：服务器名 参数3：端口号 |
| Jedis getResource() | 从连接池中获取连接对象 |
| void close() | 关闭连接池 |
ResourceBundle类的使用
代码
jedis.properties的内容
# 连接池的最大连接数
maxTotal=10
# 最长等待时间为2秒钟
maxWaitMillis=2000
# 服务器名字
host=localhost
# 端口号
port=6379
使用ResourceBundle类：
package com.itheima;
import java.util.ResourceBundle;
public class Demo3Resource {
public static void main(String[] args) {
//1. 通过静态方法读取属性文件，参数是：属性文件的主文件名，没有扩展名
ResourceBundle bundle = ResourceBundle.getBundle("jedis");
//2. 获取属性值，通过键获取值
String host = bundle.getString("host");
//3.输出值
System.out.println(host);
}
}
小结
| java.util.ResourceBundle类 | 功能 |
|---|---|
| static ResourceBundle getBundle(“配置文件基名”) | 读取配置文件，得到对象。参数是主文件名 |
| String getString(“键名”) | 通过键获取值 |
Jedis连接池工具类的实现
jedis.properties配置文件
# 主机名
host=localhost
# 端口号
port=6379
# 最大连接数
maxTotal=20
# 最长等待时间
maxWaitMillis=3000
JedisUtils.java
package com.itheima.utils;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;
import java.util.ResourceBundle;
/**
* Jedis连接池工具类
*/
public class JedisUtils {
private static JedisPool pool;
//在静态代码块中创建连接池
static {
//读取配置文件
ResourceBundle bundle = ResourceBundle.getBundle("jedis");
//读取属性值
int maxTotal = Integer.parseInt(bundle.getString("maxTotal"));
int maxWaitMillis = Integer.parseInt(bundle.getString("maxWaitMillis"));
int port = Integer.parseInt(bundle.getString("port"));
String host = bundle.getString("host");
//创建连接池配置对象
JedisPoolConfig config = new JedisPoolConfig();
//设置连接池的参数
config.setMaxTotal(maxTotal);
config.setMaxWaitMillis(maxWaitMillis);
//创建连接池
pool = new JedisPool(config, host, port);
}
/**
* 获取连接对象
*/
public static Jedis getJedis() {
return pool.getResource();
}
}
使用工具类
package com.itheima;
import com.itheima.utils.JedisUtils;
import redis.clients.jedis.Jedis;
//使用连接池工具类
public class Demo4Use {
public static void main(String[] args) {
//从连接池中获取连接对象
Jedis jedis = JedisUtils.getJedis();
//添加键和值
jedis.set("car", "BWM");
//取出
String car = jedis.get("car");
//输出
System.out.println(car);
//关闭连接
jedis.close();
}
}
持久化
利用永久性存储介质将数据进行保存，在特定的时间将保存的数据进行恢复的工作机制称为持久化 持久化用于防止数据的意外丢失，确保数据安全性
永久化介质保存数据,存在硬盘
RDB
直接拷贝数据从内存到磁盘上
RDB优点
-
RDB是一个紧凑压缩的二进制文件，存储效率较高
-
RDB内部存储的是redis在某个时间点的数据快照，非常适合用于数据备份，全量复制等场景
-
RDB恢复数据的速度要比AOF快很多
-
应用：服务器中每X小时执行bgsave备份，并将RDB文件拷贝到远程机器中，用于灾难恢复。
RDB缺点
-
RDB方式无论是执行指令还是利用配置，无法做到实时持久化，具有较大的可能性丢失数据
-
bgsave指令每次运行要执行fork操作创建子进程，要牺牲掉一些性能
-
Redis的众多版本中未进行RDB文件格式的版本统一，有可能出现各版本服务之间数据格式无法兼容现象
AOF
-
AOF(append only file)持久化：以独立日志的方式记录每次写命令，重启时再重新执行AOF文件中命令 达到恢复数据的目的。与RDB相比可以简单理解为由记录数据改为记录数据产生的变化
-
AOF的主要作用是解决了数据持久化的实时性，目前已经是Redis持久化的主流方式
Three strategies for AOF writing data (appendfsync)
-
always (every time): Each write operation is synchronized to the data in the AOF file with zero error, the performance is low, and it is not recommended.
-
everysec (per second): Synchronize the instructions in the buffer to the AOF file every second. In the case of sudden system downtime, the data within 1 second is lost. The data accuracy is high and the performance is high. It is recommended to use, and it is the default configuration
-
no (system control): The overall process of the cycle that is synchronized to the AOF file is not controlled by the operating system
RDB VS AOF
The confusion of choice between RDB and AOF
-
Very sensitive to data, it is recommended to use the default AOF persistence scheme
-
The AOF persistence strategy uses everysecond, fsync once every second. This strategy redis can still maintain good processing performance. When a problem occurs, data within 0-1 seconds can be lost at most.
-
Note: Due to the large storage volume of AOF files, the recovery speed is slow
-
The validity of the data presentation stage, it is recommended to use the RDB persistence scheme
-
The data can be well ensured that there is no loss in the stage (the stage is manually maintained by the developer or operation and maintenance personnel), and the recovery speed is relatively fast. The stage point data recovery usually adopts the RDB scheme
-
Note: The use of RDB to achieve compact data persistence will make Redis very low. Summarize carefully:
-
Comprehensive comparison
-
The choice of RDB and AOF is actually a trade-off. Each has its advantages and disadvantages, such as not being able to withstand data loss within a few minutes, and being very sensitive to business data. Choose AOF.
-
If you can withstand data loss within a few minutes and pursue the recovery speed of large data sets, choose RDB
-
RDB for disaster recovery
-
Double insurance strategy, open RDB and AOF at the same time, after restart, Redis will give priority to using AOF to recover data, reducing the amount of lost data
The above content is biased, please correct me!