Database basic finishing

MySQL

• Login: mysql -uroot -p
• Exit: exit;
• Start: /etc/inint.d/mysqld start
• Stop: /etc/inint.d/mysqld stop
• Restart: /etc/inint.d/mysqld restart

User addition and deletion check and authorization

# 修改当前登录用户密码
set password = password('新密码');

# 查看用户
select user,host from mysql.user;

# 创建用户
create user '用户名'@'loaclhost' identified by '密码';
# 授权
grant all on 数据库名.* to '用户名'@'loaclhost';
# 授权root权限[ with grant option ] 代表此用户可以给其他用户授权
grant all on *.* to '用户名'@'localhost' with grant option;

# 修改用户密码
set password for 用户名@localhost = password('新密码')

# 删除用户
drop user 用户名@'%';

# 刷新
flush privileges;

Data Definition Language (DDL)

• View library:, show databases;will display all the databases on the current server
• Create a library:, create database test charset=utf8;create a database test, specify the character set
• Delete the library:, drop database test;delete the database test
• Select the library:, use test;select the database test
  • View the current database
    • show tables;
    • select database();
• View all tables:, show tables;view all data tables under the current database
• Create a table:create table user(username varchar(20), password char(32));
• View the table structure:desc user;
• View creation method:
  • View library:show create database test;
  • View the table:show create table user;
• Delete the table:drop table user;
• Specify the character set:
  • Specify at creation:create table xxx() charset=utf8;
• Modify the field:
  • alter table user modify username varchar(30);, The field name cannot be modified
  • alter table user change email em varchar(32);, You can modify the field name
• Add fields:
  • The default is added at the end:alter table user add age int(3);
  • Add after the specified field:alter table user add email varchar(60) after password;
  • Add a field at the beginning:alter table user add id int(11) first;
• Delete field:alter table user drop age;
• Modify the location and name of the field: add / modify / change
  • alter tablr user modify em varchar(32) first;
• Modify the table name:alter table user rename new_user;

Data Manipulation Language (DML)

• Note: In most operations, CURD is used.

• Prepare: a table for testing

  mysql> create table tb_name(
      -> id int auto_increment,
      -> name varchar(20) not null,
      -> money float not null,
      -> province varchar(20) default null,
      -> age tinyint unsigned not null,
      -> sex tinyint not null,
      -> primary key(id)
      -> )engine=innodb default charset=utf8;
  

• Insert data:

  • Method 1: No fields are specified, all fields need to be written when adding data

  insert into tb_name values(1, '刘亦菲', 20000000, '武汉', 28, 1);
  

  You can insert multiple pieces of data at once, and one piece of data needs to be included using one ().

  • Method 2: Specify the field, only need to pass the value of the specified field

  insert into tb_name(name, money, age, sex, province) values('赵丽颖', 8000000, 31, 1, '河北');
  

  The order of inserting data must be consistent with the field names specified earlier.

  • Note: when inserting data, you can leave the field without value
    • Self-increasing field
    • With default
    • Can be empty
  • Note: When inserting data, the second method is often used.

• Query data:select * from tb_name;

• Query specified data:select * from tb_namewhere id=1;

• change the data:update tb_nameset age=31, money=10000000 where id=1;

  • Warning: Do not forget to specify the conditions for the modification operation, otherwise the consequences will be at your own risk.

• delete data:delete from tb_namewhere id=2;

  • Warning: The delete operation must not forget to specify the conditions, otherwise the consequences will be at your own risk.

Query Data Language (DQL)

• Basic query:select * from tb_name;

• Specify field query:select name, money, province from tb_name;

• Delete duplicate records:select distinct province, sex from tb_name;

  • The field specified by distinct cannot be repeated. When specifying multiple fields, the field combination cannot be repeated

• Specify condition query:

• • Examples:

  select * from tb_name where age > 30;
  select * from tb_name where age > 30 and sex=1;
  select * from tb_name where age > 30 or province='河南';
  select * from tb_name where age [not] between 30 and 40;
  select * from tb_name where id [not] in(1,3,5);
  select * from tb_name where province like '%北%';  # 包含北 的地区
  select * from tb_name where province like '北_';  # 北x 的地区
  select * from tb_name where province like '*北';  # 以北 开头的地区
  select * from tb_name where length(name)<6;
  

• Wildcard: use like fuzzy query

  • %: Represents any number of characters, can match any type and length of characters
  • _: Represents a single arbitrary character, commonly used to limit the character length of expressions (can represent a Chinese character)
  • *: Start or end with a certain character

• Sort the result set:

  • Examples:

   select * from tb_name order by age asc;	
  select * from tb_name order by money desc;
  

  • • Multi-field sorting: when the current field value is the same, sort according to the field specified later

    select * from tb_name order by sex asc, age desc;
    

  • Limit the result set:

    • limit: used to limit the number of result sets

  • Examples:

  select * from tb_name limit 5;						# 取前5条数据
  select * from tb_name limit 5 offset 2;			# 偏移2条，取5条数据
  select * from tb_name limit 2, 5; 					# 功能同上
  省略offset的写法  就是将偏移量放到前面  将限制每页条数的放在后面
  

  • Paging query:

  每页10条数据,用pageSize，page是当前页面，请写出对于页码的查询条件
  第一页：limit 10
  第二页：limit 10, 10
  第三页：limit 20, 10
  page页：limit (page-1) * pageSize, pageSize
  
  

• Common aggregation functions:

  function	Explanation
  count	Statistics
  sum	Sum
  avg	average value
  max	Maximum
  min	Minimum value

• Examples:

  select count(*) as c from tb_name;
  select max(money) as max_money from tb_name;

as can alias the query structure field

• Grouping and filtering:

  • Examples:

  select * from tb_name group by sex;		# 只分组，结果集中每个组中只有一条数据
  select count(*) as c, sex from tb_name group by sex;	# 分组并统计信息
  select count(*) as c, province from tb_name group by province having c>1; 
  having后面做的是分组后的显示过滤
  # 分组后条件不能使用where，只能使用having
  
  

Create a multi-table relationship:

foreign key(o_buyer_id)references s_user(u_id),

foreign key(o_seller_id)references s_user(u_id)

alter table emp add constraint foreign key emp(deptno) refetences dept(deptno);

FOREIGN KEY(o_buyer_id) REFERENCES s_user(u_id),

FOREIGN KEY(o_seller_id) REFERENCES s_user(u_id)

ALTER TABLE EMP ADD CONSTRAINT FOREIGN KEY EMP(DEPTNO) REFERENCES DEPT 	(DEPTNO);

Multi-table joint query

select distinct: Query clear

• Implicit internal connection: no joinkeywords appear

  • Examples:select username, name from user, goods where user.gid=goods.gid;
  • Description: See which user bought which product

• Explicit inner join: a joinkeyword appears in the SQL statement

  • Examples:select username,name from user join goods on user.gid=goods.gid;
  • Description: Same function as above
  • You can add inneror crosskeyword before join , you can omit

• Left outer connection: mainly to the left table,left outer

  • Examples:select username, name from user left outer join goods on user.gid=goods.gid;
  • Description: Display all the data in the left table, display the data in the right table, and display NULL if there is no corresponding

• Right outer connection: live on the right table,right outer

  • Examples:select username, name from user right outer join goods on user.gid=goods.gid;
  • Description: All data in the table is displayed, the data is displayed in the left table, and NULL is not displayed in the corresponding table

• Sub (nested) query

  select * from user where gid in (1,3,5);
  select * from user where gid in (select gid from goods);
  
  

• Record union

  • grammar:select 语句1 union select 语句2
  • Description:
    • union all: directly stitch together the query results on both sides
    • union: is the deduplication result of union all

• Update data jointly

  • Examples:update user u, goods g set u.gid=0, g.price=g.price-200 where u.gid=g.gid and u.id=7;
  • Description: You can alias the operation table, and you can only perform operations on the original field

Transaction Processing Language (DTL)

• Description: The storage engine of the tested table should support transactions (InnoDB)

• Open transaction: prohibit automatic submission

  • set autocommit=0;

• Operation rollback: usually used when an operation exception occurs

  • rollback;

• Commit operation: no problem during the entire transaction

  • commit;

Start transaction:
start transaction

Commit transaction:
commit

Rollback operation:
rollback

Data Control Language (DCL)

• View authorization:

  • format:show grants [for 'user'@'host'];
  • Examples:show grants for 'root'@'localhost';
  • Description: You can not specify the user and host when viewing the authorization of the currently logged in user

• Create user:

  • format:create user 'user'@'host' identified by 'password';
  • Examples:create user 'test'@'10.8.156.%' identified by '123456';
  • Explanation:% means wildcard, any

• User authorization:

  • format:grant 权限 privileges on 库名.表名 to 'user'@'host' identified by 'password';
  • Examples:grant all privileges on *.* to 'test'@'10.8.156.%' identified by '123456';
  • Note: The permissions can be select, delete, insert, update, etc., all means all permissions; * means all

• Refresh permissions:flush privileges;

• Cancel authorization:

  • format:revoke 权限 privileges on 库名.表名 from 'user'@'host';
  • Examples:revoke delete privileges on test.* from 'test'@'10.8.156.%';
  • Description: Withdraw the delete permission of all tables under the test database of the test user in the current LAN

• delete users:

  • format:drop user 'user'@'host';
  • Examples:drop user 'test'@'10.8.156.%';

• Remote login under linux:

  • Modify the /etc/mysql/mysql.conf.d/mysqld.cnf file,

  将bind-address=127.0.0.1改为：bind-address=0.0.0.0

  • Add the operation authority of the specified user on the specified host:

  grant all privileges on *.* to 'root'@'%' identified by '123456';
  
  

Backup and restore

• Backup:
  • Description: It is to save the data (SQL statement) in the database to a file
  • Examples:mysqldump -uroot -p test > test.sql
• restore:
  • Description: parse the file that holds the SQL statement and execute the SQL statements one by one
  • Examples:mysql -uroot -p test < test.sql

Python operation MySQL

• Install the extension:pip install pymysql
• View the installation package:pip list

import MySQLdb  # sqlite3

# 打开数据库连接
db = MySQLdb.connect("localhost", "testuser", "test123", "TESTDB", charset='utf8' )

# 使用cursor()方法获取操作游标 
cursor = db.cursor()

# 使用execute方法执行SQL语句
cursor.execute("SELECT VERSION()")

# 使用 fetchone() 方法获取一条数据；fetchall()获取全部；fetchmany()获取一条，返回列表
data = cursor.fetchone()

print("Database version : %s " % data)

# 关闭数据库连接
db.close()

Redis

Introduction to Redis

• Redis is an open source log-type, Key-Value database written in ANSI C language, supporting network, memory-based and persistent, and provides API in multiple languages.
• A type of non-relational database that is often used as a cache database server.
• Official website: http://www.redis.io/ Chinese website: http://www.redis.cn/
• Has rich data types: string, hash, list, collection, ordered collection
• Port: 6379

Features of redis:

Redis支持数据的持久化，可以将内存中的数据保存在磁盘中，重启的时候可以再次加载进行使用。
Redis不仅仅支持简单的key-value类型的数据，同时还提供list，set，zset，hash等数据结构的存储。
Redis支持数据的备份，即master-slave模式的数据备份
性能极高  支持每秒读的次数为110000次/s,写的速度是81000次/s

Redis commonly used commands

• Common management commands:

  sudo service mongod start：重启服务
  mongo：连接redis
  ping：测试连接情况，默认恢复'PONG'
  quit/exit：退出客户端
  auth：密码认证
  config：配置命令，可以查看和设置配置信息，参考密码查看与设置
  info：查看服务器信息，可以指定具体模块查看
  command：查看支持的命令
  
  select：选择库，总共16个，默认为0数据库；提示符后会有库号的提示，没有就是默认的库(0)
  dbsize：查看当前库key的数量
  flushdb：清空当前库
  flushall：清空所有库
  save：前台执行持久化存储
  bgsave：后台执行持久化存储
  
  

• Keys:

  exists：判断指定的键是否存在
  del：删除指定的键值对
  keys *：查看指定样式的键，keys *表示查看所有
  ttl：查看指定键的剩余有效时间，单位为秒，不存在键返回-2，永久返回-1
  expire：设置指定键的生存时间
  persist：移除指定键的生存时间，之后该键永久有效
  move：移动指定的键到指定的库
  rename：修改指定键的名字
  
  

• String

  set：设置，存在就修改，不存在就添加
  get：获取，获取指定键的值
  mset：同时设置多个键值对
  mget：同时获取多个键对应的值
  getset：设置新值，同时返回旧值
  setex：设置值及过期时间，单位为秒
  incr：数字值加1
  decr：数字值减1
  incrby：数字值加上指定值
  decrby：数字值减去指定值
  incrbyfloat：数字值加上一个浮点数
  append：键存在，值为字符串，追加指定的内容到原值的末尾
  strlen：返回键对应值的字符长度
  
  

• Hash

  hset：设置单个属性
  hget：获取单个属性
  hmset：设置多个属性
  hmget：获取多个属性
  hgetall：获取所有属性
  hexists：判断指定键是否有指定的字段
  hdel：删除指定键的指定字段
  hkeys：获取指定键的所有字段
  hvals：获取指定键的所有字段值
  hlen：获取指定键的字段个数
  hincrby：在指定键的指定字段上增加一个整数
  hincrbyfloat：在指定键的指定字段上增加一个浮点数
  
  

• List

  lpush：从左边(头部)插入数据
  lpop：从左边(头部)删除并返回数据
  lrange：获取区间内的数据，0 -1通常可以表示所有范围
  lindex：根据索引获取元素
  llen：统计列表中元素个数
  lset：设置指定索引的元素值
  lrem：删除指定数量的元素
  linsert：在指定元素的前/后插入元素（before）
  ltrim：修剪列表元素(保留指定范围内地，删除其他)
  rpop：从右边(尾部)删除并返回数据
  rpush：从右边(尾部)插入数据
  
  

• set

  sadd: 向集合中添加一个元素
  scard: 获取集合中的成员数
  smembers:返回集合中的所有成员
  spop:移除并返回集合中的一个随机元素
  
  

Redis publish and subscribe:

Redis 发布订阅(pub/sub)是一种消息通信模式：发送者(pub)发送消息，订阅者(sub)接收消息。
Redis 客户端可以订阅任意数量的频道。

当有新消息通过 PUBLISH 命令发送给频道 channel1 时， 这个消息就会被发送给订阅它的三个客户端：
创建订阅频道名
subscribe name
通过publish name去往频道名字里面推送信息
然后客户端就能收到此订阅信息

type of data

• String, hash, list

• Set

  sadd：向集合中添加元素(集合不存在会自动创建)
  scard：统计集合中元素的个数
  smembers：返回所有元素
  sismember：判断某个元素是否是集合的成员
  spop：随机删除并返回指定个数的元素，不指定个数默认一个元素
  srandmember：随机获取指定个数的元素
  srem：移除指定元素
  smove：将指定的元素从一个集合移动到另一个集合中
  sinter：求交集
  sinterstore：求交集并保存到一个集合中
  sdiff：求差集
  sdiffstore：求差集并保存到一个集合中
  sunion：求并集
  sunionstore：求并集并保存到一个集合中
  
  

• Sorted set

  zadd：添加元素
  zcard：统计元素个数
  zcount：指定分数区间统计
  zrange：返回指定索引范围内的元素(升序)，zrevrange：降序
  zrangebyscore：返回指定分数区间的元素(升序)，zrevrangebyscore：降序
  zrank：返回指定元素的索引(升序)，zrevrank：降序
  zrem：移除元素
  zscore：返回元素的分数
  zincrby：将元素的分数增加一个值
  zinterstore：求交集并保存
  zunionstore：求并集并保存
  
  

Python operation Redis

• Install the extension:pip install redis

• Introduction:

  redis扩展库中有两个类，Redis和StrictRedis；StrictRedis实现了官方的命令，Redis是它的子类，兼容老版本。扩展中没有提供select方法，可以在连接时通过参数指定库。
  
  

• operating:

  • Simple connection: import the class library, create Redis objects, then all the naming methods are Redis objects
  • Connection pool: Multiple Redis use the same connection pool connection, reducing the overhead of frequent connection and disconnection of the database
  • Use pipes: You can record multiple commands and then send them to the server at one time, avoiding the overhead of operating the server multiple times

Redis installation:

第一种方法:
	1.首先在官网上下载redis最新的版本，命令如下：

	wget http://download.redis.io/releases/redis-4.0.8.tar.gz

	2.解压下载的redis压缩包
	tar -vxzf redis-4.0.8.tar.gz
	
	3.进入src目录并make
	make
	4.编译完成后可以使用如下命令进行测试
	make test
	注：如果make过程中出现了如下错误：
	error: jemalloc/jemalloc.h: No such file or directory 
	就执行make MALLOC=libc 就OK了，不过在执行这一句之前最好先执行 
	make clean清理一下。

	启动方法  在src目录下   ./redis-server

第二种方法:
	sudo apt-get update
	sudo apt-get install redis-server
	启动 Redis
	 redis-server
	查看 redis 是否启动？
	 redis-cli
	以上命令将打开以下终端：
	redis 127.0.0.1:6379>
	127.0.0.1 是本机 IP ，6379 是 redis 服务端口。现在我们输入 PING 命令。
	redis 127.0.0.1:6379> ping
	PONG

	有时候会有中文乱码  连接的时候需要  redis-cli --raw

MongoDB

Introduction to MongoDB

• MongoDB is a database based on distributed file storage. Written by C ++ language. Designed to provide scalable high-performance data storage solutions for WEB applications.

• It is a product between relational database and non-relational database. It is the most similar relational database among non-relational databases and the most feature-rich non-relational database.

• Installation and uninstallation:
  1. Import the public key of the software source

  sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
  
  

  2.为mongodb创建软件源list文件 
  ubuntu12.04:
  echo "deb http://repo.mongodb.org/apt/ubuntu precise/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
  
  ubuntu14.04:
  echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
  
  ubuntu16.04:
  echo "deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
  
  3.更新软件源并安装mongodb
  sudo apt-get update
  sudo apt-get install -y mongodb-org
  
  	  如果想要安装特定的版本，使用下面命令：
  	  sudo apt-get install -y mongodb-org=3.2.9 mongodb-org-server=3.2.9 mongodb-org-shell=3.2.9 mongodb-org-mongos=3.2.9 mongodb-org-tools=3.2.9
  4.配置启动文件 
  如果是ubuntu16.04的版本，需要手动新建/lib/systemd/system/mongod.service文件，并写入下面内容：
  [Unit]
  Description=High-performance, schema-free document-oriented database
  After=network.target
  Documentation=https://docs.mongodb.org/manual
  
  [Service]
  User=mongodb
  Group=mongodb
  ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
  
  [Install]
  WantedBy=multi-user.target
  
  

	5.启动、重启和关闭命令
	sudo service mongod start
	sudo service mongod restart
	sudo service mongod stop

	6.mongodb的完全卸载 

	先停止运行mongodb
	sudo service mongod stop

	再卸载软件
	sudo apt-get purge mongodb-org*

	删除数据库和日志文件
	sudo rm -r /var/log/mongodb
	sudo rm -r /var/lib/mongodb

	如果想要使用远程连接mongo  那么需要
	sudo vim /etc/mongo.conf
	找到net   将 127.0.0.1  改为0.0.0.0即可

	连接到mongodb
	直接在终端输入 mongo

Comparison between MySQL and MongoDB

• Related concepts

  MySQL	MongoDB
  database	db (database)
  table (data table)	collection
  row	document
  column	field (field / field)

db operation

• View all databases:, show dbsif there is no data in the database, it will not be displayed
• View the current database: dbordb.getName()
• Create and switch databases:use python1806
  • There is a direct switch in the database, and there is no switch after creation
  • When there is no data in the database, it show dbswill not be displayed
• Delete the current database:db.dropDatabase()
• View help:help
• Exit: exitorquit()

collection operation

• View all collections under the current database:show collections
• Create a collection:
  • Simply create:, db.createCollection('user')will create an empty collection
  • Create on demand:, db.stu.insert({name:'shuorui', age:30})directly operate the non-existent collection, the system will automatically create
• Delete collection:db.stu.drop()

document operation

• Add documentation:

  • insert

  # 插入一条数据
  db.user.insert({name:'jiwei', age:29, height:175, isDelete:0})
  # 插入多条数据
  db.user.insert([{name:'zhihui',age:18, height:168, isDelete:0},{name:'minghui', age:28, height: 173, isDelete:0}])
  
  

  • save

  # 保存的数据没有_id字段，会插入一个新的文档
  db.user.save({name:'wenke', age:40, height:178, isDelete:0})
  # 保存的数据有_id字段，会修改对应的数据
  db.user.save({_id:ObjectId("5be3ecf105c088a265a0b578"), name:'wenke',age:30, height: 178, isDelete:0})
  
  

• Update documentation

  • save

  # 保存的数据有_id字段，会修改对应的数据，就相当于更新操作
  db.user.save({_id:ObjectId("5be3ecf105c088a265a0b578"), name:'wenke',age:25, height: 178, isDelete:0})
  
  

  • update：db.集合.update(query, update, {upsert:<boolean>, multi:<boolean>})
    • query: indicates the query condition
    • update: means to update the content of settings
      • $ set: indicates setting
      • $ inc: indicates an increase
    • upsert: the updated query result does not exist, is it inserted as new data, the default is false
    • multi: whether all the eligible data is updated; true means all updates, false means one update, the default is false

  db.user.update({age: 25}, {$set:{age:26}})			# 只会更新一条
  
  db.user.update({height:175}, {$inc:{height: 1}}, {multi: true})		# 会更新全部
  db.user.updateMany({isDelete: 1}, {$set:{isDelete: 0}})		# 更新全部
  
  

• Delete document

  • remove：db.集合.remove(query，{justOne:<boolean>})

  db.user.remove({height: 176})		# 默认删除所有数据
  db.user.remove({isDelete:0}, {justOne:true})	# 只会删除一条数据
  
  

  • delete: Official recommendation

  db.user.deleteOne({isDelete: 0})		# 删除一条数据
  db.user.deleteMany({isDelete: 0})		# 删除所有数据
  
  

• Query document

  • format:db.集合.find(query, {key1:0/1, key2:0/1， keyn:0/1})
    • query: query condition
    • key: indicates the field name
    • The second parameter is either 1 for both displayed fields or 0 for excluded fields

  db.user.find({},{name:1, age:1}).pretty()			# 只显示name和age
  db.user.find({},{name:0, age:0}).pretty()			# 不显示name和age
  
  

  • pretty (): formatted display data (similar to dictionary display)

  db.user.find().pretty()
  
  

  • findOne

   db.user.findOne({isDelete:0})	# 只返回条数据
  
  

Virtual environment

什么是虚拟环境:
	一个隔离了外界干扰的,独立的环境
虚拟环境的搭建:
1.pip install virtualenv
2.pip install virtualenvwrapper
3.whereis virtualenvwrapper.sh
4.source 上面的路径信息
5.source ~/.bashrc

mkvirtualenv name   创建虚拟环境名称(name就是自定义的虚拟环境名字)
rmvirtualenv name   删除虚拟环境名称
workon name 使用某个虚拟环境
deactivate  退出虚拟环境

6.重新打开终端  然后 sudo vim ~/.bashrc

export=/home/huguanyu/.virtualenvs
source /usr/share/virtualenvwrapper

然后强制保存

7.重新source ~/.bashrc

pip install --user virtualenv