To prove safety data warehousing -Shell command three

First, Last Course Review

Two, Linux basic commands three

• 2.1, users and groups
• 2.2 Use personal environment variables (choice of .bashrc and .bash_profile) and global environment variables (/ etc / profile) and aliases
• 2.3, su sum su - target-ku, separately
• The focus of analysis 2.4, / etc / passwd file
• 2.5, the permissions (chmod, chown) && case presentations
• 2.6, file, folder view size
• 2.7, find the search command

Third, work

First, Last Course Review

• Bowen connection: https: //blog.csdn.net/SparkOnYarn/article/details/104793896

• Future components of the deployment of large data which must be about whether to take effect under verification, under caution there is no problem; in fact aliases to simplify operations; work with the delete command frequency is relatively high, be regarded as high-risk command; history view the history, you can command execution history;

Two, Linux basic commands two

2.1, users and groups

1. Create a user

1、关于用户、用户组的几种使用方式：增加、删除、修改
[root@hadoop001 sbin]# ll /usr/sbin/user*
-rwxr-x--- 1 root root 118192 Nov  6  2016 /usr/sbin/useradd
-rwxr-x--- 1 root root  80360 Nov  6  2016 /usr/sbin/userdel
-rwxr-x--- 1 root root 113840 Nov  6  2016 /usr/sbin/usermod
-rwsr-xr-x 1 root root  11296 Apr 13  2017 /usr/sbin/usernetctl
[root@hadoop001 sbin]# ll /usr/sbin/group*
-rwxr-x--- 1 root root 65480 Nov  6  2016 /usr/sbin/groupadd
-rwxr-x--- 1 root root 57016 Nov  6  2016 /usr/sbin/groupdel
-rwxr-x--- 1 root root 57064 Nov  6  2016 /usr/sbin/groupmems
-rwxr-x--- 1 root root 76424 Nov  6  2016 /usr/sbin/groupmod

2、我们创建一个用户为ruoze，默认创建这个名称的用户组ruoze，且设置这个用户的主组还是ruoze
[root@hadoop001 sbin]# useradd ruoze
[root@hadoop001 sbin]# id ruoze
uid=1001(ruoze) gid=1001(ruoze) groups=1001(ruoze)

//举例：老板的小姨子既属于财务部，又属于人力资源部；但是主要部门是财务部==》类比于财务部就是主组

2. User and group storage directory:

1、用户存储文件目录：
cat /etc/passwd

2、用户组存储文件目录：
cat /etc/group

3、创建的用户默认会在home目录下创建自己的目录：
[root@hadoop001 home]# ll
total 8
drwx------ 7 hadoop hadoop 4096 Mar 11 16:18 hadoop
drwx------ 2 ruoze  ruoze  4096 Mar 12 10:47 ruoze
权限			所属用户	所属用户组	文件夹大小不准	创建时间（ll -rt进行时间排序）

3. Test delete a user:

1, because when ruoze this group, only a ruoze user, so the user deletes ruoze time will also delete ruoze this user group, other members of the group, this group will not be deleted ruoze

[root@hadoop001 home]# id ruoze
uid=1001(ruoze) gid=1001(ruoze) groups=1001(ruoze)
[root@hadoop001 home]# userdel ruoze
[root@hadoop001 home]# id ruoze
id: ruoze: no such user
[root@hadoop001 home]# cat /etc/passwd | grep ruoze 
[root@hadoop001 home]# cat /etc/group | grep ruoze
[root@hadoop001 home]# ll
total 8
drwx------ 7 hadoop hadoop 4096 Mar 11 16:18 hadoop
drwx------ 2   1001   1001 4096 Mar 12 10:47 ruoze

4. After deleting ruoze users continue to try to add a user, how to style after loss recovery

1、这段话的意思是home目录下的.bash文件已经存在，用户和用户组目录都已经存在了ruoze用户，
[root@hadoop001 home]# useradd ruoze
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
Creating mailbox file: File exists
[root@hadoop001 home]# cat /etc/passwd|grep ruoze
ruoze:x:1001:1001::/home/ruoze:/bin/bash
[root@hadoop001 home]# cat /etc/group | grep ruoze
ruoze:x:1001:

2、使用root用户切换到ruoze用户，删除.bash隐藏文件
[root@hadoop001 ~]# su - ruoze
[ruoze@hadoop001 ~]$ ll -a
total 20
drwx------  2 ruoze ruoze 4096 Mar 12 10:47 .
drwxr-xr-x. 4 root  root  4096 Mar 12 10:47 ..
-rw-r--r--  1 ruoze ruoze   18 Dec  7  2016 .bash_logout
-rw-r--r--  1 ruoze ruoze  193 Dec  7  2016 .bash_profile
-rw-r--r--  1 ruoze ruoze  231 Dec  7  2016 .bashrc
[ruoze@hadoop001 ~]$ rm -rf .bash*
[ruoze@hadoop001 ~]$ ll -a
total 8
drwx------  2 ruoze ruoze 4096 Mar 12 11:17 .
drwxr-xr-x. 4 root  root  4096 Mar 12 10:47 ..

3、删除完.bash隐藏文件后新开session，再切换到ruoze用户，发现样式丢失==>说白了就是丢失.bash文件
[root@hadoop001 ~]# su - ruoze
Last login: Thu Mar 12 11:17:05 CST 2020 on pts/1
-bash-4.2$ 
-bash-4.2$ 

4、样式丢失的话怎么办呢？
//拷贝这个目录下的文件到ruoze用户目录
[root@hadoop001 skel]# cp /etc/skel/.* /home/ruoze/
[root@hadoop001 skel]# chown -R ruoze:ruoze /home/ruoze/.bash*

//最简单的就是直接拷贝其它用户下的.bash文件到ruoze目录下，并且修改用户和用户组

5. Increase the user group

1、增加bigdata用户组
groupadd bigdata

2、添加ruoze用户到bigdata用户组
[root@hadoop001 skel]# id ruoze
uid=1001(ruoze) gid=1001(ruoze) groups=1001(ruoze)
[root@hadoop001 skel]# usermod -a -G bigdata ruoze
[root@hadoop001 skel]# id ruoze
uid=1001(ruoze) gid=1001(ruoze) groups=1001(ruoze),1002(bigdata)

3、usermod --help查看命令帮助
Usage: usermod [options] LOGIN

-g, --gid GROUP               force use GROUP as new primary group
强制使用组作为新组

-G, --groups GROUPS           new list of supplementary GROUPS
新增补充组的名单

-a, --append                  append the user to the supplemental GROUPS
                                mentioned by the -G option without removing
                                him/her from other groups
将用户追加到补充组中，将-G选项中提到，但没有删除来自其它组的他 

6. Requirements: Modify bigdata to ruoze user's primary group, the user must simultaneously ruoze in ruoze this group:

1、没有修改前，ruoze用户的组和主组是ruoze，还在bigdata组中
[ro[root@hadoop001 skel]# id ruoze
uid=1001(ruoze) gid=1001(ruoze) groups=1001(ruoze),1002(bigdata)

2、强制修改ruoze用户的组和主组
[root@hadoop001 skel]# usermod -g bigdata ruoze
[root@hadoop001 skel]# id ruoze
uid=1001(ruoze) gid=1002(bigdata) groups=1002(bigdata)

3、ruoze用户的主组是bigdata，把ruoze这个组另外加进去
[root@hadoop001 skel]# usermod -a -G ruoze ruoze
[root@hadoop001 skel]# id ruoze
uid=1001(ruoze) gid=1002(bigdata) groups=1002(bigdata),1001(ruoze)

2.2, set a password for the average user, sudo privileges to ordinary users

1, set a password for the user, the root user to set:

[root@hadoop001 skel]# passwd ruoze
Changing password for user ruoze.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@hadoop001 skel]# su - ruoze
Last login: Thu Mar 12 11:26:07 CST 2020 on pts/1

2, set a password for the average user is mainly to be used when the remote connection:

• file -> quick connetc -> Enter the hostname

3, it is not behind passwd to change the password for the current user with the name

• Changing password for ruoze.
  (current) UNIX password:

The maximum authority to the average user root and no password

 1、使用root用户，vi /etc/sudoers，编辑这个文件，找到如下行数在91行后添加如下信息：hadoop是我的用户，NOPASSWD:ALL意思是无需密码
 90 ## Allow root to run any commands anywhere（允许root用户在任何地方执行命令）
 91 root    ALL=(ALL)       ALL
 92 hadoop  ALL=(ALL)       NOPASSWD:ALL

2.3, su sum su - target-ku, separately

1, su sum su - target-ku, separately

1、使用su的时候切换到ruoze用户还是在原用户的目录
[root@hadoop001 ~]# su ruoze
[ruoze@hadoop001 root]$ pwd
/root
[ruoze@hadoop001 root]$ exit;
exit

2、su - 的时候代表着切换的时候也在ruoze用户的家目录，且执行ruoze用户的环境变量文件
[root@hadoop001 ~]# su - ruoze
Last login: Thu Mar 12 11:56:52 CST 2020 on pts/2
[ruoze@hadoop001 ~]$ pwd
/home/ruoze

2, how to verify su - ruoze, will perform ruoze user environment variables, and su ruoze not take effect

此处验证到的一点是.bash_profile文件，su ruoze不会执行
.bashrc 对于su ruoze和su - ruoze都会执行

• It concluded, after we switch the user to use the su - user environment variables to configure this in .bashrc configuration file.

2.4, / etc / passwd focus of analysis

1, View modify / etc / passwd file contents:

1、查看文件的最后两行
ldap:x:55:55:OpenLDAP server:/var/lib/ldap:/sbin/nologin
ruoze:x:1001:1002::/home/ruoze:/bin/bash

2、修改/bin/bash为/bin/false和/sbin/nologin
修改为这两种以后都是不能登录的

2, the production environment to explain:

在CDH中有很多的组件：
hdfs组件有hdfs用户
hbase组件有hbase用户，
yarn组件有yarn用户；
存在的情况：切换用户切换不过去，只要对应的在/etc/passwd文件中修改最后一行内容为/bin/bash，这样才是没有问题的。

2.5, the permissions (chmod, chown) && case presentations

chmod -R 777 file or folder
chown -R User: User Group file or folder

Linked with the permission error: Permission denied

1、举例：
[root@hadoop001 hadoop]# ll
total 20
drwxrwxr-x 2 hadoop hadoop 4096 Mar  5 15:58 app

rwx 第一组权限之和为7，代表的是文件和文件夹的用户root，它对这个文件有读写执行的权限
rwx  第二组权限之和为7，代表文件或文件夹的用户组root，读写执行
r-x 第三组权限之和为5，代表其他组的所属用户对这个文件或文件夹的权限，具有读和执行的权限

chmod

d：directory目录
r：read读，代表的数字4
w：write写，代表的数字是2
x：执行，代表的数字是1

7 = 4 + 2 + 1，代表r + w + x

777代表着任意的用户和用户组都有读写执行权限

Case:
There are a linux on the / tmp directory, permission to the / tmp directory is 777, we create a rz.log file in the / tmp directory;

1、查看/tmp目录，查看创建的rz.log文件权限
drwxrwxrwt. 12 root root       4096 Mar 12 17:08 tmp

2、查看rz.log的文件权限：
-rw-r--r-- 1 root root   24 Mar 12 17:05 rz.log

3、我们测试使用root用户是没有问题的，但是使用hadoop用户进入编辑，编辑rz.log的时候提示如下： warning： changing a readonly file，使用wq!强制保存的时候提示如下：can't open file for writing.


4、提示如3中的不能写入文件的时候，我们直接输入ctrl+z来中断写入，此时相当于产生了一个缓存文件，我们ll -a查看后进行删除，rm -f .rz.log.swap
-rw-r--r--   1 root   root      24 Mar 12 17:05 rz.log
-rw-------   1 hadoop hadoop 12288 Mar 12 17:10 .rz.log.swp

5、我们修改vi.log的权限：chmod -R 746 rz.log
此时再使用Hadoop用户进行编辑：保存退出就没有问题了，
因为id hadoop查看到hadoop用户不是root用户，也不是root用户所在的组，而是属于其它用户用户组，所以赋予746权限：
[root@hadoop001 tmp]# id hadoop
uid=1000(hadoop) gid=1000(hadoop) groups=1000(hadoop)

Note that the implementation of the production ctrl + z, corresponding to the broken file directly, will produce swap file, then directly ll -a, then delete the file.

• We tested the hadoop user to the root user group:

1、原先hadoop所属的用户和用户组：
[root@hadoop001 tmp]# id hadoop
uid=1000(hadoop) gid=1000(hadoop) groups=1000(hadoop)

2、把hadoop这个用户加到root这个用户组中：
[root@hadoop001 tmp]# usermod -a -G root hadoop
[root@hadoop001 tmp]# id hadoop
uid=1000(hadoop) gid=1000(hadoop) groups=1000(hadoop),0(root)

3、/tmp/rz.log的这个文件还是644权限，此时测试编辑能不能用
仍然提示不能保存

4、修改rz.log的权限为chmod -R 664 rz.log进行测试：
需要新开一个session或者重新连接当前窗口，vi rz.log能够顺利写入并且保存退出。

2.6, file, folder view size

1, view file size:

[root@hadoop001 software]# ll php-7.2.28.tar.gz 
-rw-r--r-- 1 root root 19935034 Mar  5 13:54 php-7.2.28.tar.gz
[root@hadoop001 software]# du -sh php-7.2.28.tar.gz 
20M     php-7.2.28.tar.gz

2, View Folder Size:

[root@hadoop001 hadoop]# du -sh software/
866M    software

2.7, search the find command

history history
ps -ef see the process
top system case

1, our hands got a new machine, such as a large data components hadoop, we need to find themselves deployed under which path?

• Maximum use of root privileges, find / -name "hadoop"

1、从根目录开始搜索到名字为hadoop的文件：
[root@hadoop001 ~]# find / -name "hadoop"
/home/hadoop
/var/db/sudo/hadoop
/var/spool/mail/hadoop

2、可以指定在/home下查找：
[root@hadoop001 ~]# find /home -name "*hadoop*"
/home/hadoop

3、可以对需要查找的内容进行模糊匹配：前后都加上*号：
[root@hadoop001 ~]# find / -name "*hadoop*"
/home/hadoop
/var/db/sudo/hadoop
/var/spool/mail/hadoop

Third, this coursework

Assignment:
1, organize users, user groups
2, sudo command
3, the user why not login (/ etc / passwd file related)
4, permissions (rwx), chown and chmod command
Note: Use only the -R command on linux chown and chmod command