Linux bash script using shell PowerShell, alibaba clound toolkit, OpenSSH, common shell script

## linux bash and windows powershell old cmd (command)

A, bash shell uses

运维命令：
linux bash：linux 使用bash 发布java项目、报警预告、监听文件并自动重启tomcat
windows powershell：如jenkins进行自动发布，微软新工具，主要版本 4.0(server 2012 R2)， 5.0（server2016 或者 2019之后的 server 2012 R2 ）
windows cmd: 老旧的windows脚本，基本不维护了

1. Suffix .sh: linux bash file extension, you can use intellij idea to edit
2. Suffix .bat: windows veteran cmd script file
3. Suffix .PS1, windows trendy script file powershell, to adapt to cloud computing operation and maintenance has been open source, can be used in linux. Powerful alias, you can directly use the linux command. In order to distinguish and photoshop, only named suffix ps1

Reference: 1 Continuous Integration (CI) - hands free - java developers use Jenkins
Reference: fast learning Bash
Reference: runoob bash
Reference: alibaba "Cloud Toolkit the User Guide" , Cloud Toolkit to deploy applications to Windows Server , Cloud Toolkit FAQ , application deployment to Windows server

1. Restart tomcat shell script

 文件名：tomcat-restart.sh
 执行方式如下，这两行可以一次性执行的
 sudo chmod 777 /usr/local/sh/tomcat-restart.sh;    
 sh /usr/local/sh/tomcat-restart.sh 

#!/usr/bin/env bash
## 1： 查看进程id,示例1
## echo `ps -ef | grep $CATALINA_HOME/bin/tomcat-juli.jar | grep -v grep | tr -s " "|cut -d" " -f2`f
## 1： 查看进程id,示例2
## echo `ps -ef | grep tomcat8-8081 | grep -v grep | awk '{print $2}'`

### 第一步找到服务器上面 tomcat的进程id, 并且杀掉 tomcat进程
PID=$(ps -ef | grep tomcat8-8081 | grep -v grep | awk '{print $2}')
#判断字符串是否存，-z 不存在
if [ -z "$PID" ];then
    echo Application is already stopped !!!! pid is empty
else
    echo pid = $PID is killed !!!!...
    kill $PID
fi


#远程sleep就执行失败，没反应了
#sleep 1

## 第二步 删除之前的日志
rm -rf /usr/local/tomcat8-8081/logs/*
rm -rf /usr/local/tomcat8-8081/bin/logs/mall/*
echo logs delete success !!!
#sleep 1

### 第三步 重新启动tomcat
## 启动.jar文件 示例
## nohup java -jar /root/javademo/javademo-0.0.1-SNAPSHOT.jar > nohup.log 2>&1 &
## 启动 .war文件 示例
sh /usr/local/tomcat8-8081/bin/startup.sh
#sleep 1
echo application startup dazer !!!!....

##最后我们需要的话，查看日志
##tail -f /usr/local/tomcat8-8081/logs/catalina.out
##tail -f /usr/local/tomcat8-8081/bin/logs/mall/mall_business.log
##tail -f /usr/local/tomcat8-8081/webapps/logs/mall/mall_business.log

####移动复制
#sleep 3
#echo 1
#echo 1
#echo i am move copy tomcat8-8081 war to tomcat8
#cp /usr/local/tomcat8-8081/webapps/mall.war /usr/local/tomcat8/webapps/mall.war
#echo copy sucesss!!!

Reference: 18 classic case of Linux Shell Scripting

2. Detection of directories, files

我们使用 **inotifywait**，这个需要提前安装，
可以用来监听文件。比如解决目录同步、数据同步问题、防止挖矿病毒
前置条件：yum install inotify-tools -y

1. linux file synchronization between the server; rsync + inotifywait; synchronize multiple directories ,
2. inotifywait monitor implementation file
3. How to use the command inotifywait monitor file changes?

#!/bin/bash
# 同步两个目录的文件,  linux shell 脚本
MON_DIR=/usr/local/temp
inotifywait -mqr --format %f -e create $MON_DIR |\
while read files; do
   rsync -avz /usr/local/temp /usr/local/temp22
   #echo "$(date +'%F %T') create $files" | mail -s "dir monitor" [email protected]
done

3. nginx access log split

Under Windows nginx timed split the log
file name: nginx.split.sh
nginx default access an access log, but the log will not automatically cut all the logs will be very large, often all on the G, to view the log very inconvenient.
the chmod 777 /usr/local/sh/nginx.split.sh the sudo;
SH /usr/local/sh/nginx.split.sh

#!/bin/bash
# 1： 先移动nginx现在的访问日志，删除老的访问日志
# 2： 给nginx 发送 kill 指令，让nginx 重新生成访问日志 acess.log
# nginx.split.sh

LOG_DIR=/usr/local/nginx/logs
YESTERDAY_TIME=$(date -d "yesterday" +%F)
LOG_MONTH_DIR=$LOG_DIR/$(date +"%Y-%m")
LOG_FILE_LIST="access.log"

for LOG_FILE in $LOG_FILE_LIST; do
    [ ! -d $LOG_MONTH_DIR ] && mkdir -p $LOG_MONTH_DIR
    mv $LOG_DIR/$LOG_FILE $LOG_MONTH_DIR/${LOG_FILE}_${YESTERDAY_TIME}
done

# cat 动态获取 nginx 的进程id,可能位置有变化; 这里特指主进程编号。
# kill -USR1 $(cat /var/run/nginx.pid)
kill -USR1 $(cat  /usr/local/nginx/logs/nginx.pid)

4. mysql database backups on a regular basis

Under Windows MySQL database backup plan
file name: mysql_backup.sh
nginx default access an access log, but the log will not automatically cut all the logs will be very large, often all on the G, to view the log very inconvenient.
the chmod 777 /usr/local/sh/mysql_backup.sh the sudo;
SH /usr/local/sh/mysql_backup.sh
MySQL backup sub-library sub-table

#!/bin/bash
# mysql自动备份功能
DATE=$(date +%F_%H-%M-%S)
HOST=localhost
USER=root
PASS=root007xX
BACKUP_DIR=/data/db_backup
# egrep 是排除
DB_LIST=$(mysql -h$HOST -u$USER -p$PASS -s -e "show databases;" 2>/dev/null |egrep -v "Database|information_schema|mysql|performance_schema|sys")

for DB in $DB_LIST; do
    BACKUP_NAME=$BACKUP_DIR/${DB}_${DATE}.sql
    ## 说明 /dev/null 是空设备，像一个垃圾桶，任何不要的东西都放进去。当然也可以变成自己的文件
    BACKUP_LOG_NAME=/usr/local/sh/db_error.txt
    if ! mysqldump -h$HOST -u$USER -p$PASS -B $DB > $BACKUP_NAME 2>$BACKUP_LOG_NAME; then
        echo "$BACKUP_NAME 备份失败!,请自行执行，查看错误日志"
   else
    	# 进行压缩成.zip
    	cd ${BACKUP_DIR};
    	zip ${DB}_${DATE}.sql.zip ${DB}_${DATE}.sql;
    	rm -rf $BACKUP_NAME
   fi
done

5. Configure E-mail function linux

 服务器监控的时候进行要用到报警通知，这里先来配置一下centos上面的邮件
 这里按照mailx
 mailx类似Foxmail是一个邮件客户端
 centos7 使用指定邮箱发送邮件 https://www.cnblogs.com/benjamin77/p/8571902.html
 ss   -tnl: 查看主机上面的25端口是否在监听；
 前置条件：端口，如25、465发送端口一定要能访问到外面; aliyun 服务器处于反垃圾邮件的初衷关停了25号端口。
  telnet smtp.163.com 465
  telnet smtp.163.com 25  如果在服务器上面访问不了，就换其他端口把

#!/bin/bash
# yum install mailx -y
# 最后面添加如下，配置；这里通过163邮箱配置举例。其他邮箱服务器，请跟进情况设置
# 163邮箱配置  163邮箱--设置---pop3/smtp/imap
# qq邮箱配置
# 阿里云邮箱配置  https://mailhelp.aliyun.com/freemail/detail.vm?knoId=5869705
# vi /etc/mail.rc
# 一排可以设置多个参数
# set smtp=smtp.163.com:25
# set smtp=smtp.163.com:465

set [email protected]
#set smtp=smtp.aliyun.com:25
set smtp="smtps://smtp.aliyun.com:465"
set [email protected] smtp-auth-password=7518479160.Ab.
set smtp-auth=login
set ssl-verify=ignore
# 为了解决阿里云主机， 25端口不同的情况，这里只能使用ssl的端口了
set nss-config-dir=/root/.certs

Configuration is complete, we send test messages, since spam Ali cloud strategy, port 25 is shut down. We can only use the ssl port

#这里生成阿里云邮箱的ssl证书
mkdir -p /root/.certs/
echo -n | openssl s_client -connect smtp.aliyun.com:465 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/.certs/qq.crt
certutil -A -n "GeoTrust SSL CA" -t "C,," -d ~/.certs -i ~/.certs/qq.crt
certutil -A -n "GeoTrust Global CA" -t "C,," -d ~/.certs -i ~/.certs/qq.crt
certutil -L -d /root/.certs
# 如果使用生成的证书一直报错，但是可以发送出去邮件，就不用处理

# 使用管道符|传递发送内容你通过
echo "this is test mail info"|mail -s "monitor" [email protected]

If the configuration is complete, there are problems, to participate in Linux Mailx Mail installation configuration and 8 most common questions

6. Check whether the site properly

 文件名：website_monitor.sh
 sudo chmod 777 /usr/local/sh/website_monitor.sh;    
 sh /usr/local/sh/website_monitor.sh
 这里通过定时器进行检查网站是否正常，给管理员发送邮件

#!/bin/bash
# /usr/local/sh/website_monitor.sh
#一个网站访问三次，三次都是失败，就发送警告邮件给管理员
URL_LIST="https://shop.dfww.com.cn/bobo/login.html https://www.jl-media.cn/hiber/login.html www.baidu.com"
for URL in $URL_LIST; do
    FAIL_COUNT=0
    for ((i=1;i<=3;i++)); do
        HTTP_CODE=$(curl -o /dev/null --connect-timeout 3 -s -w "%{http_code}" $URL)
        if [ $HTTP_CODE -eq 200 ]; then
            echo "$URL OK"
            break
        else
            echo "$URL retry $FAIL_COUNT"
            let FAIL_COUNT++
        fi
    done
    if [ $FAIL_COUNT -eq 3 ]; then
        echo "Warning: $URL Access failure!"
        echo "Warning: $URL Access failure!"|mail -s "monitor" [email protected]
    fi
done

7. Regular cleaning of useless or log backup to prevent disk full support

 文件名：delete_file.ps1 清理windows日志。

#delete old tmp files,just save files in 15 days~
#windows 使用powershell 定时删除日期文件或者过时备份文件
#定时任务也可以直接通过powershell添加，见下面链接
$TimeOutDays=10    
$filePath="C:\soft\script\db_backup"     
$allFiles=get-childitem -path $filePath     
foreach ($files in $allFiles)     
{       
   $daypan=((get-date)-$files.lastwritetime).days       
   if ($daypan -gt $TimeOutDays)       
   {         
     remove-item $files.fullname -Recurse -force       
    }     
}

Reference: 1 using powershell file before deleting the specified date and added to the Scheduled Tasks
Reference: 18 classic case of Linux Shell Scripting

Second, Microsoft powershell

What is 1. powershell

微软早期的脚本文件是cmd 批处理bat文件，微软好多年都不更新， 取之代之 是 powershell。windows server R2 是4.0、windows server 2016是5.0。
windows 7是3.0， windows 8是4.0
支持DOC命令
如果命令提示符前面显示：**ps**,恭喜你就在使用powershell 了

1. powershell tutorial site
2. microsoft PowerShell official website documentation, sample code
3. Windows PowerShell ISE Integrated Scripting Environment
4. View PowerShell version
5. Windows PowerShell: Managing Server Startup close
6. Powershell Quick Start (a) install and use

2. Common Commands

常见命令在powerShell中。

1. Get-HostView ps version, and now the windows 2012R2 basically a powershell 5.0+
2. Get-AliasView all aliases, such as: lsLinux, dircmd command you will find that actually can be used
3. Get-CommandGet all of the functions, get all the command line Get-Command -CommandType Alias,Get-Command -CommandType Function

3. PS supports four types of command

四种命令类型

1. cmdletCommand Line
2. aliasAliases, and linux portion DOC command are aliases
3. function函数Combining a plurality of command, as a function of a combination of
4. applicationYou can execute run applications, such as: notepad

4. PS Help System

使用常用的帮助系统命令类型

1. Help system man, for example, to see the man ls command or the Get-Help cp , LS - dir -, cp - Copy-Item - Copy,
2. Get-Command lsView original command order, here ls alias the original command in powershell to view the details of the order Get-Command ls | fl *;, obtain all documents related to command and the Get-Item * -Noun the Command, path | the SELECT the Name ,
3. Get-AliasGet command of the original command, such as viewing, ls, cp, man's original command Get-Alias ls
4. powershell常见命令大全 powershell command Daquan
5. View command in the official website of parameters, such as we see the Get-Content cat tail

5. PS used commands, PSV5 version

#Copy-Item 'C:\ActTemp\a.txt' -Destination 'D:\soft\tomcats-8992\webapps\mall.war'  ##copy文件
#Remove-Item -Path 'C:\ActTemp\mall.md' -Force ;  ##remove 文件

#net stop Tomcat8992 ;  ##启动或者停止服务
#echo 'stop......'; 
#net start Tomcat8992; 
#echo 'start ....' ; 

##查看一个文件的内容，别名cat 类似 linux tail -f  ,这里别名 cat ，原始命令：Get-Content
cat  'D:\Program Files\tomcat8081\logs\mall\mall_business.log' -wait -encoding utf8 -Tail 400; ##-wait 每秒输出一次， 并制定格式防止中文乱码;并输出最后400行

###静默删除所有，类似shell rm -rf *
rm * -Recurse -Force  或者 Remove-Item * -Recurse -Force

创建 ZIP 压缩文件, PSV5+
Compress-Archive -Path D:\Tools -DestinationPath E:\Tools_bakcup.zip
解压 ZIP 包
Expand-Archive -Path E:\Tools_bakcup.zip -DestinationPath F:\Tools

###前端代码先上传zip、删除服务器zip、解压zip; 解压缩是PS5新增的
Remove-Item -Path 'D:\nginx-1.14.2\html\book' -Recurse -Force ;  ###静默强制删除zip，别名 del
Expand-Archive -Path 'D:\nginx-1.14.2\html\book.zip' -DestinationPath 'D:\nginx-1.14.2\html\'; ###解压缩

6. PS execution policy

经常执行powershell发现执行不了，那个是因为微软增加了脚本执行策略，
防止随意执行给用户造成的损失。
这行 策略查看：Get-ExecutionPolicy
查看该命令的所有帮助：Get-Help Set-ExecutionPolicy -full

power shell执行策略(ExecutionPolicy)： 
Restricted——默认的设置， 不允许任何script运行
AllSigned——只能运行经过数字证书签名的script
RemoteSigned——运行本地的script不需要数字签名，但是运行从网络上下载的script就必须要有数字签名
Unrestricted——允许所有的script运行

How 7. Windows server using powershell

经常执行powershell发现执行不了，那个是因为微软增加了脚本执行策略，

1. One download and install OpenSSH OpenSSH in Windows
2. Download and install OpenSSH two, use openSSH, windows computer can be connected and managed using a common computer linux ssh way. eg: and as linux command line operations, ftp upload; other Installation Guide ~ ~ ~ Windows install OpenSSH service ; https://github.com/PowerShell/Win32-OpenSSH/releases page on the OpenSSH-Win64.zip
3. Configuring openSSH root directory, you can access the account and other windows openssh set the root directory
4. Regedit registry settings to the default shell default shell from cmd => powershell Windows OpenSSH supports up! Search "DefaultShell"; recommends using powershell command, add a line of code directly, free registry to find inside trying to get in regedit. Run as Administrator powershell execution> New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PropertyType String -Force
5. FAQ: The configuration file, found that does not work; you need to uninstall openSSH, reinstall it goes into effect
6. After installation, see the port number 22 has started listening:netstat -an | findstr :22
7. The above test server first: check whether ssh can connect a second: shell checking ssh using cmd or powershell ssh administrator@localhostthen enter a password on it