shell脚本定时推送钉钉战报

一、目的与演示效果

解析服务端返回的如下字符串，获取今日数据、历史数据，实现钉钉定时推送战报效果。

{
    
    
    "code": "00000",
    "msg": "",
    "success": true,
    "data": {
    
    
        "2023-07-19": [
            "1673945805117063168",
            "1671561317942689792",
            "1658787124574552064",
            "1675750158471659520"
        ],
        "2023-07-20": [
            "1673945805117063168",
            "1671561317942689792",
            "1658787124574552064",
            "1675750158471659520"
        ]
    },
    "ref": null
}

二、请求网络

1、curl获取数据并赋值变量

在shell中直接使用curl http://www.baidu.com会出现下载器，因此我们拼接一个-s。

对于接收数据的变量api_result：其后面的等号前后都不要有空格。

对于要执行的函数：用反单引号包裹起来

api_result=`curl -s http://www.baidu.com`

也可以使用eval函数来执行。以下为等价写法：

api_result=`eval 'curl -s http://www.baidu.com'`

2、带参数

对于整个地址用引号（单、双都可）拼接起来，要不然不识别拼接的参数。示例：

api_result=`eval 'curl -s "http://www.baidu.com?s=1&c=ANDROID"'`

等价eval写法：

api_result=`curl -s 'http://www.baidu.com?s=1&c=ANDROID'`

3、封装网络请求方法

对于请求网络，往往是为了拿到里层的data数据。因此可以考虑封装方法来复用。这里含有以下知识点：

1、方法参数传递使用$1 $2 $3等，代表第一个参数、第二个参数的意思。

调用的时候把参数放方法后面，中间以空格连接即可。

2、echo为返回，不要用return，return只能返回数字类型。

#!/bin/bash
# 请求网络，拿到里层data数据
# {"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392"]},"ref":null}
checkUploadResult(){
    
    
  api_result=`eval $1` # 执行api_result，拿到返回的结果
  data=$(echo $api_result | jq -r '.data')
  echo $data
}
# 调用方法
androidTotalTodo=`checkUploadResult "curl -s 'http:/www.baidu.com?s=1&c=ANDROID'"`

三、json解析

shell的json解析，这里以jq解析为主，以封装的getJsonValuesByAwk解析为辅。主要是jq解析获取key的时候要求key不能是数字

1、根据key获取value (key非数字)

【key非数字】从json数据api_result中取data，即可这样写

api_result='{"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392"]},"ref":null}'
data=$(echo $api_result | jq -r '.data')
echo $data

# 打印数据：{ "2023-07-12": [ "1679009176166203392" ] }

【key为数字】这个时候使用jq解析会报错

json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
data=$(echo $json | jq -r '.2023-07-12')
echo $data
# 打印数据：-18.7977

然后我们使用getJsonValuesByAwk解析就正常了

### 方法简要说明：
### 1. 是先查找一个字符串：带双引号的key。如果没找到，则直接返回defaultValue。
### 2. 查找最近的冒号，找到后认为值的部分开始了，直到在层数上等于0时找到这3个字符：,}]。
### 3. 如果有多个同名key，则依次全部打印（不论层级，只按出现顺序）
### @author lux feary
###
### 3 params: json, key, defaultValue
function getJsonValuesByAwk() {
    
    
    awk -v json="$1" -v key="$2" -v defaultValue="$3" 'BEGIN{
        foundKeyCount = 0
        while (length(json) > 0) {
            # pos = index(json, "\""key"\""); ## 这行更快一些，但是如果有value是字符串，且刚好与要查找的key相同，会被误认为是key而导致值获取错误
            pos = match(json, "\""key"\"[ \\t]*?:[ \\t]*");
            if (pos == 0) {if (foundKeyCount == 0) {print defaultValue;} exit 0;}

            ++foundKeyCount;
            start = 0; stop = 0; layer = 0;
            for (i = pos + length(key) + 1; i <= length(json); ++i) {
                lastChar = substr(json, i - 1, 1)
                currChar = substr(json, i, 1)

                if (start <= 0) {
                    if (lastChar == ":") {
                        start = currChar == " " ? i + 1: i;
                        if (currChar == "{" || currChar == "[") {
                            layer = 1;
                        }
                    }
                } else {
                    if (currChar == "{" || currChar == "[") {
                        ++layer;
                    }
                    if (currChar == "}" || currChar == "]") {
                        --layer;
                    }
                    if ((currChar == "," || currChar == "}" || currChar == "]") && layer <= 0) {
                        stop = currChar == "," ? i : i + 1 + layer;
                        break;
                    }
                }
            }

            if (start <= 0 || stop <= 0 || start > length(json) || stop > length(json) || start >= stop) {
                if (foundKeyCount == 0) {print defaultValue;} exit 0;
            } else {
                print substr(json, start, stop - start);
            }

            json = substr(json, stop + 1, length(json) - stop)
        }
    }'
}

json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
data=`getJsonValuesByAwk "$json" '2023-07-12' '没有数据'`
echo $data

# 打印数据：["1679009176166203392"]

2、获取数组长度

list='["1679009176166203392"]'
arr=($(echo $list | tr -d '[],'))
echo "${
     
     #arr[@]}"

# 打印数据：1

3、删除json中的某个key

json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
json=$(echo "$json" | jq 'del(.["2023-07-12"])')
echo "$json"

# 打印数据：
#{
    
    
#  "2023-07-13": [
#    "1679009176166203392"
#  ]
#}

4、删除空格与引号

钉钉要求文案中不能有空格与引号，否则推送报错。

# 删除空格
dingtalkResult="${dingtalkResult// /}"
# 删除引号"
dingtalkResult=${dingtalkResult//"\""/}

5、判断数据是否为空

比如对未声明的变量json。前面判断是否是字符串null，后边"-z"选项来检查变量是否为空。

#!/bin/bash

# null字符串或者空
if [ "$json" = "null" ] || [ -z "$json" ]; then
    echo '数据空'
else
    echo '数据不空'
fi

# 打印数据：数据空

四、钉钉推送

提供两种，一种是带At一种不带

# 钉钉通知
notifyDingTalk(){
    
    
  curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
    -H 'Content-Type: application/json' \
    -d '{
          "msgtype": "text",
          "text": {
                   "content": "'$1'"
               },
      }'
}

# 钉钉通知带at
notifyDingTalkWithAt(){
    
    
  curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
    -H 'Content-Type: application/json' \
    -d '{
          "msgtype": "text",
          "text": {
                   "content": "'$1'"
               },
               "at": {
                   "atMobiles": [
                       "1875819063x",
                       "1580366544x"
                   ],
                   "isAtAll": false
               }
      }'
}

使用如下

notifyDingTalk '这是发给钉钉的文案'
notifyDingTalkWithAt '这是发给钉钉的文案'

配置安全文案即可

五、jenkins配置

jenkins配置定时器如下

H 17 * * *

会提示构建时间

六、源码

敏感信息部分以xxx打码，替换自己的后不影响运行。

#!/bin/bash
# 工作台视频上传定时任务

# 当前时间（示例：2023-07-15）
cur_time=`date +"%F"`
#钉钉key
DKEY="65f276f7747278212d5fb85729117037bb62485b7573eb6b1ba028acf2ef6fxx"

# 默认为null字符串
emptyValue=null

# 检测是否是定时任务触发的构建
# 如果 $toDingTalk 为空，则默认发送钉钉消息
if [ -z "$toDingTalk" ]; then
  toDingTalk=true
fi

### 方法简要说明：
### 1. 是先查找一个字符串：带双引号的key。如果没找到，则直接返回defaultValue。
### 2. 查找最近的冒号，找到后认为值的部分开始了，直到在层数上等于0时找到这3个字符：,}]。
### 3. 如果有多个同名key，则依次全部打印（不论层级，只按出现顺序）
### @author lux feary
###
### 3 params: json, key, defaultValue
function getJsonValuesByAwk() {
    
    
    awk -v json="$1" -v key="$2" -v defaultValue="$3" 'BEGIN{
        foundKeyCount = 0
        while (length(json) > 0) {
            # pos = index(json, "\""key"\""); ## 这行更快一些，但是如果有value是字符串，且刚好与要查找的key相同，会被误认为是key而导致值获取错误
            pos = match(json, "\""key"\"[ \\t]*?:[ \\t]*");
            if (pos == 0) {if (foundKeyCount == 0) {print defaultValue;} exit 0;}

            ++foundKeyCount;
            start = 0; stop = 0; layer = 0;
            for (i = pos + length(key) + 1; i <= length(json); ++i) {
                lastChar = substr(json, i - 1, 1)
                currChar = substr(json, i, 1)

                if (start <= 0) {
                    if (lastChar == ":") {
                        start = currChar == " " ? i + 1: i;
                        if (currChar == "{" || currChar == "[") {
                            layer = 1;
                        }
                    }
                } else {
                    if (currChar == "{" || currChar == "[") {
                        ++layer;
                    }
                    if (currChar == "}" || currChar == "]") {
                        --layer;
                    }
                    if ((currChar == "," || currChar == "}" || currChar == "]") && layer <= 0) {
                        stop = currChar == "," ? i : i + 1 + layer;
                        break;
                    }
                }
            }

            if (start <= 0 || stop <= 0 || start > length(json) || stop > length(json) || start >= stop) {
                if (foundKeyCount == 0) {print defaultValue;} exit 0;
            } else {
                print substr(json, start, stop - start);
            }

            json = substr(json, stop + 1, length(json) - stop)
        }
    }'
}

# 请求网络，拿到里层data数据
# {"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392","1676130723649683456","1675059618608447488","1674345317023219712","1672586086360154112","1673586172305211392","1673950865947492352","1675757864595095552","1670766831293562880","1674976668499968000","1676513810044813312","1676512377715163136","1675146378722017280","1673687264389103616","1675283959501684736","1674213349681922048","1675545501522591744","1674269196629966848","1669551312678813696","1675366041963855872","1675445816296341504","1668133451263508480","1673610261883387904","1673945805117063168","1671561317942689792","1658787124574552064","1675750158471659520"]},"ref":null}
checkUploadResult(){
    
    
  api_result=`eval $1` # 执行cmd，拿到返回的结果
  data=$(echo $api_result | jq -r '.data')
  echo $data
}

# 获取今天的数据
getTodayList(){
    
    
  json=`getJsonValuesByAwk "$1" $cur_time $emptyValue`
  echo $json
}

getArrayLengthFromString(){
    
    
  # null或者null字符串
  if [ "$1" = "null" ] || [ -z "$1" ]; then
      echo 0
  else
    arr=($(echo "$1" | tr -d '[],'))
    echo "${
     
     #arr[@]}"
  fi
}

# 删除今日数据
deleteTodayData(){
    
    
  # null或者null字符串
  if [ "$1" = "null" ] || [ -z "$1" ]; then
      echo {
    
    }
  else
    # 转正常json
      # 使用 jq 命令将 JSON 字符串变量转换为 JSON 变量
      # json=$(echo "$1" | jq .)
      # 删除今日数据
      json=$(echo "$1" | jq 'del(.["'$cur_time'"])')
      echo "$json"
  fi
}

# 总计
androidTotalTodo=`checkUploadResult "curl -s 'https://pre-api.xx.com/na/api/manual/v2/getVideoInfo?s=1&c=ANDROID'"`
androidTotalFinish=`checkUploadResult "curl -s 'https://pre-api.xx.com/na/api/manual/v2/getVideoInfo?s=2&c=ANDROID'"`
# 今日
androidTodayTodo=`getTodayList "$androidTotalTodo"`
androidTodayFinish=`getTodayList "$androidTotalFinish"`

# 总计
iosTotalListTodo=`checkUploadResult "curl -s 'https://pre-api.xx.com/na/api/manual/v2/getVideoInfo?s=1&c=IOS'"`
iosTotalListFinish=`checkUploadResult "curl -s 'https://pre-api.xx.com/na/api/manual/v2/getVideoInfo?s=2&c=IOS'"`
# 今日
iosTodayTodo=`getTodayList "$iosTotalListTodo"`
iosTodayFinish=`getTodayList "$iosTotalListFinish"`

# 总共未完成
totalTodo=`checkUploadResult "curl -s 'https://pre-api.xx.com/na/api/manual/v2/getVideoInfo?s=1'"`

dingtalkResult+="\n一、今日待上传"
dingtalkResult+="\n1、安卓今日待上传：【"`getArrayLengthFromString "$androidTodayTodo"`"个】\n"$androidTodayTodo
dingtalkResult+="\n2、ios今日待上传：【"`getArrayLengthFromString "$iosTodayTodo"`"个】\n"$iosTodayTodo

dingtalkResult+="\n\n二、今日完成"
dingtalkResult+="\n1、安卓完成上传：【"`getArrayLengthFromString "$androidTodayFinish"`"个】"
dingtalkResult+="\n2、ios完成上传：【"`getArrayLengthFromString "$iosTodayFinish"`"个】"

dingtalkResult+="\n\n三、历史待上传"
androidTotalTodo=$(deleteTodayData "$androidTotalTodo")
iosTotalListTodo=$(deleteTodayData "$iosTotalListTodo")
totalTodo=$(deleteTodayData "$totalTodo")

# 使用 jq 命令将 JSON 对象转换为字符串
sum_totalTodo=$(echo "$totalTodo" | jq -c .)
# 判断整数值是否为 0
if [ "${
     
     #sum_totalTodo}" -eq 0 ]; then
  dingtalkResult+="\n{}"
else
  dingtalkResult+="\n"$sum_totalTodo
fi

# 删除空格
dingtalkResult="${dingtalkResult// /}"
# 删除引号"
dingtalkResult=${dingtalkResult//"\""/}

# 钉钉通知
notifyDingTalk(){
    
    
  if [ $toDingTalk == true ]; then
    echo $1
    curl -s "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
        -H 'Content-Type: application/json' \
        -d '{
              "msgtype": "text",
              "text": {
                       "content": "'$1'"
                   },
          }'
  else
    echo $1
  fi
}

# 钉钉通知带at
notifyDingTalkWithAt(){
    
    
  if [ $toDingTalk == true ]; then
      echo $1
      curl -s "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
          -H 'Content-Type: application/json' \
          -d '{
                "msgtype": "text",
                "text": {
                         "content": "'$1'"
                     },
                     "at": {
                         "atMobiles": [
                             "187581906xx",
                             "158036654xx"
                         ],
                         "isAtAll": false
                     }
            }'
  else
      echo $1
  fi
}

if test "$androidTodayTodo" != null && test "$iosTodayTodo" != null; then
    notifyDingTalk "【"$cur_time"】今天andriod、ios视频都未上传完！以下是具体数据：${dingtalkResult}"
elif test "$androidTodayTodo" != null; then
    notifyDingTalk "【"$cur_time"】今天andriod视频未上传完！以下是具体数据：${dingtalkResult}"
elif test "$iosTodayTodo" != null; then
    notifyDingTalk "【"$cur_time"】今天ios视频未上传完！以下是具体数据：${dingtalkResult}"
else
    notifyDingTalk "【"$cur_time"】今日视频上传完毕！辛苦了！${dingtalkResult}"
fi