【YAML】Python中YAML文件的操作【原创】

Python中YAML文件的操作

一、概要

在开发容器发布系统的时候，由于需要调用k8s的API，故需要和k8s的deployment、service等YAML内容格式打交道。

先来介绍一下YAML

YAML 全名 YAML Ain’t Markup Language，主要设计目标是对人类可读性高。

YAML 1.2 是 JSON 的超集，也就是说合法的 JSON 扔给 YAML 1.2 解析器是可以被完美解析的

1. yaml文件规则

• 区分大小写；
• 使用缩进表示层级关系；
• 使用空格键缩进，而非Tab键缩进
• 缩进的空格数目不固定，只需要相同层级的元素左侧对齐；
• 文件中的字符串不需要使用引号标注，但若字符串包含有特殊字符则需用引号标注；
• 注释标识为#

2. yaml文件数据结构

• 对象：键值对的集合（简称 “映射或字典”）
  • 键值对用冒号 “:” 结构表示，冒号与值之间需用空格分隔
• 数组：一组按序排列的值（简称 “序列或列表”）
  • 数组前加有 “-” 符号，符号与值之间需用空格分隔
• 纯量(scalars)：单个的、不可再分的值（如：字符串、bool值、整数、浮点数、时间、日期、null等）
  • None值可用null可 ~ 表示

3. 使用场景

YAML一般来说，做序列化和做配置文件的场景比较多。

另外，YAML是可以转化为Json格式的

二、Python中的使用

在Python中操作yaml有两种做法：

• pyyaml：pip3 install pyyaml
• ruamel（推荐）：pip3 install ruamel.yaml

1. pyyaml

pyyaml是python自带的官方包，官方文档为：https://pyyaml.org/wiki/PyYAML

安装：

pip3 install pyyaml

读取：yaml.load

import yaml

# 读取yaml内容
yaml_content = yaml.load("""
name: Vorlin Laruknuzum
sex: Male
class: Priest
title: Acolyte
hp: [32, 71]
sp: [1, 13]
gold: 423
inventory:
- a Holy Book of Prayers (Words of Wisdom)
- an Azure Potion of Cure Light Wounds
- a Silver Wand of Wonder
""", Loader=yaml.FullLoader)
print(yaml_content['name'])

# 读取yaml文件
with open(r'test.yaml', 'r') as f:
    yaml.load(f, Loader=yaml.FullLoader)

写入：yaml.dump

import yaml

# 读取yaml内容
yaml_content = yaml.load("""
name: Vorlin Laruknuzum
sex: Male
class: Priest
title: Acolyte
hp: [32, 71]
sp: [1, 13]
gold: 423
inventory:
- a Holy Book of Prayers (Words of Wisdom)
- an Azure Potion of Cure Light Wounds
- a Silver Wand of Wonder
""", Loader=yaml.FullLoader)

yaml_content['name'] = 'New Name'
new_yaml = yaml.dump(yaml_content)
print(new_yaml)

但是呢，yaml.dump是有点问题的：

• 不能处理字典嵌套字典的数据
• 保存后的并不是yaml的标准形式
• 会丢失掉双引号

比如这里有个例子：

yaml内容为：

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  annotations:
    creator: admin
    description: mbs-statement-service
    owner_name: mbs-statement-service
    sidecar.istio.io/inject: "false"
  creationTimestamp: null
  labels:
    app: mbs-statement-service
    version: "07071418"
  name: mbs-statement-service-07071418
  namespace: bankapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mbs-statement-service
      version: "07071418"
  strategy: {}
  template:
    metadata:
      annotations:
        creator: admin
        description: mbs-statement-service
        owner_name: mbs-statement-service
        sidecar.istio.io/inject: "false"
      creationTimestamp: null
      labels:
        app: mbs-statement-service
        version: "07071418"
      name: mbs-statement-service-07071418
      namespace: bankapp
    spec:
      containers:
      - command:
        - /bin/sh
        - -c
        - /usr/local/jdk1.8.0_172/bin/java -jar  $(JAVA_OPTS) $(GCLOG_OPTS)  $(SHOOTING_OPTS)
          $(APOLLO_OPTS) 
          mbs-statement-service.jar
        image: vb-harbor.in.xx/xxx/mbs-statement-service:uat_20200528_v2
        imagePullPolicy: IfNotPresent

使用PyYaml来处理的话：

import yaml

deployment_obj = yaml.load(yaml_content, Loader=yaml.FullLoader)
for key, containers in enumerate(deployment_obj['spec']['template']['spec']['containers']):
    deployment_obj['spec']['template']['spec']['containers'][key]['image'] = "$IMAGE_URL"


k8s_deployment = yaml.dump(deployment_obj)
print(k8s_deployment)

打印出来后，格式有问题，version的双引号丢失了：

apiVersion: apps/v1beta1
kind: Deployment
metadata:
  annotations:
    creator: admin
    description: mbs-statement-service
    owner_name: mbs-statement-service
    sidecar.istio.io/inject: 'false'
  creationTimestamp: null
  labels:
    app: mbs-statement-service
    version: 07071418
  name: mbs-statement-service-07071418
  namespace: bankapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mbs-statement-service
      version: 07071418
  strategy: {}
  template:
    metadata:
      annotations:
        creator: admin
        description: mbs-statement-service
        owner_name: mbs-statement-service
        sidecar.istio.io/inject: 'false'
      creationTimestamp: null
      labels:
        app: mbs-statement-service
        version: 07071418
      name: mbs-statement-service-07071418
      namespace: bankapp
    spec:
      containers:
      - command:
        - /bin/sh
        - -c
        - /usr/local/jdk1.8.0_172/bin/java -jar  $(JAVA_OPTS) $(GCLOG_OPTS)  $(SHOOTING_OPTS)
          $(APOLLO_OPTS) mbs-statement-service.jar
        image: $IMAGE_URL
        imagePullPolicy: IfNotPresent

使用default_style来进行处理

import yaml

deployment_obj = yaml.load(yaml_content, Loader=yaml.FullLoader)
for key, containers in enumerate(deployment_obj['spec']['template']['spec']['containers']):
    deployment_obj['spec']['template']['spec']['containers'][key]['image'] = "$IMAGE_URL"


k8s_deployment = yaml.dump(deployment_obj, default_style='"')
print(k8s_deployment)

打印出来问题更大，会出现："replicas": !!int "1"

"apiVersion": "apps/v1beta1"
"kind": "Deployment"
"metadata":
  "annotations":
    "creator": "admin"
    "description": "mbs-statement-service"
    "owner_name": "mbs-statement-service"
    "sidecar.istio.io/inject": "false"
  "creationTimestamp": !!null "null"
  "labels":
    "app": "mbs-statement-service"
    "version": "07071418"
  "name": "mbs-statement-service-07071418"
  "namespace": "bankapp"
"spec":
  "replicas": !!int "1"
  "selector":
    "matchLabels":
      "app": "mbs-statement-service"
      "version": "07071418"
  "strategy": {}
  "template":
    "metadata":
      "annotations":
        "creator": "admin"
        "description": "mbs-statement-service"
        "owner_name": "mbs-statement-service"
        "sidecar.istio.io/inject": "false"
      "creationTimestamp": !!null "null"
      "labels":
        "app": "mbs-statement-service"
        "version": "07071418"
      "name": "mbs-statement-service-07071418"
      "namespace": "bankapp"
    "spec":
      "containers":
      - "command":
        - "/bin/sh"
        - "-c"
        - "/usr/local/jdk1.8.0_172/bin/java -jar  $(JAVA_OPTS) $(GCLOG_OPTS)  $(SHOOTING_OPTS)\
          \ $(APOLLO_OPTS) mbs-statement-service.jar"
        "image": "$IMAGE_URL"
        "imagePullPolicy": "IfNotPresent"

所以得使用下面的ruamel，使用下面的ruamel的话并不会去掉version后面的引号

2. ruamel

PyYaml应用最广泛，但是封装的API不够简单，而且不支持YAML 1.2最新版，所以可以使用ruamel来替代，ruamel.yaml是PyYaml的衍生版，封装的API简单而且支持YAML 1.2最新版

用法基本上也和PyYAML差不多

安装

pip3 install ruamel.yaml

读取：yaml.load

from ruamel import yaml

# 读取yaml内容
yaml_content = yaml.load("""
name: Vorlin Laruknuzum
sex: Male
class: Priest
title: Acolyte
hp: [32, 71]
sp: [1, 13]
gold: 423
inventory:
- a Holy Book of Prayers (Words of Wisdom)
- an Azure Potion of Cure Light Wounds
- a Silver Wand of Wonder
""", Loader=yaml.Loader)
print(yaml_content['name'])

# 读取yaml文件
with open(r'test.yaml', 'r') as f:
    yaml.load(f, Loader=yaml.Loader)

写入：yaml.dump

from ruamel import yaml

# 读取yaml内容
yaml_content = yaml.load("""
name: Vorlin Laruknuzum
sex: Male
class: Priest
title: Acolyte
hp: [32, 71]
sp: [1, 13]
gold: 423
inventory:
- a Holy Book of Prayers (Words of Wisdom)
- an Azure Potion of Cure Light Wounds
- a Silver Wand of Wonder
""", Loader=yaml.Loader)

yaml_content['name'] = 'New Name'
new_yaml = yaml.dump(yaml_content, Dumper=yaml.RoundTripDumper)
print(new_yaml)

三、参考

• python – pyyaml并仅使用字符串引号

• python：yaml模块

• python笔记20-yaml文件写入(ruamel.yaml)