[Community Library] "The Python Road of Network Engineers: Network Operation and Maintenance Automation (2nd Edition)"

book preface

Time flies like an arrow, and time flies like a shuttle. In the blink of an eye, three years have passed since the earliest e-book publication of this book. Thanks to the love of readers, the e-book and the first edition of the printed book have far exceeded my expectations. This also makes me feel the need to update the content and republish this book for readers.
IT technology is changing rapidly, and the world of Python and NetDevOps is no exception. The second edition is based on the most stable Python 3.10.6 before the deadline. It has modified, updated, supplemented or deleted every chapter in the first edition based on Python 3.8.6, and removed some that are no longer popular or The author has publicly announced that the modules that are no longer maintained, such as pyping, pyntc, netdev, etc., have added 5 chapters of TextFSM Detailed Explanation, Netmiko Detailed Explanation, Nornir Detailed Explanation, NETCONF Detailed Explanation and RESTCONF Detailed Explanation on the basis of the 6 chapters in the first edition. Content, so that the content of the second edition and the thickness of the book are completely doubled compared with the first edition.
After the release of the first edition of this book, Python and NetDevOps technologies quickly became popular in domestic computer network circles, and domestic technical articles related to NetDevOps have sprung up on the Internet like mushrooms after rain. In view of the limitations of the author's working environment, all the experimental content of the first edition is completely based on Cisco equipment. Many readers of the first edition made corresponding supplements to the content of this book with domestic equipment such as Huawei and H3C. What stands out is the reader Mr. Zhu Jiasheng. In the two years since the first edition of this book was released, he has been working tirelessly. In his Zhihu personal column, he "rewrote" almost all the content of the author's first edition with Huawei equipment. He added a lot of his personal understanding and experience, and patiently answered various questions from newcomers in the readership. In view of this, the second edition specially invited Mr. Zhu Jiasheng as a co-author. Mr. Zhu Jiasheng will add the instructions for using Huawei equipment in the experimental chapter of this book, and the detailed explanation of TextFSM in Chapter 7 is also completely written by him. In addition, I am also very grateful to Teacher Li Xiumei from the Electronic Industry Press for her enthusiastic support and various assistance in publishing this book. It is an author's luck to work with such an capable and responsible editor.
Human society is developing and science and technology are progressing. Although domestic NetDevOps technology started relatively late and generally lags behind foreign advanced countries, China has a vast land, rich resources and abundant talents. Since the first edition of this book was released two years ago, a large number of in-depth studies have emerged in this field. , Domestic readers who are willing to share. China has gradually grown from a NetDevOps learning "desert" before 2019 to a NetDevOps learning "big country". However, to grow from a "big country" learning NetDevOps to a "powerful country" learning NetDevOps, domestic colleagues still need to have the courage to explore, communicate extensively, seek truth and be pragmatic, put an end to quick success and quick gains, and abandon impetuous tendencies. If so, our goal will not only be achieved, but will not be far away. I hope that this book can serve as a catalyst for us to achieve this goal, and provide a more accurate, systematic and conceptual NetDevOps textbook for the vast number of young network engineering students and professionals in China. Concise and concise. This is the purpose of the first edition and the guideline for the second edition.

book introduction

The book's Python-based NetDevOps technical knowledge is changing rapidly. The second edition will add TextFSM detailed explanation, Netmiko detailed explanation, Nornir detailed explanation, NETCONF detailed explanation and RESTCONF detailed explanation on the basis of the first edition. 5 chapters, and remove pyping, pyntc , netdev and other modules that are no longer popular or the author has publicly announced that they are no longer maintained.

• Introduction to the book

The first edition of "The Python Road of Network Engineers: Network Operation and Maintenance Automation (Second Edition)" was published in November 2020 and became a great success. Python and NetDevOps technologies quickly became popular in the domestic computer network circle. Relevant technical articles have sprung up like mushrooms on the Internet, and there are many network engineer positions that require NetDevOps technology in the industry, and the demand is increasing. NetDevOps is a skill that "next-generation network engineers" must master. and an irreversible fact. The technical knowledge of NetDevOps based on Python is changing rapidly. The second edition will add 5 chapters on the basis of the first edition, including TextFSM detailed explanation, Netmiko detailed explanation, Nornir detailed explanation, NETCONF detailed explanation and RESTCONF detailed explanation, and remove pyping, pyntc, netdev Wait for some module content that is no longer popular or the author has publicly announced that it will no longer be maintained. In addition, the second edition will add the content of Huawei equipment on the basis of Cisco equipment, written by Mr. Zhu Jiasheng, I believe it will be more beneficial to domestic network engineers who mainly use domestic equipment.

Book author, profile

• author:

Wang Yin , Zhu Jiasheng

• About the Author:

Wang Yin, Zhihu column "Internet Walker" writer, Zhihu ID: Yixin, senior network engineer at King Abdullah University of Science and Technology (KAUST), 14 years of overseas experience, CCIE #40245. Since 2009 in Singapore, he has successively worked in the US telecom operator AT&T, the US data center company Equinix, the Singapore Land Transport Authority (LTA), the Singapore Stone Forest IT Consulting Company (SFIT), the US Apple, and the Royal Bank of Scotland (RBS) ), has rich practical experience in network design and operation and maintenance of large parks and financial industries.

Zhu Jiasheng, the author of the Zhihu column "Networking Skills", graduated from South China University of Technology in 2010. He has been a network engineer for 13 years and has been rooted in the front line of network operation and maintenance for a long time. The communication network ranges from 2G to 5G, and the Internet has gone from almost blank construction operation and maintenance to a scale of millions of users. We have rich practical experience in large-scale networks in these fields. In his spare time, he loves reading and writing, is good at sorting out knowledge, and is willing to share and help novices. In recent years, he has been deeply involved in the field of Python network automation. After the release of the first edition of this book, I adapted the Cisco equipment experiments to Huawei equipment experiments, and added a lot of my own understanding and thinking. Applied to other domestic equipment.

Book review

The rating of this book is still very high;

Personally, I also recommend that it is suitable for operation and maintenance personnel to read, and a certain python foundation is also required.

book catalog

Chapter 1 Installation and use of Python 1
1.1 Install Python 1
1.1.1 Install Python 3.10.6 under Windows 1
1.1.2 Install Python 3.10.6 under Linux 5
1.2 Use Python 3.10.6 under Windows 8
1.2.1 Interactive Mode 8
1.2.2 Script Mode 10
1.2.3 Running Python Scripts 13
1.3 Using Python Under Linux 3.10.6 16
1.3.1 Interactive Mode 16
1.3.2 Scripting Mode 17
1.3.3 Running Python Scripts 19
1.3.4 Shebang Symbols 19 Chapter 2 Python Basic Syntax 21
2.1 Variables 21
2.2 Comments 24
2.3 Methods and Functions 25
2.4 Data Types 26
2.4.1 Strings 27
2.4.2 Integers and Floats 36
2.4.3 Lists 39
2.4.4 Dictionaries 44
2.4.5 Boolean type 48
2.4.6 Sets, tuples, null values ​​51 Chapter 3 Python advanced grammar 54
3.1 Conditional (judgment) statement 54
3.1.1 Judgment by comparison operator 54
3.1.2 String method + logical operator making judgments 56
3.1.3 Judgment by member operator 58
3.2 Loop statement 60
3.2.1 while statement 60
3.2.2 for statement 63
3.3 Text file reading/writing 65
3.3.1 open() function and its mode 65
3.3.2 File reading Fetching 66
3.3.3 File writing 74
3.3.4 with statement 77
3.4 Custom function 78
3.4.1 Function creation and calling 78
3.4.2 Function value return 80
3.4.3 Nested function 81
3.5 Module 82
3.5.1 Modules without custom functions 82
3.5.2 Modules with custom functions 83
3.5.3 Python built-in modules and third-party modules 84
3.5.4 from...import... 87
3.5.5 if name == ' main ': 87
3.6 Regular Expressions 88
3.6.1 What is Regular Expressions 89
3.6.2 Validation of Regular Expressions 89
3.6.3 Rules of Regular Expressions 90
3.6.4 Application of Regular Expressions in Python 97
3.7 Exception Handling 102
3.8 Classes 106
3.8.1 How to Create a Class 107
3.8.2 Method 109
3.8.3 Inheritance 110 Chapter 4 Python Network Operation and Maintenance Experiment (Network Simulator) 112
4.1 Experimental Operating Environment 112
4.1.1 Experimental Operating System 112
4.1.2 Cisco Experimental Topology 113
4.1.3 Huawei experimental topology 114
4.2 Telnet and SSH modules in Python 114
4.2.1 Telnetlib 115
4.2.2 Paramiko and Netmiko 121
4.3 Experiment 1 input() function and getpass module (Cisco equipment) 135
4.3.1 Experiment purpose 136
4.3 .2 Experiment preparation 136
4.3.3 Experiment code 137
4.3.4 Code segmentation explanation 138
4.3.5 Verification 139
4.4 Experiment 1 input() function and getpass module (Huawei equipment) 145
4.5 Experiment 2 batch login switches with discontinuous addresses ( Cisco equipment) 147
4.5.1 Experiment purpose 147
4.5.2 Experiment preparation 147
4.5.3 Experiment code 148
4.5.4 Code segmentation explanation 149
4.5.5 Verification 150
4.6 Experiment 2 batch login switches with discontinuous addresses (Huawei equipment) 152
4.7 Experiment 3 Application of exception handling (Cisco equipment) 155
4.7.1 Experiment Purpose 156
4.7.2 Experiment Preparation 156
4.7.3 Experiment Code 157
4.7.4 Code Segmentation Explanation 159
4.7.5 Verification 160
4.8 Experiment 3 Application of Exception Handling (Huawei Equipment) 162
4.9 Experiment 4 Using Python to Implement the Network Device configuration backup (Cisco equipment) 164
4.9.1 Experiment purpose 164
4.9.2 Experiment preparation 164
4.9.3 Experiment code 167
4.9.4 Code segment explanation 168
4.9.5 Verification 170
4.10 Experiment 4 Use Python to configure network devices Backup (Huawei equipment) 172 Chapter 5 Python network operation and maintenance (real machine) 175
5.1 Experiment 1 Large-scale batch modification of switch QoS configuration (Cisco equipment) 175
5.1.1 Experiment background 176
5.1.2 Experiment purpose 177
5.1.3 Experiment preparation 177
5.1.4 Experiment code 178
5.1.5 Code segment explanation 179
5.1.6 Verification 181
5.2 Experiment 2 How to use pythonping (Cisco equipment) 184
5.2.1 Experiment background 185
5.2.2 Experiment purpose 185
5.2.3 Experiment Idea 185
5.2.4 Experiment preparation - script 1 186
5.2.5 Experiment code - script 1 187
5.2.6 Script 1 Code Segmentation Explanation 187
5.2.7 Script 1 Verification 190
5.2.8 Experiment Preparation - Script 2 190
5.2.9 Experimental Code - Script 2 191
5.2.10 Script 2 Code Segmentation Explanation 193
5.2.11 Script 2 Verification 196
5.3 Experiment 3 Use Python script to check switch configuration (Cisco equipment) 197
5.3.1 Experiment background 197
5.3.2 Experiment purpose 198
5.3.3 Experiment idea 198
5.3.4 Experiment preparation - script 1 199
5.3.5 Experiment code - script 1 199
5.3.6 Script 1 code section explanation 201
5.3.7 Script 1 verification 206
5.3.8 Experiment preparation - script 2 206
5.3.9 Experiment code - script 2 207
5.3.10 Script 2 code Segmentation Explanation 208
5.3.11 Script 2 Verification 210
5.4 Experiment 4 Excessive Command Echo Processing in Live Network (Huawei Equipment) 211
5.4.1 Experiment Background 211
5.4.2 Experiment Purpose 212
5.4.3 Experiment Ideas 212
5.4.4 Experiment Preparation - Script 1 213
5.4.5 Script 1 Validation 214
5.4.6 Experiment Preparation - Script 2 214
5.4.7 Script 2 Validation 216
5.5 Experiment 5 Custom ping tool and exe packaging (Huawei equipment) 216
5.5.1 Experiment background 216
5.5.2 Experiment purpose 217
5.5.3 Experiment idea 217
5.5.4 Experiment preparation - script 218
5.5.5 Script verification 219
5.5. 6 Script Packaging 220
5.5.7 Packaging Verification 220 Chapter 6 Detailed Explanation of Python Built-in Modules and Third-Party Modules 222
6.1 JSON 223
6.1.1 JSON Basics 223
6.1.2 Using JSON in Python 224
6.2 Pain Points of Regular Expressions 226
6.3 TextFSM and ntc-templates 227
6.3.1 TextFSM installation 228
6.3.2 TextFSM template creation and application 230
6.3.3 ntc-templates 234
6.4 NAPALM 238
6.4.1 What is NAPALM 239
6.4.2 Advantages of NAPALM 239
6.4. 3 Disadvantages of NAPALM 241
6.4.4 Installation of NAPALM 242
6.4.5 Application of NAPALM 242
6.5 asyncio 248
6.5.1 Synchronous and asynchronous 248
6.5.2 Application of asynchronous in Python 249
6.6 Multithreading 252
6.6.1 Single thread and multithreading 252
6.6.2 Application of multithreading in Python 253
6.6.3 Application of multithreading in Netmiko 257
6.7 CSV and Jinja2 260
6.7.1 CSV configuration file and csv module in Use in Python 260
6.7.2 Jinja2 configuration template 262
6.7.3 Use of Jinja2 in Python 264
6.7.4 Upload the generated configuration command to the switch and execute it 267 Chapter 7 Detailed explanation
of TextFSM 273 7.1 TextFSM installation and examples 274
7.1 .1 TextFSM installation 274
7.1.2 TextFSM reference example analogy 275
7.1.3 TextFSM reference example detailed explanation 276
7.2 Experiment 1 single-line echo single-line rule 280
7.2.1 Install tabulate module 280
7.2.2 Create experiment folder 281
7.2.3 Prepare output. txt 282
7.2.4 Prepare template.txt 282
7.2.5 Prepare Python script 283
7.2.6 Execute Python script 286
7.2.7 Template matching process 286
7.2.8 Experiment summary 286
7.3 Experiment 2 Multi-line echo single-line rule 286
7.3.1 Prepare output.txt 287
7.3.2 Prepare template.txt 288
7.3.3 Prepare Python script 289
7.3.4 Execute Python script 289
7.3.5 Template matching process 290
7.3.6 Experiment summary 291
7.4 Experiment 3 Multi-line echo Multi-line rule, new keywords 291
7.4.1 Prepare output.txt 291
7.4.2 Prepare template.txt 292
7.4.3 Prepare Python script 293
7.4.4 Experiment debugging 294
7.4.5 Template matching process 298
7.4.6 Experiment summary 299
7.5 Experiment 4 Keyword List and Action Continue.Record 299
7.5.1 Prepare output.txt 299
7.5.2 Prepare template.txt 300
7.5.3 Prepare Python script 300
7.5.4 Experiment commissioning 301
7.5.5 Template matching process 308
7.5.6 Experimental summary 308
7.6 TextFSM scene combing and expansion 309 Chapter 8 Netmiko detailed explanation 310
8.1 Experiment 1 Log in to a switch (Cisco device) through Netmiko 311
8.2 Experiment 1 Log in to a switch (Huawei device) through Netmiko 314
8.3 Experiment 2 Configure the device through Netmiko (Cisco device) 315
8.4 Experiment 2 Configure the device through Netmiko (Huawei device) 319
8.5 Experiment 3 Use Netmiko with TextFSM or Genie to format the echo (Cisco device) 321
8.6 Experiment 3 Use Netmiko with TextFSM or Genie to format the echo (Huawei equipment) 327
8.7 Experiment 4 Connect multiple switches through Netmiko (Cisco equipment) 330
8.8 Experiment 4 Connect multiple switches through Netmiko (Huawei equipment) 335
8.9 Experiment 5 Netmiko with Jinja2 The configuration template configures the device (Cisco device) 336
8.10 Experiment 5 Netmiko cooperates with the Jinja2 configuration template to configure the device (Huawei device) 340
8.11 Experiment 6 Use the enable password in Netmiko to enter the privileged mode of the device (Cisco device) 342
8.12 Experiment 6 in Use the enable password in Netmiko to enter the privileged mode of the device (Huawei equipment) 346
8.13 Experiment 7 Use Netmiko to transfer files to the device (Cisco equipment) 348
8.14 Experiment 7 Use Netmiko to transfer files to the device (Huawei equipment) 351
8.15 Experiment 8 Use Netmiko to process device prompts Command (Cisco device) 354
8.16 Experiment 8 Use Netmiko to process device prompt commands (Huawei device) 357
8.17 Experiment 9 Use Netmiko to obtain the device hostname (Cisco device) 358
8.18 Experiment 9 Use Netmiko to obtain the device hostname (Huawei device) 360
8.19 New features introduction of Netmiko 4 361 Chapter 9 Nornir detailed explanation 368
9.1 Nornir experiment preparation (Cisco equipment, CentOS system) 370
9.2 Nornir experiment preparation (Huawei equipment, Windows system) 374
9.3 Experiment 1 Call nornir_napalm to obtain the facts and interfaces information of the device (Cisco equipment) 378
9.4 Experiment 1 Call nornir_napalm to obtain device facts and interfaces information (Huawei equipment) 381
9.5 Experiment 2 Call nornir_netmiko to obtain device information (Cisco equipment) 383 9.6 Experiment
2 Call nornir_netmiko to obtain device information (Huawei equipment) 385
9.7 Experiment 3 Use filter() with F() for advanced filtering (Cisco equipment) 386
9.8 Experiment 3 Use filter() with F() for advanced filtering (Huawei equipment) 391
9.9 Experiment 4 Use filter() for simple filtering (Cisco equipment ) 393
9.10 Experiment 4 Use filter() to do simple filtering (Huawei equipment) 399
9.11 Experiment 5 Use lambda in filter() to filter single or multiple devices (Cisco equipment) 400
9.12 Experiment 5 Use lambda in filter() to filter a single or multiple devices (Huawei equipment) 403
9.13 Experiment 6 Use Nornir to configure equipment (Cisco equipment) 405
9.14 Experiment 6 Use Nornir to configure equipment (Huawei equipment) 407
9.15 Experiment 7 Use Nornir to save and backup equipment configuration (Cisco equipment) ) 408
9.16 Experiment 7 Use Nornir to save device configuration (Huawei equipment) 411
9.17 Experiment 8 Use Nornir with Jinja2 to configure the device (Cisco equipment) 412
9.18 Experiment 8 Use Nornir with Jinja2 to configure the device (Huawei equipment) 420
9.19 Experiment 9 Nornir 3 + Scrapli (Cisco equipment) 423
9.20 Experiment 9 Nornir 3 + Scrapli (Huawei equipment) 432
9.21 Experiment 10 Nornir 3 + TextFSM (Cisco equipment) 434
9.22 Experiment 10 Nornir 3 + TextFSM (Huawei equipment) 437
9.23 Experiment 11 Nornir 3 + ipdb (Cisco equipment) 438
9.24 Experiment 11 Nornir 3 + ipdb (Huawei equipment) 444
9.25 Experiment 12 Nornir's Inventory (Cisco equipment) 447
9.26 Experiment 12 Nornir's Inventory (Huawei equipment) 451
9.27 Experiment 13 Nornir's Task (Cisco equipment) 455
9.28 Experiment 13 Nornir's Task (Huawei equipment) 461
9.29 Experiment 14 Use Nornir to modify switch configurations in batches on demand (Cisco equipment) 463
9.30 Experiment 14 Use Nornir to modify switch configurations in batches on demand (Huawei equipment) 467 Chapter 10 NETCONF Detailed Explanation 474
10.1 Theoretical part of NETCONF 475
10.2 Theoretical part of YANG 476
10.2.1 What is a data model476
10.2.2 YANG model477
10.2.3 YANG module478
10.2.4 Download YANG module from GitHub478
10.2.5 pyang module481
10.3 Experimental part of NETCONF482
10.3.1 Experimental topology and experimental environment482
10.3.2 Experimental steps 483
10.4 ncclient 490
10.4.1 Introduction to ncclient 491
10.4.2 ncclient practical application (get_config) 492
10.4.3 ncclient practical application (edit_config) 503
10.5 NETCONF experiment (Huawei) 510
10.5.1 Experimental topology 510
10. 5. 2 Experiment purpose 511
10.5.3 Start NETCONF 511
10.5.4 Link ncclient 515 Chapter 11 RESTCONF detailed explanation 519
11.1 Introduction to RESTCONF 519
11.1.1 Correspondence between HTTP method and CRUD 520
11.1.2 Postman 520
11.2 RESTCONF experiment (Postman) 52 0
11.2 .1 Experimental environment 520
11.2.2 Switch initial configuration 521
11.2.3 Postman initial configuration 522
11.2.4 Obtain switch configuration by GET method 525
11.2.5 Change switch configuration by PATCH method 529
11.2.6 Replace switch configuration by PUT method 532
11.2.7 Delete device configuration by DELETE method 536
11.3 RESTCONF experiment (Requests) 538
11.3. 1 Introduction to the Requests module 538
11.3.2 HTTP basic knowledge review 538
11.3.3 Requests experimental environment 542
11.3.4 Get the switch configuration through the GET method 543
11.3.5 Use ?depth= ​​to modify the depth 547
11.3.6 Change the switch configuration through the PATCH method 549
11.3.7 Replace switch configuration by PUT method 552
11.3.8 Add switch configuration by POST method 556
11.3.9 Delete switch configuration by DELETE method 559
11.4 RESTCONF experiment (Huawei) 561
11.4.1 Experiment topology 561
11.4.2 Experiment purpose 561
11.4 .3 Start RESTCONF 562
11.4.4 Link Postman 564

Summary: The meaning of my choice of this book

• Friendly to novices, follow the experiment step by step, you can get started quickly, and there are many actual combats
• Very suitable for operation and maintenance colleagues or python colleagues
• There are very good prospects for the operation and maintenance colleagues, and they can engage in operation and maintenance development, DevOps, automation and the like in the future;
• Personal feeling is very practical, very good.