Go Language Learning Chapter 05

Go Language Learning Chapter 05

Golang file manipulation

Documentation

• basic introduction
  • The file is 数据源, the database is essentially a file
  • Files are manipulated in the form of streams in the program

• Stream overview
  • Stream: The path data takes between the data source (file) and the program (memory)
  • Input stream: the path of data from the data source (file) to the program (memory)
  • Output stream: the path of data from the program (memory) to the data source (file)

os.File encapsulates all file-related operations, and File is a structure

use of documents

1) Open a file for read and write operations

os.Open(name string) (*File, error)

2) Close a file

File.Close()

3) Other

file对象、file指针、file文件句柄 //3种叫法

Open and close files

• file is a pointer

func Open

func Open(name string) (file *File, err error)

Open打开一个文件用于读取. If the operation is successful, the methods of the returned file object can be used to read data; the corresponding file descriptor has mode O_RDONLY. If an error occurs, the underlying error type is *PathError.

func (*File) Close

func (f *File) Close() error

Close关闭文件f, making the file unavailable for reading and writing. It returns possible errors.

the case

package main

import (
	"fmt"
	"os"
)

func main() {
    
    
	//file对象、file指针、file文件句柄
	var filePath string = "D:/axis.log"
	file, err := os.Open(filePath)
	if err != nil {
    
    
		fmt.Println("文件打开error:", err)
		return
	}
	//打开文件
	fmt.Println(file) //&{0xc000100780} file就是一个指针

	//关闭文件
	error := file.Close()
	if error != nil {
    
    
		fmt.Println("关闭文件error:", error)
	}
}

Buffered file reading

1) Read the content of the file and 显示在终端( with a buffer , suitable for 大文件), use os.Open, file.Close(),

bufio.NewReader(), reader.ReaderString Functions and Methods

• buffer:
  • read part, process part
  • for all large files

the code

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"
)

func main() {
    
    
	file, error := os.Open("D:/axis.log")
	if error != nil {
    
    
		fmt.Println("文件打开失败,", error)
		return
	}
	defer file.Close()//延时机制
	//创建一个指针内型的reader
	//默认缓冲4096字节，读一部分，取一部分=缓冲
	reader := bufio.NewReader(file)

	for {
    
    
		str, err := reader.ReadString('\n')
		if err == io.EOF {
    
    
			fmt.Println("已达文件末尾")
			break
		}
		if err != nil {
    
    
			fmt.Println("读取过程中出现错误,", err)
			break
		}
		fmt.Print(str)
	}
	fmt.Println("文件读取结束...")
}

result

2020-08-02 16:00:24,505 0      [           main] INFO  ClassPathXmlApplicationContext  - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@fb434: startup date [Sun Aug 02 16:00:24 CST 2020]; root of context hierarchy
2020-08-02 16:00:24,571 66     [           main] INFO  ry.xml.XmlBeanDefinitionReader  - Loading XML bean definitions from class path resource [applicationContext.xml]
已达文件末尾
文件读取结束...

one-time file read

• suitable for small files

under the io/ioutil package

func ReadFile

func ReadFile(filename string) ([]byte, error)

ReadFile reads data from the file specified by filename and returns the contents of the file. Successful calls return err as nil instead of EOF. Because this function is defined as 读取整个文件, it does not treat EOF returned by a read as an error that should be reported .

the code

package main

import (
	"fmt"
	"io/ioutil"
)

func main() {
    
    
	bytes, error := ioutil.ReadFile("d:\axis.log")
	if error != nil {
    
    
		fmt.Println("文件读取出错,", error)
		return
	}
	fmt.Println(string(bytes))
}

result

2020-08-02 16:00:24,505 0      [           main] INFO  ClassPathXmlApplicationContext  - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@fb434: startup date [Sun Aug 02 16:00:24 CST 2020]; root of context hierarchy
2020-08-02 16:00:24,571 66     [           main] INFO  ry.xml.XmlBeanDefinitionReader  - Loading XML bean definitions from class path resource [applicationContext.xml]

in conclusion

• ioutil.ReaderFile(path string) unbuffered, only for small files

write file operation

Constants

const (
    O_RDONLY int = syscall.O_RDONLY // 只读模式打开文件
    O_WRONLY int = syscall.O_WRONLY // 只写模式打开文件
    O_RDWR   int = syscall.O_RDWR   // 读写模式打开文件
    O_APPEND int = syscall.O_APPEND // 写操作时将数据附加到文件尾部
    O_CREATE int = syscall.O_CREAT  // 如果不存在将创建一个新文件
    O_EXCL   int = syscall.O_EXCL   // 和O_CREATE配合使用，文件必须不存在
    O_SYNC   int = syscall.O_SYNC   // 打开文件用于同步I/O
    O_TRUNC  int = syscall.O_TRUNC  // 如果可能，打开时清空文件
)

func OpenFile

func OpenFile(name string, flag int, perm FileMode) (file *File, err error)

OpenFile is a more general file opening function, and most callers use Open or Create instead of this function. It will use the specified options (such as O_RDONLY, etc.), the specified mode (such as 0666, etc.) to open the file with the specified name. If the operation is successful, the returned file object can be used for I/O. If an error occurs, the underlying error type is *PathError.

name是文件

flag是打开的方式

FileMode只在Unix、Linux有效

write file new 01

• Create a new file and write 5 sentences "hello go"
  • os.Open
  • Use the cached method: bufio.NewWriter()

the code

package main

import (
	"bufio"
	"fmt"
	"os"
	"strconv"
)

func main() {
    
    
	filePath := "d:/carter.txt"
	file, error := os.OpenFile(filePath, os.O_WRONLY | os.O_CREATE | os.O_APPEND, 0666)

	if error != nil {
    
    
		fmt.Println("文件打开失败,", error)
		return
	}

	defer file.Close()

	//准备写入的内容
	str := "hello go"

	//带缓冲的写入
	writer := bufio.NewWriter(file)

	for i := 0; i < 5; i++ {
    
    
        //有些不能识别\n换行符
		writer.WriteString(str + strconv.Itoa(i) + "\r\n")
	}

	//writer是带缓存的
	//内容先写入缓存，使用Flush真正写入文件中
	//否则文件没有数据，数据还在缓存中
	writer.Flush()
	fmt.Println("文件写入成功...")
}

result

hello go0
hello go1
hello go2
hello go3
hello go4

in conclusion

• bufio.NewWriter is cached
• The content is written to the cache first, but not to the data source (file)
• Use Flush( ), actually write to the file

write file overwrite 02

1) Overwrite existing files

• train of thought
  • write only + clear:os.O_WRONLY | os.O_TRUNC
  • Newline:\r\n

write file append 03

1) Add text to the original file

• train of thought
  • Only write + append:os.O_WRONLY | os.O_APPEND
  • Newline:\r\n

Read and write file append 04

1) Open an existing file, read the original file and display it on the terminal, and add 5 sentences "hello world"

• train of thought
  • read-only + append:os.O_RDONLY | os.O_APPEND
  • Newline:\r\n

the code

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"
	"strconv"
)

func showFile (file *os.File) {
    
    
	fmt.Println("文件读取中...")
	//读取源文件类型
	reader := bufio.NewReader(file)
	for {
    
    
		str, err := reader.ReadString('\n')
		if err == io.EOF {
    
    
			break
		}
		//显示在终端
		fmt.Print(str)
	}
}

func main() {
    
    
	filePath := "d:/carter.txt"
	file, error := os.OpenFile(filePath, os.O_RDONLY | os.O_APPEND, 0666)

	if error != nil {
    
    
		fmt.Println("文件打开失败,", error)
		return
	}

	defer file.Close()

	//显示文件
	showFile(file)

	//准备写入的内容
	str := "hello world"

	//带缓冲的写入
	writer := bufio.NewWriter(file)

	for i := 1; i < 5; i++ {
    
    
		writer.WriteString(str + strconv.Itoa(i) + "\r\n")
	}

	//writer是带缓存的
	//内容先写入缓存，使用Flush真正写入文件中
	//否则文件没有数据，数据还在缓存中
	writer.Flush()
	fmt.Println("OK~")
}

Reading and writing between files

1) Write a program to write the content of one file into another file.

NOTE: These two files already exist

illustrate:

​ Use ioutil.ReadFile, ioutil.WriteFile to complete

• are disposable

Case 2

• File 1 -> File 2
  • Read: iouilt.ReadFile[disposable]
  • Open: os.OpenFile()[with cache]
  • closure:file.Close()
  • write+append:os.O_WRONLY | os.O_APPEND
  • bufio.NewWriter(name string)[with cache]
  • Refresh the cache:writer.Flush()

package main

import (
	"bufio"
	"fmt"
	"io/ioutil"
	"os"
)

func main() {
    
    
	file1Path := "d:/axis.log"
	file2Path := "d:/carter.txt"

	//1、读取文件1
	//ioutil中是一次性的，不需要关闭文件
	data, error1 := ioutil.ReadFile(file1Path)
	if error1 != nil {
    
    
		fmt.Println("file1文件读取有误,error=", error1)
		return
	}

	//2、打开文件2（写+追加）
	//os中是带缓存的，需要Close()
	file2, error2 := os.OpenFile(file2Path, os.O_WRONLY | os.O_APPEND, 0666)
	if error2 != nil {
    
    
		fmt.Println("文件打开出错,", error2)
		return
	}
	defer file2.Close()//关闭文件资源

	//3、写文件
	writer := bufio.NewWriter(file2)
	length, error3 := writer.Write(data)
	fmt.Println("length:", length)
	if error3 != nil {
    
    
		fmt.Println("写入过程有误,", error3)
		return
	}

	//4、刷新缓存
	writer.Flush()
	fmt.Println("成功写入到carter.txt中")
}

Does the file exist?

• os.Stat()function
  • The returned error is nil, indicating that the file (folder) exists
  • The returned error is IsNotExist( ), indicating that the file (folder) does not exist
  • Other types are returned, indicating that the file (folder) is not sure whether it exists

file exists function

package main

import (
	"fmt"
	"os"
)

func FileExists(filePath string) (bool, error)  {
    
    
	fileInfo, error := os.Stat(filePath)
	if error == nil {
    
    
		if fileInfo.IsDir() {
    
    
			fmt.Println("这是个文件夹...")
		}
		return true, nil
	}
	if os.IsNotExist(error) {
    
    
		return false, nil
	}
	return false, error
}

func main() {
    
    
	flag, err := FileExists("D:/222s")
	if flag {
    
    
		fmt.Println("文件存在...")
	} else if err == nil {
    
    
		fmt.Println("文件不存在...")
	} else {
    
    
		fmt.Println(err)
	}
}

copy picture

If the file does not exist, create it yourself

func Copy

func Copy(dst Writer, src Reader) (written int64, err error)

Copies data from src to dst until EOF is reached on src or an error occurs. Returns the number of bytes copied and the first error encountered.

On successful calls, the return value of err is nil instead of EOF, because Copy is defined to read from src until EOF, it does not consider reading to EOF as an error that should be reported. If src implements the WriterTo interface , this function will call src.WriteTo(dst) to copy; otherwise, if dst implements the ReaderFrom interface , this function will call dst.ReadFrom(src) to copy.

the code

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"
)

//编写函数接收两个文件路径
func CopyFile(dstFileName string, srcFileName string) (written int64, err error) {
    
    
	srcFile, srcError := os.Open(srcFileName)
	if srcError != nil {
    
    
		fmt.Println("Error opening file,", srcError)
		return
	}
	defer srcFile.Close()//戒得关闭句柄
	//通过srcFile获取到Reader
	reader := bufio.NewReader(srcFile)
	//只写+创建
	dstFile, dstError := os.OpenFile(dstFileName, os.O_WRONLY | os.O_CREATE, 0666)
	if dstError != nil {
    
    
		fmt.Println("Error opening file,", dstError)
		return
	}
	defer dstFile.Close()//防止内存泄露
	//通过dstFileName获取到writer
	writer := bufio.NewWriter(dstFile)
	return io.Copy(writer, reader)//目标文件writer+源文件reader
}

func main() {
    
    
	srcFile := "C:/Users/Pictures/4K壁纸/美女.jpg"
	dstFile := "C:/Users/Desktop/abc.jpg"
	_, error := CopyFile(dstFile, srcFile)
	if error != nil{
    
    
		fmt.Println("Error Coping file,", error)
	} else {
    
    
		fmt.Println("Copy completed...")
	}
}

Statistics character type

1) Count the number of English, numbers, spaces and other characters

illustrate:

​ Count the number of English, numbers, spaces and other characters contained in a file

• train of thought
  • open a file
  • Create a Reader with cache
  • Statistics per Reader row
  • save the result in a structure

the code

package main

import (
	"bufio"
	"fmt"
	"io"
	"os"
)

type CharCount struct {
    
    
	EnglishCount int
	NumberCount int
	SpaceCount int
	OtherCount int
}

func main() {
    
    
	filePath := "d:/carter.txt"
	file, error := os.Open(filePath)
	defer file.Close()
	if error != nil {
    
    
		fmt.Println("Error opening file,", error)
		return
	}

	var charCount CharCount

	reader := bufio.NewReader(file)

	//循环读取文件内容
	for {
    
    
		str, err := reader.ReadString('\n')//换行符为准，进行分隔
		if err == io.EOF {
    
    
			break
		}
		if err != nil {
    
    
			fmt.Println("文件读取错误...")
			break
		}
		//兼容中文字符
		strSlice := []rune(str)
		for _, value := range strSlice {
    
    
			switch {
    
    //注意：这个地方不要key，分支结构
			case value >= 'a' && value <= 'z':
				charCount.EnglishCount++
			case value >= 'A' && value <= 'Z':
				charCount.EnglishCount++
			case value == ' ' || value == '\t':
				charCount.SpaceCount++
			case value > '0' && value < '9':
				charCount.NumberCount++
			default:
				charCount.OtherCount++
			}
		}
	}
	fmt.Println("字母个数:", charCount.EnglishCount)
	fmt.Println("数字个数:", charCount.NumberCount)
	fmt.Println("空格个数:", charCount.SpaceCount)
	fmt.Println("其它字符个数:", charCount.OtherCount)
}

result

字母个数: 4
数字个数: 4
空格个数: 4
其它字符个数: 5

in conclusion

Pay attention to the branch usage of switch

Args parses command line arguments

We hope to be able to obtain various parameters entered in the password line, how to deal with it?

os.ArgsIt is a string slice, counted by spaces

the code

package main

import (
	"fmt"
	"os"
)

func main() {
    
    
	fmt.Println("参数个数:", len(os.Args))
	for index, value := range os.Args {
    
    
		fmt.Printf("args[%v]=%v\n", index, value)
	}
}

run

D:\Work\Goland\Go\src\showComannd\main>go build -o test.exe main.go
D:\Work\Goland\Go\src\showComannd\main>test.exe d:/aaa/bbb/axis.log 88
参数个数: 3
args[0]=test.exe
args[1]=d:/aaa/bbb/axis.log
args[2]=88

The flag package parses command line arguments

template:

mysql -u carter -p 1234 -h localhost -P 3306

the code

package main

import (
	"flag"
	"fmt"
)

func main() {
    
    
	var username string
	var password string
	var host string
	var port int

	flag.StringVar(&username, "u", "", "获取用户名，默认为空")
	flag.StringVar(&password, "p", "", "获取密码，默认为空")
	flag.StringVar(&host, "h", "localhost", "主机默认是localhost")
	flag.IntVar(&port, "P", 3306, "端口默认3306")

	//必须调用该方法
	flag.Parse()

	//输出结果
	fmt.Printf("username=%v\n", username)
	fmt.Printf("password=%v\n", password)
	fmt.Printf("host=%v\n", host)
	fmt.Printf("port=%v\n", port)
}

result

D:\Work\Goland\Go\src\showComannd\flag\main>main.exe -u carter -p 1122
username=carter
password=1122
host=localhost
port=3306

Golang JSON

Basic introduction to JSON

1) JSON (JavaScript Object Notation) is a lightweight data exchange format

2) JSON was promoted and used in 2001, and it is now the mainstream data format

3) JSON is easy for machine parsing and generation, and effectively improves network transmission efficiency

4) Usually, during network transmission, the data (structure, map, etc.) will be serialized into a json string, and when the receiver gets the json string, it will be deserialized and restored to the original data type

Json data format description

In the JS language, everything is an object .

Use of Json serialization

• introduce

Json serialization refers to the operation of serializing data types with key-value structures (such as structures, maps, slices) into json strings

• Applications

Serialize structs, maps, and slices

• code demo

package main

import (
	"encoding/json"
	"fmt"
)

// Define a structure
type Student struct {
    
    
	Name string `json:"name"`
	Age int	`json:"age"`
	Birthday string `json:"birthday"`
	Salary float64	`json:"salary"`
	Skill string	`json:"skill"`
}

func testStruct() {
    
    
	var stu Student = Student{
    
    
		Name:     "孙悟空",
		Age:      700,
		Birthday: "1211-11-12",
		Salary:   10000.50,
		Skill:    "筋斗云",
	}
	// Serialization
	bytes, error := json.Marshal(&stu)
	if error != nil {
    
    
		fmt.Println("Serialization error，error=", error)
		return
	}
	fmt.Println("Student序列化后的结果：", string(bytes))
}

func testMap() {
    
    
	// 1、Define a map
	var map01 map[string]interface{
    
    }
	// 2、Use map ，need to make
	map01 = make(map[string]interface{
    
    })
	map01["name"] = "唐僧"
	map01["sex"] = "男"
	map01["address"] = "东土大唐"
	data, error := json.Marshal(map01)
	if error != nil {
    
    
		fmt.Println("Serialization error,error=", error)
		return
	}
	fmt.Println("map序列化后的结果：", string(data))
}

// Slice contains multiple maps
func testSlice() {
    
    
	var slice []map[string]interface{
    
    }
	var map1 map[string]interface{
    
    }
	// Need to make before use
	map1 = make(map[string]interface{
    
    })
	map1["name"] = "carter"
	map1["age"] = 18
	map1["sex"] = "man"

	var map2 map[string]interface{
    
    }
	map2 = make(map[string]interface{
    
    })
	map2["name"] = "廖述幸"
	map2["age"] = 20
	map2["sex"] = "男"

	slice = append(slice, map1, map2)
	// Slice serialization
	data, error := json.Marshal(slice)
	if error != nil {
    
    
		fmt.Println("Serialization error,error=", error)
		return
	}
	fmt.Println("slice序列化后的结果：", string(data))
}

func main() {
    
    
	// 1、Structure serialization
	testStruct()
	// 2、Map serialization
	testMap()
	// 3、Slice serialization
	testSlice()
}

• result

Student序列化后的结果： {
    
    "name":"孙悟空","age":700,"birthday":"1211-11-12","salary":10000.5,"skill":"筋斗云"}
map序列化后的结果： {
    
    "address":"东土大唐","name":"唐僧","sex":"男"}
slice序列化后的结果： [{
    
    "age":18,"name":"carter","sex":"man"},{
    
    "age":20,"name":"廖述幸","sex":"男"}]

Basic data type Json serialization

• After serialization, it is itself, meaningless
• code demo

package main

import (
	"encoding/json"
	"fmt"
)
func testBasicDataTypes() {
    
    
	var num float64 = 123.45
	data, error := json.Marshal(num)
	if error != nil {
    
    
		fmt.Println("Serialization error,error=", error)
		return
	}
	fmt.Println("num float64序列化后的结果：", string(data))
}
func main() {
    
    
	// Serialization of basic data types
	testBasicDataTypes()
}

• result

num float64序列化后的结果： 123.45

deserialization

basic introduction

• introduce

Json deserialization refers to the operation of deserializing Json strings into corresponding data types (such as structures, maps, slices)

Deserialize Json into structures, maps, and slices

• Notice
  • The type is consistent before and after deserialization
  • The make operation is encapsulated in Unmarshal
  • json.Unmarshal([]byte("serialized value"), & type)

the code

package main

import (
	"encoding/json"
	"fmt"
)
// Define a structure
type Student struct {
    
    
	Name string `json:"name"`
	Age int	`json:"age"`
	Birthday string `json:"birthday"`
	Salary float64	`json:"salary"`
	Skill string	`json:"skill"`
}

func unMarshalStruct() {
    
    
	str := `{"name":"孙悟空","age":700,"birthday":"1211-11-12","salary":10000.5,"skill":"筋斗云"}`
	//define a Student instance
	var stu Student
	// Parameter is pointer
	error := json.Unmarshal([]byte(str), &stu)
	if error != nil {
    
    
		fmt.Println("Struct反序列化失败,error", error)
		return
	}
	fmt.Println("Struct反序列化的结果：", stu)
}

func unMarshalMap() {
    
    
	map01 := `{"address":"东土大唐","name":"唐僧","sex":"男"}`
	//Need not to make, Unmarshal has make operate
	var mapResult map[string]interface{
    
    }
	error := json.Unmarshal([]byte(map01), &mapResult)
	if error != nil {
    
    
		fmt.Println("Map反序列化失败,error", error)
		return
	}
	fmt.Println("Map反序列化的结果：", mapResult)
}
func unMarshalSlice() {
    
    
	slice := `[{"age":18,"name":"carter","sex":"man"},{"age":20,"name":"廖述幸","sex":"男"}]`
	//Need not to make, Unmarshal has make operate
	var sliceResult []map[string]interface{
    
    }
	error := json.Unmarshal([]byte(slice), &sliceResult)
	if error != nil {
    
    
		fmt.Println("Slice反序列化失败,error", error)
		return
	}
	fmt.Println("Slice反序列化的结果：", sliceResult)
}

func main() {
    
    
	// 1、Unmarshal structure
	unMarshalStruct()
	// 2、Unmarshal map
	unMarshalMap()
	// 3、Unmarshal slice
	unMarshalSlice()
}

result

Struct反序列化的结果： {
    
    孙悟空 700 1211-11-12 10000.5 筋斗云}
Map反序列化的结果： map[address:东土大唐 name:唐僧 sex:男]
Slice反序列化的结果： [map[age:18 name:carter sex:man] map[age:20 name:廖述幸 sex:男]]