The following content comes from Shang Silicon Valley. I wrote this series of articles mainly to facilitate my own subsequent viewing. I don’t have to carry around a PDF to find it, which is too troublesome!
Chapter 16 How the time series database stores user names and passwords
1. InfluxDB comes with a BlotDB written in Go language. BlotDB is a key-value database. Its functions are relatively limited. It basically focuses on storing and reading values. At the same time, because of its limited functions, it can also be made small and lightweight.
2. InfluxDB stores username, password, token and other information in such a key-value database. By default, BlotDB data will be stored in a separate file, which will be in the ~/.influxdbv2/ path and named influxd.bolt. The path of this file can be modified through the bolt-path configuration item in influxd.
Chapter 17 Migrating data from InfluxDB OSS
17.1 Export data from InfluxDB
1. To export InfluxDB data, you must use the influxd command (note, not the influx command). In InfluxDB2.x, data export is based on buckets. Here are sample commands:
influxd inspect export-lp \
--bucket-id 12ab34cd56ef \
--engine-path ~/.influxdbv2/engine \
--output-path path/to/export.lp \
--start 2022-01-01T00:00:00Z \
--end 2022compress-01-31T23:59:59Z \
--compress
2. Parameter explanation:
- influxd inspect, influxd is a command line tool that can operate the InfluxDB service process, inspect is a subcommand of the influxd command
- export-lp is the abbreviation of export xxx to line protocol, which means exporting data to line protocol. It is a subcommand of inspect.
- bucket-id, a required parameter of inspect. bucket id
- engine-path, a required parameter of inspect, but has a default value
~/.influxdbv2/engine. So if your data directory is~/.influxdbv2/engine, then it is okay not to specify this parameter. - output-path, a required parameter of inspect, specifies the location of the output file.
- start, optional, the start time of exporting data
- end, optional, the end time of exported data.
- compress, it is recommended to enable it. If enabled, influxd will use gzip to compress the output data.
17.2 Example: Export data from InfluxDB
1. This time, we try to export the data of test_init. As of now, the data in this bucket should be the most.
2. First, you can use influx-cli or Web UI to view the ID corresponding to the bucket we want to export. Here, choose to use the Web UI, and you can see that the ID of the test_init bucket is 0a2e821ccd12854a.
3. So, we run the following command and try to export the data.
./influxd inspect export-lp --bucket-id a2e821ccd12854a --out-path 0./oh.lp
4. This command will export the data in the test_init bucket to the oh.lp file in the current directory in the format of InfluxDB row protocol. Under normal circumstances, the program will output a series of read and write information.
5. Use the following command to view the files and their sizes in the current path. ls -lh, the h parameter of ls, can print the number of bytes of the file into easier-to-read MB and GB units. As you can see, the data file oh.lp we exported is 1.5G in size.
6. Now, we use the tail command to view the contents of the file. The command outputs the last 15 lines of the file, and you can see that they are all InfluxDB line protocol data.
tail -15 ./oh.lp
7. However, we should pay attention to a characteristic of the InfluxDB row protocol. In fact, for the entire file, the measurements of multiple pieces of data are actually repeated. The repetition rate of tagset is not low, and the change of filed is not great. This highly repetitive data is actually very suitable for compression algorithms.
17.3 Example: Compression when exporting data
1. Now, we re-run the data export command, this time adding the –compress parameter at the end of the command. Don't worry about the oh.lp file already existing in the directory, the program will overwrite it directly.
./influxd inspect export-lp --bucket-id a2e821ccd12854a --out-path 0./oh.lp - -compress
2. Use the ls command to check the file size again. You can see that the file has changed from the previous 1.5G to the current 9.1M, and the compression rate is very high.
ls -lh
