Python reads binary files (simple white version) Take sea temperature data reading of PIOMAS as an example

total code

import numpy as np
import struct

def get_data_level1(datafile, n1=360, n2=120, levels=10):
    data = np.zeros(n1 * n2 * levels * 12)
    i = 0
    #"rb"二进制读取
    fh = open(datafile, "rb")
    bys = fh.read(4)
    while bys:
        #每读一个数，存为一个元组a=(3.14，)
        bys_unpk = struct.unpack("f", bys)
        data[i] = bys_unpk[0]
        i += 1
        bys = fh.read(4)

    fh.close()
    data = data.reshape(12, levels, n1 * n2)
    data_level1 = data[:, 0, :]

    return data_level1

def get_grid_lonlat(gridfile, n1=360, n2=120):
    lonlat_arr = np.zeros(n1 * n2 * 2)
    i = 0
    fh = open(gridfile)
    for line in fh:
        linestrs = line.split()
        for s in linestrs:
            lonlat_arr[i] = float(s)
            i += 1

    fh.close()
    lon_arr = lonlat_arr[:n1 * n2]
    lon_arr[lon_arr > 180] = lon_arr[lon_arr > 180] - 360
    lat_arr = lonlat_arr[n1 * n2:]

    return lon_arr, lat_arr


if __name__ == "__main__":
    gridfile = "/g/aaa/sst_dir/grid.dat"
    datafile = "/g/aaa/sst_dir/otemp1_10.H2002"

    lon_arr, lat_arr = get_grid_lonlat(gridfile)
    otemp1 = get_data_level1(datafile)

    #输出经纬度和温度数据
    print(lon_arr, lat_arr)
    print(otemp1)

Code explanation:

There are two binary data I read, one is the temperature data of PIOMAS (otemp1_10.H2002), and the other is the latitude and longitude data corresponding to the temperature data (grid.dat).

1. Temperature data of PIOMAS (otemp1_10.H2002)

Binary files are directly stored in bytes in a certain order, so the whole file is a bunch of 0s and 1s, we need to decode them, how to decode them, we have to go to the website to find the data type of the data , to decode according to different data types.

According to the information on the webpage, it can be seen that the single-precision floating-point number stored in otemp1_10.H2002, that is, 4 bytes is a temperature data, so first we need to read the file sequentially in units of 4 bytes.

(In addition, since the grid point of otemp1_10.H2002 is 360*120, there are 10 different depths, 12 months, so I first created a matrix of 360 * 120 * 10 * 12, you can modify it according to your own data dimension and size )

After reading, put it into the pre-created matrix, and reshape the matrix according to the storage order of the binary file data. (This also needs to be found on the website. For example, my binary crisis is stored in the order of month-depth-grid, so I also reshape the matrix in this order)

def get_data_level1(datafile, n1=360, n2=120, levels=10):
    data = np.zeros(n1 * n2 * levels * 12)
    i = 0
    #"rb"二进制读取
    fh = open(datafile, "rb")
    bys = fh.read(4)
    while bys:
        #每读4个字节即为一个温度数据，存为一个元组 eg:a=(3.14，)
        bys_unpk = struct.unpack("f", bys)
        data[i] = bys_unpk[0]
        i += 1
        bys = fh.read(4)

    fh.close()
    data = data.reshape(12, levels, n1 * n2)
    data_level1 = data[:, 0, :]

    return data_level1

2. Latitude and longitude data (grid.dat)

This is not a binary file. The storage order of the file is: first save all longitudes, then save all latitudes, open it with Notepad and see:

Just read each line of split directly

def get_grid_lonlat(gridfile, n1=360, n2=120):
    lonlat_arr = np.zeros(n1 * n2 * 2)
    i = 0
    fh = open(gridfile)
    for line in fh:
        linestrs = line.split()
        for s in linestrs:
            lonlat_arr[i] = float(s)
            i += 1

    fh.close()
    lon_arr = lonlat_arr[:n1 * n2]
    #将0-360经度转为-180-180
    lon_arr[lon_arr > 180] = lon_arr[lon_arr > 180] - 360
    lat_arr = lonlat_arr[n1 * n2:]

    return lon_arr, lat_arr