Detailed explanation of tick data, handicap data, and transaction details data

Recently, I started to learn order book strategy and order flow strategy. Sorting out the data is the first step.

1. Transaction details

Use various software to watch the market. In addition to the most commonly used time sharing and K-line, the market data/transaction details will also be displayed. The picture below is pushed by wind and multicharts before the closing of the thread main force shfe.rb2401 on October 24, 2023. Transaction details.

The first problem I encounter is if I want to use these data for analysis, how to download and save this part of the data?
I used wind a lot before, but I didn’t find how to download this part of the data on wind. Tongdaxin software can download historical time-sharing data, but it downloads it day by day. Manual downloading of data for multiple days is very time-consuming. hour.
After careful study and understanding, I discovered that this part of the data can be directly processed and converted from the tick data. Understanding the data generation can greatly improve the efficiency and facilitate the subsequent use of this part of the data.

2. Tick data

• Tick ​​quotations, also known as tick-by-tick quotes, are tick-by-tick data on the entire market. For example, a new order from an investor will form a market price, and a new transaction matched by the exchange will also form a market price. The tick market records the data of every event in the market, which is the most detailed and complete data.

• Snapshot market is also called slice (snapshot) market. As its name suggests, it is sliced ​​data of tick market data at a certain moment. For example, the common 500ms market price of futures is a snapshot of the highest price, lowest price, trading volume, etc. within this time period every 500ms.

At present, there is no real tick data in the domestic futures market. The tick data mentioned at this time refers to the snapshot market.
The snapshot market can be divided into level1 and level2 data. The main difference is the quotation depth.

• Level 1 market is also called level 1 market or basic market. Literally, it means that there is only one level in the market information book, that is, one price for buying and one price for selling. In practical applications, it is generally used to refer to a relatively basic market situation, which only includes the lowest level of buying and selling, transaction data, and the lowest update frequency.

• Level 2 market, also known as multi-level market or in-depth market, can be considered as an upgraded version of level 1 market, which is reflected in the order book having more levels, rich data, or faster update frequency. This kind of market is usually charged separately.

Taking rb2401 as an example, Tianqin was used to download the tick data (actually snapshot data) on October 24, 2023 (only the day disk was downloaded, and the 10.23 night disk was not downloaded, which does not affect subsequent analysis). market data).

The downloaded tick data looks like this, mainly including transaction and order data. You can download the details by yourself. You can find that the current position, position difference, and opening direction provided by the software are not original data, but are calculated using the last_price, volume, open_interst, bid_price1, and ask_price1 fields.

It can also be found that compared with the data returned by the exchange, the transaction details of the software vendor are missing a few items. They are only pushed when the transaction volume changes. Subsequently, the last two lines of data when the transaction volume changes (data marked yellow in the figure) ) as an example to perform sample calculations.

3. Use tick data to calculate transaction details

1. Ready to hand

The volume field is the cumulative trading volume field. The current data is easy to calculate. It is the difference between volume, that is, the next data volume - the previous data volume. The calculation of the yellow marked data on the chart is 1443112-1443011=101, which is consistent with the data pushed by the software.

2. Warehouse difference

The open_interest field is the number of positions, and the position difference is the difference between open_interest. The figure is 1862177-1862180=-3, which is consistent with the data pushed by the software.

3. Kaiping

The direction of opening and closing is related to the current position, increasing positions, and the direction of price changes. For details, you can see the following tables:

• Kaiping direction
  
• Price movement direction and internal and external market direction
  
• Summary
  

4. python implementation

import pandas as pd
import numpy as np

# 读取数据
rb_data = pd.read_csv("RB2401_tick_1024.csv")
# 处理列名
cols = [i.replace("SHFE.rb2401.", "").strip() for i in rb_data.columns]
rb_data.columns = cols


def get_ticks_info(df):
    df_pre = df.copy().shift(1)
    df_pre["price_diff"] = df["last_price"] - df_pre["last_price"]
    df_pre["oi_diff"] = df["open_interest"] - df_pre["open_interest"]
    df_pre["vol_diff"] = df["volume"] - df_pre["volume"]
    
    df_pre["pc"] = np.where(
        df["last_price"] <= df_pre["bid_price1"], -1,
        np.where(df["last_price"] >= df_pre["ask_price1"], 1,
                 np.sign(df_pre["price_diff"])))
    pc_g = df_pre["pc"] > 0

    condition_buy = (df["last_price"] >= df_pre["ask_price1"]) | (df["last_price"] >= df["ask_price1"])
    condition_sell = (df['last_price'] <=df_pre['bid_price1']) |(df['last_price'] <= df['bid_price1'])
    direction_values = np.where(condition_buy, '主动买', np.where(condition_sell, '主动卖', '其他'))
    df_pre['direction'] = pd.Series(direction_values, index=df.index)

    df_pre["info"] = pd.Series(
        np.where(
            df_pre["oi_diff"] > 0, np.where(pc_g, "多开", "空开"),
            np.where(
                df_pre["oi_diff"] < 0, np.where(pc_g, "空平", "多平"),
                np.where(df_pre["oi_diff"] == 0, np.where(pc_g, "多换", "空换"),
                         ""))))
    df_pre.loc[df_pre["pc"] == 0, "info"] = "换手"
    df_pre.loc[(df_pre["oi_diff"] < 0) &
               (df_pre["oi_diff"] + df_pre["vol_diff"] == 0), "info"] = "双平"
    df_pre.loc[(df_pre["oi_diff"] > 0) &
               (df_pre["oi_diff"] == df_pre["vol_diff"]), "info"] = "双开"
    df_pre.loc[df_pre["vol_diff"] == 0, "info"] = ""

    return df_pre.shift(-1)

data = get_ticks_info(rb_data)

Tick ​​sample data and code follow the WeChat public account: Quantitative Learning J (public account WeChat ID: quant_study_june), reply to tick data processing, and send the network disk link