-
Preface
I had completed the analysis of the Binary market before, and then continued to study the FAST (STEP) market, but it took nearly a month to finally get it. As mentioned before, the market in Binary format is not very intuitive, so it is a bit difficult for beginners. Only after coming into contact with FAST did I know what "completely incomprehensible" means. Fortunately, the Internet is great, and the occasional words left by the big guys are a rare enlightenment for us.
At first I wanted to analyze human flesh, but I really didn’t understand it. The reference materials I found are as follows:
GitHub - kuangtu/fixfast: Analysis of fast protocol through OpenFast
The information in the above link is very comprehensive, but I’m sorry that I really didn’t understand it. I would like to recommend another information:
SSE Level 2 Vendor Interface Specification (FAST)-Chapter 15-FAST Decoder (FAST Analysis-Part 4)_wqfhenanxc's Blog-CSDN Blog (This is only a part of this blog There are other articles in the same series)
Combining the two, I finally figured out "stop bit" and "map", but the actual analysis was still wrong, so I gave up and started studying openfast.
In fact, the information about openfast is very rare, and most of them only mention a few words and are completely unusable. However, after my unremitting efforts, I still found some useful information.
Openfast download:
OpenFAST download | SourceForge.net
A very useful example:
With the previous binary experience and the blessing of the above information, I finally figured out the FAST market. However, so far, I have only used openfast to parse the content and have not done in-depth research, so it is only for your reference.
Also attached is the official reference document:
"Shanghai Stock Exchange Low Delay Distribution System (LDDS) Interface Instructions"
"Shanghai Stock Exchange LDDS System Level-1 FAST Market Interface Instructions"
"IS120_Shanghai Stock Exchange Quotation Gateway STEP Data Interface Specification"
-
FAST interface
To understand FAST, you must first understand the fix and step. The reference materials are as follows:
Domestic exchange protocol FIX STEP FAST Binary_wqfhenanxc's blog-CSDN blog_binary protocol
To put it simply, FIX is a pair of data like KEY=VALUE, separated by a special symbol. It is easier to parse and it is also easier to add and modify interfaces. But the disadvantage is that it is too lengthy, and the transmission of financial market data takes up a lot of bandwidth, which is not very scientific.
STEP looks similar to FIX, but some localization has been done. I haven't studied it in detail and don't know what modifications have been made, so I won't go into details.
Since FIX and STEP are relatively lengthy, FAST was created. First of all, FAST is also derived from the format of KEY=VALUE, and then compressed on this basis, so the so-called FAST analysis is to restore it to KEY=VALUE. The general method of FAST compression is as follows:
- Key is canceled and template is used to agree on the meaning of each field.
- Use binary instead of string form to transmit
- On the basis of binary, a very special (and incomprehensible) compression method is used to further compress it.
When the Shanghai Stock Exchange issued the FAST quotation, it also put a STEP shell on him, and it was a double shell, as shown in the picture below:
The content of STEP can be output directly into a string, which is very readable, but the content of FAST is all garbled because it is binary and compressed. Although the FAST protocol itself is difficult to understand, with the help of openfast we can easily parse the data, so the question now is how to obtain the content of FAST.
-
Log in
Similar to binary, you only need to send information to VDE in the specified format. This can be easily accessed by referring to the official documentation. Among them, "\u0001" is the delimiter SOH.
-
STEP analysis
If you directly output the content obtained from the interface as a string, you can see that the STEP part is easy to process, but the FAST part is all garbled, so the FAST part must be completely intercepted and parsed with openfast. If you use SOH to split it, it won't work, because there are a lot of data like "00000001" in FAST, so you have to parse the length of FAST first, and then read the byte array according to the length, read the byte array that is exactly that long, and directly complain Just parse it in openfast.
Here I am lazy and directly find the length of FAST by counting SOH, which is tag95. Note that because of the matryoshka relationship, it is the value of the second tag95.
-
FAST analysis
I read out the contents of FAST earlier. For analysis, just use openfast. The specific use of openfast seems to be quite diverse. I just wrote it casually with reference to the previous blog post without going into depth.
The output is as follows:
Since Chinese characters have to be specially processed, what is displayed here is garbled characters. In addition, for the sake of simplicity, I read them in string format, which is not suitable. You can do your own research. You can look at the openfast documentation to see what functions are available.
The complete code is as follows:
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.OutputStream;
import java.io.*;
import java.net.Socket;
import java.nio.ByteBuffer;
import java.util.Arrays;
import org.openfast.template.MessageTemplate;
import org.openfast.template.loader.XMLMessageTemplateLoader;
import org.openfast.Context;
import org.openfast.codec.FastDecoder;
import org.openfast.Message;
public class connSHFAST2{
public static String getCheckSum(String str){
byte[] strBytes = str.getBytes();
int sum = 0;
for(byte b : strBytes){
int bInt = b&0xff;
sum += bInt;
sum = sum&0xff;
}
String sumStr = sum + "";
if(sumStr.length()==1){
sumStr = "00"+sumStr;
}else if(sumStr.length()==2){
sumStr = "0"+sumStr;
}
return sumStr;
}
/**
* 把byte转化成2进制字符串
* @param b
* @return
*/
public static String getBinaryStrFromByte(byte b){
String result ="";
byte a = b; ;
for (int i = 0; i < 8; i++){
byte c=a;
a=(byte)(a>>1);//每移一位如同将10进制数除以2并去掉余数。
a=(byte)(a<<1);
if(a==c){
result="0"+result;
}else{
result="1"+result;
}
a=(byte)(a>>1);
}
return result;
}
static class ReadThread extends Thread{
InputStream readStream;
public ReadThread(InputStream readStream){
this.readStream = readStream;
}
@Override
public void run(){
try{
System.out.println("thread run...");
while(true){
byte[] buffer = new byte[102400];
int len = readStream.read(buffer,0,102400);
//读到的数据太少
if(len<28){
System.out.println("len<28");
Thread.sleep(1000);
continue;
}
//读到的数据太多
if(len>=102400){
System.out.println("len>=102400");
continue;
}
//截取有效部分
byte[] step = Arrays.copyOf(buffer,len);
//逐一读取byte，数SOH，数到第27个
int count = 0;
int start = 0;
len = 0;
for(int i=0; i<step.length; i++){
byte b = step[i];
int bInt = b & 0xFF;
if(bInt==1){
count = count + 1;
if(count == 1){
len = i - start;
byte[] BeginString = new byte[len];
System.arraycopy(step,start,BeginString,0,len);
System.out.println("BeginString="+new String(BeginString));
start = i+1;
}
if(count == 2){
len = i - start;
byte[] BodyLength = new byte[len];
System.arraycopy(step,start,BodyLength,0,len);
System.out.println("BodyLength="+new String(BodyLength));
start = i+1;
}
if(count == 3){
len = i - start;
byte[] MsgType = new byte[len];
System.arraycopy(step,start,MsgType,0,len);
System.out.println("MsgType="+new String(MsgType));
start = i+1;
if(!"35=UA5302".equals(new String(MsgType))){
System.out.println("MsgType!=UA5302");
break;
}
}
if(count == 26){
start = i+1;
}
if(count==27){
len = i - start;
byte[] Tag95 = new byte[len];
System.arraycopy(step,start,Tag95,0,len);
System.out.println("Tag95="+new String(Tag95));
byte[] FastLenBG = new byte[len-3];
System.arraycopy(Tag95,3,FastLenBG,0,len-3);
int FastLen = Integer.parseInt(new String(FastLenBG));
System.out.println("FastLen="+FastLen);
start = i+1;
//读取FAST内容
byte[] FAST = new byte[FastLen];
System.arraycopy(step,start+3,FAST,0,FastLen);
//System.out.println("FAST="+new String(FAST));
//解析FATS内容
InputStream inputStream = new FileInputStream("template.xml");
MessageTemplate template = new XMLMessageTemplateLoader().load(inputStream)[0];
Context context = new Context();
context.registerTemplate(4001, template);
InputStream sbs = new ByteArrayInputStream(FAST);
FastDecoder fastDecoder = new FastDecoder(context, sbs);
Message msg111 = fastDecoder.readMessage();
System.out.println("MDStreamID="+msg111.getString("MDStreamID"));
System.out.println("SecurityID="+msg111.getString("SecurityID"));
System.out.println("Symbol="+msg111.getString("Symbol"));
System.out.println("NumTrades="+msg111.getString("NumTrades"));
System.out.println("TradeVolume="+msg111.getString("TradeVolume"));
System.out.println("TotalValueTraded="+msg111.getString("TotalValueTraded"));
System.out.println("PrevClosePx="+msg111.getString("PrevClosePx"));
System.out.println("PrevSetPx="+msg111.getString("PrevSetPx"));
System.out.println("TotalLongPosition="+msg111.getString("TotalLongPosition"));
}
}
}
}
}catch(Exception e){
System.out.println(e.getMessage());
}
}
}
public static void main(String[] args){
try{
//head
String BeginString = "8=STEP.1.0.0\u0001";
String BodyLength = "9=56\u0001";//除了 8、9、10 域，所 有其他域长度总和
String MsgType = "35=A\u0001";
String SenderCompID = "49=VSS\u0001";
String TargetCompID = "56=VDE\u0001";
String MsgSeqNum = "34=0\u0001";
String SendingTime = "52=20220916-11:00:00\u0001";
//body
String EncryptMethod = "98=0\u0001";
String HeartBtInt = "108=0\u0001";
//check body，用于做校验和的
String CheckBody = BeginString+BodyLength+MsgType+SenderCompID
+TargetCompID+MsgSeqNum+SendingTime
+EncryptMethod+HeartBtInt;
//checksum 除了10域本身，对所有其他域的每个字节累加后取256的余数。余数不足3位的，前补0
String CheckSum = "10="+getCheckSum(CheckBody)+"\u0001";
String MsgBody = CheckBody + CheckSum;
//connect
Socket socket = new Socket("你的IP",端口);
OutputStream outStream = socket.getOutputStream();
outStream.write(MsgBody.getBytes());
outStream.flush();
//listion thread
ReadThread readThread = new ReadThread(socket.getInputStream());
readThread.start();
while(true){
Thread.sleep(3000);
System.out.println("every 3000...");
}
}catch(Exception e){
System.out.println(e.getMessage());
}
}
}