Original | A slight improper use of FastJson can lead to StackOverflow

Original | A slight improper use of FastJson can lead to StackOverflow

△Hollis, a person with a unique pursuit of Coding△

This is Hollis’s 235th original sharing

Author l Hollis
Source l Hollis (ID: hollischuang)
For the majority of developers, FastJson must be familiar to everyone.

FastJson ( https://github.com/alibaba/fastjson) is Alibaba’s open source JSON parsing library. It can parse strings in JSON format, support serialization of Java Beans into JSON strings, and reverse JSON strings. Serialize to JavaBean.

It has the characteristics of fast speed, wide use, complete testing and simple use. However, although there are so many advantages, it does not mean that you can use it casually, because if you use it in an incorrect way, it may cause StackOverflowError. And StackOverflowError is undoubtedly a disaster for the program.

The author encountered this situation during the first use of FastJson. Later, after in-depth source code analysis, I understood the principle behind this. This article comes from the reappearance of the scene, showing you where the pit is and how to avoid it.

Problem reappears

FastJson can help developers convert between Java Bean and JSON strings, so it is a way often used in serialization.

There are many times when we need to save some redundant fields in a table in the database, and these fields are generally stored in the form of JSON strings. For example, we need to redundant some basic buyer information in the order table, such as JSON content:

{
    "buyerName":"Hollis",
    "buyerWechat":"hollischuang",
    "buyerAgender":"male"
}

Because these fields are redundant, there must be some place to read the values ​​of these fields. Therefore, for the convenience of use, a corresponding object is generally defined.

I recommend an IDEA plug-in—JsonFormat, which can generate a JavaBean from a JSON string with one click. We get the following Bean:

public class BuyerInfo {

    /**
     * buyerAgender : male
     * buyerName : Hollis
     * buyerWechat : [email protected]
     */
    private String buyerAgender;
    private String buyerName;
    private String buyerWechat;

    public void setBuyerAgender(String buyerAgender) { this.buyerAgender = buyerAgender;}
    public void setBuyerName(String buyerName) { this.buyerName = buyerName;}
    public void setBuyerWechat(String buyerWechat) { this.buyerWechat = buyerWechat;}
    public String getBuyerAgender() { return buyerAgender;}
    public String getBuyerName() { return buyerName;}
    public String getBuyerWechat() { return buyerWechat;}
}

Then in the code, you can use FastJson to convert between JSON strings and Java Beans. Such as the following code:

Order order = orderDao.getOrder();

// 把JSON串转成Java Bean
BuyerInfo buyerInfo = JSON.parseObject(order.getAttribute(),BuyerInfo.class);

buyerInfo.setBuyerName("Hollis");

// 把Java Bean转成JSON串
order.setAttribute(JSON.toJSONString(buyerInfo));
orderDao.update(order);

Sometimes, if there are multiple places that need to be converted to each other in this way, we will try to encapsulate a method in BuyerInfo to specifically convert the object into a JSON string, such as:

public class BuyerInfo {

    public String getJsonString(){
        return JSON.toJSONString(this);
    }
}

However, if we define such a method, we will have problems when we try to convert BuyerInfo into a JSON string, such as the following test code:

public static void main(String[] args) {

    BuyerInfo buyerInfo = new BuyerInfo();
    buyerInfo.setBuyerName("Hollis");

    JSON.toJSONString(buyerInfo);
}

operation result:

As you can see, after running the above test code, StackOverflow was thrown when the code was executed.

From the above screenshot, we can see that the exception stack is mainly caused by the execution of the getJsonString method of BuyerInfo.

So, why does such a problem occur? This is related to the realization principle of FastJson.

The realization principle of FastJson

For the basic knowledge of serialization and deserialization, you can refer to the serialization and deserialization of Java objects, which will not be repeated here.

The serialization process of FastJson is to convert a Java Bean in memory into a JSON string. After the string is obtained, it can be persisted through the database and other methods.

So, how does FastJson convert a Java Bean into a string? There are many properties and methods in a Java Bean. Which properties should be retained and which should be removed? What kind of principle does it follow?

In fact, for the JSON framework, if you want to convert a Java object into a string, you have two options:

• 1. Based on attributes.
• 2. Based on setter/getter

关于Java Bean中的getter/setter方法的定义其实是有明确的规定的，参考JavaBeans(TM) Specification

In our commonly used JSON serialization framework, when FastJson and Jackson serialize an object into a json string, they do so by traversing all the getter methods in the class. Gson does not do this, he traverses all the attributes in the class through reflection and serializes its values ​​into json.

Different frameworks for different choices have different thinking. If you are interested, the follow-up text can specifically introduce it.

So, let's dive into the source code next to verify if this is the case.

When analyzing the problem, the best way is to follow the exception stack information and look a little bit. Let's look back at the stack of the previous exception:

We simplify, we can get the following call chain:

BuyerInfo.getJsonString 
    -> JSON.toJSONString
        -> JSONSerializer.write
            -> ASMSerializer_1_BuyerInfo.write
                -> BuyerInfo.getJsonString

It is because of an infinite loop when FastJson converts Java objects into strings, which causes StackOverflowError.

The ASMserializer_1_BuyerInfo in the call chain is actually a Serializer generated by FastJson for BuyerInfo using ASM, and this Serializer is essentially the JavaBeanSerizlier built into FastJson.

Readers can experiment by themselves. For example, by degbug in the following way, you can find that ASMserializer_1_BuyerInfo is actually JavaBeanSerizlier.

之所以使用ASM技术，主要是FastJson想通过动态生成类来避免重复执行时的反射开销。但是，在FastJson中，两种序列化实现是并存的，并不是所有情况都需要通过ASM生成一个动态类。读者可以尝试将BuyerInfo作为一个内部类，重新运行以上Demo，再看异常堆栈，就会发现JavaBeanSerizlier的身影。

So, since the StackOverflowError is caused by the cyclic call, we will focus on why the cyclic call occurs.

JavaBeanSerizlier serialization principle

We already know that in the process of FastJson serialization, JavaBeanSerizlier will be used, so let's see what JavaBeanSerizlier does and how it helps FastJson to serialize.

FastJson will call the write method of JavaBeanSerizlier during the serialization process. Let's take a look at the content of this method:

public void write(JSONSerializer serializer, Object object, Object fieldName, Type fieldType, int features) throws IOException {
    SerializeWriter out = serializer.out;
    // 省略部分代码
    final FieldSerializer[] getters = this.getters;//获取bean的所有getter方法
    // 省略部分代码
    for (int i = 0; i < getters.length; ++i) {//遍历getter方法
        FieldSerializer fieldSerializer = getters[i];
        // 省略部分代码
        Object propertyValue;
        // 省略部分代码
        try {
            //调用getter方法，获取字段值
            propertyValue = fieldSerializer.getPropertyValue(object);
        } catch (InvocationTargetException ex) {
            // 省略部分代码
        }
        // 省略部分代码
    }
}

After omitting most of the above code, we can see that the logic is relatively simple: first obtain all getter methods of the object to be serialized, and then traverse the methods to execute, and the view obtains the value of the corresponding property through the getter method.

However, when the getJsonString method defined by us is called, JSON.toJSONString(this) will be called, and then the write of JavaBeanSerizlier will be called again. Such reciprocation forms an endless loop, and then a StackOverflowError occurs.

So, if you define a Java object, define a getXXX method, and call the JSON.toJSONString method in this method, a StackOverflowError will occur!

How to avoid StackOverflowError

By checking the source code of FastJson, we have basically located the problem, so how to avoid this problem?

Still starting from the source code, since the write method of JavaBeanSerizlier will try to get all the getter methods of the object, then let's see how he gets the getter method, and which methods will be recognized as "getters" by him, and then we will prescribe the right medicine .

In the write method of JavaBeanSerizlier, getters are obtained as follows:

final FieldSerializer[] getters;

if (out.sortField) {
    getters = this.sortedGetters;
} else {
    getters = this.getters;
}

It can be seen that whether it is this.sortedGetters or this.getters, they are all properties in JavaBeanSerizlier, then continue to look up to see how JavaBeanSerizlier is initialized.

By tracing the source of the call stack, we can find that JavaBeanSerizlier is obtained in the member variable serializers of SerializeConfig, so we need to see how SerializeConfig is initialized, that is, how the JavaBeanSerizlier corresponding to BuyerInfo is stuffed into serializers.

Through the call relationship, we found that SerializeConfig.serializers is plugged in by the SerializeConfig.putInternal method:

And there is a call to putInternal in getObjectWriter:

putInternal(clazz, createJavaBeanSerializer(clazz));

Here comes the JavaBeanSerializer we mentioned earlier. We know how createJavaBeanSerializer creates JavaBeanSerializer and how to set the setters.

private final ObjectSerializer createJavaBeanSerializer(Class<?> clazz) {
    SerializeBeanInfo beanInfo = TypeUtils.buildBeanInfo(clazz, null, propertyNamingStrategy);
    if (beanInfo.fields.length == 0 && Iterable.class.isAssignableFrom(clazz)) {
        return MiscCodec.instance;
    }

    return createJavaBeanSerializer(beanInfo);
}

The point is here, TypeUtils.buildBeanInfo is the point, and here is what we are looking for.

buildBeanInfo calls computeGetters, dive into this method, and see how the setters are identified. Part of the code is as follows:

for (Method method : clazz.getMethods()) {
    if (methodName.startsWith("get")) {
            if (methodName.length() < 4) {
                continue;
            }

            if (methodName.equals("getClass")) {
                continue;
            }

            ....
    }
}

This method is very long and very long. The above is just a part of it. The above is just a simple judgment. Whether the method starts with'get', and then the length is less than 3, in judging whether the method name is getClass, etc. Wait for a series of judgments.

Below I simply drew a picture that lists the core judgment logic:

Then, through the above figure, we can see that the computeGetters method has a certain logic when filtering the getter method. As long as we find a way to use these logic, we can avoid StackOverflowError.

I want to mention here that the several methods that will be introduced below are all trying to make the target method not participate in serialization, so pay special attention to it. But then again, who would serialize a JavaBean's toJSONString?

1. Modify the method name

First of all, we can solve this problem by modifying the method name. Let's change the name of the getJsonString method, as long as it does not start with get, such as toJsonString.

public class Main {
    public static void main(String[] args) {
        BuyerInfo buyerInfo = new BuyerInfo();
        buyerInfo.setBuyerName("Hollis");
        JSON.toJSONString(buyerInfo);
    }
}

class BuyerInfo {
    private String buyerAgender;
    private String buyerName;
    private String buyerWechat;

    //省略setter/getter

    public String toJsonString(){
        return JSON.toJSONString(this);
    }
}

2. Use JSONField annotation

In addition to modifying the method name, FastJson also provides two annotations for us to use. First, introduce the JSONField annotation. This annotation can be applied to the method. If the parameter serialize is set to false, then this method will not be recognized as a getter method. , It will not participate in serialization.

public class Main {
    public static void main(String[] args) {
        BuyerInfo buyerInfo = new BuyerInfo();
        buyerInfo.setBuyerName("Hollis");
        JSON.toJSONString(buyerInfo);
    }
}

class BuyerInfo {
    private String buyerAgender;
    private String buyerName;
    private String buyerWechat;

    //省略setter/getter

    @JSONField(serialize = false)
    public String getJsonString(){
        return JSON.toJSONString(this);
    }
}

3. Use JSONType annotation

FastJson also provides another annotation-JSONType, this annotation is used to modify the class, you can specify ignores and includes. As in the following example, if you use @JSONType(ignores = "jsonString") to define BuyerInfo, StackOverflowError can also be avoided.

public class Main {
    public static void main(String[] args) {
        BuyerInfo buyerInfo = new BuyerInfo();
        buyerInfo.setBuyerName("Hollis");
        JSON.toJSONString(buyerInfo);
    }
}

@JSONType(ignores = "jsonString")
class BuyerInfo {
    private String buyerAgender;
    private String buyerName;
    private String buyerWechat;

    //省略setter/getter    

    public String getJsonString(){
        return JSON.toJSONString(this);
    }
}

to sum up

FastJson is a very widely used serialization framework that can convert between JSON strings and Java Beans.

But pay special attention when using it, do not call the JSON.toJSONString method in the getXXX method of Java Bean, otherwise it will cause StackOverflowError.

The reason is because when FastJson is serialized, it will obtain all the getter methods in an object according to a series of rules, and then execute them in turn.

If you must define a method, call JSON.toJSONString, to avoid this problem, you can use the following methods:

• 1. The method name does not start with get
• 2. Use @JSONField(serialize = false) to modify the target method
• 3. Use @JSONType to modify the Bean, and ignore the attribute name corresponding to the method (getXxx -> xxx).
  Finally, the author wrote this article because he really encountered this problem in his work.

When a problem occurred, I immediately thought of changing the method name and changing getJsonString to toJsonString to solve the problem. Because I have seen the simple principle of FastJson before.

Later I thought, since FastJson is designed to serialize through getters, he must provide a hole for developers to specify that certain methods beginning with get do not participate in serialization.

The first time I thought that this kind of opening is usually realized through annotations, so I opened the source code of FastJson and found the corresponding annotations.

Then, taking advantage of the weekend, I dug through the source code of FastJson, and thoroughly figured out the true underlying principles.

The above is the whole process that I found the problem -> analyze the problem -> solve the problem -> problem sublimation, I hope it will be helpful to you.

Through this incident, the author realized a truth:

I have read too many development specifications, but I still write BUG!

I hope that through such a small article, you can get a basic impression of this problem. In case you encounter a similar problem one day, you can immediately think that Hollis seems to have written such an article. That's enough!

Welcome everyone to pay attention to the Java Way public account and publish original Java technical articles regularly~

• MORE | More exciting articles-

• Arthas-the ultimate tool for Java online problem location processing
• If someone asks you why MySQL uses B+ tree as index, send this article to her
• On Double Eleven, even Dangdang officials hoped that your wool was discovered by me!
• I didn't understand Nginx after reading this article, then I cried!

If you like this article,
please press

and hold the QR code and follow Hollis. Forward it to the circle of friends. This is my greatest support.

Forward + watching, let more people see it.