Original Title: Defining Smart Contract Defects on Ethereum
Original Authors: Jiachi Chen, Xin Xia, David Lo, John Grundy, Xiapu Luo and Ting Chen
Original Institutions: School of Information Technology, Monash University, Melbourne, Victoria, Australia, School of Information Systems, Singapore Management University,
Department of Computer Science, Hong Kong Polytechnic University and School of Computer Science and Engineering, University of Electronic Science and Technology of
China
This paper presents an empirical study of defining smart contract flaws on the Ethereum platform. Some previous work has focused on improving the quality of smart contracts from a security perspective, and has not verified whether practitioners consider these contract flaws to be harmful. This is the first paper aiming to systematically study contract flaws from five aspects of security, usability, performance, maintainability, and reusability. To address these limitations, we conducted a study on 17128 Ethereum. The main contributions of this paper are:
- This article defines 20 contract defects for smart contracts from five aspects of security, usability, performance, maintainability and reusability, lists the symptoms of each contract defect and gives a code example
- This article analyzes the impact of defined contract deficiencies and summarizes 5 common ones that can help in prioritizing deficiencies removal.
- The work in this paper is the first empirical study on the flaws of smart contracts. The goal is to determine their importance and to gather the views of practitioners. This work is a requirement engineering step of a practical contract defect detection tool.
1.1 Smart contract defects
This article crawled 17,128 posts from the StackExchange website. The author found 66 important keywords by reading the solidity document and filtered 4,141 articles, and then manually filtered posts that were not related to contract defects, and only kept those related to contract defects. post. The Card Sorting method is used to analyze and classify the filtered posts related to contract defects. A card with information about the bug title, description, and notes was created for each post. The general process is as follows
First, 20% of the cards are randomly selected. Read the title and description of the card first to understand the flaws discussed in the article. Then read the comments to understand how to resolve the defect. The root cause of the defect is then divided into the first five categories shown in the figure below.
Next, the two authors independently sorted the remaining 80% of the cards following the same method described in the previous section. Then an "Inappropriate Standard" category was discovered, which is common among other cards. Then compare the results and discuss the differences. Finally, the defects are grouped into 6 themes; the details are shown in the figure below. Cohen's Kappa was used to measure the agreement between the two authors.
Their overall Kappa value is 0.82, which indicates a strong agreement.
The picture below shows the classification of smart contract defects,
and then look for more details on the posts to define contract defects, and finally summarize them into 16 types of contract defects.
All 17,013 verified smart contracts were grabbed from Etherscan, and then 600 smart contracts were randomly selected. After filtering out some smart contracts without any functions, there were 587 smart contracts left in the data set.
Then, through code differences, the defects of smart contracts in the data set are classified, and four types of defects are defined.
Finally, a total of 20 contract defects are defined.
1.2 Results
According to the definition, defects can be divided into security defects, performance defects, usability defects, maintainability defects and reusability defects.
2. Practitioner's perspective
This article created an online survey to gather opinions from real-world smart contract developers.
The survey respondents were partners working or studying in world-renowned companies or academic institutions and 1,489 practitioners who contributed to open source smart contract-related projects on GitHub. A total of 84 reviews were received from 32 different countries, with an average smart contract development experience of 1.95 years.
5 points for very important and 1 point for very unimportant. Almost all deficiencies exceeded 4 points, with an average score of 4.22.
3. Distribution and impact of contract defects
This paper summarizes 5 influences and manually annotates 587 smart contracts to show their distribution in real-world smart contracts.
The two authors have relatively rich experience in smart contract-related development, and their overall Kappa value is 0.71, indicating that they have great consistency. They jointly draw a conclusion about the impact level of contract defects.
The characteristics of each degree of influence are considered from three dimensions, namely contract dimension (unwanted behavior), attacker dimension (attack vector) and user dimension (usability), among which IP1 is the highest and IP5 is the lowest. Contract flaws with impact levels 1-2 can lead to severe unwanted behavior such as crashes or contracts being controlled by an attacker. Contract flaws with an impact level of 3 lead to major harmful behavior such as lost gas. Impact levels 4-5 may cause some trivial issues such as low readability, but will not affect the normal operation of the contract.
discuss
This article has a lot of inspiration for researchers, practitioners and educators. At present, there are many tools that can realize the function of detecting the above defects. But none of the four tools below take into account the other two aspects that practitioners consider equally important. Therefore, more effort can be put on developing tools capable of detecting the other 13 contract defects in addition to the one shown below.
