Abstract

Grey box fuzz has made remarkable progress in recent years, from heuristic-based random mutation evolution to solving single branch constraints. However, they are difficult to solve path constraints that contain deeply nested conditional statements, which are common in image and video decoders, network packet analyzers, and checksum tools. We have proposed a solution to this problem. First, we identify all control flow related conditional statements of the target conditional statement. Next, we select the conditional statements related to pollution flow. Finally, we use three strategies to find input that satisfies all conditional statements at the same time. We implemented this method in a tool called Matryoshka1 and compared its effectiveness on 13 open source programs with other state-of-the-art fuzzers. Matryoshka's cumulative survey line and branch coverage are significantly higher than AFL, QSYM and Angora. We manually classified the collisions discovered by Matryoshka into 41 unique new vulnerabilities and obtained 12 CVEs. Our evaluation shows the key technology of Matryoshka's impressive performance: In the nested constraints of the target conditional statement, Matryoshka only collects those constraints that may cause the target to be unreachable, which greatly simplifies the path constraints it must solve.

introduction

Fuzzing is an automated software testing technology, which successfully found many defects in real software. Among the various types of blurring techniques, the coverage-based greybox blurring technique is especially popular. It prioritizes branch exploration to efficiently trigger bugs in branches that are difficult to reach. Compared with symbolic execution, gray box fuzzy avoids expensive symbolic constraint solving, so it can handle large and complex programs.

AFL [2] is a basic gray box fuzz. It lets the program report whether the current input explores a new state at runtime. If the current input triggers a new program state, the fuzzer saves the current input as a seed for further mutation [35]. However, because AFL only uses rough heuristics to randomly mutate the input, it is difficult to achieve high code coverage. Newer fuzzers use program states to guide input variation and show significant performance improvements over AFL, such as Vuzzer [30], Steelix [26], QSYM [41], and Angora [13]. Take Angora as an example. It uses dynamic taint tracking to determine which input bytes flow into the conditional statement that protects the target branch, and then only mutates these related bytes, rather than mutating the entire input, to greatly reduce the search space. Finally, the gradient descent method is used to find the solution of the branch constraint.

However, these fuzzers face difficulties in solving path constraints that contain nested conditional statements. Branch constraints are predicates in conditional statements that protect branches. The branch is reachable only when (1) the conditional statement is reachable and (2) the branch constraint is satisfied. Path constraints satisfy these two conditions. When a conditional statement s is nested, s is only reachable if some previous conditional statements P on the execution path are reachable. If $\begin{Bmatrix} s \end{Bmatrix}\cup p$ the branch constraint in shares the common input byte, then when the fuzzer mutates the input to satisfy the constraint in s, it may invalidate the constraint in P, thus making s inaccessible. This problem plagues the aforementioned fuzzers because they cannot track control flow and pollute the flow dependency between conditional statements. Nested conditional statements are common in image and video encoders and decoders, network packet analyzers, and checksum verification programs, and they have a rich history of vulnerabilities. Although mixed execution can solve some nested constraints. The results show that the hybrid execution engine will have over-constraint problems, which makes the cost of solving constraints too high [41], especially in actual programs.

Figure 1 shows such an example in the program readpng. The predicate on line 6 is nested in the predicate on line 4. It is difficult for the fuzzifier to find the input that reaches the false branch in line 6 because the input must also satisfy the false branch in line 4. When the fuzzer tries to change the predicate on line 6, it only changes the input bytes that flow into the buffer [0], but this will almost certainly cause the CRC check in png_CRC_finish () to fail, which will cause line 4 to accept the true branch And return.

In order to evaluate whether the current fuzzer is difficult to solve the path constraints containing nested conditional statements, we use Angora as a case study. We run it on 13 open source programs that read structured input, so there may be many nested conditional statements. Table 1 shows that in all programs, most unresolved path constraints involve nested conditional statements. In 5 programs, more than 90% of unresolved constraints involve nested conditional statements. This shows that solving these constraints will significantly improve the coverage of the fuzzer.

We designed and implemented a method that allows fuzzers to explore deeply nested conditional statements. Take the program in Figure 2 as an example. Suppose the current input runs the false branch on line 6, and the fuzzer wants to explore the true branch on line 6.

Determine the control flow dependency between conditional statements. The first task is to identify all conditional statements before line 6 on the trace, which may make line 6 inaccessible. They include lines 2, 3, and 4, because if any of these lines use different branches, then line 6 will not be accessible. Section 3.3 will describe how we can use post-dominant trees within and between procedures to find these conditional statements.
Determine the dependence of the pollution flow between conditional statements. Among the conditional statements determined in the previous step, only the statements in lines 2 and 3 have a polluted flow dependency relationship with line 6. This is because when we mutate on line 6, this may change the branch selection on line 3, making line 6 inaccessible. In order to avoid this problem, we must keep the branch selection of line 3, which may require us to change both x and y, but this may change the branch selection of line 2. Therefore, Line 2 and Line 3 are dependent on Line 6 for pollution flow. In contrast, when we mutated in order to explore the true branch of Line 6, the branch selection of Line 4 never changed, so it has no dependency on the pollution flow of Line 6. Section 3.4 will describe how we find these conditional statements that rely on polluting flows.
Resolve constraints. Finally, we need to mutate the input to satisfy multiple dependency statements at the same time. In other words, we need to find a new input that reaches line 6 while satisfying its true branch. We propose three strategies.

The first strategy conservatively assumes that if we mutate any byte in any conditional statement that line 6 depends on, line 6 will become inaccessible. Therefore, when blurring line 6.4 (Section 3.5.1), this strategy avoids the variation of these bytes
The second strategy artificially retains the branch selection of all conditional statements that line 6 relies on when mutating the input bytes that flow into line 6. When it finds a satisfactory input, it verifies that the program can reach line 6 without changing the branch selection. If so, then the fuzzer successfully solved the problem. Otherwise, the fuzzer will backtrack on the track to try this strategy on lines 3 and 2. (Section 3.5.2)
The last strategy attempts to find a solution that satisfies all the conditional statements. It defines a joint constraint, which includes the constraints of each conditional statement. When the fuzzer finds an input that satisfies the joint constraint, it is guaranteed that the input satisfies the constraints in all relevant conditional statements. (Section 3.5.3)

Our method assumes that there are no special structures or attributes in the fuzzed program, such as magic bytes or checksum functions. On the contrary, our general method of solving nested conditional statements can handle these special structures naturally.

We implemented our method in a tool called Matryoshka and compared its effectiveness on 13 open source programs with other state-of-the-art fuzzers. Matryoshka discovered a total of 41 unique new vulnerabilities and obtained 12 CVEs in 7 of them. Matryoshka's impressive performance lies not only in its ability to solve nested constraints, but also in how it constructs these constraints. Traditional symbolic execution collects predicates in all conditional statements on the path. In contrast, Matryoshka only collects predicates in conditional statements where the target branch depends on both control flow and polluted flow. Our evaluation shows that the latter accounts for only a small part of all conditional statements on the path, which greatly simplifies the constraints that Matryoshka must resolve.

Background

Greybox fuzzing is a popular method of program testing. It combines program status monitoring with random input mutation, and the effect is remarkable. However, currently the most advanced greybox fuzzers cannot reliably and effectively solve nested conditional statements. Fuzzers that use heuristics (for example, AFL) or principled mutation methods (for example, Angora) do not have enough information about the dependence of control and pollution flows between conditional statements to design inputs that satisfy all relevant branch constraints. Other fuzzers that use hybrid-hybrid execution, such as drillers, will encounter performance problems because they specify the entire symbolic constraints of the path [41, 42]. QSYM is a practical hybrid execution fuzzer, but it is specifically customized to solve the last constraint on the path, so it faces the same challenge as Angora to solve nested conditional statements.

Taking Angora as an example, we evaluated the impact of nested conditional statements on Angora performance, and analyzed the constraints in 8 programs that Angora failed to solve in Table 1, where each constraint corresponds to a unique branch in the program. The second column shows the percentage of nested unresolved constraints, which depends on the control flow and other conditional statements that pollute the flow (Section 3.4). The third column shows the percentage nested in all constraints (resolved and unresolved). Table 1 shows that most unresolved constraints are nested, ranging from 57.95% to nearly 100%. Research also shows that nested constraints account for a large portion of all constraints, from 44.14% to 89.50%. These results show that solving nesting constraints can greatly improve the coverage of greybox fuzzers

Design

3.1.problem

The latest coverage-guided fuzzers, such as Angora [13], QSYM [41], VUzzer [30], and REDQUEEN [4], explore new branches by solving branch constraints, where branch constraints are in conditional statements that protect branches predicate. This usually includes the following steps. First, use dynamic pollution analysis or similar techniques to determine the input bytes that affect each conditional statement. Then, determine how the input bytes should be mutated, such as calculating the gradient of the predicate and using gradient descent, matching magic numbers, or using symbols to execute the solver. Finally, use the mutated input to execute the program and verify that this triggers another branch in the conditional statement.

Although this method is effective in solving many branch constraints, it fails when the target conditional statement becomes inaccessible during input mutation.

Figure 2 shows an example. Let variables x, y, and z contain different input bytes. Suppose the current input executes the false branch on line 6 and the goal is to explore the true branch on line 6. The fuzzer then determines through dynamic byte-level pollution analysis that it needs to change the bytes in y. Consider two different initial values of x and y.

x = 0, y = 1. If the fuzzer mutates y to 3, then the program will no longer reach line 6, because line 3 will take a different (false) branch. This makes the ambiguous person unable to solve the branch predicate even if there is a satisfactory assignment y = 2.
x = 1 and y = 1. In this case, none of the y values can satisfy the true branch of lines 2, 3, and 6 at the same time, unless we also mutated x. However, since x does not flow into the conditional statement on line 6, the fuzzer does not know that it should mutate x, so no matter what, it cannot find a satisfactory assignment to explore the algorithm on line 6 to solve the constraint .

This example shows that to perform unexplored branches, sometimes it is not enough to mutate only the input bytes that flow into the conditional statement, because doing so may make the statement inaccessible. People can naively mutate all input bytes, but this will increase the search space by many orders of magnitude, making this method too expensive and not practical

3.2.solution overview

To overcome the problems in Section 3.1, our key insight is that when we obfuscate conditional statements, we must find inputs that both satisfy branch constraints and keep the statements accessible. Most fuzzy operators that explore branches by solving branch constraints only consider the satisfaction criterion, but not the reachability criterion. We propose the following steps to meet these two criteria while changing the input. Let's track the program on this input as a conditional statement. Our goal is to change the input and let us take different branches. We call the target conditional statement and say that the new input satisfies s.

Determine the control flow dependency between conditional statements. Identifies all conditional statements that may make s inaccessible before s. For example, if s is on line 6 in Figure 2, then if any of the conditional statements on lines 2, 3, and 4 take different branches, then line 6 will not be accessible. We call this a priori conditional statement of s, which depends on the control flow. In contrast, no matter which branch line 5 goes, line 6 can always be reached. Section 3.3 will describe this step in detail.
Determine the dependence of the pollution flow between conditional statements. In the previous conditional statement of s, identify those corresponding input bytes that may have to be mutated to satisfy the conditional statement of s. For example, let s be the sixth line in Figure 2. Among its three a priori conditional statements, only those statements on lines 2 and 3 contain bytes (x and y) that may have to be mutated to satisfy s. We call these valid prior conditional statements, which depend on the pollution flow. In contrast, line 4 does not contain input bytes that must be mutated to satisfy s. Section 3.4 will describe this step in detail.
Resolve constraints. Mutate the bytes in a valid prior conditional statement to satisfy s. Section 3.5 will describe this step in detail.

Figure 3 shows the overall design of Matryoshka and illustrates how these strategies are used in the fuzzification process.

3.3. Determine control flow dependency among conditional statements

For each conditional statement s, we want to identify all the conditional statements before it. If these conditional statements use different branches, s may not be accessible. The immediately preceding precondition statement of s on the path is the last precondition statement of s, that is, the precondition statement without s between r and s. Note that if s is a precondition statement of t and t is in u, then s is a precondition statement of u, which allows us to find all precondition statements of s by passing: starting from s, we repeat Find the direct precondition statement, and then take the union of all these statements.

We propose two different methods to find the immediately preceding statement in the same function and in different functions, respectively. In our optimized implementation, we cached all the found dependencies to avoid double counting.

3.3.1 Precondition statement immediately following the program

Starting from the conditional statement s, we return to tracking. When we find the first conditional statement r, 1. in the same function, and 2. that is not postdominated. Then r is the immediate preceding statement of s, and our implementation uses the post-dominant tree generated by LLVM

If we cannot find such r, then s does not have a direct prior conditional statement within the procedure, and we will search for its direct prior conditional statement between programs, as described in Section 3.3.2.

3.3.2. Immediate precondition statements between programs

It is very simple to use the post-procedure post-domination tree for effective processing, but unfortunately, LLVM does not provide such information, so we designed the following method to find the pre-procedure conditional statement r of s that meets all the following conditions: 1. r and s are in different functions (we call it fr), and when s is executed, fr is still on the stack (that is, it has not returned), and let rc be the last call instruction executed by fr. rc must exist because r is in a deeper stack frame than s. If rc does not dominate r (note that r and rc are in the same function), then r is the immediate priority statement between s's procedures.

3.3.3 Irregular inter-program control flow

In addition to function calls, programs can also exhibit irregular inter-process control flows, such as those involving exit and longjmp instructions. If at least one branch of a conditional statement r points to a basic block containing irregular flows, then even if its frame is not on the stack, we consider it to be the priority conditional statement for all subsequent statements. If s is a conditional statement after r, we add the prior conditional statements of r and r to the set of prior conditional statements of s. In LLVM, basic blocks containing irregular inter-process control flow are terminated with unreachable instructions.

3.4 Determining the dependence of pollution flow between conditional statements

For each conditional statement, Section 3.3 finds all its previous conditional statements p (s). Let b (s) be the set of input bytes that flow into s, where s is one or more conditional statements. When we mutate the input, as long as there are no conditional statements in p (s) that accept different branches, it is guaranteed that s is reachable. This seems to indicate that we should avoid changing any bytes in b (p (s)).

On the other hand, avoiding changing each byte in b (p (s)) may prevent the fuzzer from finding a satisfactory assignment for s, as described in Section 3.1. Take Figure 2 as an example. Let's simulate program execution to line 6. According to Section 3.3.1, we determined that p (s) consists of lines 2, 3, and 4. Therefore, b (p (s)) = {x, y, z} ;. If we keep all bytes in b (p (s)) immutable, then when trying to find an input that satisfies s, there are no input bytes that can be mutated.

The problem arises because Section 3.3 only considers the control flow dependency between conditional statements, but it does not consider whether there is a polluting flow dependency relationship between conditional statements. We define the effective a priori conditional statements of s, e (s) as a subset of the a priori conditional statements of s. If we want to find the input that satisfies s, we may need to count some bytes that flow into the statement in e (s) Make mutations. In other words, if the prior conditional statement r of s is not a valid prior conditional statement of s, then the bytes flowing into r need not be mutated to satisfy s, which means we can only consider valid prior conditional statements and ignore invalid prior Conditional statement.

Algorithm 1 gives an algorithm for calculating valid prior conditional statements. The algorithm depends on the following properties: if r is a valid prior conditional statement for s, q is a priori conditional statement for s, and q and r share a common input byte, Then q is also a valid prior conditional statement for s.

3.5 Resolve constraints

Section 3.4 determines the effective priority conditional statement for each conditional statement s. On the one hand, if we freely change the bytes flowing into any of them, s may become inaccessible. But on the other hand, we may need to mutate some of these bytes to satisfy the unexplored branch of s. Therefore, we need to determine which relevant input bytes of the sentence may be mutated. We propose the following three alternative strategies. Matryoshka tries in this order and sets a time budget for each strategy to ensure overall efficiency. (1) Prioritize accessibility (2) Prioritize satisfaction (3) Joint optimization of reachability and satisfiability

Each strategy identifies constraints on a set of input bytes. Then, the gradient-based optimization method is used to solve the constraint conditions. These strategies only provide benefits when s is nested, that is, s has a valid prior conditional statement. If S is not nested, Matryoshka only uses existing strategies from Angora or other fuzzy bodies to resolve branch constraints. Therefore, Matryoshka shows better performance when solving nested conditional statements, and has the same ability as other fuzzers to solve non-nested conditional statements.

3.5.1 Prioritize accessibility

This strategy pessimistically assumes that if we mutate the bytes of any valid prior conditional statement flowing into any conditional statement s, then s may become inaccessible. Therefore, this strategy ensures that s is always reachable by avoiding mutating any byte of any valid prior conditional statement flowing into s. Formally, let b (s) be the byte flowing into s, and b (e (s)) be the byte of the valid prior conditional statement flowing into s. Angola mutates all bytes in b (s), which may cause s to become inaccessible. In contrast, Matryoshka's strategy only mutates the bytes in b (s) \ b (e (s)), that is, all the bytes of valid priors that flow into s but not s.

Take the program in Figure 2 as an example. When we blur s to line 3, its only valid conditional statement is t on line 2. b (s) = {x, y}. b (e (s)) = {x}. Using this strategy, the fuzzer only mutates the bytes in b (s) \ b (e (s)) = {y}.

However, when we blur lines s to 6, this strategy fails. In this case, its valid previous statement consists of the statements on lines 2 and 3, so b (s) = {y}, b (e (s)) = {x, y}. But b (s) \ b (dp (s)) = $\phi$ . With this strategy, Matryoshka will not be able to obfuscate s because it cannot find the bytes to mutate.

3.5.2 Priority satisfaction

The strategy optimistically hopes that the mutated input that satisfies the conditional statement s can also reach s. It has a forward stage and a backward stage. In the forward phase, it mutates the bytes flowing into s, while artificially retaining the branch selection of all valid prior conditional statements of s, thereby ensuring that s is always reachable. If it finds an input that satisfies the target branch of S, then it runs the input program normally (without artificially fixing the branch selection). If this trace still reaches and selects the target branch, it will succeed. Otherwise, it will enter the backtracking stage. In this stage, it starts with s and then blurs each valid a priori statement of s in this order backwards. When it obfuscates such a statement r, it avoids mutating any valid prior conditional statements that may flow into any byte of s or s after r. This process succeeds when the fuzzer successfully blurs all these valid prior conditional statements. Algorithm 2 shows the algorithm.

Take the program in Figure 2 as an example. When we blur s to line 6, its valid prior conditional statements are on lines 3 and 2. Set the correct input to x = 1; y = 1. Under this input, both the second and third lines take the true branch, and the sixth line takes the false branch. Our goal is to find the real branch on line 6. Using this strategy, in the forward stage, the fuzzer will mutate y, and artificially force the program to adopt true branches on Line 2 and Line 3. If the fuzzer finds that the assignment y = 2 satisfies the true branch of line 6, but because x = 1; y = 2 does not satisfy line 3, it will enter the backtracking stage. At this stage, it will first blur Line 3. Although this line is affected by two values fx; yg, because y flows into line 6, the fuzzer will only change x. If a satisfactory assignment x = 0 is found, it will try to run the program with x = 0; y = 2 without artificially forcing branch selection. Since this input reaches line 3 and meets the target (true) branch, the fuzzification is successful.

In contrast, suppose the fuzzer finds a satisfactory assignment y = 3 when blurring line 6. In the backtracking stage, when line 3 is blurred, it can only mutate x, so it cannot find a satisfactory assignment. Therefore, the fuzzification of s fails.

3.5.3 Joint optimization of accessibility and satisfiability

The two strategies in Section 3.5.1 and Section 3.5.2 search for a constrained solution at a time. Section 3.5.1 only mutates input bytes that will not make the target conditional statement inaccessible, while Section 3.5.2 attempts to satisfy the target conditional statement and its valid previous conditional statement once. However, they did not find a solution, and we must jointly optimize multiple constraints.

Let s be the target conditional statement. Let $f_{i}(x)\leq 0,\forall i\in [1,n]$ represent the constraints of the effective prior conditional statement of $f_{0}(x)\leq 0$ s , and represent the constraints of s. x is a vector representing input bytes. Table 2 shows how to convert each type of comparison to $f(x)\leq 0$ . Our goal is to find one that meets all $f_{i}(x)\leq 0,i\in [0,n]$ . Note that each $f_{i}(x)$ is a black box function, and the expression in the conditional statement i represents the calculation of the input x. Since $f_{i}(x)$ the analytical form is not available, many commonly used optimization techniques, such as Lagrange multipliers, are not applicable.

We propose a solution to the optimization problem. Definition $g(x)= \sum_{i=0}^{n}R(f_{i}(x)))$ . Among them corrected $R(x)=0\vee x$ (the binary $\vee$ operator outputs the larger value of its operand). Therefore, $g(x)=0$ only if $f_{i}(x)\leq 0,\forall i\in [1,n]$ . In other words, we combine n optimizations into one optimization. Now we can use the gradient descent algorithm, similar to the gradient descent algorithm used in Angola, to find a solution $g(x)=0$ . Note that when we use differentiation to calculate the gradient of g (x), we need to manually maintain a valid prerequisite statement for branch selection to ensure that this is achievable.

Let's revisit the program in Figure 2. Let [x, y] = [1,3] be the initial input. When we blur the target conditional statement s in line 6 to explore the true branch, we cannot solve the branch constraint only by mutating y. Using joint optimization, we combine the branch constraints of s on lines 3 and 2 and their effective prior conditional statements (through Equation 1 and Table 2):

$g([x,y])=R(x-2+\varepsilon )+R(x+y-3+\varepsilon )+R(1-y+\varepsilon ) (\varepsilon=1)$

At the initial input [x, y] = [1,3], g ([x, y]) = 2. Using gradient descent, we will find the solution of g ([x, y]) = 0, where [x, y] = [0,2].

3.6 Detecting implicitly valid a priori conditional statements

If we cannot find all control flow and pollution flow dependencies between conditional statements, then the mutation strategy in Section 3.5 may fail. Sections 3.3 and 3.4 describe the algorithm for finding all explicit control flow and pollution flow dependencies, respectively. However, they cannot find implicit streams. Figure 4 shows such an example. The conditional statement on line 13 causes the implicit pollution flow into fun-ptr in function foo, and then implicitly determines whether the control flow is a program call to a function bar or another function. In addition, line 3 causes the implicit pollution stream to enter the variable k, whose value will determine the value of the predicate in line 6. Therefore, the conditional statements on lines 13 and 3 should both be valid precondition statements on the target statement on line 7. However, because the pollution streams are implicit, the algorithm in Section 3.4 cannot find them.

Implicit pollution flow can be identified with a control flow graph [23]. If the predicate is polluted, then this method pollutes all variables that get new values in any branch of the conditional statement. For byte-level stain tracking, this method adds the stain label of the predicate to each of the above variables. For example, consider the predicate on line 3 of Figure 4. Since the variables k and n are assigned new values in a branch of this conditional statement, this method adds the tainted label of the predicate (that is, the tainted label of the variable y) to the variables k and n. However, this method often results in excessive stains or stain explosions because it may add contamination labels that are useless for analysis. For example, in the above example, although the pollution label added to the variable k captures the implicit pollution flow dependency from lines 3 to 6, the pollution label added to the variable n is useless because it is not helpful It is used to identify the new pollution flow dependency between conditional statements. To make matters worse, these useless pollution labels will further spread to other parts of the program, causing pollution explosions.

We propose a new method to identify the implicit control flow and pollution flow dependencies between conditional statements without generating huge analysis overhead or pollution explosions. The insight is that we don't need to identify all implicit streams, but only those streams that cause the target conditional statement to become inaccessible during input mutation. Let us be the target conditional statement, which is reachable on the original input, but not reachable on the mutated input. We run the program twice. First, we run the program on the original input and record the branch selection of all conditional statements on the path before s. Then, we use a special processing method to run the program on the mutated input: when we encounter a conditional statement, we record its branch selection, but force it to accept the branch selection from the previous run (on the original input). Therefore, the paths run twice have the same sequence of conditional statements. We check all conditional statements on the path from the beginning of the program to s in reverse chronological order. For each such statement t, if it is not already an explicitly valid priority statement recognized by the algorithm in Section 3.4, and if it is run on the first run (on the original input) and the second run (on the mutated input) The branch selection in (above) is different. It has a potential control flow or taint flow dependency with S to test whether this dependency really exists. We use a special process to run the program on the mutant input: we force All the following conditional statements are branched like the first run: (1) All conditional statements before t on the path (2) All explicitly valid priori conditional statements (3) All implicitly valid priori conditional statements

If the program no longer reaches s, then t does have an implicit control flow or pollution flow dependency on s, and we mark it as an implicitly valid prior conditional statement for S.

The complexity of the algorithm is linear in the number of conditional statements before S, which is affected by the mutation bytes, but does not depend on the control flow of S. However, because MatRysHKA changes the input by gradient descent on a small portion of the input; the number of statements we should test may be very small