Triangle Attack: A Query-efficient Decision-based Adversarial Attack
Triangle Attack: A Query-efficient Decision-based Adversarial Attack
Abstract
Decision-based attacks pose a serious threat to real-world applications because it treats the target model as a black box, accessing only hard-predicted labels. Significant efforts have been made recently to reduce the number of queries; however, existing decision-based attacks still require thousands of queries to generate high-quality adversarial examples. In this work, we find that a benign example, the current and the next adversarial example can naturally construct a triangle in the subspace for any iterative attack . Based on the law of sinusoids, we propose a new triangle attack (TA) to optimize perturbations by exploiting the geometric information that the longer side in any triangle is always opposite the larger angle. However, directly applying such information to the input image is ineffective because it cannot thoroughly explore the neighborhood of input samples in high-dimensional space. To address this issue, TA optimizes the perturbation of the low-frequency space, effectively reducing dimensionality due to the generality of this geometric property. Extensive evaluations on the ImageNet dataset show that TA achieves higher attack success rates within 1,000 queries, and compared to existing decision-based attacks, requires The number of queries is much lower. Through such high efficiency, we further verified the applicability of TA on the real API, that is, the Tencent Cloud API.
1. Introduction
Despite the unprecedented progress of deep neural networks (DNNs) [27, 24, 25], the vulnerability of adversarial examples [47] poses a serious threat to security-sensitive applications, e.g., face recognition [42, 48, 20, 30,56,50,15,37,62], autonomous driving [7,19,4,61,40], etc. In order to safely deploy DNNs in various real-world applications, it is necessary to deeply analyze the intrinsic properties of adversarial instances, which inspires a large number of adversarial attacks [36, 6, 3, 11, 8, 17, 5, 52] and defenses [ 34,23,64,57,58,53]. Existing attacks can be divided into two categories: white-box attacks have sufficient knowledge of the target model (often exploiting gradients) [21, 6, 34, 17], while black-box attacks only have access to model outputs, which are more applicable to real-world scenarios .
Black-box attacks can be implemented in different ways. Transfer-based attacks [32, 17, 60, 55] utilize an adversary generated on a surrogate model to directly fool the target model. Score-based attacks [9, 26, 2, 31] assume that the attacker has access to output logs, while decision-based (aka hard-label) attacks [5, 11, 10, 29, 35] only have access to predictions (top- 1) Labels.
Among black-box attacks, decision-based attacks are more challenging and practical due to the minimal information required for the attack. In decision-based attacks, the number of queries to the target model often plays an important role, since in practice access to the victim model is usually limited. Although recent work has managed to reduce the total number of queries from millions to thousands of requests [5, 29, 38], this is still not enough for most practical applications [35].
Existing decision-based attacks [5, 29, 38, 35] first generate a large adversarial perturbation, and then minimize the perturbation through various optimization methods while maintaining the adversarial properties.
As shown in Figure 1, we find that at the t-th iteration, a benign sample x, the current adversarial example xadvt and the next adversarial example xadvt+1 can naturally construct a triangle for any iterative attack. According to the sine law, at (t + 1) iterations, the adversarial example xadvt+1 should satisfy βt+2 αt > π to ensure that the disturbance is reduced, that is, δt+1 =∥xadvt+1−x∥p < δt = ∥xadvt−x∥p (when βt+2·αt = π, it is an isosceles triangle, that is, δt+1 = δt).
Figure 1: Schematic diagram of candidate triangles at any iteration of TA. In the t-th iteration, TA constructs a triangle with angle αt such that βt + 2αt > π in the sampled subspace, finds a new adversarial example xadvt+1, and updates αt accordingly. Note that, unlike existing decision-based attacks [5, 38, 35], TA does not constrain xadvt on the decision boundary, but exploits geometric properties to minimize perturbations in the low-frequency space; making TA itself query-efficient
Based on the above geometric properties, we propose a novel and query-efficient decision-based attack called Triangle attack (TA). Specifically, in the th iteration, we randomly select a direction line passing through a benign sample x to determine a 2D subspace in which we base on the current adversarial example x adv t , the benign sample x, learning angle α t , and search angle β t iteratively construct a triangle until the third vertex of the constructed triangle is an adversarial vertex. Using geometric information, we can perform TA in the low-frequency space generated by discrete cosine transform (Discrete cos Transform, DCT) [1], effectively reducing dimensionality and improving efficiency. And α t is further updated to adapt to the perturbation optimization of each constructed triangle. Unlike most existing decision-based attacks, it does not require x adv t to be constrained on the decision boundary , nor does it need to estimate gradients at each iteration, making TA queries efficient.
Our main contributions are summarized as follows:
- To the best of our knowledge, this is the first work that directly optimizes perturbations in frequency space via geometric information without constraining the adversary on the decision boundary, resulting in high query efficiency.
- Extensive evaluations on the ImageNet dataset show that TA exhibits a higher attack success rate within 1000 queries, and on 5 models, TA requires only a smaller number of queries than existing SOTA attacks. Achieve the same attack success rate under the same perturbation budget [11, 12, 8, 29, 38, 35].
- TA generates more adversarial instances on Tencent Cloud API with imperceptible perturbations, showing its industrial-grade applicability.
2. Related work
Since the adversarial examples were identified by Szegedy et al. [47], large-scale adversarial attacks have been proposed to fool DNNs. White-box attacks, such as single-step gradient attack [21], iterative gradient attack [36, 28, 34, 14], optimization attack [47, 6, 3], etc., often utilize gradients and have good attack performance. They have been widely used to evaluate the model robustness of defenses [34, 64, 41, 13, 16], but are difficult to apply in the real world due to limited information. In order to make adversarial attacks practical, various black-box attacks, including transfer-based attacks [17, 60, 51, 52, 59], score-based attacks [9, 26, 49, 2, 18, 63, 65], and decision-based attacks [5, 12, 8, 38, 35], have received increasing attention. Among them, the decision-based attack is the most challenging since it only has access to the predicted labels. In this work, we aim to leverage geometric information to improve the query efficiency of decision-based attacks and provide a brief overview of existing decision-based attacks.
The boundary attack [5] is the first decision-based attack that initializes a large perturbation and performs a random walk on the decision boundary while maintaining adversariality. This paradigm is widely adopted in subsequent decision-based attacks. OPT [11] defines the decision-based attack problem as a heavy-valued optimization problem with zero-order optimization. Whereas Sign-OPT [12] further computes the sign of the directional derivative instead of the magnitude of the fast convergence. HopSkipJumpAttack (HSJA) [8] enhances boundary attack by estimating gradient directions from binary information on the decision boundary. QEBA [29] enhances HSJA to obtain better gradient estimates using perturbations sampled from different subspaces, including spatial, frequency and eigencomponents. To further improve the query efficiency, qFool [33] assumes that the boundary curvature around the pairs is small and employs multiple perturbation vectors for efficient gradient estimation. BO [43] employs Bayesian optimization to find adversarial perturbations in low-dimensional subspaces and maps them back to the original input space to obtain the final perturbations. GeoDA [38] approximates a local decision boundary with a hyperplane, and searches for the closest point to a benign sample on the hyperplane as an adversary. Surfree [35] iteratively constructs a circle on the decision boundary, and uses binary search to find the intersection of the constructed circle and the decision boundary as the opponent without gradient estimation.
Most existing decision-based attacks limit the adversarial examples at each iteration on the decision boundary, and usually attack with different gradient estimation methods. In this work, we propose the triangle attack, which enables an efficient decision-based attack by exploiting the law of sinusoids to directly minimize adversarial perturbations in the low-frequency space without gradient estimation or restricting adversarial examples to the decision boundary. .
3. Method
In this section, we first provide preliminary content. We then introduce our motivation and proposed triangle attack (TA).
3.1 Preparations
Given a classifier f with parameters θ, benign samples x ∈ X, and original labels y ∈ Y, where x represents all images and y is the output space. Adversarial attacks find an adversary x adv ∈ X to mislead the target model:
where ε is the perturbation budget. Decision-based attacks usually first generate a large adversarial perturbation δ and then minimize the perturbation as follows:
Existing decision-based attacks [11, 12, 29] tend to estimate the gradient to minimize the perturbation, which is time consuming. In recent years, several studies employ geometric properties to estimate gradients or optimize perturbations directly. Here we detail two geometry-inspired decision-based attacks.
GeoDA [38] argues that a decision boundary near a data point x can be locally approximated by a hyperplane passing through a boundary point xB near x with a normal vector w. Therefore, Equation (1) can be locally linearized:
here x B is a data point on the boundary, which can be found by binary search with multiple queries, and GeoDA randomly samples several data points to estimate w to optimize the perturbation of each iteration .
Surfree [35] assumes that the boundary can be locally approximated by a hyperplane around the boundary point x+δ. In each iteration, it represents the opponent in polar coordinates and searches for an optimal θ to update the perturbation: u is the unit vector
from x to x adv t and v is the orthogonal vector of u.
3.2 Motivation
Unlike most decision-based attacks that employ gradient estimates [11, 12, 29, 38] or random walks on the decision boundary [5, 35], our goal is to exploit geometric properties to optimize perturbations without any inquiries. After generating a large adversarial perturbation, the decision-based attack moves the adversarial samples closer to the benign samples, i.e., reduces the adversarial perturbation δt while maintaining the adversarial properties in each iteration. In this study, as shown in Figure 1, we found that at the t-th iteration, the benign sample x, the current adversarial example x adv t and the next adversarial example x adv t+1 can be naturally in the subspace for any Iterative attack constructs a triangle.
Therefore, searching for the next less perturbed adversary example x adv t+1 is equivalent to searching a triangle based on x and x adv t , where the third data point x' is adversarial and satisfies ||x '− x||p <||x adv t −x||p. This inspires us to exploit the relationship between the angles and side lengths in a triangle to search for a suitable triangle to minimize the perturbation at each iteration. However, as shown in Section 4.4, applying such geometric properties directly on the input image leads to very poor performance. Due to the generality of this geometric property, we optimize the perturbation produced by DCT [1] in the low-frequency space for effective dimensionality reduction, exhibiting a huge attack efficiency as shown in Section 4.4.
Furthermore, since Brendel et al. [5] proposed Boundary Attack, most decision-based attacks [11, 12, 8, 38, 35] follow the setting that the adversarial examples for each iteration should be on the decision boundary . We argue that such a restriction is unnecessary in decision-based attacks, but would introduce too many queries on the target model to get close to the bounds. Therefore, we do not adopt this constraint in this study and verify this argument in Section 4.4.
3.3 Triangle attack
In this work, we make the following assumptions that adversarial examples exist for any deep neural classifier f:
Assumption 1: Given a benign sample x and a perturbation budget ϵ, there exists an adversarial perturbation to the decision boundary ∥δ∥ p ≤ ϵ, the adversarial perturbation will mislead the target classifier f. This is a general assumption that
we can find adversarial examples x adv of an input sample x, which has been verified by many works [21, 6, 3, 5, 54]. If this assumption does not hold, the target model is ideally robust, so we cannot find any adversarial examples within the perturbation budget, which is beyond the scope of our discussion. Therefore, we follow existing decision-based attack frameworks by first randomly generating a large adversarial perturbation, and then minimizing this perturbation. In keeping with previous work, we use binary search [29, 38, 35] to generate a random perturbation near the decision boundary and focus primarily on perturbation optimization.
In any two consecutive iterations of the perturbation optimization process of any adversarial attack, that is, the tth and (t+1)th iterations, the input sample x, the current adversarial example x adv t and the next adversarial example x adv t+1 can naturally construct a triangle in a subspace of the input space x. Therefore, reducing the perturbation to generate xadvt + 1 , as shown in Figure 1, amounts to searching for an appropriate triangle with three vertices x, xadvt , and xadvt + 1 .
Theorem 1 (sin theorem): Suppose a, b and c are the side lengths of a triangle, and α, β and γ are the opposite angles, we have a/sin α = b/sin β = c/sin γ.
From Theorem 1, we can get the relationship between the side length and diagonal angle of the triangle in Figure 1:
In order to minimize the disturbance δ t , the tth triangle should satisfy this condition.
Therefore, the disturbance at the tth iteration can be reduced by finding a It is realized by the triangle formed by the input sample x, the current adversarial sample x adv t and angles β t and α t , the triangle satisfies β t + 2α t > π and the third vertex is the adversarial vertex. We denote such a triangle as a candidate triangle and T (x, x adv t , α t , β t , S t ) as the third vertex, where St is a sampled subspace. Based on this, we propose a new decision-based attack called Triangle Attack (TA), which searches for candidate triangles in each iteration and adjusts the angle αt accordingly .
The two-dimensional subspace S of the sampling frequency space. The input image is usually in a high-dimensional space, such as 224×224×3 in ImageNet [27], which is too large for the attack to effectively explore the neighborhood to minimize the adversarial perturbation. Previous studies [22, 29, 35] show that exploiting information in different subspaces can improve the efficiency of decision-based attacks. For example, QEBA [29] samples random noise, performs gradient estimation in spatially transformed space or low-frequency space, but uses the estimated gradient to minimize perturbations in the input space. Surfree [35] optimizes the perturbation in a subspace of the input space determined by a unit vector randomly sampled in the low-frequency space. Usually, the low-frequency space contains the most critical information of the image. Due to the poor performance of TA in the input space (as shown in Section 4.4) and the generality of geometric properties (as shown in Figure 2), we directly optimize the perturbation in the frequency space at each iteration to achieve effective dimensionality reduction . We randomly draw a d-dimensional line across the benign samples in the low-frequency space (top 10%). Sampling lines, direction lines from benign samples x and current adversary x adv t can determine a unique 2D subspace S in the frequency space where we can construct candidate triangles to minimize perturbations. The final adversary is transformed into the input space by inverse DCT (IDCT).
Figure 2: Schematic diagram of the whole process of TA attack at the t-th iteration. We construct triangles in frequency space to efficiently craft adversarial examples. Note that here we use DCT for demonstration, but we don't need to use it on x in every iteration. Due to the one-to-one mapping of the DCT, we still take x and x adv t in frequency space , without ambiguity
Search for candidate triangles . Given a subspace S t , candidate triangles only depend on angle β, since α is updated during optimization. As shown in Fig. 3, if we search for an angle β and do not find an adversarial example (x adv t+1,1 ), we can further construct a symmetric triangle with the same angle in the opposite direction to check the relationship with x adv t+ 1,1 data point x adv t+1,2 with the same perturbation magnitude but different direction . For symmetrical triangles without ambiguity, we denote the angles as −β. Note that under the same angle α, a larger angle β will bring the third vertex closer to the input sample x, i.e., a smaller perturbation. After determining the subspace S t , we first check the angle β t,0 = max(π−2α, β'), where β' = π/16 is a predefined small angle. If T (x, x adv t , α t , β t,0 , S t ) and T (x, x adv t , α t , - β t,0 , S t) are not adversarial, we discard this subspace since it does not bring any benefit. In addition, we use the binary search method to find an optimal angle β*∈[max(π−2α, β'), min(π−α, π/2)] that makes the perturbation as small as possible. Here we place an upper bound on β, since T(x, x adv t , α t , β, S t ) will be in opposite directions. X guarantees a valid triangle for β > π/2 and π−α.
Figure 3: Illustration of symmetric candidate triangles (x, x adv t and x adv t+1,2 ). When the angle β cannot lead to an adversarial example (x adv t+1,1 ), we further construct a symmetrical triangle based on the line = (x, x adv t ) to check the data point x adv t+1,2
Adjust the angle α . Intuitively, the angle α balances the size of the perturbation and the difficulty of finding adversarial examples:
Proposition 1 . In the case of the same β angle, the smaller the α angle, the easier it is to find adversarial examples, and the larger the α angle, the smaller the perturbation.
Intuitively, as shown in Figure 4, the smaller the angle α, the larger the perturbation, but the more likely it is to cross the decision boundary and it is easier to search for adversarial examples, and vice versa. It is difficult to find the optimal α every iteration, let alone various input images and target models. Therefore , we adaptively adjust the angle α according to carefully crafted adversarial examples:
where x adv t,i+1 = T (x, x adv t , α t,i , β t,i , S t ) is The adversarial examples generated by i , γ is the rate of change, λ is a constant, and τ limits the adversarial examples generated by the upper and lower bounds of α. During the perturbation optimization process, considering that there are more failures than successes, we adopt λ < 1 to prevent the angle from falling too fast. Note that a larger angle α makes it harder to find adversarial examples.
Figure 4: The effect of the magnitude of α on candidate triangles in TA. For the same sampling angle β, the larger the angle α, the smaller the disturbance, but the more likely it is to cross the decision boundary
However, if the α angle is too small, the boundary of β will be lowered a lot, which also makes T (x, x adv t , α t , β t , S t ) far away from the current adversarial example x adv t , reducing the probability of finding an adversarial example . Therefore, we add an upper bound on α to constrain it to an appropriate range.
TA iteratively searches for candidate triangles in the subspace St sampled in the low-frequency space, finds adversarial examples, and updates the angle α accordingly. The whole TA algorithm is summarized in Algorithm 1.
4. Experiment
In this section, we perform extensive evaluation on the standard ImageNet dataset using five models and Tencent Cloud API to evaluate the effectiveness of TA. The code is available at https://github.com/xiaosen-wang/TA.
4.1 Experimental setup
dataset . To verify the effectiveness of the proposed TA, we randomly sample 200 correctly classified images from the ILSVRC 2012 validation set after the Surfree [39] setup to evaluate on the corresponding model.
model . We consider five widely adopted models, namely VGG-16 [44], Inceptionv3 [45], ResNet-18 [24], ResNet-101 [24] and DenseNet-121 [25]. To verify its applicability in the real world, we evaluate TA on Tencent Cloud API3.
baseline . We baseline various decision-based attacks, including four gradient estimation-based attacks, namely OPT [11], siginpt [12], HSJA [8], QEBA [29], and one optimization-based attack, namely BO [43], and two geometry-based attacks, namely GeoDA [38], Surfree [35].
evaluation index . According to the standard setting in QEBA [29], we use the root mean square error (RMSE) between the benign sample x and the adversarial sample x adv to measure the magnitude of the perturbation:
where w, h, c are the width of the input image, height and number of channels. We also employ the attack success rate, which is the percentage of adversarial instances that reach a certain distance threshold.
Hyper-parameters. For comparison, all attacks are initialized with the same adversarial perturbation method as in [35], and the baseline hyperparameters are exactly the same as in the original paper. For our TA, we adopt a maximum number of iterations per subspace N = 2, dimensionality of direction lines d = 3, γ = 0.01, update angle λ = 0.05, τ = 0.1.
4.2 Evaluation of the Standard Model
To evaluate the effectiveness of TA, we first compare the performance of different decision-based attacks on five popular models and report the attack success rates at different RMSE thresholds (i.e., 0.1, 0.05 and 0.001).
We first evaluate the attack within 1000 queries, which has been widely adopted in recent work [8, 38, 35]. Table 1 summarizes the attack success rate, which means that if 1000 queries are made without reaching the given threshold, the attack will fail to generate adversarial examples for the input image. We can observe that TA consistently achieves better attack success rates than existing decision-based attacks on models with five different architectures under different perturbation budgets. For example, at RMSE thresholds of 0.1, 0.05, 0.01 on ResNet-101 (widely used to evaluate decision-based attacks), TA outperforms the runner-up attack by a significant margin of 1.0%, 7.5%, and 13.0%. In particular, the TA proposed in this paper significantly outperforms two geometry-based attacks, GeoDA [38] and Surfree [35], showing the best attack performance among the baselines. This convincingly verifies the high effectiveness of the proposed TA. Moreover, among five models, Inceptionv3 [46] shows better robustness than other models against both baseline and TA under different perturbation budgets, which is rarely studied in decision-based attacks. Therefore, it is necessary to thoroughly evaluate decision-based attacks on various architectures, not just ResNet models.
Table 1: The attack success rate (%) of the 5 models under different RMSE thresholds. The maximum number of queries is set to 1000. We highlight the highest attack success rate in bold
To further verify the high efficiency of TA, we investigate the number of queries achieving various attack success rates under RMSE thresholds of 0.1, 0.05, and 0.01, respectively. The maximum number of queries is set to 10,000, and the result summary on ResNet-18 is shown in Figure 5. As shown in Figures 5a and 5b, when the RMSE thresholds are 0.1 and 0.05, respectively, the number of queries required by TA to achieve various attack success rates is much less, which shows that our method has higher query efficiency. For a smaller threshold of 0.01, as shown in Figure 5c, our TA still requires fewer queries when achieving an attack success rate of less than 50%, but fails to achieve an attack success rate higher than 60%. Note that, as shown in Figure 6 and Table 1, the RMSE threshold of 0.01 is very strict, so perturbations are difficult to detect, but it is also difficult to generate adversarial examples for decision-based attacks. Since we mainly focus on the attack query efficiency based on geometric information, the attack performance under the RMSE threshold of 0.01 is acceptable since such a high number of queries is unrealistic when attacking real applications.
Figure 5: Number of queries achieving a given attack success rate on ResNet-18 for the attack baseline and the proposed TA under different perturbation budgets. The maximum number of queries is 10,000
Moreover, since TA aims to leverage triangle geometry to improve query efficiency, global optimality may be worse than existing gradient estimation-based attacks when more queries are allowed. Other geometric heuristics also perform worse than QEBA [29] without gradient estimation. However, this is not the goal of TA and can be easily solved with gradient estimation. Due to the high efficiency of TA, we can use TA as a warm-up for precise gradient estimation attacks (such as QEBA [29]) to obtain high attack performance with a lower number of queries when the number of queries is acceptable. We integrate the gradient estimation used in QEBA [29] into TA after 2000 queries, called TAG. With a perturbation budget of 0.01, TAG achieves a 95% attack success rate using 7000 queries, outperforming the best baseline of 92% using 9000 queries.
4.3 Real-world application evaluation
With the excellent performance and unprecedented progress of DNN, many companies have deployed DNN to various tasks and provided commercial APIs (application programming interfaces) for different tasks. Developers can pay for these services to integrate APIs into their applications. However, the vulnerability of DNNs to adversarial examples, especially the flourishing of decision-based attacks that do not require any target model information, poses a serious threat to these real-world applications. The practical attack applicability of the algorithm is verified by using Tencent Cloud API. Due to the high cost of commercial APIs, we randomly sample 20 images from the ImageNet validation set with a maximum number of queries of 1000.
Table 2: Number of adversarial examples successfully generated by various attack baselines and proposed TA on Tencent Cloud API in 200/500/1000 queries. Due to the high cost of the online api, 20 images were randomly sampled from correctly classified images in ImageNet to evaluate the results
The number of images successfully attacked is shown in Table 2. We can observe that under different RMSE thresholds, in the range of 200, 500 and 1000 queries, TA successfully generates more adversarial examples than the attacking baseline. In particular, our TA can generate more adversarial examples in 500 queries than the best attack baseline in 1000 queries, showing the advantage of TA. We also visualize some adversarial examples generated by TA in Fig. 6. We can see that TA can successfully generate high-quality adversarial examples for various categories with few queries (≤200), verifying the high applicability of TA in the real world. Especially when the number of queries is 200, the adversarial examples generated by TA are almost visually imperceptible to humans, which highlights the vulnerability of current commercial applications.
Figure 6: Adversarial examples made by TA against Tencent Cloud API. #Q. Indicates the number of queries of the attack, and RMSE indicates the RMSE distance between benign samples and adversarial samples. We report the correct and predicted labels on the leftmost and rightmost columns, respectively (zoom in for details).
4.4 Ablation experiment
In this section, we conduct a series of ablation studies on ResNet-18, namely the subspace selected by TA, the ratio of the low-frequency subspace, and the rate of change γ and λ of the update angle α. Parametric studies on the dimensionality of the sampling line d and the bound τ on α are summarized in Appendix B.
on the subspace chosen by TA . Unlike existing decision-based attacks, the generality of the geometric properties used by TA makes it possible to directly optimize perturbations in frequency space. To study the effectiveness of frequency space, we implement TA in a different space, the input space (TAI), sampling the direction lines in the frequency space but optimizing the input used by Surfree [35] and the full frequency space (TAF) Perturbations in space (TAFI). As shown in Table 3, due to the high-dimensionality of the input space, TAI cannot efficiently explore the neighborhood of input samples to find good perturbations, and the performance is very poor. For sampling information from frequency space to subspace, TAFI exhibits better results than TAI. When the perturbation is optimized in the whole frequency space, TAF can obtain a higher attack success rate than TAFI, which reflects the advantage of frequency space. When using low-frequency information to sample the subspace, TA obtains better performance than other attacks, which supports the necessity and rationality of TA's choice of subspace.
Table 3: Ablation studies of ResNet-18 in different spaces, namely input space (TAI), line sampling frequency space (TAFI) and maskless full frequency space (TAF)
On the ratio of the low-frequency subspace . The low frequency domain plays a key role in improving TA efficiency. However, since low frequencies correspond to high frequencies, there is no criterion for identifying low frequencies, and high frequencies are usually determined by the lower half of the frequency domain for a given ratio. Here, we investigate the effect of this ratio on the performance of TA attacks. As shown in Figure 7, at smaller RMSE thresholds, the effect of the ratio on the attack success rate is more significant. In general, increasing the ratio will roughly reduce the attack performance, because it will make TA pay more attention to the high-frequency domain, which contains less key information of the image. Therefore, we use the lower 10% part as the low-frequency subspace, which is more efficient, which also helps TA effectively reduce dimensionality and is easier to be attacked.
Figure 7: Under 3 RMSE thresholds and different ratios of low-frequency subspaces, the attack success rate (%) of TA within 1000 queries on ResNet-18
Update angle α with respect to rate of change γ and λ . As mentioned in Section 3.3, the angle α plays a key role in selecting better candidate triangles, but it is difficult to find a consistent optimal α for different iterations and input images. We assume that the larger the α angle, the more difficult it is to find candidate triangles, but the smaller the perturbation. As in equation (3), if we successfully find a triangle, we will increase α by γ. Otherwise, α decreases as λγ changes. We investigate the effect of various γ and λ in Figure 8. Here we only report the results with RMSE = 0.01, the results with RMSE = 0.1/0.05 show the same trend. In general, γ = 0.01 has better attack performance than γ = 0.05/0.005. When we increase λ by γ = 0.01, the attack success rate increases to λ = 0.05 and then decreases. We also study the effect of τ controlling the α bound in Eq. (3), which shows stable performance at 1000 queries, but to take effect at 10,000 queries, we simply take τ = 0.1. In our experiments, we employ γ = 0.01, λ = 0.05 and τ = 0.1.
Figure 8: Under RMSE = 0.01, using different γ and λ to update α, the attack success rate (%) of TA on ResNet-18 for 1000 queries
4.5 Further discussion
Boundary attack [5] employs random walks on the decision boundary to minimize the perturbation of decision-based attacks, and follow-up work often follows this setting to limit adversarial examples on the decision boundary. We do not think such a restriction is necessary, and our TAs do not employ it. To test this thesis, after we have found candidate triangles, we also perform a binary search, moving adversarial examples to the decision boundary at each iteration, to investigate the benefit of this restriction. As shown in Figure 9, when the number of binary search iterations is 0, the attack success rate of vanilla TA is the highest. As we increase Nbs, binary search accepts more queries in each iteration, which reduces the total number of iterations for a given total number of queries. In general, as Nbs increases, the attack success rate decreases steadily, especially when RMSE = 0.01, which means that binary search is not worth the cost (i.e. query) of limiting adversarial examples on the decision boundary. This limitation is likely to be unreliable and unreasonable for most decision-based attacks, especially for geometry-based attacks. We hope this draws more attention to discussing the necessity of constraining adversarial examples on decision boundaries and sheds new light on designing more powerful decision-based attacks.
Figure 9: TA limits the adversary’s attack success rate (%) on the decision boundary of each iteration using binary search (Nbs) with different iterations
5. Summary
In this work, we show that for any iterative attack, benign samples, current and next adversarial examples can naturally construct a triangle in the subspace at each iteration. Based on this observation, we propose a new decision-based attack called Triangle Attack (TA), which exploits the geometric information that the longer side is opposite the larger angle in any triangle. Specifically, in each iteration, TA randomly samples a direction line on benign samples to determine a subspace in which TA iteratively searches for a candidate triangle to minimize the adversarial perturbation. Taking advantage of the generality of geometric properties, TA directly optimizes the adversarial perturbation generated by DCT in the low-frequency space which is many dimensions lower than the input space, significantly improving the query efficiency. Extensive experiments prove that TA achieves a higher attack success rate within 1000 queries and requires fewer queries to achieve the same attack success rate. The actual application on Tencent Cloud API has also verified the superiority of TA.