作者: Florian Kramer, Dan Boneh [Stanford University] [CLR 2019]
Abstract
To protect the machine learning data integrity and privacy, usually a trusted execution environment may be utilized (Trusted Execution Environment), using protection of hardware and software to enable the computing sensitive data independent, but there is a loss in efficiency in this way . Therefore, this paper proposes a trusted device assigned to the calculation process and untrusted devices to high-performance execution depth neural network. Slalom is, all the linear calculations outer layers of the DNN untrusted device, but quickly, while ensuring the privacy and DNN integrity.
Three motivating scenarios
- Outsourced ML. When the ML tasks outsourced to cloud service providers, to fully trust cloud services will be executed correctly. The results of malicious tampering cloud (A dishonest cloud provider) may use a weaker model, a method using unfair to different users, or malicious. Even sensitive data (sensitive data) will be leaked.
- Federated learning. (Joint Learning) It works like this: your device downloads the current model, improves it by learning from data on your phone, and then summarizes the changes as a small focused update Only this update to the model is sent to the cloud, using. encrypted communication, where it is immediately averaged with other user updates to improve the shared model. All the training data remains on your device, and no individual updates are stored in the cloud. joint learning privacy can be guaranteed through an encrypted way, the integrity of the need to ensure that a malicious user (malicious user) and the infected device will not be uploaded to the cloud model is wrong.
- Infected hosts. Malicious software may tampering local machine produce erroneous results.
These three ways of protecting privacy is not necessary (2,3 training model and data exist locally), but data integrity is necessary.
Background
U $ $ behalf of the user, DNN by $ F (x): Xrightarrow Y $ represents, $ P_T $ processor means credible, $ P_U $ Representative untrusted and efficient processor.
- Integrity, users can and $ P_T large column thesis notes [Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware] probability $ interaction, receive the correct output $ y = F (x) $ , or an error is very small .
- Privacy, $ P_U $ can not obtain the information of the input $ x $
Freivald’s Algorithm
$ N times n $ matrix $ A $, $ B $ and $ tilde {C} $ in the domain $ mathbb {F} $, verification $ tilde {C} = AB $ is established can use a random vector $ R & lt $ to verify $ tilde {C} r = A (Br) $. Comparison matrix multiplication, it takes $ O (n ^ 3) $ operation times (can be improved to $ O (n ^ {2.81}) $). Freivald algorithm by three matrix - vector multiplication, only cost $ O (n ^ 2) $ operation times.
Order $ D: = tilde {C} -AB $, $ R & lt vectors from $ $ S n $ ^ in the randomly selected, False Positive (FP, positive predictive model is negative samples) Probability:
$$ Pr [tilde are {C} r = A (Br ) | tilde {C} neq AB] = Pr [Dr = 0 | Dneq 0] leq cfrac {1} {| S |} $$
proof . By repeating the test $ k $ times, so that the probability of error of less than $ 1 / {| S | ^ k} $.
Private linear operation
Privacy can be protected by the input homomorphic encryption mode, but large overhead. As shown in, $ $ P_T inputs of each layer adding a random vector, and the modified input p_u $ $ to do calculations, the results $ tilde {Z} _i $ back to $ P_T $. Calculation $ Z_i = tilde {Z} _i-R_iW_i $, $ R_i $ can be calculated in advance $ $ P_T. The final output is $ Z_n $. Paper substantially scheme shown below, the following discussion of specific embodiments.
Quantization
In their frame, they perform floating-point processing, converted to an integer, an integer in the range of $ mathbb {Z} _p $. Conversion of a floating point $ X $ represents a fixed point (fixed-point representation) $ tilde {x} = FP (x; k): = left lfloor {2 ^ kcdot x} right rceil $, representative of the nearest integer. $ Tilde {W} = FP (W, k) $, emphasis $ tilde {b} = FP (W, 2k) $. The results of reduced $ 2 ^ k $ times. Transformation of the input data and the parameters, all calculations DNN value not exceeding $ 2 ^ {24} $ to ensure that no loss of precision. Thesis VGG16 and MobileNet two models to assess the accuracy of the model has about less than 0.5% loss.