This article published in NIPS2016, paper Code: https://github.com/mp2893/retain .
The core model is shown in this paper:
The article sequence of events modeled as predictors of heart failure diagnosis method shows that complex models can provide higher precision and more accurate interpretation capabilities. Taking into account RNNs ability to sequence data analysis, the article presents the RETAIN , in keeping RNN while the predictive power allows a higher degree of interpretation. RETAIN core idea is to generate attention through a complex process to improve the prediction accuracy, while maintaining representation learning part is easy to explain, so that the whole algorithm accurately interpreted. Retention time series reverse two training the neural network, to efficiently generate the appropriate variable attention.
This article relatively new place as follows:
1. Use two sets of attentional mechanisms, namely visit-level and variable-level , where the visit-level attention given to patients with the disease to predict more helpful to visit more weight, and variable-level attention given to the once a more influential variables in forecasting disease patient visit ( diagnosis, drug intervention ) .
2. In the generation worthy of the attention during the course record in reverse chronological order input, because of the recent admission records the diagnosis more helpful
By the above two binding mimetic clinician diagnostic process, to achieve higher prediction accuracy.
3. For the sake of explanation model having a higher degree, encode using RNN , focus use of simple values the MLP , the prediction context vector generation is very simple:
Prediction using SoftMax , using cross-entropy loss function, experiments show that the proposed method has good explanatory.