版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/hellonlp/article/details/78083656
实验1. Adam+LN（LN没有对context vector和source side的input做）
BEST在update 75000
20000 BLEU score = 0.2947
21000 BLEU score = 0.3159
22000 BLEU score = 0.3109
23000 BLEU score = 0.3298
24000 BLEU score = 0.3196
25000 BLEU score = 0.3305
26000 BLEU score = 0.3203
27000 BLEU score = 0.3209
28000 BLEU score = 0.3145
29000 BLEU score = 0.3241
30000 BLEU score = 0.3071
31000 BLEU score = 0.3236
32000 BLEU score = 0.3184
33000 BLEU score = 0.3107
34000 BLEU score = 0.3239
35000 BLEU score = 0.3197
36000 BLEU score = 0.3306
37000 BLEU score = 0.3371
38000 BLEU score = 0.3326
39000 BLEU score = 0.3438
40000 BLEU score = 0.3402
41000 BLEU score = 0.3183
42000 BLEU score = 0.3335
43000 BLEU score = 0.3277
44000 BLEU score = 0.3257
45000 BLEU score = 0.3107
46000 BLEU score = 0.3256
47000 BLEU score = 0.3117
48000 BLEU score = 0.3266
49000 BLEU score = 0.3276
50000 BLEU score = 0.3288
51000 BLEU score = 0.3253
52000 BLEU score = 0.3275
53000 BLEU score = 0.3110
54000 BLEU score = 0.3300
55000 BLEU score = 0.3259
56000 BLEU score = 0.3341
57000 BLEU score = 0.3472
58000 BLEU score = 0.3279
59000 BLEU score = 0.3406
60000 BLEU score = 0.3515
61000 BLEU score = 0.3489
62000 BLEU score = 0.3537
63000 BLEU score = 0.3388
64000 BLEU score = 0.3466
65000 BLEU score = 0.3317
66000 BLEU score = 0.3484
67000 BLEU score = 0.3411
68000 BLEU score = 0.3358
69000 BLEU score = 0.3534
70000 BLEU score = 0.3415
71000 BLEU score = 0.3585
72000 BLEU score = 0.3500
73000 BLEU score = 0.3402
74000 BLEU score = 0.3581
75000 BLEU score = 0.3622
76000 BLEU score = 0.3490
77000 BLEU score = 0.3610
78000 BLEU score = 0.3498
79000 BLEU score = 0.3563
80000 BLEU score = 0.3504
81000 BLEU score = 0.3499
82000 BLEU score = 0.3514
83000 BLEU score = 0.3536
84000 BLEU score = 0.3433
85000 BLEU score = 0.3441
86000 BLEU score = 0.3472
87000 BLEU score = 0.3465
88000 BLEU score = 0.3545
89000 BLEU score = 0.3489
90000 BLEU score = 0.3361
91000 BLEU score = 0.3349
92000 BLEU score = 0.3437
93000 BLEU score = 0.3622
94000 BLEU score = 0.3433
95000 BLEU score = 0.3435
96000 BLEU score = 0.3482
97000 BLEU score = 0.3466
98000 BLEU score = 0.3505
99000 BLEU score = 0.3473
100000 BLEU score = 0.3367
101000 BLEU score = 0.3576
102000 BLEU score = 0.3283
103000 BLEU score = 0.3280
104000 BLEU score = 0.3385
105000 BLEU score = 0.3434
106000 BLEU score = 0.3450
107000 BLEU score = 0.3373
108000 BLEU score = 0.3416
109000 BLEU score = 0.3372
110000 BLEU score = 0.3440
111000 BLEU score = 0.3457
112000 BLEU score = 0.3328
113000 BLEU score = 0.3439
114000 BLEU score = 0.3475
115000 BLEU score = 0.3506
116000 BLEU score = 0.3547
117000 BLEU score = 0.3482
118000 BLEU score = 0.3461
119000 BLEU score = 0.3521
120000 BLEU score = 0.3483
121000 BLEU score = 0.3405
122000 BLEU score = 0.3331
123000 BLEU score = 0.3417
124000 BLEU score = 0.3508
125000 BLEU score = 0.3518
126000 BLEU score = 0.3444
127000 BLEU score = 0.3512
128000 BLEU score = 0.3552
129000 BLEU score = 0.3470
130000 BLEU score = 0.3414
131000 BLEU score = 0.3506
132000 BLEU score = 0.3490
133000 BLEU score = 0.3548
134000 BLEU score = 0.3254
135000 BLEU score = 0.3621
136000 BLEU score = 0.3548
137000 BLEU score = 0.3574
138000 BLEU score = 0.3518
139000 BLEU score = 0.3470
140000 BLEU score = 0.3455
141000 BLEU score = 0.3444
142000 BLEU score = 0.3364
143000 BLEU score = 0.3510
144000 BLEU score = 0.3414
145000 BLEU score = 0.3424
146000 BLEU score = 0.3345
147000 BLEU score = 0.3463
148000 BLEU score = 0.3349
149000 BLEU score = 0.3517
150000 BLEU score = 0.3500
151000 BLEU score = 0.3454
152000 BLEU score = 0.3284
153000 BLEU score = 0.3452
154000 BLEU score = 0.3469
155000 BLEU score = 0.3519
156000 BLEU score = 0.3398
157000 BLEU score = 0.3419
158000 BLEU score = 0.3529
159000 BLEU score = 0.3460
160000 BLEU score = 0.3399
161000 BLEU score = 0.3386
162000 BLEU score = 0.3410
163000 BLEU score = 0.3395
164000 BLEU score = 0.3418
165000 BLEU score = 0.3326
166000 BLEU score = 0.3452
167000 BLEU score = 0.3456
168000 BLEU score = 0.3398
169000 BLEU score = 0.3408
170000 BLEU score = 0.3489
171000 BLEU score = 0.3440
172000 BLEU score = 0.3492
173000 BLEU score = 0.3418
174000 BLEU score = 0.3489
175000 BLEU score = 0.3406
176000 BLEU score = 0.3342
177000 BLEU score = 0.3435
178000 BLEU score = 0.3522
179000 BLEU score = 0.3501
180000 BLEU score = 0.3517
181000 BLEU score = 0.3476
182000 BLEU score = 0.3456
183000 BLEU score = 0.3545
184000 BLEU score = 0.3378
185000 BLEU score = 0.3496
186000 BLEU score = 0.3306
187000 BLEU score = 0.3486
188000 BLEU score = 0.3409
189000 BLEU score = 0.3535
190000 BLEU score = 0.3433
191000 BLEU score = 0.3342
192000 BLEU score = 0.3387
193000 BLEU score = 0.3518
194000 BLEU score = 0.3581
195000 BLEU score = 0.3422
196000 BLEU score = 0.3333
197000 BLEU score = 0.3425
198000 BLEU score = 0.3474
199000 BLEU score = 0.3424
200000 BLEU score = 0.3404
210000 BLEU score = 0.00
211000 BLEU score = 0.3465
212000 BLEU score = 0.3394
213000 BLEU score = 0.3346
214000 BLEU score = 0.3342
215000 BLEU score = 0.3400
216000 BLEU score = 0.3330
217000 BLEU score = 0.3451
218000 BLEU score = 0.3419
219000 BLEU score = 0.3316
220000 BLEU score = 0.3357
221000 BLEU score = 0.3369
222000 BLEU score = 0.3318
223000 BLEU score = 0.3362
224000 BLEU score = 0.3200
225000 BLEU score = 0.3340
226000 BLEU score = 0.3469
227000 BLEU score = 0.3480
228000 BLEU score = 0.3467
229000 BLEU score = 0.3467
230000 BLEU score = 0.3437
231000 BLEU score = 0.3400
232000 BLEU score = 0.3477实验2. Adam+LN（LN同时也对context vector和source side的input做）
BEST在update 77000
5000 BLEU score = 0.2185
6000 BLEU score = 0.2319
7000 BLEU score = 0.2630
8000 BLEU score = 0.2580
9000 BLEU score = 0.2750
10000 BLEU score = 0.2739
11000 BLEU score = 0.2765
12000 BLEU score = 0.2908
13000 BLEU score = 0.2906
14000 BLEU score = 0.2896
15000 BLEU score = 0.3180
16000 BLEU score = 0.3163
17000 BLEU score = 0.3177
18000 BLEU score = 0.3229
19000 BLEU score = 0.3222
20000 BLEU score = 0.3253
21000 BLEU score = 0.3299
22000 BLEU score = 0.3308
23000 BLEU score = 0.3133
24000 BLEU score = 0.3276
25000 BLEU score = 0.3309
26000 BLEU score = 0.3440
27000 BLEU score = 0.3267
28000 BLEU score = 0.3262
29000 BLEU score = 0.3291
30000 BLEU score = 0.3074
31000 BLEU score = 0.3300
32000 BLEU score = 0.3295
33000 BLEU score = 0.3289
34000 BLEU score = 0.3415
35000 BLEU score = 0.3235
36000 BLEU score = 0.3352
37000 BLEU score = 0.3399
38000 BLEU score = 0.3418
39000 BLEU score = 0.3395
40000 BLEU score = 0.3370
41000 BLEU score = 0.3227
42000 BLEU score = 0.3360
43000 BLEU score = 0.3267
44000 BLEU score = 0.3445
45000 BLEU score = 0.3286
46000 BLEU score = 0.3450
47000 BLEU score = 0.3464
48000 BLEU score = 0.3312
49000 BLEU score = 0.3376
50000 BLEU score = 0.3374
51000 BLEU score = 0.3252
52000 BLEU score = 0.3293
53000 BLEU score = 0.3305
54000 BLEU score = 0.3341
55000 BLEU score = 0.3468
56000 BLEU score = 0.3221
57000 BLEU score = 0.3455
58000 BLEU score = 0.3427
59000 BLEU score = 0.3382
60000 BLEU score = 0.3451
61000 BLEU score = 0.3601
62000 BLEU score = 0.3532
63000 BLEU score = 0.3437
64000 BLEU score = 0.3428
65000 BLEU score = 0.3383
66000 BLEU score = 0.3550
67000 BLEU score = 0.3538
68000 BLEU score = 0.3492
69000 BLEU score = 0.3517
70000 BLEU score = 0.3417
71000 BLEU score = 0.3488
72000 BLEU score = 0.3641
73000 BLEU score = 0.3474
74000 BLEU score = 0.3605
75000 BLEU score = 0.3467
76000 BLEU score = 0.3344
77000 BLEU score = 0.3688
78000 BLEU score = 0.3477
79000 BLEU score = 0.3623
80000 BLEU score = 0.3476
81000 BLEU score = 0.3616
82000 BLEU score = 0.3457
83000 BLEU score = 0.3506
84000 BLEU score = 0.3535
85000 BLEU score = 0.3529
86000 BLEU score = 0.3458
87000 BLEU score = 0.3589
88000 BLEU score = 0.3540
89000 BLEU score = 0.3470
90000 BLEU score = 0.3360
91000 BLEU score = 0.3407
92000 BLEU score = 0.3471
93000 BLEU score = 0.3557
94000 BLEU score = 0.3539
95000 BLEU score = 0.3453
96000 BLEU score = 0.3495
97000 BLEU score = 0.3410
98000 BLEU score = 0.3455
99000 BLEU score = 0.3427
100000 BLEU score = 0.3459
101000 BLEU score = 0.3464
102000 BLEU score = 0.3254
103000 BLEU score = 0.3403
104000 BLEU score = 0.3533
105000 BLEU score = 0.3538
106000 BLEU score = 0.3367
107000 BLEU score = 0.3293
108000 BLEU score = 0.3348
109000 BLEU score = 0.3515
110000 BLEU score = 0.3414
111000 BLEU score = 0.3448
112000 BLEU score = 0.3288
113000 BLEU score = 0.3506
114000 BLEU score = 0.3440
115000 BLEU score = 0.3515
116000 BLEU score = 0.3566
117000 BLEU score = 0.3439
118000 BLEU score = 0.3394
119000 BLEU score = 0.3640
120000 BLEU score = 0.3493
121000 BLEU score = 0.3457
122000 BLEU score = 0.3432
123000 BLEU score = 0.3458
124000 BLEU score = 0.3440
125000 BLEU score = 0.3477
126000 BLEU score = 0.3451
127000 BLEU score = 0.3459
128000 BLEU score = 0.3513
129000 BLEU score = 0.3511实验3：四层Encoer（Residual Networks）
11000 BLEU score = 0.0894
12000 BLEU score = 0.1453
13000 BLEU score = 0.1795
14000 BLEU score = 0.2038
15000 BLEU score = 0.2390
16000 BLEU score = 0.2352
17000 BLEU score = 0.2612
18000 BLEU score = 0.2720
19000 BLEU score = 0.2849
20000 BLEU score = 0.2792
21000 BLEU score = 0.2891
22000 BLEU score = 0.3002
23000 BLEU score = 0.3061
24000 BLEU score = 0.3049
25000 BLEU score = 0.3051
26000 BLEU score = 0.3123
27000 BLEU score = 0.3061
28000 BLEU score = 0.3052
29000 BLEU score = 0.3059
30000 BLEU score = 0.3025
31000 BLEU score = 0.3139
32000 BLEU score = 0.3171
33000 BLEU score = 0.3204
34000 BLEU score = 0.3211
35000 BLEU score = 0.3219
36000 BLEU score = 0.3244
37000 BLEU score = 0.3277
38000 BLEU score = 0.3306
39000 BLEU score = 0.3372
40000 BLEU score = 0.3221
41000 BLEU score = 0.3147
42000 BLEU score = 0.3304
43000 BLEU score = 0.3079
44000 BLEU score = 0.3241
45000 BLEU score = 0.3247
46000 BLEU score = 0.3375
47000 BLEU score = 0.3264
48000 BLEU score = 0.3396
49000 BLEU score = 0.3252
50000 BLEU score = 0.3374
51000 BLEU score = 0.3267
52000 BLEU score = 0.3146
53000 BLEU score = 0.3168
54000 BLEU score = 0.3373
55000 BLEU score = 0.3303
56000 BLEU score = 0.3293
57000 BLEU score = 0.3425
58000 BLEU score = 0.3423
59000 BLEU score = 0.3471
60000 BLEU score = 0.3508
61000 BLEU score = 0.3579
62000 BLEU score = 0.3556
63000 BLEU score = 0.3427
64000 BLEU score = 0.3434
65000 BLEU score = 0.3442
66000 BLEU score = 0.3515
67000 BLEU score = 0.3398
68000 BLEU score = 0.3431
69000 BLEU score = 0.3449
70000 BLEU score = 0.3488
71000 BLEU score = 0.3476
-------------------------
78000 BLEU score = 0.3426
79000 BLEU score = 0.3640
80000 BLEU score = 0.3479
81000 BLEU score = 0.3637SYS DEV %% %% %% %% %% AVG
BASELIEN 34.57 36.68 34.81 37.68 34.68 25.82 33.93
LN_weak 36.22 38.66 36.16 38.50 35.69 27.29 35.26
LN_strong 36.88 38.50 36.97 39.46 36.04 27.90 35.77
LN(80) 37.60 38.20 36.27 39.34 36.11 29.56 35.90
4Layer Encoder + LN 37.55 38.78 36.88 39.13 36.16 28.84 35.96
4Layer Encoder + LN(80) 37.85 38.91 37.88 39.94 37.27 28.24 36.45
4Layer ED +LN (80) 38.23 39.54 37.30 40.28 37.21 29.39 36.74