WMT’16 English-Romanian

Encoder

Vocabulary

BLEU

(Sennrich et al., 2016a)

BiGRU

BPE 90K

28.1

Single-layer decoder

BiLSTM

80K

27.5

Convolutional

80K

27.1

Deep Convolutional 8/4

80K

27.8

WMT’15 English-German

Encoder

Vocabulary

BLEU

(Jean et al., 2015) RNNsearch-LV

BiGRU

500K

22.4

(Chung et al., 2016) BPE-Char

BiGRU

Char 500

23.9

(Yang et al., 2016) RNNSearch + UNK replace

BiLSTM

50K

24.3

+ recurrent attention

BiLSTM

50K

25.0

Single-layer decoder

BiLSTM

80K

23.5

Deep Convolutional 8/4

80K

23.6

Two-layer decoder

Two-layer BiLSTM

80K

24.1

Deep Convolutional 15/5

80K

24.2

WMT’14 English-French (12M)

Encoder

Vocabulary

BLEU

(Bahdanau et al., 2015) RNNsearch

BiGRU

30K

28.5

(Luong et al., 2015b) Single LSTM

6-layer LSTM

40K

32.7

(Jean et al., 2014) RNNsearch-LV

BiGRU

500K

34.6

(Zhou et al., 2016) Deep-Att

Deep BiLSTM

30K

35.9

Single-layer decoder

BiLSTM

30K

34.3

Deep Convolutional 8/4

30K

34.6

Two-layer decoder

2-layer BiLSTM

30K

35.3

Deep Convolutional 20/5

30K

35.7