A Cooperative Lightweight Translation Algorithm Combined with Sparse-ReLU

Table 7

Comparison of translation results of different structures between CNN and Transformer (512 convolution output channels).

Structural details	Convolution type	Convolution kernel size	BLEU
Structure 1: Figure 6(a)	2d convolution	Same as structure 7	34.32
Structure 2: Figure 6(a)	1d convolution	in_ch = 512, kernel = (5, 1), stride = (1, 0), pad = 0	34.15
Structure 3: Figure 6(b)	2d convolution	in_ch = 1, kernel = (3, 512), stride = (1, 0), pad = (1, 0)	31.46
Structure 4: Figure 6(b)	2d convolution	Same as structure 7	33.61
Structure 5: Figure 6(c)	2d convolution	Same as structure 7	30.86
Structure 6: Figure 6(d)	2d convolution	in_ch = 1, kernel = (7, 512), stride = (1, 0), pad = (3, 0)	34.37
Structure 7: Figure 6(d) (This research adopts)	2d convolution	in_ch = 1, kernel = (5, 512), stride = (1, 0), pad = (2, 0)	35.24