A Cooperative Lightweight Translation Algorithm Combined with Sparse-ReLU

Table 8

Influence of Sparse-ReLU on model sparsity.

Sparse-ReLU parameters	Formula	Relative loss value	Relative sparsity
a = 1, b = 1, c = 0, d = 0	Y = max(0, x)	100% (Baseline)	100% (Baseline)
a = 1, b = 1, c = 0.2, d = 0.2	Y = max(0, x-0.2)	100.92%	127.09%
a = 1, b = 1, c = 0.4, d = 0.4	Y = max(0, x-0.4)	114.68%	150.96%
a = 0.25, b = 1, c = 0.2, d = 0.4	Y = max(0,1/4(x-0.2), x-0.4)	57.80%	150.95%