Medical Image Description Based on Multimodal Auxiliary Signals and Transformer

Table 1

Performance of different methods on the IU X-ray and COV-CTR datasets.

Datasets	Methods	BLEU1	BLEU2	BLEU3	BLEU4	METEOR	ROUGE_L	CIDEr
IU-X-ray	Transformer	0.422	0.264	0.177	0.120	0.164	0.338	0.421
	CoAtt	0.455	0.288	0.205	0.154	—	0.369	0.277
	HRGR-Agent	0.438	0.298	0.208	0.151	—	0.322	0.343
	PPKED	0.483	0.315	0.224	0.168	0.190	0.376	0.351
	KERP	0.482	0.325	0.226	0.162	0.187	0.339	0.280
	M2Transformer	0.463	0.318	0.214	0.155	0.192	0.335	—
	ASGMD	0.489	0.326	0.232	0.173	0.206	0.397	—
	R2Gen (base)	0.470	0.304	0.219	0.165	0.187	0.371	0.398
	MDAK (our)	0.480	0.328	0.231	0.172	0.201	0.369	0.424
	MDAKF (our)	0.494	0.318	0.229	0.174	0.194	0.389	0.371
COV-CTR	CoAtt	0.709	0.645	0.603	0.552	—	0.748	—
	SAT	0.697	0.621	0.568	0.515	—	0.723	—
	ASGK	0.712	0.659	0.611	0.570	—	0.746	—
	AdaAtt	0.676	0.633	0.596	0.514	—	0.726	—
	R2Gen	0.725	0.641	0.580	0.528	0.399	0.677	1.358
	MDAK (our)	0.723	0.652	0.586	0.545	0.403	0.676	1.452
	MDAKF (our)	0.726	0.651	0.583	0.539	0.401	0.683	1.354

The bold values indicate that the model performance of the algorithm is optimal in a certain type of dataset.