A General Method for Transferring Explicit Knowledge into Language Model Pretraining

Table 2

Results of various models on various open-domain tasks (accuracy, %).

Models/datasets	Book_review		Shopping		Weibo		XNLI		LCQMC		MSRA-NER
	Dev	Test	Dev	Test	Dev	Test	Dev	Test	Dev	Test	Dev	Test
XLNet	88.71	87.69	96.82	96.73	98.04	97.98	76.87	76.33	88.79	87.25	95.08	94.97
K-XLNet (HowNet)	88.83	88.67	97.04	97.14	98.17	98.05	77.18	77.13	89.02	87.37	96.29	96.26
K-XLNet (CN-DBpedia)	88.76	87.71	96.89	96.77	98.12	98.41	76.97	76.39	88.87	87.31	96.31	96.24