VulDistilBERT: A CPS Vulnerability Severity Prediction Method Based on Distillation Model

Table 9

Results of pooling effect comparison using fine-tuned DistilBERT model.

Models	Pool	Best epoch	Loss	Accuracy	Precision	Recall	F1 scores
Fine-tuned DistilBERT	Mean	3	0.585	0.77	0.78	0.64	0.67
	Max	4	0.588	0.77	0.75	0.64	0.67
	CLS	4	0.571	0.78	0.76	0.68	0.70