NKT: Evidence-aware Inference for Neutralized Zero-shot Transfer

Cross-domain knowledge transfer, which has received growing research attention in natural language processing (NLP), is a promising approach for various NLP tasks such as evidence-aware inference. However, the presence of biased language in well-known benchmarks notably misleads predictive models due to the hidden false correlations in the linguistic corpus. In this paper, we propose Neutralized Knowledge Transfer framework (NKT) to equip pre-trained language models with neutralized transferability. Specifically, we construct debiased multi-source corpora (CV and EL) for two exemplary knowledge transfer tasks: claim verification and evidence learning, respectively. To counteract biased language, we design a neutralization mechanism in the presence of label skewness. We also design a label adaptation mechanism in light of the mixed label systems in the multi-source corpora. In extensive experiments, the proposed NKT framework shows effective transferability contrarily to the disability of dominant baselines, particularly in the zero-shot cross-domain transfer setting.