Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation
Di Wu, Jia-Chen Gu, Fan Yin, Nanyun Peng, and Kai-Wei Chang, in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
Download the full text
Abstract
This paper proposes SynCheck, a lightweight monitor that detects unfaithful sentences in retrieval-augmented language models (RALMs). By integrating fine-grained decoding dynamics, SynCheck outperforms existing baselines in faithfulness detection. We also introduce FOD, a faithfulness-oriented decoding algorithm that significantly improves the faithfulness of long-form generation outputs.
Bib Entry
@inproceedings{wu2024synchronous,
title = {Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation},
author = {Wu, Di and Gu, Jia-Chen and Yin, Fan and Peng, Nanyun and Chang, Kai-Wei},
booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
year = {2024}
}