Evaluating alignment techniques for enhancing LLM performance in a closed-domain application: a RAG bench-marking study

Ahmad Al-Zuraiqi*, Des Greer*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Downloads (Pure)

Abstract

In the field of natural language processing (NLP), Large Language Models (LLMs) such as MISTRAL have been pre-trained on diverse tasks and datasets to achieve comprehensive language understanding. Nevertheless, fine-tuning small closed-domain datasets presents limitations, including overfitting, restricted generalization, data representation bias, and catastrophic forgetting issues. To tackle these challenges and align LLMs with ethical standards and specific requirements within closed-domain datasets, we apply a methodology that involves training four prominent LLMs - GEMMA, LLaMA, MISTRAL, and SOLAR - on a small closed-domain dataset using three distinct optimization techniques: Contrastive Preference Optimization (CPO), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO). We evaluate their proficiency by employing the Retrieval Augmented Generation (RAG) benchmark system to assess accuracy. Our research provides researchers with comprehensive insights into the effectiveness of these training methods in improving language model performance within closed-domain datasets while addressing the aforementioned challenges faced by LLMs in specific domains with limited datasets
Original languageEnglish
Title of host publication2024 International Conference on Machine Learning and Applications (ICMLA): Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1170-1175
DOIs
Publication statusAccepted - 18 Sept 2024
Event2024 International Conference on Machine Learning and Applications (ICMLA) - Hyatt Regency Coral Gables, Miami, United States
Duration: 18 Dec 202420 Dec 2024
https://www.icmla-conference.org/icmla24/

Conference

Conference2024 International Conference on Machine Learning and Applications (ICMLA)
Abbreviated titleICMLA24
Country/TerritoryUnited States
CityMiami
Period18/12/202420/12/2024
Internet address

Fingerprint

Dive into the research topics of 'Evaluating alignment techniques for enhancing LLM performance in a closed-domain application: a RAG bench-marking study'. Together they form a unique fingerprint.

Cite this