Abstract
In the field of natural language processing (NLP), Large Language Models (LLMs) such as MISTRAL have been pre-trained on diverse tasks and datasets to achieve comprehensive language understanding. Nevertheless, fine-tuning small closed-domain datasets presents limitations, including overfitting, restricted generalization, data representation bias, and catastrophic forgetting issues. To tackle these challenges and align LLMs with ethical standards and specific requirements within closed-domain datasets, we apply a methodology that involves training four prominent LLMs - GEMMA, LLaMA, MISTRAL, and SOLAR - on a small closed-domain dataset using three distinct optimization techniques: Contrastive Preference Optimization (CPO), Direct Preference Optimization (DPO), and Kahneman-Tversky Optimization (KTO). We evaluate their proficiency by employing the Retrieval Augmented Generation (RAG) benchmark system to assess accuracy. Our research provides researchers with comprehensive insights into the effectiveness of these training methods in improving language model performance within closed-domain datasets while addressing the aforementioned challenges faced by LLMs in specific domains with limited datasets
Original language | English |
---|---|
Title of host publication | 2024 International Conference on Machine Learning and Applications (ICMLA): Proceedings |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 1170-1175 |
DOIs | |
Publication status | Accepted - 18 Sept 2024 |
Event | 2024 International Conference on Machine Learning and Applications (ICMLA) - Hyatt Regency Coral Gables, Miami, United States Duration: 18 Dec 2024 → 20 Dec 2024 https://www.icmla-conference.org/icmla24/ |
Conference
Conference | 2024 International Conference on Machine Learning and Applications (ICMLA) |
---|---|
Abbreviated title | ICMLA24 |
Country/Territory | United States |
City | Miami |
Period | 18/12/2024 → 20/12/2024 |
Internet address |