Fine-tuning LLMs encourages hallucinations

Рет қаралды 232

Ай бұрын

arxiv.org/abs//2405.05904
0:00 Introduction and recap of previous paper
0:29 Fine-tuning LLMs can lead to hallucination
1:18 Constructing an experiment to test the conjecture
1:51 Categorizing knowledge into four categories
2:59 Fine-tuning with different percentages of unknown examples
3:31 Impact of unknown items on fine-tuning accuracy
4:02 Fine-tuning improves utilization of pre-existing knowledge
4:19 Conclusion and wrap-up
Video on RAG vs fine-tuning: kzfaq.info/get/bejne/mNtxlNh-1bXLoZs.html

Пікірлер: 6

@willtipton1698 Ай бұрын

Nice video ty

@hosseinmohammadi4574 Ай бұрын

Interesting! Tnx

@thankqwerty 27 күн бұрын

Thanks for sharing the paper. In my experience with using Llama3-8B, in my benchmark dataset, I noticed that LLM has learned an incorrect fact or in contradiction with my application. I tried to clarify that in the prompt, but noticed the LLM is actually quite stubborn, and lead to quite fragile responses, i.e. the LLM sometimes get it right sometimes get it wrong with minimal changes in the prompt, could be as small as adding spaces. I wonder if you have come across similar situation or papers that discuss this behavior. Thanks.

@VivekHaldar 22 күн бұрын

Yes that kind of brittleness is a common issue unfortunately.

@gilinachum Ай бұрын

But why is the paper's fine tuning different than the original pre-training and alignment fine tuning that came before it. All expose the model to a mix of existing and new data...

@VivekHaldar Ай бұрын

You are correct -- in principle fine-tuning works the same way as pre-training (updating weights), so FT can be thought of as continued PT. Difference is in data used. One will FT when they have a domain-specific set of data that's very different from the PT data.