Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? 논문을 읽다가 의문이 가는 문구를 발견했다.
"Greedy decoding always predicts the correct answer."
"Greedy decoding sometimes (but not always) predicts the correct answer."
Greedy decoding을 사용하면 동일한 input에 대해 항상 output이 같을 텐데, 왜 이런 문장을 썼을까?
물론 나도 이런 경험을 한 적이 있어서 의문이 든 김에 조사해봐야겠다고 생각했다.
다행히 검색해보니 누군가가 트윗(현재 X)을 남겼다.
https://twitter.com/cwolferesearch/status/1671984854601220098
X의 Cameron R. Wolfe, Ph.D.님(@cwolferesearch)
The temperature setting is a numerical "lever" that controls the randomness of an LLM's decoding process. Low temperature yields a more deterministic output and vice versa. But, setting your temperature to zero does NOT mean the LLM's output will always be
twitter.com
"decoding discrepancies. When the LLM is used to compute the probability distribution over potential next tokens, numerical instabilities/discrepancies may occur in this process. This means that even if we are using a greedy decoding strategy (i.e., always pick the highest-probability next token), slight deviations in floating point operations could lead the next token that is produced to be different between calls."
해석해보면 계산 상의 불안정성으로 인해 항상 같은 결과를 얻는 것은 불가능하다는 얘기다.
'ML 이론' 카테고리의 다른 글
| Variational Inference - Intro (0) | 2022.03.21 |
|---|