Asked about Sampling from scratch in NumPy
Why is greedy decoding the same as temperature zero?
An answer that works
Dividing the logits by a temperature approaching zero sends every gap between logits to infinity, so the softmax converges to one on the largest logit and zero everywhere else. Sampling from that distribution always returns the argmax. In code you special-case it rather than dividing by zero, but it is a limit and not an exception.
Reading an answer is not being able to give one. The section that teaches this is Sampling from scratch in NumPy, 3 hours of Sampling & decoding.
Learn this properlyAlso asked about Sampling & decoding
What is the difference between temperature and top-p?Is temperature 0 deterministic?You need consistent JSON output. What settings do you use?What is self-consistency?Your answers are sometimes cut off mid-sentence. Why?Implement top-p sampling given a vector of logits.Why use argpartition rather than argsort for top-k?