Turning the Knobs of LLM Text Generation
Turning the Knobs of LLM Text Generation Ever wonder how much control you actually have over the text an LLM produces? In this post, we will look at three simple but powerful knobs you can tweak to push a model toward more deterministic output or toward something more creative. We are talking about top_k, top_p and temperature. But before describing them, we need to understand the two main behaviors we can get from an LLM when it is sampling tokens:...