LLM Sampling Parameter Planner — Temperature / top_p / top_k by task × provider

Sampling parameter quick-reference matrix

The planner above computes provider-specific T-normalized values because raw Temperature does not mean the same thing across OpenAI (range 0-2), Anthropic (range 0-1), Google Gemini (range 0-2), Mistral (range 0-1), and Llama (range 0-2). Paste your task category + target provider; the planner emits the four-parameter tuple pre-normalized for direct SDK use.

Why sampling parameters are under-tuned in production

Quick answer: most teams set Temperature once at deployment (often 0.7 or the provider default) and never revisit it. This is a mistake. Different task types have radically different optimal settings — a classification endpoint at Temperature 0.7 will show output variance across identical calls (unacceptable for audit); a creative-writing endpoint at Temperature 0.2 produces template-y repetitive output (unacceptable for engagement). Tuning Temperature + top_p per endpoint gives 10-30% quality improvement on most production benchmarks at zero compute cost.

Why Temperature 0 is not actually deterministic

Quick answer: in theory Temperature 0 picks the single highest-probability token at each step. In practice: (1) tokenization ties — when two tokens have exactly equal probability, the tie-breaker is floating-point ordering which varies with GPU batch layout. (2) Provider batching — your request may be batched with others and tensor-parallel shards produce slightly different floating-point reductions. (3) Temperature floor — some providers clamp T to 0.001 internally to avoid log-sum-exp NaN. For reproducible evaluation, set seed (where available) + Temperature 0 + pin model version + accept providers may change model weights silently. OpenAI + Google + Mistral + Llama expose seed; Anthropic does not.

top_p vs top_k — which is right for your task

Quick answer: top_p (nucleus sampling, Holtzman 2019) truncates to the smallest set of tokens whose cumulative probability exceeds p, then samples. Adapts to distribution shape: sharp distributions get a small nucleus, flat distributions get a larger nucleus. top_k (Fan 2018) truncates to a fixed number of highest-probability tokens regardless of shape. For most tasks top_p is better — it adapts. Exception: hallucination-reduction in narrow-domain factual tasks where you want a hard cap on token selection; top_k 10-20 with Temperature 0 prevents rare-token excursions. Provider note: OpenAI gpt-4o does not expose top_k; Anthropic Claude + Google Gemini + Mistral + Llama do.

Frequency vs presence penalty — they solve different problems

Quick answer: frequency_penalty subtracts n × penalty from the logit of each token, where n is occurrence count in context. Discourages repetition proportional to how often the token appeared. presence_penalty subtracts a flat penalty for any token that appeared at least once, regardless of count. Discourages topic-repetition + encourages topic-novelty. For creative writing use both (0.5 each) to avoid both word-repetition and topic-repetition. For code gen use 0 for both — legitimate code repeats identifiers and patterns. For factual Q&A use 0 — repeating key terms is correct content. Values above 1.5 cause tortured phrasing: the model avoids the best word because it was already used. Anthropic does not expose these; OpenAI + Mistral + Llama do.

Provider-specific temperature-range normalization

Quick answer: Temperature 0.7 on OpenAI (range 0-2) is at 35% of max range. Temperature 0.7 on Anthropic (range 0-1) is at 70% of max range. These produce noticeably different output diversity. When cross-provider A/B testing, do NOT use the same T value — normalize. A reasonable heuristic: OpenAI T=1.0 ≈ Anthropic T=0.7 ≈ Google T=1.0 ≈ Mistral T=0.6 ≈ Llama T=0.9. Exact calibration depends on task; run a diverse-output test (e.g. "write 5 alternative taglines") and tune each provider T to produce similar output diversity before comparing quality.

What this planner does NOT do

Quick answer: it does not run your prompts through the providers. It does not benchmark quality. It does not handle streaming-specific parameters (stop sequences, stream-mode). It does not cover prompt-engineering techniques (few-shot, chain-of-thought, tree-of-thought) — those are a separate concern from sampling. It does not cover fine-tuned model parameters (LoRA rank, RLHF reward shaping). It does not estimate cost (use the model-cost-calculator for that). It does not advise on model choice — that is a separate workflow involving quality + cost + latency + availability tradeoffs. For production eval + A/B testing, use Promptfoo + LangSmith + Humanloop; this tool configures the sampling layer that eval tooling then tests.