Interface IPromptQueryConfig_UseSampling

public interface IPromptQueryConfig_UseSampling
Mixin for a query connection configuration model that allows the decide whether the model should use sampling or greedy decoding when generating text. If sampling is not used, the model uses greedy decoding, always picking the most likely next token. If sampling is used, the model samples from the probability distribution according to parameters such as temperature or top-p, allowing for more diverse and creative outputs.

Use of this mixin is optional, but ensures a consistent name for the property. When different prompt service handlers use the same property name, switching between different prompt service handlers preserves the configured system prompt.

See also the <xp:useSampling/> Faces component from the http://www.xima.de/taglib/xfc/prompt namespace, which provides a user interface for editing the data defined by this mixin.

Since:
8.5.0

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    ATTR_USE_SAMPLING
    The name of the use sampling attribute.

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    isUseSampling()
    Gets whether the model should use sampling or greedy decoding when generating text.
    void
    setUseSampling(boolean useSampling)
    Sets whether the model should use sampling or greedy decoding when generating text.

  • Field Details

    • ATTR_USE_SAMPLING

      static final String ATTR_USE_SAMPLING
      The name of the use sampling attribute. If you implement a custom serialization / deserialization, use this name for the attribute to increase compatibility with other prompt service handlers.
      See Also:

  • Method Details

    • isUseSampling

      boolean isUseSampling()
      Gets whether the model should use sampling or greedy decoding when generating text.
      Returns:
      Whether to use sampling.

    • setUseSampling

      void setUseSampling(boolean useSampling)
      Sets whether the model should use sampling or greedy decoding when generating text.
      Parameters:
      useSampling - Whether to use sampling.