Interface IPromptQueryConfig_CumulativeProbabilityThreshold

public interface IPromptQueryConfig_CumulativeProbabilityThreshold
Mixin for a query connection configuration model that allows the user to enter an optional cumulative probability threshold for nucleus sampling, often referred to as "top-p". Top-p is a value that controls the diversity of the output by limiting the selection of tokens to a subset whose cumulative probability exceeds the top-p value.

This is an alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with "top-p" probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered.

Use of this mixin is optional, but ensures a consistent name for the property. When different prompt service handlers use the same property name, switching between different prompt service handlers preserves the configured system prompt.

See also the <xp:cumulativeProbabilityThreshold/> Faces component from the http://www.xima.de/taglib/xfc/prompt namespace, which provides a user interface for editing the data defined by this mixin.

Since:
8.5.0

  • Field Details

    • ATTR_CUMULATIVE_PROBABILITY_THRESHOLD

      static final String ATTR_CUMULATIVE_PROBABILITY_THRESHOLD
      The name of the cumulative probability threshold attribute. If you implement a custom serialization / deserialization, use this name for the attribute to increase compatibility with other prompt service handlers.
      See Also:

  • Method Details

    • getCumulativeProbabilityThreshold

      Double getCumulativeProbabilityThreshold()
      Gets the cumulative probability threshold for the prompt query. This is a value that controls the diversity of the output by limiting the selection of tokens to a subset whose cumulative probability exceeds the top-p value.
      Returns:
      The top probability.

    • setCumulativeProbabilityThreshold

      void setCumulativeProbabilityThreshold(Double cumulativeProbabilityThreshold)
      Sets the cumulative probability threshold for the prompt query. This is a value that controls the diversity of the output by limiting the selection of tokens to a subset whose cumulative probability exceeds the top-p value.
      Parameters:
      cumulativeProbabilityThreshold - The top probability to set.