Interface IPromptQueryConfig_CandidateTokenLimit

public interface IPromptQueryConfig_CandidateTokenLimit
Mixin for a query connection configuration model that allows the user to enter an optional candidate token limit, often referred to as "top-k". Top-k is the number of highest probability vocabulary tokens to keep for top-k-sampling. Top-k sampling considers the set of topK most probable tokens.

Use of this mixin is optional, but ensures a consistent name for the property. When different prompt service handlers use the same property name, switching between different prompt service handlers preserves the configured system prompt.

See also the <xp:candidateTokenLimit/> Faces component from the http://www.xima.de/taglib/xfc/prompt namespace, which provides a user interface for editing the data defined by this mixin.

Since:
8.5.0

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    ATTR_CANDIDATE_TOKEN_LIMIT
    The name of the candidate token limit attribute.

  • Method Summary

    Modifier and Type
    Method
    Description
    Long
    getCandidateTokenLimit()
    Gets the candidate token limit for the prompt query.
    void
    setCandidateTokenLimit(Long candidateTokenLimit)
    Sets the candidate token limit for the prompt query.

  • Field Details

    • ATTR_CANDIDATE_TOKEN_LIMIT

      static final String ATTR_CANDIDATE_TOKEN_LIMIT
      The name of the candidate token limit attribute. If you implement a custom serialization / deserialization, use this name for the attribute to increase compatibility with other prompt service handlers.
      See Also:

  • Method Details

    • getCandidateTokenLimit

      Long getCandidateTokenLimit()
      Gets the candidate token limit for the prompt query. This is the maximum number of tokens to consider when sampling.
      Returns:
      The candidate token limit.

    • setCandidateTokenLimit

      void setCandidateTokenLimit(Long candidateTokenLimit)
      Sets the candidate token limit for the prompt query. This is the maximum number of tokens to consider when sampling.
      Parameters:
      candidateTokenLimit - The candidate token limit to set.