Package de.xima.fc.prompt.service.support

Interface IPromptQueryConfig_CandidateTokenLimit

  • public interface IPromptQueryConfig_CandidateTokenLimit
    Mixin for a query connection configuration model that allows the user to enter an optional candidate token limit, often referred to as "top-k". Top-k is the number of highest probability vocabulary tokens to keep for top-k-sampling. Top-k sampling considers the set of topK most probable tokens.

    Use of this mixin is optional, but ensures a consistent name for the property. When different prompt service handlers use the same property name, switching between different prompt service handlers preserves the configured system prompt.

    See also the <xp:candidateTokenLimit/> Faces component from the http://www.xima.de/taglib/xfc/prompt namespace, which provides a user interface for editing the data defined by this mixin.

    Since:
    8.5.0

    • Field Detail

      • ATTR_CANDIDATE_TOKEN_LIMIT

        static final String ATTR_CANDIDATE_TOKEN_LIMIT
        The name of the candidate token limit attribute. If you implement a custom serialization / deserialization, use this name for the attribute to increase compatibility with other prompt service handlers.
        See Also:
        Constant Field Values

    • Method Detail

      • getCandidateTokenLimit

        Long getCandidateTokenLimit()
        Gets the candidate token limit for the prompt query. This is the maximum number of tokens to consider when sampling.
        Returns:
        The candidate token limit.

      • setCandidateTokenLimit

        void setCandidateTokenLimit​(Long candidateTokenLimit)
        Sets the candidate token limit for the prompt query. This is the maximum number of tokens to consider when sampling.
        Parameters:
        candidateTokenLimit - The candidate token limit to set.