vllm.model_executor.models.adapters — vLLM (original) (raw)

Contents

• Module Contents
  • Functions
  • API
    * as_classification_model()
    * as_embedding_model()
    * as_reward_model()

vllm.model_executor.models.adapters#

Module Contents#

Functions#

API#

vllm.model_executor.models.adapters.as_classification_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T[source]#

Subclass an existing vLLM model to support classification.

By default, the class probabilities are extracted from the softmaxed hidden state corresponding to the last token.

.. note::

We assume that the classification head is a single linear layer stored as the attribute score of the top-level model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_embedding_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T[source]#

Subclass an existing vLLM model to support embeddings.

By default, the embeddings of the whole prompt are extracted from the normalized hidden state corresponding to the last token.

.. note::

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.

vllm.model_executor.models.adapters.as_reward_model(cls: vllm.model_executor.models.adapters._T) → vllm.model_executor.models.adapters._T[source]#

Subclass an existing vLLM model to support reward modeling.

By default, we return the hidden states of each token directly.

.. note::

We assume that no extra layers are added to the original model; please implement your own model if this is not the case.