llm package - github.com/fermyon/spin/sdk/go/v2/llm - Go Packages (original) (raw)

Package llm provides the interface to use Large Language Models in Spin.

• type EmbeddingsResult
• • func GenerateEmbeddings(model string, text []string) (*EmbeddingsResult, error)
• type EmbeddingsUsage
• type InferencingParams
• type InferencingResult
• • func Infer(model, prompt string, params *InferencingParams) (*InferencingResult, error)
• type InferencingUsage

This section is empty.

This section is empty.

This section is empty.

type EmbeddingsResult struct {

Embeddings [][][float32](/builtin#float32)

Usage *[EmbeddingsUsage](#EmbeddingsUsage)

}

EmbeddingsResult of generating embeddings.

GenerateEmbeddings generates the embeddings for the supplied list of text.

type EmbeddingsUsage struct {

PromptTokenCount [int](/builtin#int)

}

Embeddings is usage related to an embeddings generation request.

InferenceParams is the optional request parameters.

type InferencingResult struct {

Text [string](/builtin#string)

Usage *[InferencingUsage](#InferencingUsage)

}

InferencingResult is the result of an inference.

Infer performs inferencing using the provided model and prompt with the given optional parameters.

type InferencingUsage struct {

PromptTokenCount [int](/builtin#int)


GeneratedTokenCount [int](/builtin#int)

}

InferencingUsage represents information related to the inferencing result.