LiteLLM

LiteLLM

Bases: BaseLLM

A wrapper class for interacting with a LiteLLM-compatible large language model (LLM). For more information, see: https://docs.litellm.ai/.

Attributes:

Name	Type	Description
model	str	The identifier of the LLM model to use (e.g., "gpt-4", "llama-3").
temperature	float	Sampling temperature to use. Must be between 0.0 and 1.0. Higher values result in more random outputs, while lower values make the output more deterministic. Default is 1.0.
max_tokens	int	The maximum number of tokens to generate in the completion.
api_key	str	API key used for authenticating with the LLM provider.
additional_kwargs	dict[str, Any]	A dictionary of additional parameters passed to the LLM during completion. This allows customization of the request beyond the standard parameters.
callback_manager	BaseObservability | None	(ModelMonitor, optional): The callback manager is used for observability.

completion

completion(prompt: str, **kwargs: Any) -> CompletionResponse

Creates a completion for the provided prompt and parameters. Using OpenAI's standard endpoint (/completions).

Parameters:

Name	Type	Description	Default
prompt	str	The input prompt to generate a completion for.	required
**kwargs	Any	Additional keyword arguments to customize the LLM completion request.	{}

chat_completion

chat_completion(messages: list[ChatMessage | dict], **kwargs: Any) -> ChatResponse

Creates a chat completion for LLM. Using OpenAI's standard endpoint (/chat/completions).

Parameters:

Name	Type	Description	Default
messages	list[ChatMessage]	A list of chat messages as input for the LLM.	required
**kwargs	Any	Additional keyword arguments to customize the LLM completion request.	{}