ElixirLLM.Chat (ElixirLLM v0.3.0)

Copy Markdown View Source

Represents a conversation with an LLM.

The Chat struct maintains conversation state including messages, model configuration, tools, and callbacks. It is immutable - all operations return a new Chat struct.

Example

chat = ElixirLLM.new()
|> ElixirLLM.model("gpt-4o")
|> ElixirLLM.instructions("You are a helpful assistant")
|> ElixirLLM.temperature(0.7)

{:ok, response, chat} = ElixirLLM.ask(chat, "Hello!")

Summary

Functions

Adds a message to the chat history.

Adds multiple messages to the chat history.

Adds a system instruction message. Use replace: true to replace existing instructions.

Sets the maximum tokens for the response.

Sets the model for the chat.

Creates a new empty chat.

Sets a callback for when a tool is called.

Sets a callback for when a tool returns a result.

Sets additional provider-specific parameters.

Explicitly sets the provider for the chat.

Sets a schema for structured output. The response will be parsed into the schema struct.

Sets the temperature (creativity) for responses. Range: 0.0 to 2.0.

Registers a tool with the chat. Accepts a module implementing the Tool behaviour or an inline tool map.

Registers multiple tools at once. Use replace: true to clear existing tools first.

Enables response caching.

Enables circuit breaker protection.

Enables client-side rate limiting.

Enables automatic retry with exponential backoff.

Types

callback()

@type callback() :: (any() -> any())

t()

@type t() :: %ElixirLLM.Chat{
  cache: boolean(),
  circuit_breaker: boolean(),
  max_tokens: non_neg_integer() | nil,
  messages: [ElixirLLM.Message.t()],
  model: String.t() | nil,
  on_chunk: callback() | nil,
  on_tool_call: callback() | nil,
  on_tool_result: callback() | nil,
  params: map(),
  provider: module() | nil,
  rate_limit: boolean(),
  retry: keyword() | false,
  schema: module() | nil,
  temperature: float() | nil,
  tools: [module() | map()]
}

Functions

add_message(chat, message)

@spec add_message(t(), ElixirLLM.Message.t()) :: t()

Adds a message to the chat history.

add_messages(chat, messages)

@spec add_messages(t(), [ElixirLLM.Message.t()]) :: t()

Adds multiple messages to the chat history.

instructions(chat, content, opts \\ [])

@spec instructions(t(), String.t(), keyword()) :: t()

Adds a system instruction message. Use replace: true to replace existing instructions.

max_tokens(chat, tokens)

@spec max_tokens(t(), non_neg_integer()) :: t()

Sets the maximum tokens for the response.

model(chat, model_id)

@spec model(t(), String.t()) :: t()

Sets the model for the chat.

new(opts \\ [])

@spec new(keyword()) :: t()

Creates a new empty chat.

on_tool_call(chat, callback)

@spec on_tool_call(t(), callback()) :: t()

Sets a callback for when a tool is called.

on_tool_result(chat, callback)

@spec on_tool_result(t(), callback()) :: t()

Sets a callback for when a tool returns a result.

params(chat, params)

@spec params(t(), map()) :: t()

Sets additional provider-specific parameters.

provider(chat, provider)

@spec provider(t(), atom() | module()) :: t()

Explicitly sets the provider for the chat.

This overrides the auto-detected provider from the model name. Accepts either a provider atom (:openai, :anthropic, etc.) or a provider module directly.

Examples

chat = ElixirLLM.new()
|> ElixirLLM.provider(:anthropic)
|> ElixirLLM.model("claude-3-opus-20240229")

# Use OpenRouter for any model
chat = ElixirLLM.new()
|> ElixirLLM.provider(:openrouter)
|> ElixirLLM.model("anthropic/claude-3-opus")

# With a custom provider module
chat = ElixirLLM.new()
|> ElixirLLM.provider(MyApp.CustomProvider)

schema(chat, schema_module)

@spec schema(t(), module()) :: t()

Sets a schema for structured output. The response will be parsed into the schema struct.

temperature(chat, temp)

@spec temperature(t(), float()) :: t()

Sets the temperature (creativity) for responses. Range: 0.0 to 2.0.

tool(chat, tool)

@spec tool(t(), module() | map()) :: t()

Registers a tool with the chat. Accepts a module implementing the Tool behaviour or an inline tool map.

tools(chat, tool_list, opts \\ [])

@spec tools(t(), [module() | map()], keyword()) :: t()

Registers multiple tools at once. Use replace: true to clear existing tools first.

with_cache(chat)

@spec with_cache(t()) :: t()

Enables response caching.

Caches responses based on model, messages, and settings to reduce API costs. Cache configuration (TTL, max entries) is set in application config.

Example

chat
|> ElixirLLM.with_cache()
|> ElixirLLM.ask("What is 2+2?")  # First call hits API
|> ElixirLLM.ask("What is 2+2?")  # Second call returns cached response

with_circuit_breaker(chat)

@spec with_circuit_breaker(t()) :: t()

Enables circuit breaker protection.

Prevents cascading failures by temporarily blocking requests to failing providers. Configuration (failure threshold, recovery timeout) is set in application config.

Example

chat
|> ElixirLLM.with_circuit_breaker()
|> ElixirLLM.ask("Hello!")

with_rate_limiting(chat)

@spec with_rate_limiting(t()) :: t()

Enables client-side rate limiting.

Automatically acquires tokens before making requests to prevent hitting provider rate limits. Rate limits are configured per provider in application config.

Example

chat
|> ElixirLLM.with_rate_limiting()
|> ElixirLLM.ask("Hello!")

with_retry(chat, opts \\ [])

@spec with_retry(
  t(),
  keyword()
) :: t()

Enables automatic retry with exponential backoff.

Options

  • :max_attempts - Maximum retry attempts (default: 3)
  • :base_delay_ms - Initial delay between retries (default: 1000)
  • :max_delay_ms - Maximum delay between retries (default: 30000)
  • :jitter - Add random jitter to delays (default: true)
  • :on_retry - Callback (attempt, error) -> any() called before each retry

Examples

chat
|> ElixirLLM.with_retry()
|> ElixirLLM.ask("Hello!")

chat
|> ElixirLLM.with_retry(max_attempts: 5, base_delay_ms: 500)
|> ElixirLLM.ask("Hello!")