Token bucket rate limiter for LLM API calls.
Provides client-side rate limiting to prevent hitting provider rate limits. Uses ETS for efficient concurrent access.
Configuration
config :elixir_llm, :rate_limiter,
enabled: true,
openai: [requests_per_minute: 60],
anthropic: [requests_per_minute: 50],
openrouter: [requests_per_minute: 100],
ollama: [requests_per_minute: 1000]Example
# Wait for a token before making a request
:ok = ElixirLLM.RateLimiter.acquire(:openai)
# Non-blocking check
case ElixirLLM.RateLimiter.try_acquire(:openai) do
:ok -> # proceed
{:error, :rate_limited} -> # handle rate limiting
end
# Check remaining tokens
tokens = ElixirLLM.RateLimiter.available(:openai)
Summary
Functions
Acquires a token for the given provider, blocking if necessary.
Returns the number of available tokens for a provider.
Returns a specification to start this module under a supervisor.
Resets the rate limiter for a provider.
Resets all rate limiters.
Starts the rate limiter.
Attempts to acquire a token without blocking.
Types
@type provider() :: atom()
Functions
Acquires a token for the given provider, blocking if necessary.
Returns :ok when a token is available.
@spec available(provider()) :: non_neg_integer()
Returns the number of available tokens for a provider.
Returns a specification to start this module under a supervisor.
See Supervisor.
@spec reset(provider()) :: :ok
Resets the rate limiter for a provider.
@spec reset_all() :: :ok
Resets all rate limiters.
Starts the rate limiter.
@spec try_acquire(provider()) :: :ok | {:error, :rate_limited}
Attempts to acquire a token without blocking.
Returns :ok if successful, {:error, :rate_limited} otherwise.