Struct GenAIProvider 

pub struct GenAIProvider { /* private fields */ }

Thread-safe LLM provider implementation using Arc<RwLock<>>.

This provider can be cheaply cloned and shared across multiple agents. Each clone shares the same underlying client and rate limiting state.

Implementations

impl GenAIProvider

pub fn new() -> Result<Self>

Creates a new GenAI provider with automatic configuration.

pub fn new_with_config( provider_type: Option<&str>, api_key: Option<&str>, ) -> Result<Self>

Creates a new GenAI provider with explicit configuration.

pub fn from_config( config: &Config, cli_model: Option<&str>, ) -> Result<(Self, ResolvedProvider)>

Build a provider from a Config, merging in environment detection and an optional CLI model override, and return the effective provider/model.

Precedence:

• provider: config.provider > environment detection
• model: cli_model > config.model > provider default
• api_key: config.api_key > ambient environment
• base_url: config.base_url > ambient environment

The returned client is bound to the resolved adapter, so custom/local OpenAI-compatible model names (e.g. phi-4-npu-ov) route correctly while recognized names still resolve by prefix. Model names are passed through verbatim so genai namespacing (openai::model) keeps working.

pub async fn get_total_tokens_used(&self) -> usize

Get total tokens used across all requests

pub async fn get_request_count(&self) -> usize

Get total request count

pub async fn add_tokens(&self, count: usize)

Add tokens to the total count

pub async fn get_available_models(&self, provider: &str) -> Result<Vec<String>>

Retrieves all available models for a specific provider.

pub async fn generate_response_simple( &self, model: &str, prompt: &str, ) -> Result<LlmResponse>

Generates a simple text response without streaming. Includes exponential backoff retry for rate limits and transient errors.

pub async fn generate_response_with_retry( &self, model: &str, prompt: &str, max_retries: usize, ) -> Result<LlmResponse>

Generates a response with configurable retry count and exponential backoff.

pub async fn generate_response_stream_to_channel( &self, model: &str, prompt: &str, tx: UnboundedSender<String>, ) -> Result<()>

Generates a streaming response and sends chunks via mpsc channel.

pub fn get_supported_providers() -> Vec<&'static str>

Get a list of supported providers

pub async fn get_available_providers(&self) -> Result<Vec<String>>

Get all available providers

pub async fn test_model(&self, model: &str) -> Result<bool>

Test if a model is available and working

pub async fn validate_model( &self, model: &str, provider_type: Option<&str>, ) -> Result<String>

Validate and get the best available model for a provider

impl GenAIProvider

pub async fn chat_turn_with_tools( &self, model: &str, messages: &[CoreMessage], tools: &[CoreToolSpec], choice: CoreToolChoice, ) -> Result<CoreTurnOutput>

Execute one tool-calling chat turn against model.

Returns the model’s tool calls when any were issued, otherwise its text. The caller records the output as an observation before acting on it (R2); this driver records nothing.

pub async fn chat_turn_configured( &self, model: &str, messages: &[CoreMessage], tools: &[CoreToolSpec], choice: CoreToolChoice, config: CoreGenerationConfig, ) -> Result<CoreTurnOutput>

Execute a tool-calling turn with explicit sampling controls.

pub async fn stream_tool_turn( &self, model: &str, messages: &[CoreMessage], tools: &[CoreToolSpec], choice: CoreToolChoice, emit: impl FnMut(CoreTurnStreamEvent), ) -> Result<CoreTurnOutput>

Stream one tool-aware turn while retaining the complete tool calls.

Reasoning is forwarded as it arrives. Answer text is held until the terminal event proves that the model did not select a tool, preventing tool-call preambles and protocol fragments from leaking into the chat.

Trait Implementations

impl Clone for GenAIProvider

fn clone(&self) -> GenAIProvider

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.