TensorZero

[!CAUTION] Breaking Changes

• Streaming Inference + Tool Use: During streaming inferences, raw_name in a tool call chunk represents a delta that should be accumulated. If the tool name has finished streaming, this field will contain an empty string. Previously, TensorZero returned the same raw_name in every subsequent chunk for that tool call. The new behavior matches the OpenAI API's behavior.

Bug Fixes

• Return null instead of an empty string when missing service_tier in the OpenAI-compatible inference endpoint

New Features

• Allow inference containing files with arbitrary MIME types
• Add [timeouts] to model provider configuration for granular timeout functionality
• Support templates without schemas; add built-in system_text, assistant_text, and user_text template variables
• Support tags in OpenAI-compatible inference endpoint (tensorzero::tags)
• Add experimental_list_inferences method to the client for retrieving historical inferences

& multiple under-the-hood and UI improvements (thanks @vr-varad!)