Inference API

There are two API surfaces for inference. The Management API (/api/inference, authenticated with your platform API key) creates and controls endpoints. The Inference API is the OpenAI-compatible data plane you call to run the model, authenticated with the per-endpoint key.

Management API

Authenticate every management call with your platform API key via the X-API-Key header.

Inference API (OpenAI-compatible)

Once an endpoint is Running, call it with any OpenAI SDK. Use the endpoint's Base URL, the per-endpoint key as the Bearer token, and the endpoint id as the model name. The route depends on the model type: /chat/completions,/completions,/embeddings, or/rerank.

Tool calling and JSON output

Tool (function) calling follows the model's own chat template, not its name. When you deploy a chat model, the platform reads the template that ships with that exact model version and decides how tool calls are emitted and parsed. Fine-tunes carry their own template, so a custom model that speaks a known format gets full tool support automatically. You can override the detection per deployment underSettings → Tool calling and output: pick a specific format, turn tools off, or paste a custom chat template.

• Native: the template defines a known tool format. Tool calls come back parsed in message.tool_calls with finish_reason: "tool_calls".
• Best effort: the template speaks tools in a format we don't recognize (custom fine-tunes). Tool requests are accepted; when the output can be read as tool calls you get them parsed, otherwise you get the model's raw text in message.content.
• Unavailable: the template defines no tool calling (or the model has no chat template and nothing marks it as instruction-tuned). Requests with tools are rejected with a clear reason. JSON output still works.

The never-break contract: once a tool request is accepted, you always get an answer. If the model's output cannot be read as tool calls, the response carries the raw text inmessage.content with notool_calls array and an honestfinish_reason. It is never an error.

Structured output works on every chat model, independent of tool support: passresponse_format and the engine constrains decoding so the reply is always valid JSON (or matches your schema exactly).

curl https://api.usbios.ai/v1/chat/completions \
  -H "Authorization: Bearer <api_key_or_access_token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "dep_abc123",
    "messages": [{"role": "user", "content": "Weather in Paris as JSON"}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "weather",
        "schema": {
          "type": "object",
          "properties": {"city": {"type": "string"}, "temperature_c": {"type": "number"}},
          "required": ["city", "temperature_c"]
        },
        "strict": true
      }
    }
  }'