Models
DocAI Fabric uses AI models for every intelligent step of the pipeline: OCR, splitting, classification, extraction, redaction, and the in-app assistant. The Models page is where a tenant administrator reviews the available models, brings their own, and chooses which model serves each task.
It has three tabs:
| Tab | Purpose |
|---|---|
| Models | Every model and OCR engine available to the tenant. |
| Defaults | Which model is used by default for each task. |
| Usage | Estimated cost tracking and budgets (see Cost & Budgets). |
The Models page is available to tenant administrators (anyone with the
tenant.manage permission).
System vs. tenant models
The Models tab lists two kinds of models, each with a badge:
- System models (system): provided by the platform and shared across tenants: the built-in OCR engines and the default LLMs. They are read-only for tenants; you can use them and test their endpoints, but only a platform administrator can change them or their cost rates.
- Tenant models (tenant): your own bring-your-own (BYO) LLM endpoints, which you add, configure, and remove. DocAI Fabric load-balances across each model's endpoints using the same routing pool that serves system models.
Each row shows the model's tasks, modalities, context window, endpoint health, and its estimated per-unit cost (see Cost & Budgets).
Bring your own model
To register a custom model, open the Models tab and click Add model. A model is a logical entry that can be served by one or more endpoints (see below).
Copy settings from at the top of the dialog starts the new model from one that already works: a system model, or another model of your tenant. It fills in the supported tasks, modalities, context window, image limit, estimated costs and the endpoint settings (protocol, API version, API path, extra request body; for a tenant model also its endpoint URLs, tiers, priorities and custom header names). It never copies a deployment name, an API key or a header value, and never a system model's endpoint URL, so fill in the fields left blank, then press Test. Copying from a system model on the same provider is the quickest way to get the API version and extra request body right, for example a system GPT model when you add your own Azure OpenAI deployment of a GPT model.
Model settings
| Field | Meaning |
|---|---|
| Model id | Identifies the model in per-task defaults and usage logs, so it cannot be the id of a system model. An OpenAI-compatible endpoint also sends it as the model on every request, unless the endpoint gives a provider model name (Azure endpoints send their deployment name instead). Where you can, use the provider's exact spelling: an Ollama tag such as qwen3.8:27b, an org-prefixed name such as meta-llama/llama-3.3-70b, or a URI such as Yandex Cloud's gpt://<folder-id>/yandexgpt/latest. When the provider's name is a system model's id (your own OpenAI key for gpt-5.4-mini, say), choose another id such as gpt-5.4-mini-own and put the provider's name in the endpoint's provider model name. Lowercase letters, digits, ., _, -, : and /. Immutable after creation. |
| Display name | Human-friendly name shown in the UI. |
| Context window (tokens) | The model's maximum context size; used when fitting prompts. |
| Supported tasks | Which pipeline stages this model may serve: split, classify, extract, redact (the Redact step's AI finding pass), assistant, and ocr (only if the model does OCR natively). |
| Modalities | Text, Image (needed for image-based extraction), and Streaming (for the assistant). |
| Usage counted as | Which counters this model contributes to on the Usage tab: pages, input tokens, output tokens. LLMs typically count all token metrics; an OCR-style model may count only pages. |
| Estimated cost | Optional per-unit rates used for cost tracking and budgets (see Cost & Budgets). Leave blank to count as 0. |
| Notes | Free-text notes for your own reference. |
At least one endpoint is required. After saving, you can make the model a per-task default on the Defaults tab.
Endpoints, and why add more than one
An endpoint is a single network destination that serves the model (a cloud deployment, a gateway, a self-hosted server). A model can have several, and DocAI Fabric load-balances and fails over across them automatically. Reasons to add more than one:
- Throughput & rate limits: spreading load across several endpoints avoids hitting one endpoint's quota (HTTP 429) under heavy batches.
- Resilience: if an endpoint fails or its circuit breaker trips (after repeated errors), the others keep serving with no manual intervention.
- Cost control: keep cheaper endpoints in the Normal tier and pricier ones in Overflow / Emergency, so the expensive capacity is only used when needed.
- Regional compliance: pin specific endpoints to regions so a project that restricts processing to a region only uses eligible endpoints.
Endpoint settings
| Field | Meaning |
|---|---|
| Name | A label for the endpoint. |
| Protocol | OpenAI-compatible, Azure OpenAI, or Azure AI Inference. |
| Endpoint URL / API path | The base URL. OpenAI-compatible endpoints use an API path (default /v1/chat/completions) and an optional provider model name, sent as the model (blank sends the model id); Azure endpoints use a deployment name and API version instead. The app speaks the Chat Completions API, so keep the path on /v1/chat/completions even when a provider's sample code shows /v1/responses. The address must be one the server may reach: on our hosted service, a public internet address; on a self-hosted deployment, also an address on your own network (an Ollama server on the LAN, for example). Loopback (localhost, 127.0.0.1) and link-local addresses are refused everywhere, when you save and on every connection. |
| Authentication | On Azure OpenAI and Azure AI Inference there is nothing to choose: the dialog asks only for the API key, which is sent in the api-key header as Azure expects. On OpenAI-compatible: Bearer token or API key, which both send the key as Authorization: Bearer <key>, or custom headers (e.g. Cloudflare Access client id/secret, custom gateways). A provider that wants the key in a header of its own, such as x-api-key, needs custom headers. Secrets are stored encrypted; on edit, leave a secret blank to keep the existing value. A custom Authorization header must include its scheme, exactly as the provider's sample shows it (Api-Key <key>, Bearer <key>); a bare key is refused on save. |
| Extra request body (JSON) | Optional fields merged into every request, after the ones the app sets. For provider switches the standard request has no word for. The one that matters most: a reasoning model (Qwen 3.6 and similar "thinking" models) spends the whole token budget on its reasoning and returns an empty answer unless reasoning is turned off, for example {"reasoning_effort": "none"}. A field set to null is not sent at all, which is how you drop one the provider refuses: every request otherwise carries temperature, top_p, frequency_penalty and presence_penalty. When a provider refuses one of them, Test names it and the null to add, for example {"temperature": null}. Blank means none. Not a secret: shown as stored. |
| Region | global, or a specific region for compliance routing. |
| Tier | Routing tier (see below). |
| Priority (1 = highest) | Tiebreaker within a tier. |
| Enabled | Whether the endpoint is eligible for routing. |
Each endpoint has a Test button in the Add and Edit Model dialog and a Test link in the Endpoint health column. It checks that the endpoint is reachable and that it performs the operations the model's tasks need, and reports a verdict per task. In the dialog, Test runs against the settings as they are on screen, so you can check a new endpoint or a changed URL or key before saving it; a key field left blank on an existing endpoint is tested with the key already stored. A test that is taking too long can be stopped. The row's Performance action measures token rates, image cost and capacity under load. Both are described in Testing and Measuring Models.
Example: Azure OpenAI deployment
A GPT model deployed in your own Azure OpenAI or Azure AI Foundry resource:
| Field | Value |
|---|---|
| Copy settings from | A system GPT model, if one is listed. It fills in the rest of this table except the URL, deployment and key. |
| Model id | Your deployment name works well, for example gpt-5.4-luna. It names the model in defaults and usage logs (Azure is called with the deployment name below), and it cannot be the id of a system model such as gpt-5.4-mini. |
| Context window | The model's context window from its Azure model card (the dialog starts at 128000, below what current GPT models take) |
| Protocol | Azure OpenAI |
| Endpoint URL | The resource root only, for example https://<resource>.openai.azure.com. The app adds /openai/deployments/<deployment>/chat/completions. |
| Deployment | The deployment name exactly as the Azure portal shows it, for example gpt-5.4-luna |
| API version | The one shown next to the deployment in the portal, for example 2025-04-01-preview |
| API key | The resource key |
| Extra request body | Blank to start. Press Test: if the deployment refuses a request field (a reasoning deployment can refuse temperature), Test names it and the null to add. |
Example: Yandex Cloud AI Studio
The provider's sample code targets the Responses API with an Api-Key header and a
project header. Translate it to the endpoint form like this:
| Field | Value |
|---|---|
| Model id | gpt://<folder-id>/yandexgpt-5.1/latest (the model from the sample, as is) |
| Protocol | OpenAI-compatible |
| Endpoint URL | https://ai.api.cloud.yandex.net |
| API path | /v1/chat/completions (not the sample's /v1/responses) |
| Auth mode | Bearer token, with the API key. The service accepts the key with either scheme, so this is equivalent to a custom Authorization header set to Api-Key <key>. |
| Custom headers | Usually none: the folder is already part of the model id, so the sample's OpenAI-Project header is not required. Add it as a custom header if the provider asks for it on your account. |
| Extra request body | {"reasoning_effort": "none"} for the hosted open models that reason before answering (Qwen 3.6, for example gpt://<folder-id>/qwen3.6-35b-a3b/latest). Without it every operation in Test fails with "The reply carried no content" and extraction returns no fields. YandexGPT itself needs nothing here. |
The same recipe applies to any provider that hosts a reasoning model behind an
OpenAI-compatible endpoint; the field name depends on the provider (reasoning_effort
for most, {"chat_template_kwargs": {"enable_thinking": false}} for a vLLM server).
Test tells you when it is needed: the text and structured-output operations fail with
"The reply carried no content" while tool calls pass.
Example: Anthropic Claude
Claude models are reached through Anthropic's OpenAI-compatible route with one key from the Anthropic Console:
| Field | Value |
|---|---|
| Model id | The model's exact name, for example claude-sonnet-5 |
| Protocol | OpenAI-compatible |
| Endpoint URL | https://api.anthropic.com |
| API path | /v1/chat/completions (not the sample's /v1/messages) |
| Auth mode | Bearer token, with the Anthropic API key |
| Extra request body | {"temperature": null, "top_p": null, "response_format": null} |
The nulls are required on current models: Claude Sonnet 5 refuses temperature and
top_p as deprecated, and refuses the JSON mode that classify and extract ask for.
The walkthrough, task by task, with what Test reports and what it costs, is in
Claude Models from the Anthropic API.
How routing works
When a model is needed, DocAI Fabric chooses an endpoint like this:
- Tier order: endpoints are tried by tier:
- Normal: the default; always tried first.
- Overflow: used only when every Normal endpoint is over its quota cap or has an open circuit breaker.
- Emergency: last resort (typically the most expensive); used only when Normal and Overflow are exhausted.
- Priority within a tier: a lower priority number is preferred. Endpoints with equal priority are round-robined. Priority never crosses tiers: a Normal endpoint with priority 10 is still tried before any Overflow endpoint.
- Region: if a project restricts processing to a region, only endpoints in that region are eligible.
Defaults tab
The Defaults tab sets which model serves each task (split, classify, extract, redact, assistant) by default for the tenant. Each task can either:
- use the system default for that task, or
- use one of your tenant models that supports the task.
Individual projects can still override these per-task choices in their own workflow configuration; the Defaults tab sets the tenant-wide fallback.
Estimated cost
Every model, system or tenant, can carry an estimated cost rate (LLMs: per 1M input/output tokens; OCR engines: per page). Tenant-model rates are set in the model's Edit dialog; system-model rates are set by the platform administrator. These rates drive the cost figures and budgets on the Usage tab. See Cost & Budgets for the full picture.