Models

DocAI Fabric uses AI models for every intelligent step of the pipeline: OCR, splitting, classification, extraction, redaction, and the in-app assistant. The Models page is where a tenant administrator reviews the available models, brings their own, and chooses which model serves each task.

It has three tabs:

Tab	Purpose
Models	Every model and OCR engine available to the tenant.
Defaults	Which model is used by default for each task.
Usage	Estimated cost tracking and budgets (see Cost & Budgets).

The Models page is available to tenant administrators (anyone with the tenant.manage permission).

System vs. tenant models

The Models tab lists two kinds of models, each with a badge:

• System models (system): provided by the platform and shared across tenants: the built-in OCR engines and the default LLMs. They are read-only for tenants; you can use them and test their endpoints, but only a platform administrator can change them or their cost rates.
• Tenant models (tenant): your own bring-your-own (BYO) LLM endpoints, which you add, configure, and remove. DocAI Fabric load-balances across each model's endpoints using the same routing pool that serves system models.

Which system models you see, and for which tasks, follows your account's license. A license can include every system model, or name for each task the models your agreement provides; a task it gives none for runs on your own models. The Models tab, the step settings and the Defaults tab show only what the license includes, and testing a system model it does not include is refused. The same holds for the OCR engines: the OCR step and the OCR default offer only the engines the license includes, and an OCR step whose engine, or whose default engine, the license does not include fails and says so, rather than reading the pages with another engine. Voice input in the assistant is offered only when the license includes it. See Licensing for what your license includes.

Each row shows the model's tasks, modalities, context window, endpoint health, and its estimated per-unit cost (see Cost & Budgets).

Bring your own model

On the cloud service you can also run every task on your own models, OCR included, under an Order Form: see Bring Your Own Models for which models that takes and how to choose them.

To register a custom model, open the Models tab and click Add model. A model is a logical entry that can be served by one or more endpoints (see below).

Copy settings from at the top of the dialog starts the new model from one that already works: a system model, or another model of your tenant. It fills in the supported tasks, modalities, context window, image limit, estimated costs and the endpoint settings (protocol, API version, API path, extra request body; for a tenant model also its endpoint URLs, tiers, priorities and custom header names). It never copies a deployment name, an API key or a header value, and never a system model's endpoint URL, so fill in the fields left blank, then press Test. Copying from a system model on the same provider is the quickest way to get the API version and extra request body right, for example a system GPT model when you add your own Azure OpenAI deployment of a GPT model.

Model settings

Field	Meaning
Model id	Identifies the model in per-task defaults and usage logs, so it cannot be the id of a system model. An OpenAI-compatible endpoint also sends it as the model on every request, unless the endpoint gives a provider model name (Azure endpoints send their deployment name instead). Where you can, use the provider's exact spelling: an Ollama tag such as qwen3.8:27b, an org-prefixed name such as meta-llama/llama-3.3-70b, or a URI such as Yandex Cloud's gpt://<folder-id>/yandexgpt/latest. When the provider's name is a system model's id (your own OpenAI key for gpt-5.4-mini, say), choose another id such as gpt-5.4-mini-own and put the provider's name in the endpoint's provider model name. Lowercase letters, digits, ., _, -, : and /. Immutable after creation.
Display name	Human-friendly name shown in the UI.
Context window (tokens)	The model's maximum context size; used when fitting prompts. On Azure, keep it at or below the deployment's tokens per minute: one request larger than that is refused. DocAI Fabric holds it under the model's own maximum where its profile records one.
Max output (tokens)	The most the model may write in one answer. A call that asks for more is sent this number instead, so set it for a model whose server allows less than DocAI Fabric asks for (up to 65,536 tokens); the context window then keeps room for an answer this long. Blank means no limit.
Supported tasks	Which pipeline stages this model may serve: split, classify, extract, redact (the Redact step's AI finding pass), translate (the viewer's Translate Text), assistant, and ocr (only if the model does OCR natively). A model that served extract before translate existed serves it too, and the translate default starts as the extract one.
Modalities	Text, Image (needed for image-based extraction, and what lets the assistant see the pages of a document it reads), and Streaming (for the assistant).
Profile	What the model needs from a request, chosen from the profile catalog that ships with DocAI Fabric: request fields it refuses or needs (a reasoning model's reasoning_effort, say), how it gives JSON, and settings that differ by task. When you type the model id or display name, the dialog proposes the profile made for that model, for example claude-sonnet-5 for claude-sonnet-5-20260901 or qwen-3.6 for qwen3.6:27b, and shows what it sets. Generic changes nothing: requests go out as the endpoint settings below configure them. A profile that needs an API your endpoints do not speak is shown but cannot be chosen.
Overrides (JSON)	Optional changes to the profile for this model only, as JSON: for example {"set": {"reasoning_effort": "low"}}, or per task under "tasks", such as {"tasks": {"classify": {"set": {"reasoning_effort": "none"}}}}. Limits are not overrides: Context window and Max output above are the model's own fields. Each field replaces the profile's, so drop (the request fields left out) is the whole list: Copy the profile's drop list starts it from the profile's. Blank means none.
Usage counted as	Which counters this model contributes to on the Usage tab: pages, input tokens, output tokens. LLMs typically count all token metrics; an OCR-style model may count only pages.
Estimated cost	Optional per-unit rates used for cost tracking and budgets (see Cost & Budgets). Leave blank to count as 0.
Notes	Free-text notes for your own reference.

At least one endpoint is required. After saving, you can make the model a per-task default on the Defaults tab.

When you edit a model, Last full test shows when the model's Test last ran on every endpoint and where its result and the model disagree: the Image modality ticked while image input failed, a profile that says the model calls tools while tool calling failed, an endpoint that did not answer, or a test run before the profile, overrides or endpoints changed. The same warnings show as a ⚠ beside the model's name in the list. They never change how the model is called; you change the setting they name, then test again. It also shows what the test measured: each endpoint's tokens a minute where the endpoint reports them, what a page image costs in tokens by size, and how many characters make a token in each kind of text. These are recorded for sizing requests to the model; nothing uses them yet.

Endpoints, and why add more than one

An endpoint is a single network destination that serves the model (a cloud deployment, a gateway, a self-hosted server). A model can have several, and DocAI Fabric load-balances and fails over across them automatically. Reasons to add more than one:

• Throughput & rate limits: spreading load across several endpoints avoids hitting one endpoint's quota (HTTP 429) under heavy batches.
• Resilience: if an endpoint fails or its circuit breaker trips (after repeated errors), the others keep serving with no manual intervention.
• Cost control: keep cheaper endpoints in the Normal tier and pricier ones in Overflow / Emergency, so the expensive capacity is only used when needed.
• Regional compliance: pin specific endpoints to regions so a project that restricts processing to a region only uses eligible endpoints.

Endpoint settings

Field	Meaning
Name	A label for the endpoint.
Protocol	OpenAI-compatible, Azure OpenAI, or Azure AI Inference.
Endpoint URL / API path	The base URL. OpenAI-compatible endpoints use an API path (default /v1/chat/completions) and an optional provider model name, sent as the model (blank sends the model id); Azure endpoints use a deployment name and API version instead. Keep the path on /v1/chat/completions even when a provider's sample code shows /v1/responses: a model whose profile goes out on the Responses API (gpt-6.1-sol) takes the same path with chat/completions replaced by responses, and an Azure OpenAI endpoint its /openai/v1/responses route. Azure AI Inference has no Responses API, so such a profile cannot be chosen there. On such a profile the endpoint's extra request body goes into the Responses request, which names fields differently (reasoning: {"effort": "low"} rather than reasoning_effort): clear a body written for Chat Completions when you switch a model to it, or Test fails. The address must be one the server may reach: on our hosted service, a public internet address; on a self-hosted deployment, also an address on your own network (an Ollama server on the LAN, for example). Loopback (localhost, 127.0.0.1) and link-local addresses are refused everywhere, when you save and on every connection.
Authentication	On Azure OpenAI and Azure AI Inference there is nothing to choose: the dialog asks only for the API key, which is sent in the api-key header as Azure expects. On OpenAI-compatible: Bearer token or API key, which both send the key as Authorization: Bearer <key>, or custom headers (e.g. Cloudflare Access client id/secret, custom gateways). A provider that wants the key in a header of its own, such as x-api-key, needs custom headers. Secrets are stored encrypted; on edit, leave a secret blank to keep the existing value. A custom Authorization header must include its scheme, exactly as the provider's sample shows it (Api-Key <key>, Bearer <key>); a bare key is refused on save.
Extra request body (JSON)	Optional fields merged into every request, after the ones the app sets. For provider switches the standard request has no word for. The one that matters most: a reasoning model (Qwen 3.6 and similar "thinking" models) spends the whole token budget on its reasoning and returns an empty answer unless reasoning is turned off, for example {"reasoning_effort": "none"}. A field set to null is not sent at all, which is how you drop one the provider refuses: every request otherwise carries temperature, top_p, frequency_penalty and presence_penalty. When a provider refuses one of them, Test names it and the null to add, for example {"temperature": null}. Blank means none. Not a secret: shown as stored.
Region	global, or a specific region for compliance routing.
Tier	Routing tier (see below).
Priority (1 = highest)	Tiebreaker within a tier.
Enabled	Whether the endpoint is eligible for routing.

Each endpoint has a Test button in the Add and Edit Model dialog and a Test link in the Endpoint health column. It checks that the endpoint is reachable and that it performs the operations the model's tasks need, and reports a verdict per task. In the dialog, Test runs against the settings as they are on screen, so you can check a new endpoint or a changed URL or key before saving it; a key field left blank on an existing endpoint is tested with the key already stored. A test that is taking too long can be stopped. The row's Performance action measures token rates, image cost and capacity under load. Both are described in Testing and Measuring Models.

Example: Azure OpenAI deployment

A GPT model deployed in your own Azure OpenAI or Azure AI Foundry resource:

Field	Value
Copy settings from	A system GPT model, if one is listed. It fills in the rest of this table except the URL, deployment and key.
Model id	Your deployment name works well, for example gpt-5.4-luna. It names the model in defaults and usage logs (Azure is called with the deployment name below), and it cannot be the id of a system model such as gpt-5.4-mini.
Context window	The model's context window from its Azure model card (the dialog starts at 128000, below what current GPT models take)
Protocol	Azure OpenAI
Endpoint URL	The resource root only, for example https://<resource>.openai.azure.com. The app adds /openai/deployments/<deployment>/chat/completions.
Deployment	The deployment name exactly as the Azure portal shows it, for example gpt-5.4-luna
API version	The one shown next to the deployment in the portal, for example 2025-04-01-preview
API key	The resource key
Extra request body	Blank to start. Press Test: if the deployment refuses a request field (a reasoning deployment can refuse temperature), Test names it and the null to add.

Example: Yandex Cloud AI Studio

The provider's sample code targets the Responses API with an Api-Key header and a project header. Translate it to the endpoint form like this:

Field	Value
Model id	gpt://<folder-id>/yandexgpt-5.1/latest (the model from the sample, as is)
Protocol	OpenAI-compatible
Endpoint URL	https://ai.api.cloud.yandex.net
API path	/v1/chat/completions (not the sample's /v1/responses)
Auth mode	Bearer token, with the API key. The service accepts the key with either scheme, so this is equivalent to a custom Authorization header set to Api-Key <key>.
Custom headers	Usually none: the folder is already part of the model id, so the sample's OpenAI-Project header is not required. Add it as a custom header if the provider asks for it on your account.
Profile	Proposed from the model id: qwen-3.6 for the hosted Qwen 3.6 (gpt://<folder-id>/qwen3.6-35b-a3b/latest), which turns its reasoning off (reasoning_effort: none); yandexgpt for YandexGPT.
Extra request body	Blank. Without the qwen-3.6 profile, a hosted open model that reasons before answering needs {"reasoning_effort": "none"} here instead: otherwise every operation in Test fails with "The reply carried no content" and extraction returns no fields.

The same recipe applies to any provider that hosts a reasoning model behind an OpenAI-compatible endpoint; the field name depends on the provider (reasoning_effort for most, {"chat_template_kwargs": {"enable_thinking": false}} for a vLLM server). Test tells you when it is needed: the text and structured-output operations fail with "The reply carried no content" while tool calls pass.

Example: Anthropic Claude

Claude models are reached through Anthropic's OpenAI-compatible route with one key from the Anthropic Console:

Field	Value
Model id	The model's exact name, for example claude-sonnet-5
Protocol	OpenAI-compatible
Endpoint URL	https://api.anthropic.com
API path	/v1/chat/completions (not the sample's /v1/messages)
Auth mode	Bearer token, with the Anthropic API key
Profile	claude-sonnet-5, proposed from the model id
Extra request body	Blank

The profile leaves out what current models refuse: Claude Sonnet 5 refuses temperature and top_p as deprecated, and refuses the JSON mode that classify and extract ask for. With the generic profile, the same takes an extra request body of {"temperature": null, "top_p": null, "response_format": null}, which is what a model added before profiles carries; the two send the same request. The walkthrough, task by task, with what Test reports and what it costs, is in Claude Models from the Anthropic API.

How routing works

When a model is needed, DocAI Fabric chooses an endpoint like this:

1. Tier order: endpoints are tried by tier:
   • Normal: the default; always tried first.
   • Overflow: used only when every Normal endpoint is over its quota cap or has an open circuit breaker.
   • Emergency: last resort (typically the most expensive); used only when Normal and Overflow are exhausted.
2. Priority within a tier: a lower priority number is preferred. Endpoints with equal priority are round-robined. Priority never crosses tiers: a Normal endpoint with priority 10 is still tried before any Overflow endpoint.
3. Region: if a project restricts processing to a region, only endpoints in that region are eligible.

Defaults tab

The Defaults tab sets which model serves each task (split, classify, extract, redact, assistant) by default for the tenant. Each task can either:

• use the system default for that task, or
• use one of your tenant models that supports the task.

Individual projects can still override these per-task choices in their own workflow configuration; the Defaults tab sets the tenant-wide fallback.

Leave a step on the default unless that step needs a particular model. A step on the default works wherever the project goes: exported to another deployment, or moved from a test environment to production, it uses the defaults set there. A step that names a model needs a model with the same id on every deployment the project moves to. Give the same model the same id in each of your environments, even when its endpoints differ, so such a step finds it. Changing a default changes every project that uses it, published ones included.

A default must name a model that serves its task. A row whose model is no longer available (it was removed from the deployment, for example) is marked not available: every task that needs it fails until you choose another model or the system default. Its work never moves to another model on its own. A tenant model that is a default, or that a project's workflow step names, cannot be deleted or lose that task until you choose another model there.

From the command line

Your models and the Defaults tab can also be managed with the da command-line tool, for scripts and for setting up several environments the same way. It works on your tenant's own models and defaults, with the same rules as the page, and needs the same permission: sign in with da user login, or pass an API key whose access includes managing models. It works on the tenant you are signed into (switch with da user switch-tenant), or the API key's own tenant; --tenant-id (or the DOCAIFABRIC_TENANT_ID variable) names it when the CLI cannot tell.

da models list                                   # your models, and the default per task
da models show my-llama                          # one model's settings, as JSON
da models add my-llama --url https://llm.example.com --context-window 32768
da models update my-llama --tasks classify,extract
da models update my-llama --set-key primary      # replace an endpoint's key
da models update my-llama --set-key gw:X-Api-Key # one header of an endpoint with custom headers
da models test my-llama
da models performance my-llama --concurrency 1
da models defaults --set extract=my-llama --clear classify
da models profiles --for my-llama                # which profile fits the model
da models remove my-llama

A key is never written on the command line or in a file: add and update --set-key ask for it at a hidden prompt, and it is stored encrypted, as from the page. add -f and update -f take the model, or the fields to change, as JSON in the shape da models show prints. add and update end with the model's Test; --no-test skips it. test exits with 1 when a task is unsupported, so it can sit in a check script.

Your own OCR

Your own Azure AI Document Intelligence resource can read your pages instead of the built-in OCR engines. On the Models tab, click Add model and choose the kind OCR (Azure AI Document Intelligence): give the model an id and a name, choose Read or Layout, and for each endpoint the resource's endpoint (from its Keys and Endpoint page) and its key. Test reads a reference page on each endpoint. Then choose it as the OCR default on the Defaults tab, or in an OCR step's engine. From the command line, it is a model with the protocol azure_document_intelligence:

da models add my-ocr --protocol azure_document_intelligence \
  --url https://<resource>.cognitiveservices.azure.com/
da models add my-ocr-layout --protocol azure_document_intelligence \
  --url https://<resource>.cognitiveservices.azure.com/ --deployment prebuilt-layout
da models defaults --set ocr=my-ocr

The model runs the Read model, or Layout with --deployment prebuilt-layout; for both, add two models on the same resource. It serves OCR only. Its Test reads a reference page on each endpoint, and more than one endpoint (resources in two regions) fails over as for any model. The OCR step offers it beside the built-in engines, and the OCR default can name it. Pages then go to your resource only: when it cannot read one, the step waits and tries again, it never sends the page to another engine.

Your own voice input

The microphone in the assistant can turn speech into text on your own speech service instead of the deployment's. On the Models tab, click Add model and choose the kind Voice input (speech), then for each endpoint its Speech service:

• Azure AI Speech: the resource's endpoint and key, and the Languages people dictate in (language tags such as en-AU, en-US, at most 10; empty means en-US).
• Azure OpenAI transcription: the resource's endpoint, the transcription deployment's name (for example gpt-4o-mini-transcribe), its API version (empty takes a current one) and its key.

Test sends each endpoint a short silent clip: it checks the endpoint, the key and the deployment. Then choose the model as the Voice input default on the Defaults tab. From then on recordings go to your service only; when one endpoint fails the next is tried, and they never go to the deployment's speech service instead. With no voice input default, the microphone uses the deployment's speech service, where your license includes it.

From the command line:

da models add my-speech --protocol azure_speech \
  --url https://<resource>.cognitiveservices.azure.com/ --locales en-AU,en-US
da models add my-transcribe --protocol azure_openai --tasks assistant_voice \
  --url https://<resource>.openai.azure.com/ --deployment gpt-4o-mini-transcribe
da models defaults --set assistant_voice=my-speech

Estimated cost

Every model, system or tenant, can carry an estimated cost rate (LLMs: per 1M input/output tokens; OCR engines: per page). Tenant-model rates are set in the model's Edit dialog; system-model rates are set by the platform administrator. These rates drive the cost figures and budgets on the Usage tab. See Cost & Budgets for the full picture.