Knowledge Preparation — Administration

Knowledge asset generation runs on its own — but it is the administrator who defines the tone, who may approve, how much AI capacity a batch consumes and the moment approved content starts counting for chat users. This guide gathers everything you control: permissions, per-context prompts, generation limits, priority relative to interactive chat, the consumption on/off switch, installation resources and what to do when something gets stuck.

Architecture: Architecture — Knowledge Preparation. External export API: Knowledge Export API. User guide: Knowledge Asset Generation — User Guide.

1. Permissions

The feature's permissions live in the platform's central permission catalog, under the KNOWLEDGE category, and come mapped like this in the default roles:

The rule is enforced on the server, in two layers: the filter chain requires the matching authority before the request reaches the code, and a second permission check runs inside the service itself. The interface merely mirrors what the server already decides.

The platform's internal integrations — such as the cascade triggered when a document is superseded by a newer version — authenticate with the internal service token and receive a safe subset: view, generate and approve, never delete or export.

2. Knowledge prompts per context

Every context has four prompts of its own that govern generation:

• Automatic seeding: when a context is created, the four prompts are born with the default text. Seeding is idempotent and never overwrites an existing customization.
• Where they live: in the datta_prompts index, cached under datta:prompts:{dominio}:{tipo} with a 30-minute TTL.
• Customization: prompts are read with PROMPT_VIEW and edited with PROMPT_EDIT through the context's prompt interface, always with the option to restore the default. The three matching endpoints (read, customize, restore) are in the API reference.
• Defaults: they come from the default texts embedded in the platform. There are no per-domain knowledge prompt files in the repository — every context uses the generic default until you customize it through the screen.

Care when customizing: the prompts require the model to return JSON with the source excerpt citations per item (chunkIdsOrigem) — that is what makes source validation possible. Removing that requirement makes validation discard all the content, and the type fails with "nenhum item com citações válidas" ("no item with valid citations").

3. Generation capacity and limits

The defaults balance speed and AI consumption. All of them are adjustable at installation, under the datta.knowledge.* prefix:

And on the export-to-external-systems side:

Priority: interactive chat never waits

Batch generations run on a low-priority lane for AI access, separate from chat: a call only enters that lane when it is marked as batch — without the marking it is treated as interactive and goes straight through, no queue.

When the lane saturates — full queue or a wait above the limit — the call receives HTTP 429 with Retry-After and a body stating the reason (queue_full or wait_timeout) and how many seconds to wait. Tracking lives in the llm_lane_wait_ms{lane=batch}, llm_lane_inflight{lane} and llm_lane_rejected_total{lane,reason} metrics.

Sizing in practice: with 2 simultaneous calls in the lane, 3 documents in parallel and up to 3 types per document, knowledge preparation queues at most ~9 calls — the 100-slot queue absorbs that comfortably. The bottleneck is intentional: it sits in the concurrency semaphore, and it exists precisely so that a chat user never waits behind a batch.

4. Consumption by chat (per-context on/off)

Approving an asset makes it eligible for chat search, but chat only starts querying it when the context has knowledge asset consumption turned on in its domain configuration (useKnowledgeAssets, off by default; the endpoint that updates the context is in the API reference).

• Publication to the retrieval index happens on every approval, item by item, with a deterministic identifier per asset and order; deprecating an asset removes all of its items.
• With the option off, the index list the search consults is byte-identical to the behavior before the feature — nothing changes for users.
• With the option on, approved assets join that context's retrieval; the change propagates within 5 minutes (the datta:chat:knowledge-optin:{dominio} cache).
• Wildcard searches across several contexts always exclude the knowledge indices — a context never inherits another context's knowledge without opting into it.

Master switch: publication is on by default (datta.knowledge.retrieval.enabled=true); set to false, it is suspended without blocking approvals (they simply log the intent). Publication is best-effort: an embedding or indexing failure does not undo the approval. Watch knowledge_retrieval_publish_total{outcome=error} and re-approve or republish once the vector generator is back — republishing overwrites, it does not duplicate.

5. Where the knowledge is stored

Both the retrieval index suffix (conhecimento) and the master index name are configurable at installation.

6. Installation and resources

Minimum memory of 1Gi, no exception: the application's startup with the telemetry agent takes around 190 seconds and does not fit in 512Mi — the service dies at boot. The readiness and liveness probes sit on the platform's standard health paths.

Cache prerequisite: knowledge preparation uses logical database 17 of the platform cache (the first free one — 1 through 16 are already allocated). The installation already configures 32 logical databases; older installations capped at 16 make the service fail to connect. To check how many databases the cache exposes, run CONFIG GET databases on the cache itself.

Relevant environment variables:

Note: the installation configuration exports OPENSEARCH_URI, but knowledge preparation reads OPENSEARCH_URL. In practice the default http://opensearch:9200 applies; to point at another OpenSearch, set OPENSEARCH_URL explicitly.

The feature's calls arrive through the platform edge and are routed to knowledge preparation; the knowledge prompts have a route of their own, served by the prompt repository. All the endpoints are in the API reference.

7. Observability

Audit events come out in structured logs (the otel-logs-* indices): KNOWLEDGE.EXPORTED, KNOWLEDGE.EXPORT_DENIED, KNOWLEDGE.ASSET_EDITED and KNOWLEDGE.ASSET_STATUS_LOTE. Asset lineage (ASSET_*) is published to the lineage:events stream and consumed by the platform catalog.

8. Troubleshooting

Chained 429 errors during generation

1. Tell the origin apart. A batch lane 429 carries the reason in the body (queue_full or wait_timeout). A provider 429 (the Gemini per-minute quota) shows up in the logs of the model access layer and is retried automatically — first by the lane itself, then by knowledge preparation, with progressive backoff from 2s to 30s.
2. If it persists: lower KNOWLEDGE_MAX_CONCURRENT_GENERATIONS from 3 to 1 or shrink the batch size; check the Gemini key's quota; in isolated environments, use the local provider (datta.llm.provider=vllm).
3. Confirm with data: growth of llm_lane_rejected_total on queue_full means the batch is too large for 2 simultaneous calls.

"Stuck" generation batch

• The state lives in the instance that created the batch, with a copy in logical database 17 of the cache (datta:knowledge:task:{taskId}, 6-hour TTL). A service restart in the middle of a batch interrupts generation: the copy stops advancing, but the progress feed keeps answering the last state until the TTL expires.
• Diagnosis: search the knowledge preparation logs for "Lote de geracao" and list the datta:knowledge:task:* keys in logical database 17 of the cache.
• Recovery: fire a new batch with only the missing documents. Persistence is idempotent per version — nothing gets corrupted, and the new run creates new versions only for what it processes.

datta-knowledge-assets index missing or with the wrong mapping

• The index is created at service startup; if OpenSearch is down, creation is retried on first access. Confirm the log line "Indice datta-knowledge-assets criado com mapping explicito".
• Never let the index be born through dynamic indexing: fields become a text type and exact filters silently stop matching. If it already happened, delete the empty index and restart the service, or recreate it with the correct mapping and reindex.

Generation returns a warning instead of content

• When AI is unavailable (missing Gemini key, local provider off), the model layer returns the unavailability as friendly content, not as an HTTP error. Knowledge preparation recognizes the authentication messages and fails the item with the original text — check the key under Sistema›Data Layer›LLM / vLLM, which is the effective source.

Stale listings or indicators

• Caches: listings with 60 seconds in memory plus 5 minutes in the platform cache; indicators with 30 seconds in memory plus 60 seconds in the cache. Both are invalidated on any mutation.
• If a value stays stale beyond that, inspect the datta:knowledge:* keys in logical database 17 and delete the domain's keys selectively — never flush the whole database without explicitly selecting 17.