Knowledge Export API

The Knowledge Export API delivers the content curated in DATTA — optimized documents, FAQ, glossary and passages — to chatbots and external systems. You curated the base on the platform; now the portal chatbot, the intranet or the service desk needs that content, and without access to all of DATTA. The API solves the integration: a versioned REST API with service account authentication, incremental synchronization and request limiting.

Governance out of the box: only content approved in curation (the Knowledge Catalog's Curadoria tab) goes out and, for passages, only content in force. Drafts and superseded content are never exported. Every response carries the contract header X-Knowledge-Export-Version: 1.

Architecture and design decisions: knowledge export. General authentication and error conventions: DATTA API overview. The exact routes are in the API reference.

1. Service accounts

A machine-to-machine credential with per-domain (context) scope, the fixed KNOWLEDGE_EXPORT permission and a secret revealed only once.

1.1 Management operations

By role defaults, only administrators hold DATA_ACCESS_MANAGE_USERS and KNOWLEDGE_EXPORT.

1.2 Create (reveal-once)

Creation returns 201 with the full credential — the only time the secret is ever shown:

{ "id":"<uuid>", "nome":"bot-mec", "dominios":["MEC"],
  "clientId":"<uuid>", "clientSecret":"dsa_<base64url>",
  "aviso":"Guarde o clientSecret agora: ele não será exibido novamente." }

• The clientSecret (dsa_ + 32 random bytes) is stored only as a bcrypt hash and never reappears — store it in your secret vault at creation time.
• The account is persisted as a ServiceAccount record in the datta system database (Neo4j).
• Audit: DATA_ACCESS.SERVICE_ACCOUNT_CREATED.
• The account's permission is fixed as ["KNOWLEDGE_EXPORT"] (v1); there is no per-account lifetime — the token lifetime is global (datta.auth.service-account-token-ttl, default 900 seconds).

1.3 Obtain a token (client-credentials)

Sending clientId and clientSecret, the account receives a short-lived token:

{"accessToken":"<jwt>","tokenType":"Bearer","expiresIn":900,"scope":["MEC"]}

Token claims (HS256, the same key as user tokens): sub = svc:{id}, type = service-account, scope = [domains], permissions = ["KNOWLEDGE_EXPORT"].

Any failure — nonexistent clientId, wrong secret, revoked account — returns the same generic 401 "Credenciais inválidas" ("Invalid credentials"), by design (anti-enumeration, constant-time comparison). Above 10 exchanges per minute for the same clientId+IP, the answer is 429 with Retry-After: 60.

1.4 Revoke

Revocation requires a reason and is always soft (never a physical delete), idempotent, with the DATA_ACCESS.SERVICE_ACCOUNT_REVOKED audit event:

{"id":"<uuid>","ativo":false,"revogadoEm":"..."}

Propagation: the account's status is checked with a 60-second cache (key datta:knowledge:export:sa-status:{id}) — still-valid tokens stop working within up to 60 seconds, with 401 "Credencial revogada. Solicite uma nova credencial ao administrador." ("Credential revoked. Request a new credential from the administrator.")

Interface: service account management on the data-access page has not been implemented yet — today it is done entirely through the operations above.

2. Exported collections

Each collection is read per domain. The {dominio} accepts [a-zA-Z0-9_-]{1,64} and is compared to the credential's scope in a normalized way (lowercase, alphanumeric). Authorization happens in layers: the KNOWLEDGE_EXPORT authority → domain scope → revocation check → request limit.

In the examples below, $EXPORT is your environment's export base address (see the API reference) and $TOKEN is the token obtained in section 1.3.

2.1 Manifest — inventory + ETag

curl -H "Authorization: Bearer $TOKEN" "$EXPORT/MEC/manifest"

{
  "dominio": "MEC",
  "snapshotVersion": "9f2c3a1b0d4e5f67",
  "geradoEm": "2026-07-09T12:00:00Z",
  "colecoes": {
    "documentos": {"total": 12, "lastModified": "2026-07-08T18:20:11Z"},
    "faq":        {"total": 9,  "lastModified": "2026-07-08T18:20:14Z"},
    "glossario":  {"total": 4,  "lastModified": "2026-07-07T10:02:00Z"},
    "chunks":     {"total": 3210, "lastModified": "2026-07-08T18:19:55Z"}
  }
}

Cheap polling with If-None-Match (the snapshotVersion becomes the ETag):

curl -i -H "Authorization: Bearer $TOKEN" \
  -H 'If-None-Match: "9f2c3a1b0d4e5f67"' "$EXPORT/MEC/manifest"
# HTTP/1.1 304 Not Modified   (60s cache; no index query on a hit)

2.2 Documents (?format=json|md&since=...)

Approved optimized documents. With format=md the response is text/markdown (rendered per section); with format=json (the default) you get an array of:

{"documentCode":"url-3f2a...","titulo":"...","tituloDocumento":"...",
 "dominio":"MEC","versao":2,"geradoEm":"...",
 "secoes":[{"titulo":"Resumo","conteudo":"...","ordem":1,"chunkIdsOrigem":["..."]}]}

2.3 FAQ and glossary (?since=...)

Flattened items — one object per question/answer pair or per term:

{"documentCode":"...","versao":1,"geradoEm":"...","pergunta":"...","resposta":"...","ordem":1,"chunkIdsOrigem":["..."]}
{"termo":"FIES","definicao":"...","siglaDe":"Fundo de Financiamento Estudantil","aliases":[],"origem":"url-...","versao":1,"geradoEm":"...","chunkIdsOrigem":["..."]}

2.4 Passages (chunks) — streaming NDJSON

curl -H "Authorization: Bearer $TOKEN" \
  "$EXPORT/MEC/chunks?since=2026-07-01T00:00:00Z&includeEmbeddings=false" \
  -o chunks.ndjson
# Content-Type: application/x-ndjson — one JSON line per passage

• Only passages in force from the context's index, internally paginated by a cursor over (createdAt, _id) (internal page default of 500) — the collection is never assembled whole in memory: 100k passages stream with constant consumption.
• includeEmbeddings=true includes embedding + embeddingModel (payload ~4–8 KB more per passage; by default the field is not even fetched from the index). In the index the vector lives in the model's field (embedding_<model>); the export reads the field named by embeddingModel and delivers it under the embedding key.
• Line: {"id","documentCode","dominio","texto","secao","ordem","lastModified"[,"embedding","embeddingModel"]}.

2.5 Bundle — zip package

curl -H "Authorization: Bearer $TOKEN" -OJ "$EXPORT/MEC/bundle"
# conhecimento-MEC.zip:
#   manifest.json
#   documentos/<documentCode>-v<versao>.md
#   faq.json
#   glossario.json

The zip is assembled by streaming, with the four collections fetched in parallel (concurrency 4).

2.6 Incremental synchronization (since)

All collections accept since in ISO-8601 (an instant or with an offset — an invalid value returns 400 with an example). Recommended consumer flow:

1. Poll the manifest with If-None-Match (e.g. every 5 min) — 304 = nothing to do;
2. snapshotVersion changed → download only the deltas, passing since=<last known lastModified> on each collection;
3. Persist the new snapshotVersion + lastModified per collection.

The since filter compares against the asset's geradoEm (collections) and createdAt (passages) — when a document is regenerated, the new version appears in the delta once it is approved in curation.

Practical example — connecting an external chatbot

1. The administrator creates the bot-mec account with scope ["MEC"] (section 1.2) and hands clientId/clientSecret to the chatbot team, who store them in a vault.
2. The chatbot exchanges the credentials for a token (section 1.3) and downloads the initial load through the bundle.
3. Every 5 minutes it polls the manifest with If-None-Match. Most of the time that costs one 304.
4. When snapshotVersion changes, it downloads only the deltas with since and updates its local base.
5. If the credential leaks, the administrator revokes it with a reason (section 1.4) — within 60 seconds consumption stops, and the audit trail shows everything that was exported.

3. Request limit, errors and auditing

• Request limit: counted per identity (service account or user) in the platform cache, under the key datta:knowledge:export:rl:{serviceAccountId|user}, with a 1-minute window and a default of 60 req/min (datta.knowledge.export.rate-limit-per-minute, environment variable KNOWLEDGE_EXPORT_RATE_LIMIT). Exceeded → 429 with Retry-After and {"error": ..., "reason": "rate_limit"}.
• Structured errors (always {"error": <pt-BR message>, "reason": <stable code>}): 403 fora_do_escopo (domain outside the account's scope — audited), 401 credencial_revogada, 400 for an invalid since, format or domain.
• Auditing (recorded under datta.audit.knowledge, flowing to otel-logs-*): KNOWLEDGE.EXPORTED per served collection (author, domain, collection, items, format) and KNOWLEDGE.EXPORT_DENIED with the reason.
• Metrics: knowledge_export_requests_total{colecao,outcome=success|error|not_modified} and knowledge_export_items_total{colecao}.

4. Troubleshooting