PT EN
Back to site

Clearing a context's data (purge)

Reprocessing a corpus from scratch — a new chunking strategy, a fresh extraction round — used to mean deleting indices by hand, hunting down leftover records and hoping nothing of the system went down with them. With Clear data, you empty all of a context's content with one audited click: documents, legislation, passages, case law and entities disappear from the graph and from search, the context stays registered and ready for a new ingestion, and the system databases are untouchable by design.

Do not confuse this with Delete context, which removes the context itself from the configuration. Clearing only empties the content — the context remains.

The operation is destructive and irreversible: the erased content only comes back through a new ingestion. The context's record stays in the datta system database; only the content is erased, in Neo4j and in the context's search indices (datta_{contexto}_*).

Step by step

  1. Open SistemaContextos.
  2. On the context's row, click Limpar dados (amber icon).
  3. In the confirmation dialog, provide the reason — it is recorded in the audit trail — and confirm with Sim, limpar tudo.
  4. Follow the progress through the notices on the screen; errors appear with a message in Portuguese.

Requires the CONTEXT_PURGE permission (sensitive; granted by default only to the ADMIN role — see roles and permissions).

What the cleanup does

The platform orchestrates the graph and search cleanups in parallel, and cascades to the business processes before deleting the database:

  • Graph: erases all content of the context's graph database (documents, passages, entities, relationships), in sliced transactions so it can handle large bases.
  • Search: removes the context's search indices (pattern datta_{contexto}_*).
  • BPM processes: before the database is erased, the BPM process instances whose effective context is the purged domain are deleted along with it. Active instances are canceled first — in-flight triages are interrupted and call-activity child instances cascade. The context's auto-start baseline is cleared as well, so re-ingested cases get an instance again. Without this cascade, the BPM monitor would be left with orphaned executions pointing at cases that no longer exist — the same rationale as the passage cleanup when an individual case is deleted.
  • Ready for re-ingestion: after the cleanup, re-sending the documents safely recreates everything — reprocessing the same corpus does not create duplicates, because the graph write is idempotent and the search documents have a deterministic identifier.

The BPM cascade is best-effort: if the process engine is unavailable, the cleanup proceeds and the result reports the failure (bpmn.status=falhou). In that case, clean up the orphaned instances afterwards with the orphaned BPM instances runbook. The cascade's audit event is BPMN.INSTANCIA.EXCLUIDA_POR_CONTEXTO.

Built-in protections

  • System databases are never touched: the platform refuses to clear the internal databases (neo4j, system, datta, datta-*) and any domain that is not explicitly mapped to a context database — there is no fallback to the default database, which means there is no path to an accidental wipe of the system. On the search side, the datta-* and system patterns are refused as well.
  • Sensitive permission + reason: only holders of CONTEXT_PURGE can run it, and the reason stays in the audit trail.
  • Through the app only: the mutation requires DATTA's own origin marker (the X-Requested-With header, injected automatically by the application client) — calls made outside the app are blocked.

Practical example — reprocessing with a new chunking strategy

The team changed how documents are chunked and wants to reprocess the whole "Processos" context:

  1. In SistemaContextos, click Limpar dados on the "Processos" row.
  2. Reason: "Reprocessamento com nova estratégia de chunking — chamado 4211".
  3. Confirm with Sim, limpar tudo and wait for it to finish.
  4. Re-send the documents through the Upload screen — the context is repopulated from scratch, with the new chunking, without duplicates.

Known limitations

  • Audited in the log, not as a structured event: today the cleanup records CONTEXT.PURGED in the platform logs, with author and reason. Emitting a structured event to the audit service, as happens in other mutations, is still pending.
  • Dedicated telemetry: the cleanup operation does not yet have its own span and metric.

Diagnosis

  • 403 CSRF_BLOCKED: the call did not go through the application client (no X-Requested-With). Use the button in the interface, not a raw external call.
  • 403 for permission: the user does not have CONTEXT_PURGE (only ADMIN by default).
  • "Contexto ... nao esta mapeado a um database Neo4j — purge recusado.": the domain is not registered with a graph database of its own — review the registration in managing contexts.

The routes involved in the cleanup (orchestration, graph, search and the BPM cascade) are in the API reference.