Scheduler — Job Monitor tab

One load has just finished, another is halfway through, a third runs at dawn with nobody watching. The Job Monitor tab is the part of the Scheduler that answers the three questions an operator has: what is running now, what will run and what already ran — and, for every finished run, what exactly happened inside it.

It lives under DATTA Extract › Scheduler, tab Job Monitor.

One screen, three tabs

Old addresses still work. view=pipeline-monitor opens the Scheduler already on this tab; view=pipeline-agenda and view=datta-scheduler open on the other two.

This tab has no log viewer. It shows what the run recorded — status, duration, warnings, the error with its stage, and the per-node breakdown. If you are after the engine's line-by-line output, it is not here: see "Where to look when the detail is not enough", at the end of this page.

The summary cards

The top of the tab has four cards, and they follow you across the three inner views (Monitor, Agendados and Histórico):

Under the number, each card carries a short sentence saying what it counts. Read that sentence before drawing a conclusion: it is the scope of the count, and it is what separates "two failures" from "two failures today".

When you arrive here from a specific package (see "Coming from a package", below), the cards count that package, not the whole platform.

Monitor view — what is running now

Every active run appears as a card with:

• the package name and the time the run started;
• a stopwatch ticking while it runs;
• who triggered the load;
• the flow path, with the current stage highlighted;
• the run's progress bar.

Two buttons go with the card:

Detalhes also works for a load split across several workers: each line sums what the pieces have reported so far, and the number is a row count — never a percentage, because the engine only knows how many rows there are after reading the whole source. A box only shows as finished when every piece that went through it has finished; if one of them failed, it shows as failed, not finished. Here the numbers advance in steps of a few seconds — for finer monitoring, the place is the editor's step-by-step (see "Where to look when the detail is not enough").

Cancelling asks for confirmation first — it is not a one-way click. The cancellation is a request: the run stops at the next safe point, not at the instant of the click. The outcome shows up in history as a cancelled run — cancelled, never "failed".

The request reaches the load wherever it is. It does not matter whether the run is being carried by another instance of the platform, nor whether it was split across several workers: the request applies to all of them. Chunks that have not started are abandoned, and those in the middle stop at the next safe point.

Why it does not stop the instant you click. Writes to the target are done in batches. Cutting mid-batch would leave the target half written with nobody knowing where it stopped — and that is the only thing worse than waiting a few seconds. So the load finishes the batch in progress and only then ends. On a large load that interval can run from a few seconds to a few minutes: the card's stopwatch keeps ticking until the actual end, and it is the one that says when it is over.

What stays written when the load stops mid-way. Whatever was already written is not undone — cancelling is not rolling back. On a load into the graph that usually means nodes written without the links between them, because links are created after the nodes, and without the post-load step, which runs at the end. The run detail records how far it got. In practice: treat a cancelled load as incomplete, not as partial-but-consistent. Repeating the load is safe on targets that write by key (graph and search), because reprocessing the same row does not duplicate it; on a target that only appends rows, check what is there before triggering again.

When nothing is running, the section says so — "Nenhum pipeline em execução no momento" — instead of showing an empty list with no explanation.

Right below it, the same view already shows the history, so you do not have to switch views just to check the last run.

A run whose process died

A load can end with nobody left to report it: the process running it ceases to exist — the service restarted, the disk filled up, the machine went down. Before, that run stayed running forever, and the Em Execução card counted a load that no longer existed.

The platform now reconciles those runs: one with no live process gets a closing status with the reason, leaves the active list, and shows up in history like any other finished run. The disk space it was using for temporary files is released too.

Releasing the disk waits for the file to fall out of use — and that is deliberate. Deleting a file that someone still has open returns no space at all: the system only reclaims it when the last use ends. Deleting earlier would look like the cleanup worked while the disk stayed full, which is the worst of both worlds for whoever is diagnosing.

Agendados view — what will run

Lists the packages that have an active schedule, with:

The description and the time come from the scheduler, not from a second calculation. What the screen shows is computed by the same code that fires the load — if it says 04:30 here, 04:30 is what runs.

A repetition with no equivalent among the editor's shapes shows as "Agendamento personalizado" and keeps firing normally. To build or change the repetition, the place is the Calendário tab — this view is read-only, plus the one-off trigger. To pause the package without deleting the schedule, the place is the Motor tab.

With no scheduled package at all, the view explains that and points the way, instead of going blank.

Histórico view — what already ran

The table lists finished runs, newest first:

Above the table sit the search by package name, the status filter and the count of records left after filtering. Every header is clickable and sorts by that column; the table is paginated ten at a time.

Clicking any row opens the detail — no need to aim at the button.

On mobile the secondary columns leave the table and stay in the detail; Pipeline, Status, Duração and Ações remain visible, and the table scrolls horizontally when it has to. On mobile the Scheduler tabs themselves scroll horizontally, with a full touch target — none of them falls out of reach.

The run detail

The detail opens in a side panel, with the package name and the status at the top. It contains, in order:

Run metadata — identifier, package version, who triggered it, total duration, start and end.

Error — only present when the run failed. It carries the cause and the stage where it happened (connecting, extracting, transforming or persisting), which is what tells you where to look first.

Sources accessed — the origins the run actually read.

Warnings — the run's non-fatal warnings, each with its stage and code. A run can finish Completed and still not have done what you expected; this is where that shows up. What each warning means is in DATTA Extract.

Discarded records — how many rows fell out along the way in filters, deduplication or quality rules. It is a different number from "failed rows".

Per-node execution — how much each box of the flow processed: node, status, input rows, output rows and duration. A load split across several workers shows up here too: each line carries that box's total across the whole flow — the sum of what every piece processed — not the number of one isolated worker. The duration is that of the slowest piece of that box, not the sum of the pieces: summing several workers' time would make a twelve-minute load look like two hours.

When there is no per-node breakdown

Not every run records the per-node walk. That is the case for loads split across several workers that ran before this version: back then the coordinator did not keep the per-box split, and the data does not exist to be shown after the fact — there is no way to reconstruct it. Split runs from here on record the breakdown normally. When it is missing, the screen says exactly that:

Esta execução não registrou detalhamento por nó.

This is the most important point on the screen and it is worth being explicit: when the data does not exist, the screen shows the emptiness — never a plausible number in its place. This is the screen where you decide whether a load worked, and an invented value here becomes a wrong decision later. If you saw per-node numbers in this table before August 2026, distrust them: back then the table displayed a demo chain of nodes for every run, with fixed names and counts. None of it was measurement.

Coming from a package

In the Packages gallery, a package's Ver histórico action opens this tab already filtered by it: a chip with the package name appears next to the history title, and both cards and table start counting that package only. Clicking the chip's × brings back the view of all packages.

The package focus follows a tab change: leaving for the Calendário and coming back keeps the same package in focus, and the screen address carries both the tab and the package — you can copy the link and send it to someone else.

Refresh, connection and errors

The tab refreshes itself on a cycle, and the AO VIVO indicator in the header shows the cycle is up. The Atualizar button forces an immediate re-read.

When the screen is hidden — another browser tab, another area of the platform — the cycle stops. It resumes as soon as the screen comes back. There is no point in polling the server every ten seconds for a screen nobody is looking at. For the same reason, one Scheduler tab's cycle does not run while you are on another: only the visible tab polls.

If the read fails:

• with the screen still empty, the message "Não foi possível carregar o monitor" appears, with the reason and a Tentar novamente button;
• with data already on screen, the list stays visible and an amber banner warns that the data may be stale, with a button to refresh. The screen does not pretend to be alive when it is not.

Where to look when the detail is not enough

Since this tab has no log viewer, the path is:

1. The error in the detail — cause and stage. Solves most cases.
2. The run's warnings — a load that "succeeded" yet wrote zero is usually explained here.
3. The Motor tab — when the suspicion is that the load never fired at all: it shows the scheduler state, the last dispatch and the missed windows per package.
4. The editor's step-by-step — opening the package in DATTA Extract during (or shortly after) the run shows the node-by-node monitoring, with far more detail than the summary table. It is kept for a few hours after the end.
5. *Background executions* — the cross-cutting view, alongside screenings and uploads.

Permissions

Whoever can trigger can stop: both actions require the same pair of permissions. There is no case of someone starting a load they then cannot interrupt.

Without the read permission the platform answers with a denial explained in Portuguese — not with an empty list that reads like "there were no runs". See RBAC.

The screen can also be granted or blocked per workspace, along with the other Extract screens. Since the three former screens became one, the per-workspace grant became a single one, for the whole Scheduler: whoever already had the monitor or the schedule granted keeps access, with nobody having to redo the workspace configuration. See Workspaces.

FAQ

Where did the "Job Monitor" screen go? It became the Job Monitor tab of the Scheduler. The three inner views (Monitor, Agendados and Histórico) and the run detail are the same, and the old address opens the Scheduler already on it.

I clicked Cancelar and the run kept going for a few more seconds. That is expected. Cancellation is a request: the load stops at the next safe point — the end of the batch in progress — so the target is not left half written. On a large load that interval can exceed a few seconds. If it does not stop, the run detail keeps showing the real status — the screen does not change the status on its own.

I cancelled and the target still has data written. Is that normal? Yes. Cancelling interrupts, it does not undo. What was already written stays; what was not, never will be. On a load into the graph the typical outcome is nodes without the links between them, because links and the post-load step come after the nodes are written. Treat a cancelled load as incomplete and repeat it when you want the whole result.

I cancelled a load split across several workers. Does it stop too? It does. The request applies to the whole split: chunks that have not started are abandoned and those in the middle stop at the next safe point. The run ends as cancelled, not "failed" — cancelling is not an incident to investigate.

A run shows as "running" for hours and I know the service restarted. That is the orphan run: the process carrying it ceased to exist and had no way to write the outcome. The platform reconciles those cases — the run gets a closing status with the reason, leaves the active list, and goes to history. If it still shows as active, use Atualizar: reconciliation is not instantaneous.

The "Execução por Nó" table is empty. That run did not record a per-node breakdown. Today this happens with loads split across several workers that ran before this version: the data was not written back then and cannot be recovered. Split loads from here on carry the per-box split normally. The run's summary numbers (duration, warnings, discarded) are still valid either way.

Where do I see the load's logs? There is no log viewer on this screen. Use the sequence in "Where to look when the detail is not enough", just above.

The "Com Falha" card shows one number and the table shows another. They count different scopes: the card counts what the sentence below it announces; the table counts what survived your filter and your search. Clear the filter before comparing.

I want to change a load's time. This tab does not edit schedules. The editor lives on the Calendário tab and also on the flow editor's toolbar.

The load never even showed up as "running" at the scheduled time. Then the question belongs to the scheduler, not to the monitor: the Motor tab shows whether the package is paused, whether the schedule yields a next run and whether the window was missed while the platform was down.

An old run shows odd data. Runs recorded by earlier versions of the engine may not carry every field — what is missing appears empty, never filled in by estimation.