Repository navigation
feat(dgw): run AI analysis of a recording as a provisioner task - #2044
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 50 commits into
Conversation
Let maintainers know that an action is required on their side
|
c81d3f1 to
5b16e3a
Compare
5b16e3a to
faf9419
Compare
faf9419 to
55fef54
Compare
55fef54 to
5ff453b
Compare
5ff453b to
f2948c9
Compare
f2948c9 to
633c904
Compare
633c904 to
6083262
Compare
6083262 to
ed522eb
Compare
The caller of `POST /jet/jrec/{session_id}/ai-analysis` now picks the
language in an optional `language` field: any text of 1 to 10
characters, such as `fr-CA` or `français`; anything else is a 400.
Without it, the analysis is in English. The language is part of the
Task parameters, so the same Task ID with another language is a
`task_id_conflict`.
Gateway no longer reads anything the producer pushed: there is no
events log to take a `locale` from.
- The tests count the summary request the AI crate now sends.
- "AI analysis queued" is a debug line, since the runner already logs
the queued Task.
- The jrec and provisioner tasks intents state the language, how many
analyses a session keeps, and how retries and restarts behave.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The working files of an analysis now always name their language, so a fixture without one is not a valid workspace. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Clearing on the first attempt only missed two paths. A Task that failed before its first attempt cleared, such as on a lost key, kept the earlier analysis. A first attempt cancelled while waiting for the session retried as attempt 2 and never cleared, so the session ended with two analyses. Each attempt now takes the session lock, then clears, before anything else can fail. The Task params keep the language as a `Language`, so it's checked once, when the request comes in. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…document `params` of a `recording-ai-analysis` Task now has the `AiAnalysisParams` schema, like its payloads. The lint attributes on these schema structs are gone, since the lint never fires on them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tests move from the crate's own `tests/` to the testsuite, as AGENTS.md asks, and talk to a running Gateway and a mock AI provider instead of linking Gateway internals. They read the key leak checks from the log files and `gateway.db` in the config folder. The check that a Task of another kind is not found at this endpoint is gone: the binary only knows one kind. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…st does The full path made the OpenAPI document reference a schema that does not exist. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…a failed removal A session without recording has no AI analysis to remove, so clearing it succeeds and the attempt then reports "session has no recording". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Windows can't delete the database and log files a running Gateway holds open, so each test left its folder behind. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…er accepts The server reads a `u32`, which an `int32` schema can't hold, so the clients now get a 64-bit integer from 0 to 4294967295. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The AI access of a Task now lives in the standalone `SecretStore`, keyed by the Task ID, instead of the provisioning store, which is about sessions only. Gateway holds one store in `DgwState`, hands it to the provisioner tasks, and runs its expiry cleanup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Task secrets live in the standalone `SecretStore`, not in the provisioning store, which is about sessions only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Record the owner's rulings from the intent review of the AI analysis Task: a new Task replaces the earlier analysis only once its success is recorded, a failed Task leaves no file behind, attempt errors are recoverable or permanent, token usage is always reported, a running Task fails after 30 minutes without progress, and the base URL is a secret like the API key. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Task keeps its API key until its deadline, and that deadline now moves while the Task makes progress, so the key must follow it. An expired or removed secret stays gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Errors: an attempt fails with a closed kind and no text, recoverable
or permanent. A failed Task's payload is `{ error, usage }`, so the
caller words the error.
Usage: every payload but `queued` carries the tokens used so far, over
all attempts. Requests whose usage the provider did not report are
counted apart in `unreportedRequests`.
Time: a Task fails if it does not start within 2 hours. Once it runs,
each attempt start and each progress report moves its deadline 30
minutes ahead, and the runner cancels an attempt past it. Waiting for
a remux is not progress. The store fails an overdue Task with
`deadlinePassed` and keeps the usage of its last payload.
Success: the new log sits in the session unlisted until the success is
recorded. Ending the Task then lists it and drops the earlier analysis.
If Gateway stops in between, the job runs again at startup and redoes
that step. A failed Task removes everything it made and leaves the
earlier analysis in place.
The session manifest is written to a file of its own, then renamed
over the old one, so a crash never leaves half of it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The end of a Task, such as listing its AI analysis log, ran from the Task's own job, whose runs are shared with the attempts. Once those runs were used up, a failed end was never done again, not even at the next start. A failed end now hands the Task to a new job that ends it again five minutes later. The job is stored, so a restart runs it too. Deadlines are also cut to whole seconds, as the store keeps them, so the runner and the store agree on when one passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request was counted only once its answer came. One cut off by the deadline or a restart was never counted, so the usage looked complete when it was not. Each request now counts as unreported from the moment it is sent, and its reported tokens replace that count on answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Task whose end failed could list its log later, over the analysis of a newer Task of the session. A new Task now removes the logs earlier Tasks left unlisted, so such a late end finds nothing to list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The manifest is written to a temporary file first. Its name now starts with a dot, which jrec never serves, so a file a crash leaves behind is never downloaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenAPI document writes the error kinds out by hand, since utoipa drops the descriptions of variants without fields. An exhaustive match now stops the build until a new kind is documented too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new Task removed every log earlier Tasks left unlisted. When the end of a succeeded Task was still due, the new Task dropped an analysis whose success was recorded, and a new Task that then failed left the session without it. A new Task now does what the end of the earlier Task would: it lists the log of a succeeded Task, and removes the log of a failed or unknown one. The runner gives an attempt the state of another record for this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the success of a Task could not be recorded, any error that came after the failure was recorded was only logged. An end that could not be queued again was then lost. That error now reaches the job queue, whose retry finds the Task finished and only ends it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AI analysis of a recording, started by DVLS with a plain HTTP request (no new token type), tracked as a provisioner Task and run as a job on the existing job queue.
Provisioner tasks run through a
ProvisionerTaskRunner, the one place that keeps a task's record (#2041'sProvisionerTaskRecord) and its job in step. Each kind of work is aProvisionerTask(what one attempt does, what to drop once its record ends); the runner creates the record, queues its job, runs each attempt with its timeout, retries transient failures and records how the Task ends. The Task is the source of truth and the job only wakes it up: each Task names the one job allowed to run it (#2041), so a stale or duplicate job does nothing. A Task is reported created only once its job is stored in the job queue. At startup, before the job runner claims anything, an unfinished Task whose job is no longer stored (such as after a crash between recording the Task and storing its job) gets a new one; a Task whose job is still stored keeps it. The analysis itself is the AI crate'sanalyze_recording(#2043); Gateway hands it the manifest of the finished recording and the language of the request, and adds the log it returns to the session as its own file.API (stable, no
enable_unstable)POST /jet/jrec/{session_id}/ai-analysis, scopegateway.tasks.recording-ai-analysis.start. Body:taskId,provider,model,baseUrl(needed for openai-compatible),maxOutputTokens,language,apiKey.task_id_conflict/recording_active(a clip is still pushed) /analysis_in_progress(+ the runningtaskId)GET /jet/jrec/{session_id}/ai-analysis/{task_id}, scopegateway.tasks.recording-ai-analysis.read: the Task (state, payload, attempts, times), or 404. The 202 answer of thePOSTnames it in itsLocationheader. A Task of another kind, or the analysis of another session, is not found under that path.Each provisioner task declares its own scopes,
gateway.tasks.<kind>.startandgateway.tasks.<kind>.read, so a token for one kind of Task can neither start nor read another kind. The kind of AI analysis isrecording-ai-analysis.The two new scopes in
Devolutions.Gateway.Utilsand the regenerated OpenAPI clients are in #2054, stacked on this PR.The first commit hands the provisioner task store (#2041) to
DgwStateon its owngateway.dbconnection;DgwState::mockbecomes async for that, hence the.awaitin a few tests. It also passesjob_queue::MAX_ATTEMPTS(5, same as the current default) to the job queue explicitly, so the retry check in the AI analysis job can't drift from it.How it behaves
SecretStore(feat(dgw): add an encrypted in-memory secret store #2058); never in the job,gateway.dbor our logs. After a Gateway restart the Task fails with "the API key is no longer available"..cast,.trp) and video (.webm) recordings are analyzed (feat(dgw): analyze any recording in the AI crate #2043); video needs XMFv2026.9.25.0or later.languageis any text of 1 to 10 characters, such asfr-CAorfrançais; anything else is a 400. Without it the analysis is in English. It's part of the Task parameters, so the sametaskIdwith another language is atask_id_conflict.ai-analysis-0.slogunderartifacts.ai-analysis, with the summary on itssession.end(feat(ai): write the AI analysis in the requested language, with a summary #2118). Gateway reads nothing the producer pushed.paramskeep the AI settings without the key and the base URL, plus the language; the OpenAPI document describes them asAiAnalysisParams.Where to look hardest
provisioner_tasks/runner/mod.rs:run_attempt(cancel on timeout, retry vs. fail,on_endon every end) andresume(called fromservice.rsbefore the job runner starts)provisioner_tasks/recording_ai_analysis/mod.rs: the AIProvisionerTask, andlock_session: the working files belong to the session, so analyses of one session run one at a time (a Task past its deadline may still have an attempt running when the next Task starts)start()in the same file: the order of the checksKnown, accepted
All, reqwest/hyper's own TRACE lines show the provider's address (not the key). Our code never logs it.ai-analysis-0.slog.Tests
cargo test -p devolutions-gateway --lib: 249 passed.cargo test -p testsuite --test integration_tests ai_analysis: 10 passed. These endpoint tests drive the Gateway binary with a mock AI provider, and check that the key and the base URL reach neither the logs norgateway.db. The testsuite gains atimedev-dependency to check the RFC 3339 times.--features openapi --tests) and nightly rustfmt are clean.🤖 Generated with Claude Code