Repository navigation
feat(qualify): declarative qualify from verify.yaml (carry-over of openadapt-flow#532) - #1186
Merged
Merged
Conversation
…to checked steps Moves the four bundle fix-ups from scripts/openemr_qualification.py into the compiler. Each change is evidence-gated and recorded in compile_notes.json. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…aml (WIP) Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…_flow.qualify (WIP) Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…er qualify; tests for passes and verify.yaml Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…ify a workflow page Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
__main__ defined _cmd_seal twice. The second definition (bundle seal) won, so `openadapt-flow attach-seal <run_dir> --execution-id ...` raised AttributeError: 'Namespace' object has no attribute 'source'. Rename the attach handler to _cmd_attach_seal and test that the subcommand reaches it. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…lification-result/v1 and bind it through the admission envelope - verify.yaml takes schema_version openadapt.verify/v1 (RFC-0002): verifier (deployment effects config), effects (effect contract list) or the record shorthand that expands to it, minimum_effect_strength, optional qualification_id/environment_id; qualify adds app, save_step, inputs and campaign. The pre-RFC shape fails with a migration hint. Both configs migrated. - qualify writes qualification/result.json (qualification-result/v1): QualificationReport without workflow_name and refusal messages, plus verify_config_sha256, verifier_identity, fault_driver, per-trial cases with read-back verdicts, silent_incorrect_success_count, over_halt_count and model_calls. Certification refuses unless every case verdict matches at independent_readback with zero silent incorrect successes. - --certify binds the result digest into qualification-admission/v1 (campaign.outcomes_sha256, oracle_contract_sha256, oracle_id) signed with the local-dev qualification key, which production trust maps refuse. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…y signs Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…wer than 3 trials per case Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
… is independent_readback - result.json reports executor_model_calls and verifier_model_calls instead of one model_calls (per case: executor_model_calls). - verification_tier is independent_readback only for a Flow tier-1 read from a non-UI substrate (REST, SQL, FHIR, GraphQL, file export). A separate UI session (ui_readback, Flow #492) or any UI substrate is screen_readback. - A case passes only at independent_readback, and the normal case must reach outcome verified, so certification (clinical-write included) refuses a normal case read back through the UI. - Tests for each read kind, the ui_readback refusal, and that a record shorthand and its expanded effects list hash to the same contract digest. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…config_sha256 covers Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…lted; refusal paths carry only ids - RFC-0002 decision 4: a fault case's passed now needs the read-back to show nothing landed (REFUTED, halted_before_effect) at independent_readback AND engine_outcome HALTED. 1b4ae1e1 did not check the engine outcome or that nothing landed. An engine success that the store does not confirm now counts as a silent incorrect success. - Refusal path/step_id/case_id in result.json pass through a closed id grammar. Every path Flow builds today is dot-separated ids (audited in qualification.py); anything else is cut back to its prefix plus 'redacted', so an input value or free text cannot leave in a path. - QUALIFY_A_WORKFLOW.md: the interim report.json/read-back digests are unsalted; keep result.json from real-record campaigns inside the customer's systems until effect receipts replace them. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
… web observer in the MockMed CLI test - faults.py: narrow the screen-fault digest before assigning it. - public-artifacts.json: register benchmark/openemr_local/verify.yaml and public-demo/mockmed-verify.yaml. - test_desktop_check_it_cli_mockmed: the export script no longer has _environment_payload or _MockMedEnvironmentObserver; use openadapt_flow.qualify.environment instead. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…can verify it The static_control_identifier_crop pass gave every unlabeled web button a pixel crop as its only identity evidence and armed identity. The pixel tier's VERIFY path is off by default, so that check could only abstain or reject, and an irreversible step halted on every run (test_attached_browser_run). runtime/identity.py now says which identity evidence a runtime can turn into 'verified' on a surface (verifiable_identity_evidence). The pass asks it per step, through an optional surface_kind_for hook for multi-surface bundles, and adds a crop only when the target runtime has pixel_verify_enabled (compile --pixel-verify-enabled). The runtime is unchanged: a crop that mismatches still halts, and a crop it can't verify still never passes. Tests: a step with no surrounding text and a valid crop verifies where crops are verifiable; a crop that doesn't match halts with VERIFY on or off; a crop the runtime can't verify still halts; the pass skips the crop by default on every surface. Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
abrichr
added a commit
that referenced
this pull request
Oct 17, 2026
abrichr
added a commit
that referenced
this pull request
Oct 17, 2026
abrichr
added a commit
that referenced
this pull request
Oct 17, 2026
This was referenced Oct 17, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carry-over of OpenAdaptAI/openadapt-flow#532 (
feat/declarative-qualify, headf060c6c5) intopackages/flow.Replayed with
git format-patch --no-merges f21281fa..f060c6c5andgit am -3 --directory=packages/flowontomain(f2c4d2e7). 15 commits, applied cleanly; the one merge commit on the source branch (ac4db9d2, origin/main into the branch) is skipped because its content is already onmain.Includes the identity-crop fix (
f060c6c5, attach identity evidence only where the step's surface can verify it) and thequalify/faults.pymypy narrowing (55b075ec).Local checks on the folded tree:
ruff check openadapt_flow,ruff format --check openadapt_flow tests, andmypyare clean. Full suite runs in the Packages workflow.