Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m

## Unreleased

- fix(exec): **a write no longer leaves units that changed only in XPath whitespace for the reviewer to read.** mxcli v0.27.0's `alter entity … rename attribute` rewrote 135 units in 20 unrelated modules that way (mendixlabs/mxcli#1596; fixed upstream in v0.29.0, ako/mxcli#1506, so this is insurance for projects on v0.27–v0.28). `exec.sh` now runs `model-scope.py restore` after the write (MDL and `--patch`) and before the mxbuild gate, so the gate checks the final model; it prints the real changes per module and puts a `scope:` note in the BUILD-LOG row. It never blocks: a failure is a warning, `MXTK_NO_SCOPE=1` skips it, and it is skipped when `FORCE_EXEC` let the write run over uncommitted model changes. No pre-commit refusal (maintainer decision on #593): `model-scope.py check --staged` stays available to run by hand. Fixture `tests/wave2/test-model-scope.sh` section F. Ported from #593 onto current master — saingge
- learn(demo): **`skills/live-demo-autopilot.md` — run the demo live, not as a video**: Playwright drives the real app through scripted scenes, captions carry a WHY line, a control panel lets the presenter pause and take over, redo or skip; plus the traps that each broke a run (trial licence stops the runtime 2–4 h after start, 6-session cap and proving a logout freed its slot by the `System.Session` count, demo sign-in tiles kept out of any deployed build, focus-stealing panel pausing the run, SIGTERM closing the browser before sign-out). Field: a desktop+phone offline manufacturing demo (4/4 scenes) and a three-app process-hub demo (28/28 with an app killed mid-run and an 11 min pause), 2026-10 — Maurits Visser
- docs(bug-ledger): **32 ledger entries now say what mxcli v0.22–v0.29 changed about them, before anyone re-runs an old workaround**: 22 are fixed upstream per the mxcli changelog or source (among them BUG-121 terminators, BUG-141 workflow PersistentId, BUG-151 `rest call body $Var`, BUG-100 shared Docker stack, BUG-94 role visibility, BUG-128 module-role idempotence) and 10 changed shape or are partly fixed (BUG-154; BUG-04 no automatic module role since v0.27, BUG-WF04/WF05 named call arguments since v0.25, BUG-130/131 vs the v0.27 `exec` rollback). Each gets an "Upstream status" line naming the version, the issue and what to retest; none is closed or archived, because none was retested — the ledger archives only after a retest. Also corrects `BUG-DRAFT-audit-member-access-ce0066`: commit `2455ee9f` first shipped in v0.23.0, not v0.22. Source: mxcli v0.29 compatibility audit 2026-10-30
- fix(doctor, exec, page-scope, rehost, wf-add-path-terminators, module-close): **mxcli v0.29 compatibility, and doctor advises the latest mxcli**: on v0.29.0 (which `--install` fetches by default) four instruments read new output wrong. (1) `fix property-sets` prints `MIXED` rows and exits 1: doctor reported "could not classify" (rc 4, shown as info by sync) — now result `mixed`, a WARN naming the rows and saying the repair cannot clear them; (2) the repair also exits 1 while a MIXED name remains, so `--fix-property-sets` rolled back a repair that had landed — the apply step now passes when only MIXED is left (FAILED still fails); (3) `fix hashes` now verifies MPR v1 with new text, which exec.sh read as an error, so every `--patch` on a Mendix < 10.18 project went unverified — both v1 forms are read; (4) since v0.28 `SHOW MODULES` / `list modules` write `Protected AddOn|Solution` in Source, so a first-party module set to Protected dropped out of page-scope (FAULT, 0 modules) and out of the rehost module list — the export level no longer counts as an origin. `wf-add-path-terminators.py` refuses below Mendix 10.21 (the terminator type does not exist there; written in, the project will not load), module-close notes that v0.29 `docker check` checks the stored project (a CE0463 it now reports is real), and the stale "stay on v0.24" pin advice is replaced. **New:** the normal doctor run compares the project mxcli with the latest release (redirect of the asset URL, 5 s, nothing downloaded): WARN with release link and upgrade path when behind, a note when `MXCLI_VERSION` pins it, one quiet note offline, never a FAIL; `MXTK_NO_LATEST_CHECK=1` skips it. Golden input captured from the real v0.28.0/v0.29.0 binaries on the mxpatch fixture model and mutated copies (CAPTURE files); against master the new cases fail (exec-hash-verify 4, property-sets 7, mxpatch 1, page-scope-protected 3, rehost-list-modules 1, doctor-mxcli-latest 8). Field run: page-scope and doctor against real v0.29.0/v0.28.0 binaries in a cloud session — mxcli v0.29 audit (2026-10-30)
Expand Down Expand Up @@ -99,7 +100,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
- new(canvas): **`project-bin/mf-canvas-facts.py` prints how a microflow or nanoflow lies on the Studio Pro canvas: labels on top of each other, lines through objects, note lines leaving the side that faces away, flows that run right-to-left or curl, captions wider than their shape, and decision captions that ask the opposite of their true branch (SENSE).** It prints facts, one line each, and no verdicts; read-only (it reads units through `mxpatch.py`), exit 2 when a flow could not be read. `verify-module.sh` runs it per module as an info rung (1c, log `14-canvas-facts.log`; a missing script, Python or `.mpr`, an exit 2 or a timeout is an INFO row reading NOT MEASURED, never a gate). Fixture `tests/wave2/test-mf-canvas-facts.sh` runs on a model mxcli v0.27.0 wrote (units and `.mpr` captured); `test-verify-module-446.sh` gains V8 for the rung. Calibration is from Studio Pro 10.24 screenshots, not verified on Studio Pro 11. Field evidence, measured 2026-10-21 on one Mendix 10.24 app: 1,588 flows read in one pass; on the 36 flows an mxcli-built branch added or changed in one module, OVERLAP 19, LINE 27, NOTE-SIDE 17, LEFTWARD 3, TURN 4, SENSE 9 before repair — saingge
- learn(preflight, teamserver-alignment): **a flow's return-type change and its callers' update go in one script (STOP row 33), and Studio Pro 10.24.18's "Merge feature branch" merges `origin/<branch>`, not the local branch.** On a copy of a Mendix 10.24.18 app with mxcli v0.27.0, the first script alone left `mx check` with CE0112 at a caller stored the way Studio Pro stores a call, while `check` and `exec` stayed silent; Studio Pro's own git log shows the merge of the remote branch. Text only; baseline-routed, +101 words. — saingge
- fix(doctor, preflight, platform-link): **a clean `fix property-sets --dry-run` on Mendix below 10.24.24 is no longer reported as "every unit has the property names Studio Pro writes".** mxcli's property table starts at 10.24.24, so on 10.24.18 (mxcli v0.27.0) the dry-run printed "Every unit already has the property names Studio Pro writes" while Mendix's `mx diff` exited 129 on a unit mxcli had written (mendixlabs/mxcli#1595). `doctor.sh --property-sets`, `--if-stale` and `--quick` now WARN "NOT PROOF" and name `mx diff <base>.mpr <mine>.mpr out.json` (exit 0) as the proof; the receipt records `unproven`; exit stays 0. `learned-detection-gaps.md` and `platform-link.md` said v0.26.0 closed the gap. Fixture P2b (11 cases). Baseline-routed, +18 words. — saingge
- new(model-scope): **a write's whitespace-only units can now be found and put back.** mxcli v0.27.0's `alter entity … rename attribute` rewrote 135 units in 20 modules of a Mendix 10.24.18 app that never referenced the attribute; the only change in each was the line breaks between bracketed XPath conditions, and `mxcli diff-local` showed nothing (mendixlabs/mxcli#1596). New `project-bin/model-scope.py`: `report` sorts each changed unit into real or noise (every differing field is an XPath constraint that is equal once whitespace outside string literals is removed) and prints the real changes per module; `restore` checks the noise units out of `HEAD` and updates their ContentsHash rows through `mxpatch.py`; `check --staged` exits 1 when noise is staged (`MODEL_SCOPE_OK=1` lets one commit through). Not wired into `exec.sh` or the pre-commit hook yet. Fixture `tests/wave2/test-model-scope.sh`, with a golden before/after capture of a real v0.27.0 rename on a public model (`fixtures/model-scope/golden-v0.27.0/`) — saingge
- new(model-scope): **a write's whitespace-only units can now be found and put back.** mxcli v0.27.0's `alter entity … rename attribute` rewrote 135 units in 20 modules of a Mendix 10.24.18 app that never referenced the attribute; the only change in each was the line breaks between bracketed XPath conditions, and `mxcli diff-local` showed nothing (mendixlabs/mxcli#1596). New `project-bin/model-scope.py`: `report` sorts each changed unit into real or noise (every differing field is an XPath constraint that is equal once whitespace outside string literals is removed) and prints the real changes per module; `restore` checks the noise units out of `HEAD` and updates their ContentsHash rows through `mxpatch.py`; `check --staged` exits 1 when noise is staged (`MODEL_SCOPE_OK=1` lets one commit through). `exec.sh` calls it after each write (see the `fix(exec)` entry above). Fixture `tests/wave2/test-model-scope.sh`, with a golden before/after capture of a real v0.27.0 rename on a public model (`fixtures/model-scope/golden-v0.27.0/`) — saingge
- fix(graph-sweep, lint): **the orphan list now says a flow used only by a pluggable widget's datasource or action property may have no catalog edge, and prints the one `mxcli search` that finds the page (it needs a `REFRESH CATALOG FULL SOURCE` catalog; this sweep reads a FULL one, so the note says to rebuild).** The caveat named only layout snippet calls, so a live flow read as "strong evidence of deadness" (`graph-sweep.sh` comment, `review-report.js` orphan detail, and the `lint-that-actually-runs.md` QUAL004 row, which only said the catalog "likely misses" the ref). Observed on a Mendix 10.24 app with mxcli v0.27.0: 5 live flows used only as a Gallery's or a third-party widget's microflow datasource had 0 `refs` rows, `show references to` said "no references found", `describe page` printed each, and QUAL004 flagged each. A Gallery with a microflow datasource and no microflow `onClick` had its ref (15 of 15); with both, only the `onClick` ref (2 of 2); the two Data grid 2 microflow datasources had theirs. The QUAL004 row now states that and links [mendixlabs/mxcli#1513](https://github.com/mendixlabs/mxcli/issues/1513) (open; reported for Data grid 2 on v0.26.0); the ledger entry and the pending-issue draft now say they are filed. Pinned by `test-catalog-edge-guard.sh` case 1 (fails on 393d253). — saingge
- fix(bug-lookup): **`bin/bug-lookup.sh` also searches the project's own `bug-logs/**.md` drafts, marked `(project draft: <file>:<line>)`, after the ledger's output.** It searched only the toolkit ledger, so a bug a session had already written down in the project was invisible to the lookup run before the next experiment on the same error; the ledger's output and ranking are unchanged (drafts are ranked in a separate pass), `--full` and `--index` stay ledger-only, and `MXTK_PROJECT_DIR=` turns drafts off. Field: a Mendix 10.24 app with mxcli v0.27.0, 43 files in `bug-logs/`; the query "cannot be spliced" listed 0 project drafts at 393d253 and lists 3 (3 distinct files) now, each pointing at a `## ` line; the project was not written. Pinned by group P of `tests/wave2/test-bug-lookup.sh` (3 of its checks fail on 393d253) on synthetic drafts; the run under /bin/bash 3.2.57 passes, other awks not run. — saingge
- fix(harvest-learnings): **section 1 drafted only the first `## ` section of each bug-log file.** The awk splitter's `print buf "\x01"` put a newline after each delimiter, so every later section began with an empty line, read an empty heading and was skipped. It is `printf "%s\x01", buf` now. Field run on a Mendix 10.24 app with mxcli v0.27.0 (43 bug-log files, 54 `## ` sections): `harvest-learnings.sh` at 393d253 drafted 43, the fix drafts 54. Pinned by `tests/wave2/test-harvest-learnings.sh` T5, on a two-file fixture that keeps a captured log's structure with its text replaced (fixed: 7 of 7 sections, 20 of 20 checks; 393d253: 2 of 7, T5 fails). The splitter's delimiter bytes were checked under macOS awk 20200816 only. — saingge
Expand Down
35 changes: 34 additions & 1 deletion project-bin/exec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# exec.sh — the guard chain around a model write.
#
# STOP-row scan → concurrent-writer guard → module-brief advisory → mxcli check → snapshot → baseline → exec
# → model-scope restore (units changed only in whitespace go back to HEAD; model-scope.py)
# → mxbuild delta gate → auto-restore only on a NEW error → lint ratchet → SP reopen
#
# The mxbuild gate is a DELTA gate, not an absolute "0 errors" one: a write is kept
Expand Down Expand Up @@ -585,7 +586,7 @@ HDR2
# The gate cell is read from GATE_STATE rather than passed in, so no caller
# can omit it — a blank cell in this table reads as "fine", which is the exact
# false-green the gate exists to prevent.
_detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${ROWS_NOTE:+ · $ROWS_NOTE}${RESTATE_NOTE:+ · $RESTATE_NOTE}${GATE_NOTE:+ · $GATE_NOTE}${MXB_LOG_NOTE:+ · $MXB_LOG_NOTE}${HV_NOTE:+ · $HV_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}"
_detail="$2${CHANGE_CLASS:+ · change: $CHANGE_CLASS}${CR_ID:+ · cr: $CR_ID}${SCAN_NOTE:+ · $SCAN_NOTE}${ROWS_NOTE:+ · $ROWS_NOTE}${RESTATE_NOTE:+ · $RESTATE_NOTE}${GATE_NOTE:+ · $GATE_NOTE}${MXB_LOG_NOTE:+ · $MXB_LOG_NOTE}${HV_NOTE:+ · $HV_NOTE}${SCOPE_NOTE:+ · $SCOPE_NOTE}$_EXEC_APPROVAL_SUFFIX${STEP_TIMES:+ · steps: $STEP_TIMES}"
if [ -n "${EPOCHSECONDS:-}" ]; then printf -v _when '%(%Y-%m-%dT%H:%M:%S%z)T' -1; else _when="$(date '+%Y-%m-%dT%H:%M:%S%z')"; fi
printf '| %s | `%s` | %s | %s | %s |\n' \
"$_when" "$LOG_NAME" \
Expand Down Expand Up @@ -641,6 +642,7 @@ take_score() {
# check and the mxbuild gate still stand behind it.
SCAN_NOTE=""; HV_NOTE="" # HV_NOTE: the ContentsHash bracket's cell text, set below
RESTATE_NOTE="" # the restatement warnings' cell text (#269, #261), set after the check
SCOPE_NOTE="" # the model-scope restore's cell text, set after the write
if [ "$PATCH_MODE" != "1" ]; then
_SCAN=""
for _s in "${MXTK_ROOT:-}/bin/mdl-scan.sh" "$_TKR/bin/mdl-scan.sh"; do
Expand Down Expand Up @@ -2040,6 +2042,37 @@ if [ "$EXEC_STATUS" -ne 0 ]; then
echo " Running the gate anyway."
fi

# ── Model scope: put back units that changed only in whitespace ──────────────
# mxcli v0.27.0's `alter entity ... rename attribute` rewrote 135 units in 20 modules that never
# referenced the attribute, removing the line breaks in their XPath constraints (mxcli#1596);
# `mxcli diff-local` showed nothing. model-scope.py restore puts such units back from HEAD and
# updates their ContentsHash rows. It runs after the write (MDL and --patch alike) and before
# the mxbuild gate, so the gate checks the final model. The model was committed when this exec
# started (the refusal above), so the diff against HEAD is this write's change; when FORCE_EXEC let
# the write run over uncommitted model changes it is not, and the step is skipped. Never blocks: a
# failure is a warning, and model-scope.py says what it left (a part-way restore keeps the index
# matching). MXTK_NO_SCOPE=1 skips it.
_ms="$PROJECT_ROOT/bin/model-scope.py"
if [ "${MXTK_NO_SCOPE:-0}" != "1" ] && [ -z "$MPR_DIRTY" ] && [ -f "$_ms" ] && [ -n "${PY:-}" ] \
&& [ -d "$MODEL_DIR/mprcontents" ]; then
echo ""
echo "→ Model scope: units this write changed only in whitespace go back to HEAD..."
_ms_rc=0
_ms_out="$("$PY" -I "$(native_path "$_ms")" restore --mpr "$(native_path "$MPR")" 2>&1)" || _ms_rc=$?
printf '%s\n' "$_ms_out" | sed 's/^/ /'
if [ "$_ms_rc" -ne 0 ]; then
echo " ⚠ model-scope exited $_ms_rc (its message is above); the write is kept, with any noise not"
echo " restored still in it. To see what is left, then retry:"
echo " python3 -I bin/model-scope.py report -v" # portability-ok: command text shown to the user, not run here
echo " python3 -I bin/model-scope.py restore" # portability-ok: command text shown to the user, not run here
SCOPE_NOTE="scope: incomplete (exit $_ms_rc)"
else
# First line: "model scope vs HEAD: N unit file(s) changed, R real, K noise".
SCOPE_NOTE="$(printf '%s\n' "$_ms_out" | sed -n '1s/^model scope vs [^:]*: /scope: /p')"
case "$_ms_out" in *"restored "*) SCOPE_NOTE="$SCOPE_NOTE restored" ;; esac
fi
fi

step_done exec
echo ""
echo "→ Running mxbuild model check (catches BSON corruption before SP opens)..."
Expand Down
3 changes: 2 additions & 1 deletion project-bin/model-scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@
every unit.
restore report, then put each noise unit back from REF (`git checkout REF -- <files>`) and
set its ContentsHash row in the .mpr from the restored file (mxpatch.py's rehash), so
the index Studio Pro reads stays in step. Run it by hand; nothing calls it yet.
the index Studio Pro reads stays in step. bin/exec.sh runs this after each write to a
committed model.
check --staged: the staged index against HEAD, for the pre-commit hook. Exit 1 when a
noise unit is staged, printing the restore command. MODEL_SCOPE_OK=1 lets that one
commit through with a warning.
Expand Down
Loading
Loading