Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m
- fix(page-audit, design-audit): **the audits produce their own `.claude/loop/page-scope.json`**: when it is missing or older than the model they run `bin/page-scope.sh` (`scope-ensure.js`, 2 s) instead of FAULTing on 0 pages; a failed regeneration never blanks an existing file and names the failure. Field: scope deleted -> page-audit ran on 9 pages, `touch` of the .mpr -> regenerated once — speed experiment E2
- new(look-ledger, journey-runner, module-review): **a journey run's screenshots are LOOK evidence**: `bin/look-ledger.sh shots` (run by journey-runner after every run) maps each reached step's PNG to the one page whose widgets (`DESCRIBE PAGE`) it landed on, a URL pattern breaking ties, and lists every owed page as `OPEN` (open that PNG; the Read hook then marks the page SEEN) or `UNREACHED` (short gap-fill journey, or `page-audit.js` as before) with the line `owed N = A OPEN + B UNREACHED`; a shot older than the page's last build is not evidence and nothing is claimed when unsure. Field: 8 owed pages = 5 OPEN + 3 UNREACHED from two journey runs, no separate screenshot pass; baseline words 64,004 -> 64,193 — speed experiment E2
- fix(affected-tests): **a page, flow or navigation change no longer selects 0 journeys because the journeys name widgets, not `Module.Name`**: the affected scope now reads the widget names of every changed page/snippet and of each calling page (`DESCRIBE PAGE`, one extra warm-catalog `mxcli` call, 0.07 s) and selects each journey whose JSON holds one; a navigation or layout change selects every journey; when callers or widgets cannot be measured (`widgets NOT MEASURED` row) it selects every journey (unsure means include). Field: 067 (Request_Edit page) 0 -> J-REQ-01, 090 (navigation) 0 -> all 5 journeys — speed experiment E2
- fix(canvas): **`mf-canvas-facts.py` reads a note line's sides by the model's Mendix version, so on Mendix 11 it no longer reports NOTE-SIDE for lines that face their object.** A note numbers its sides top 0, bottom 1, left 2, right 3 on Mendix 10, but like every other object (top 0, right 1, bottom 2, left 3) on Mendix 11; the script knew only the Mendix 10 table. It now reads `_ProductVersion` from the `.mpr`'s `_MetaData` table (major 11 or higher: the activity numbering; unreadable: Mendix 11) and prints `# note sides: …` as its first line. One app's 305 hand-drawn note lines, matched across its 10.24 and 11.15 versions: the upgrade renumbered the note's side on every line (0 to 0, 1 to 2, 2 to 3, 3 to 1) and left the object's side alone, and the script's fact counts for the whole app are now identical on both. mxcli v0.28.0 (mendixlabs/mxcli#1620) writes the activity numbering whatever the project's version. Fixture section V (renumbered copy of the golden model). — saingge
- process(design-artifacts, conversion-runbook): **Stage 3 runs in parallel by default, from one brief and a mechanical task list**: the lead fixes modules, canonical names and design tokens, writes `docs/progress/stage3-brief.md`, and takes the tasks from the stage-3 lines of `bin/lib/artifact-check.sh` (so no artifact is forgotten); one subagent per 1–2 screens or artifact group, a shared opening line so the start is cached, mechanical tasks on the cheap tier, 5-line replies, the lead the only writer of `PROJECT.md`. The runbook's Stage 3 fan-out row points there, and its Stage 2 row names `facts-lock.sh resolve`. Measured on one project (single runs, Sonnet, 250k auto-compact): serial 13 min / $3.60, parallel 10 min / $4.44, parallel with a brief 6 min / $4.32; 0 name mismatches in the parallel runs (serial dropped 3 entities); both parallel runs forgot the process map and swimlanes, which the task list now covers. Baseline words 64,004 before and after — Stage 3 experiment 2026-10-28
- fix(facts-lock, gate-check): **parallel BRDs keep one spelling per identifier**: `bin/facts-lock.sh <project> resolve` freezes each casing conflict (`WIPStatus`/`WipStatus`) whose majority spelling is used by strictly more BRD sources than any other and records the count in the lock (`resolvedBy`); a tie stays contested, and `build` still never picks on its own. Stage 2 of `gate-check.sh` now reads the lock once there are 2+ BRDs: a BRD that contradicts a frozen identifier FAILs and is quoted; no lock yet is a note, never a FAIL. Measured cause: 20 parallel BRDs on PROJECT-C carried 10 casing conflicts into Stage 3 as open questions, and facts-lock.sh, documented only as advice, reached 1 of 6 projects. Fixtures: `test-facts-lock.sh` resolve cases (4), `test-bug03-gates.sh` T19 (2 of 4 fail on the previous gate-check) — speed/parallel experiments 2026-10-28
- fix(row-groups): **a build plan whose number and Kind cells are formatted as code (`` `BUILD` ``, `` `12` ``) is read like a plain one**: before, it read 0 BUILD rows, so with `Row groups: on` grouping silently did nothing (the fast-lane agent in the 2026-10-28 speed experiment had to strip the backticks by hand). Fixture: `test-row-groups.sh` G5 (both cases fail on the previous row-groups.sh) — speed experiment 2026-10-28
Expand Down
46 changes: 35 additions & 11 deletions project-bin/mf-canvas-facts.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,19 @@
CALIBRATION, measured from Studio Pro 10.24 screenshots: a variable label is ~5.5 units per
character, two lines (name, type) under the shape; an activity fits 3 caption lines of ~16
characters and no word over 14; a decision diamond fits 2 lines of ~10 characters and no word
over 10. Connection sides are stored as indexes: an annotation (note) numbers its sides top 0,
bottom 1, left 2, right 3; every other object top 0, right 1, bottom 2, left 3. The note is
always the origin of a note line. A flow line is the cubic Bezier through the two stored control
vectors, sampled at 48 points. SENSE is a prompt to read the true branch, not a verdict: on one
hand-drawn corpus 15 of its 18 hits were fine (`$Message = empty` really means "valid"), while all
9 hits in the mxcli-written flows were real.

OUTPUT: per flow `# Module.Name (n)` then one indented fact line each; last line
`# facts: <flows> flows, OVERLAP a, LINE b, ...`. Field run, measured 2026-10-21 on one Mendix 10.24 app (1,588 flows, read in one pass):
on the 36 flows one mxcli-built branch added or changed in a single module, before any repair, OVERLAP 19,
over 10. Connection sides are stored as indexes. Every object except a note numbers its sides top 0,
right 1, bottom 2, left 3. A note (annotation) numbers its own sides by the model's Mendix version:
on Mendix 10 top 0, bottom 1, left 2, right 3; on Mendix 11 like every other object. The version is
the .mpr's `_MetaData._ProductVersion`; the first output line says which numbering was used, and a
version that cannot be read counts as Mendix 11. The note is always the origin of a note line. A flow
line is the cubic Bezier through the two stored control vectors, sampled at 48 points. SENSE is a
prompt to read the true branch, not a verdict: on one hand-drawn corpus 15 of its 18 hits were fine
(`$Message = empty` really means "valid"), while all 9 hits in the mxcli-written flows were real.

OUTPUT: first `# note sides: <numbering used>`, then per flow `# Module.Name (n)` then one indented
fact line each; last line `# facts: <flows> flows, OVERLAP a, LINE b, ...`. Field run, measured
2026-10-21 on one Mendix 10.24 app (1,588 flows, read in one pass): on the 36 flows one mxcli-built
branch added or changed in a single module, before any repair, OVERLAP 19,
LINE 27, NOTE-SIDE 17, LEFTWARD 3, TURN 4, SENSE 9, CAPTION 148; after the branch's later repair
commits, OVERLAP 3, LINE 7, TURN 4, CAPTION 113 and the rest 0.

Expand All @@ -55,7 +58,8 @@
KINDS = ("OVERLAP", "LINE", "NOTE-SIDE", "LEFTWARD", "TURN", "CAPTION", "SENSE")
CH = 5.5
ACT_SIDES = {0: (0, -1), 1: (1, 0), 2: (0, 1), 3: (-1, 0)}
NOTE_SIDES = {0: (0, -1), 1: (0, 1), 2: (-1, 0), 3: (1, 0)}
NOTE_SIDES_10 = {0: (0, -1), 1: (0, 1), 2: (-1, 0), 3: (1, 0)} # a note on Mendix 10
NOTE_SIDES = ACT_SIDES # a note on Mendix 11 and up; main() sets it from the model's version
SIDE_NAME = {(0, -1): "top", (0, 1): "bottom", (-1, 0): "left", (1, 0): "right"}
NEG = re.compile(r"\b(not|no|non|none|never|missing|wrong|without|empty|invalid|inactive|incomplete|"
r"incorrect|disabled|absent|expired|refused|failed|lacks?)\b|n't\b|"
Expand All @@ -66,6 +70,23 @@
r"validated|verified)\b", re.I)


# -- which numbering the model's notes use ---------------------------------------------------------
def note_numbering(model):
"""(note side table, text for the output line), from the major version in the .mpr's _MetaData."""
try:
db = model._db()
try:
ver = db.execute("SELECT _ProductVersion FROM _MetaData").fetchone()[0]
finally:
db.close()
major = int(re.match(r"\d+", ver).group())
except Exception: # no table, no row, no number: say so, and take the newer numbering
return ACT_SIDES, "Mendix 11 numbering (model version unreadable, assumed)"
if major >= 11:
return ACT_SIDES, "Mendix 11 numbering (model version %s)" % ver
return NOTE_SIDES_10, "Mendix 10 numbering (model version %s)" % ver


# -- the unit as plain data ----------------------------------------------------------------------
def plain(t, v):
"""A mxpatch (type, key, value) value as dicts and lists; arrays keep their version marker."""
Expand Down Expand Up @@ -346,6 +367,8 @@ def main(argv):
except mxpatch.PatchError as e:
sys.stderr.write("mf-canvas-facts: %s\n" % e)
return 2
global NOTE_SIDES
NOTE_SIDES, numbering = note_numbering(model)
# model.units() would stop at the first unit that does not decode; read each candidate on
# its own so one bad unit is a named fault and every other flow is still measured.
needles = [mxpatch._bson_str(0x02, "$Type", t) for t in FLOW_TYPES]
Expand Down Expand Up @@ -375,6 +398,7 @@ def main(argv):
found = {q for q, _ in picked} | {q.split(".", 1)[1] for q, _ in picked}
missing = [n for n in names if n not in found]
totals = dict.fromkeys(KINDS, 0)
print("# note sides: %s" % numbering)
for q, mf in sorted(picked, key=lambda x: x[0]):
res = check(mf, exposed)
print("# %s (%d)" % (q, len(res)))
Expand Down
57 changes: 54 additions & 3 deletions tests/wave2/test-mf-canvas-facts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@
# (see its CAPTURE.txt). Each fact has a negative control in the same model: a parameter column
# (no OVERLAP), a decision captioned to match its true branch (no SENSE).
#
# The note numbering depends on the model's Mendix version (section V). The captured dump says
# 11.14.0; the golden facts below were measured on a Mendix 10 reading of it, so model() stamps
# 10.24.18 over it, and section V builds the Mendix 11 copies.
#
# Positive control: against a project-bin without the script every case fails -- it does not
# exist. [guard] marks regression guards.
set -uo pipefail
Expand All @@ -27,11 +31,31 @@ bad() { FAIL=$((FAIL+1)); printf ' FAIL %s\n' "$1"; [ -n "${2:-}" ] && printf '
. "$TK/bin/lib/portable.sh"; require_py
export MXPATCH_BACKUP_DIR="$WORK/bak"

# model <dir> -- the base model's units, the captured five over them, the captured .mpr.
# model <dir> [version] -- the base model's units, the captured five over them, the captured .mpr
# stamped with the Mendix version (default 10.24.18.103793, a Mendix 10 model).
model() {
rm -rf "$1"; mkdir -p "$1"; cp -r "$BASE/mprcontents" "$1/"; cp -r "$FX/overlay/mprcontents" "$1/"
"$PY" -c 'import sqlite3,sys; c=sqlite3.connect(sys.argv[1]); c.executescript(open(sys.argv[2]).read()); c.commit()' \
"$1/TestApp.mpr" "$FX/overlay/TestApp.mpr.sql"
"$PY" -c 'import sqlite3,sys; c=sqlite3.connect(sys.argv[1]); c.executescript(open(sys.argv[2]).read())
c.execute("UPDATE _MetaData SET _ProductVersion = ?", (sys.argv[3],)); c.commit()' \
"$1/TestApp.mpr" "$FX/overlay/TestApp.mpr.sql" "${2:-10.24.18.103793}"
}
# notes <dir> set <n> | remap -- the origin side of every note line: all set to n, or renumbered the way
# the Mendix 10 -> 11 upgrade does (top 0 -> 0, bottom 1 -> 2, left 2 -> 3, right 3 -> 1). Prints how many.
notes() {
"$PY" -I -c 'import sys; sys.path.insert(0, sys.argv[1]); import mxpatch
m = mxpatch.Model(sys.argv[2]); n = 0
for p in m.unit_files():
raw = open(p, "rb").read()
if b"AnnotationFlow" not in raw:
continue
u = mxpatch.Unit(p, raw)
for d in mxpatch.walk(u.doc):
if mxpatch.get(d, "$Type") == "Microflows$AnnotationFlow":
old = mxpatch.get(d, "OriginConnectionIndex")
mxpatch.set_value(d, "OriginConnectionIndex", int(sys.argv[4]) if sys.argv[3] == "set" else {0: 0, 1: 2, 2: 3, 3: 1}[old])
n += 1
m.write_unit(u, apply=True, log=lambda s: None)
print(n)' "$BIN" "$1" "$2" "${3:-}"
}
FACTS() { "$PY" -I "$BIN/mf-canvas-facts.py" "$@"; }

Expand Down Expand Up @@ -69,6 +93,33 @@ for c in sys.argv[2:]: print("%s=%s" % (c, bool(f.NEG.search(c))))' "$BIN/mf-can
printf '%s' "$OUT" | tail -1 | grep -q '^# facts: 15 flows, OVERLAP 1, LINE 1, NOTE-SIDE 1, LEFTWARD 0, TURN 0, CAPTION 0, SENSE 1$' \
&& ok "summary line counts every fact kind" || bad "summary" "$(printf '%s' "$OUT" | tail -1)"

echo "== V: the note numbering follows the model's Mendix version =="
printf '%s\n' "$OUT" | head -1 | grep -qx '# note sides: Mendix 10 numbering (model version 10.24.18.103793)' \
&& ok "the first output line names the numbering used (Mendix 10)" || bad "numbering line" "$(printf '%s\n' "$OUT" | head -1)"
M11="$WORK/m11"; model "$M11" 11.15.0
OUT11="$(FACTS "$M11" --module MyFirstModule 2>&1)"
printf '%s\n' "$OUT11" | head -1 | grep -qx '# note sides: Mendix 11 numbering (model version 11.15.0)' \
&& ok "the first output line names the numbering used (Mendix 11)" || bad "numbering line 11" "$(printf '%s\n' "$OUT11" | head -1)"
printf '%s' "$OUT" | grep -q '^ NOTE-SIDE note line Annotation@360,100 -> Action@360,200 leaves the note left' \
&& ! printf '%s' "$OUT11" | grep -q '^ NOTE-SIDE' \
&& ok "mxcli's (2,0) note above its activity: NOTE-SIDE on Mendix 10, none on Mendix 11 (bottom of the note)" \
|| bad "(2,0) above the activity, 10 vs 11" "$OUT11"
# Every origin side of that note, on a Mendix 10 model, against the same line renumbered as the upgrade does
# on a Mendix 11 model: the same NOTE-SIDE result. The note sits above its activity, so only its bottom faces it.
for n in 0 1 2 3; do
A="$WORK/n10-$n"; B="$WORK/n11-$n"; model "$A"; model "$B" 11.15.0
C1="$(notes "$A" set "$n")"; C2="$(notes "$B" set "$n")"; C3="$(notes "$B" remap)"
R10="$(FACTS "$A" --module MyFirstModule | grep -c '^ NOTE-SIDE')"; R11="$(FACTS "$B" --module MyFirstModule | grep -c '^ NOTE-SIDE')"
WANT=1; [ "$n" = 1 ] && WANT=0
[ "$C1/$C2/$C3" = "1/1/1" ] && [ "$R10" = "$WANT" ] && [ "$R11" = "$WANT" ] \
&& ok "origin side $n on Mendix 10 and the upgraded $n on Mendix 11 give the same NOTE-SIDE result ($WANT)" \
|| bad "origin side $n: lines $C1/$C2/$C3, NOTE-SIDE on 10: $R10, on 11: $R11, want $WANT"
done
MU="$WORK/mu"; model "$MU"; "$PY" -c 'import sqlite3,sys; c=sqlite3.connect(sys.argv[1]); c.execute("DROP TABLE _MetaData"); c.commit()' "$MU/TestApp.mpr"
OUTU="$(FACTS "$MU" --module MyFirstModule 2>&1)"; RC=$?
[ "$RC" -eq 0 ] && printf '%s\n' "$OUTU" | head -1 | grep -qx '# note sides: Mendix 11 numbering (model version unreadable, assumed)' \
&& ok "[guard] a version that cannot be read: Mendix 11 numbering, said on the first line, still exit 0" || bad "unreadable version rc=$RC" "$(printf '%s\n' "$OUTU" | head -1)"

echo "== S: selection and faults =="
OUT="$(FACTS "$M/mprcontents" MyFirstModule.ACT_Customer_Check ACT_Customer_Merge 2>&1)"; RC=$?
[ "$RC" -eq 0 ] && [ "$(printf '%s\n' "$OUT" | grep -c '^# MyFirstModule\.')" = 2 ] \
Expand Down
Loading