Repository navigation
mctest/mcrun: --scan_split fixes (statfactor nesting, NeXus merge) - #2891
Merged
willend merged 4 commits intoOct 23, 2026
Merged
Conversation
6253f3f (--statfactor) put its block between the %Scan branch and the --scan_split=auto that belongs to it, so the option was added to every test when --statfactor was given (a %Example then failed in mcrun with "No interval range specified"), and to no %Scan test without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each parallel step writes its own dir/i/mccode.h5 with its data as entry1. These were merged into dir/mccode.h5 with h5copy (/entry1 -> /entryN), which keeps references to entry1 and the step dir: - the 'target' attribute of each linked monitor in entry2.. pointed to /entry1/instrument/...: nexusformat/NeXpy follow it and showed entry1's data for every scan step (h5py-based readers were not affected) - Directory/name/title/file_name named the step dir dir/i - the step dirs with their mccode.h5 copies were left behind merge_nexus_steps() now copies the entries with h5py, rewrites those references (keeping fixed-length strings) and removes the step dirs, so the result matches a serial NeXus scan item by item apart from times. h5py is needed for NeXus --scan_split scans instead of the h5copy tool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n failure A failed step can leave a partial, unreadable dir/i/mccode.h5: the merge (by file existence, also with h5copy before) then crashed mcrun before mccode.dat was written. Merge only the steps with detector values (the rows of mccode.dat), numbered entry1..N as a serial NeXus scan does, and keep all step dirs when a step failed, for inspection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The step workers of a split scan run in their own processes. With the spawn start method (macOS, Windows) they lacked mcrun's log setup, so their warnings were printed bare (no "WARNING:") and their INFO messages were lost; they also numbered steps from 0 without the total. A Pool initializer now sets up mcrun's logging where the worker has none (fork, Linux, already inherits it), and the step messages read "Scan step i+1/N", as in the serial scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes around
--scan_split, so that split scans match the normal (serial) scan.1. mctest:
--scan_split=autofor%Scantests only6253f3f (
--statfactor) inserted its block between the%Scanbranch and the--scan_split=autothat belongs to it, so:--statfactor(and no MPI), every test got--scan_split=auto: a plain%Examplethen fails in mcrun withCRITICAL: No interval range specified--statfactor, no%Scantest was run splitFix: move the two lines back under
if test.scan:(as introduced in e5e7b58).mctest --no-mpi, TAS1_Diff_Slit--statfactor 0.05--statfactor2. mcrun: NeXus
--scan_splitmerge as a serial NeXus scan writes itEach parallel step writes
dir/i/mccode.h5with its data asentry1; these were merged withh5copy(/entry1→/entryN), keeping references toentry1and the step dir:targetattribute of every linked monitor inentry2..pointed to/entry1/instrument/.... nexusformat / NeXpy follow it and showed entry1's data for every scan step (h5py-based readers were not affected)Directory,name,title,file_namenamed the step dirdir/imccode.h5copies were left behindmerge_nexus_steps()now copies the entries with h5py, rewrites those references (keeping fixed-length strings) and removes the step dirs.Failed steps: a failed step can leave a partial, unreadable
dir/i/mccode.h5, on which the merge (by file existence, withh5copyas well) crashed mcrun beforemccode.datwas written. Now only the steps with detector values (the rows ofmccode.dat) are merged, numberedentry1..Nas a serial NeXus scan does, and all step dirs are kept when a step failed.mccode.datmccode.h5mccode.dat,mccode.h50/ 1/ 2/, brokenmccode.h5mccode.dat,mccode.h5+ kept0/ 1/ 2/Worker logging: split-scan steps run in
multiprocessingworkers. With the spawn start method (macOS, Windows) these lacked mcrun's log setup: warnings were printed bare (Scan step 1 failed ..., noWARNING:) and their INFO lines (e.g.call to mpirun failed) were lost. APoolinitializer now sets up mcrun's logging where the worker has none (with fork, Linux, it is inherited, so no duplicate handler), and steps are numberedi+1/Nas in the serial scan:WARNING: Scan step 2/3 failed (...). Checked on macOS with spawn and a forced fork: one formatted line per message.Tested (macOS, mcstas, same seed): 3-point scans of a 0D/1D/2D test instrument and of
Test_NeXus_output(+ a dummy scan parameter: 0D, 1D, 2D, Monitor_nD 2D/3D BINS maps, event list), normal vs--scan_split=auto:mccode.dat,mccode.h5only), andmccode.h5identical item by item (all groups, datasets, attributes incl. types) apart from times; everytargetresolves to its own entry; nexusformat reads the same data as h5pyDependency: NeXus
--scan_splitscans now needh5pyin mcrun (instead of theh5copytool). In mcstas-suite-feedstockh5pycurrently only comes indirectly (mcstas→nexpy,mcstas-vis→mcstasscript); it should be added tomcstas-core(and likewise for McXtrace).Not tested on Linux/Windows or with McXtrace (shared code).
🤖 Generated with Claude Code