Repository navigation
fix(box): Grant the standard library of the Runtime Python on CodeZip, not its prefix - #2882
SaiAbhishek98 wants to merge 1 commit into
Conversation
…, not its prefix
## Problem
A CodeZip box runtime does not start on AgentCore Runtime. box stops with this error:
`[agent] filesystem entry "/usr" is refused: ... a system root is never a grantable tree`
The Runtime profile moves the read grant and the exec grant of the uv Pythons of the local profile
to `${{ inputs.python.prefix }}`. start.py sets this input to `sys.base_prefix`. The Python of a
PYTHON_3_14 Runtime is `/usr/bin/python3.14`. Thus the value is `/usr`. Box Core refuses a grant of
`/usr`, and each Box Core release refuses it. Thus box did not start on a CodeZip Runtime before
this change. Before
[4700b62](aws@4700b62),
start.sh started the agent with no box when box did not start. That commit and
[aws#2853](aws#2853) removed this fallback. At
this time, the Runtime stops the runtime, and the deploy stops with an error. The branch
box-integration-development has the same problem.
## Solution
With this change, the uv Pythons of the local profile go to the standard library of the Runtime
Python, `${{ inputs.python.stdlib }}`, and not to its prefix.
- start.py exports `BOX_PYTHON_STDLIB`, the value of `sysconfig.get_path("stdlib")`, in place of
`BOX_PYTHON_PREFIX`. On a PYTHON_3_14 Runtime, the value is `/usr/lib/python3.14`.
- The `@box/agentcore-codezip` pack has the input `python.stdlib` in place of `python.prefix`, with
the same pattern.
- The paths rule moves `${{ home }}/.local/share/uv/python` to `${{ inputs.python.stdlib }}`.
After this change, the Runtime profile of each CodeZip template has these grants:
- `read = [code folder, standard library, "/usr/lib64", "/lib64"]`. The Python reads its modules.
`/usr/lib64` contains the libraries that its native modules use, for example `libssl.so.3`.
- `exec = [code folder, standard library]`. box mounts a folder that is not a loader folder with
no exec. Thus the Python can use the native modules in `lib-dynload` only with this grant.
- The Python itself is the command of the agent, and box gives it exec.
A read grant of the loader folder `/usr/lib` also lets the Python start. This change does not use
it, because it gives read and exec on all of `/usr/lib`, and not only on the standard library.
A CodeZip box runtime that `agentcore create` made before this change has a start.py that exports
the prefix. Copy the start.py of this change into its code folder.
## Tests
- `start.py > gives box the standard library of the Runtime's Python, and not its base prefix`
(`boxStartPy.test.ts`) starts `main()` of start.py, and replaces `os.execve` with a function that
records the environment. It makes sure that `BOX_PYTHON_STDLIB` is the standard library, and that
no value is the prefix.
- `the Runtime profile of each CodeZip box template of create` (`additions.test.ts`) derives the
Runtime profile of the six CodeZip box templates. It makes sure that the read grant and the exec
grant of the standard library are in the profile. It also makes sure that no grant has a value
on the Runtime that Box Core refuses: a system root in a list, or a loader folder in a list that
is not `read`.
- The pack test makes sure that the pack has the input `python.stdlib`. The paths test and the
test of the recorded profiles in `old-renderer-tables.json` use the standard library. In that
file, only these 204 values change.
- On the code before this change, 115 of the new and changed tests show an error.
- Five mutations of the code, and the errors that each mutation causes:
- start.py exports the prefix again: the start.py test shows an error.
- The paths rule moves the uv Pythons to the prefix again: 111 tests show an error.
- The pack has `python.prefix` again: 2 pack tests show an error.
- `runtimeCodeZipRunner` gives no exec grant of the uv Pythons: 110 tests show an error.
- `runtimeCodeZipRunner` also gives a grant of the prefix: the six template tests show an error
at the check of the refused grants.
- `bun test`: `5835 pass`, `6 skip`, `2 fail`. This change does not cause the two errors.
`boxProfileResolver.test.ts` also shows the error on box-integration with the same box
(0.1.11-preview.10) on PATH. `remove/index.test.ts` stops at its 10 s timeout when all tests
operate together. When it operates with no other test, it shows no error.
- `bun run typecheck`, `bun run lint:check`, `bun run format:check`, `bun run build`,
`bun run audit`, and `bun run secrets:check`: no errors.
- A local test in an Amazon Linux 2023 arm64 container, with box 0.1.11-preview.10 and a Python
3.14.3 that has the prefix `/usr` and the standard library `/usr/lib/python3.14`. The Python of
the Runtime has the same paths. The results:
- The zip of box-integration stops with the `/usr` refusal.
- The zip of this change starts the agent, and `/ping` gives 200.
- The zip of box-integration-development stops with the `/usr` refusal.
- A live test on AgentCore Runtime in a dev account. In the test, we deploy a new CodeZip box
project of this branch, with no change. Three invocations get status 200, and the box records
are in the log group `-DEFAULT` of the runtime. The test removes all its resources.
We did not do `bun run test:e2e`. This change has no test on the CodeZip Runtimes PYTHON_3_10 to
PYTHON_3_13, and no test of an `[mcp]` server or a `[tool]` on the Runtime Python.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## box-integration #2882 +/- ##
================================================
Coverage 96.79% 96.79%
================================================
Files 766 766
Lines 66803 66803
================================================
Hits 64665 64665
Misses 2138 2138 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
leo237
left a comment
There was a problem hiding this comment.
Approving. I found no blocking issues. The change is small, it fixes the actual cause, and it has a regression test for the exact failure.
What I checked (head 490c1972, merge-base = current box-integration e2ac9d78, so not behind base)
- The
/usrrefusal.start.pynow exportsBOX_PYTHON_STDLIB = sysconfig.get_path("stdlib")and no longer exportsBOX_PYTHON_PREFIX. The pack declaresinputs.python.stdlibwithABSOLUTE_PATH_PATTERN, and the paths rule moves the uv Pythons folder to it. The derived CodeZip grants areread = [code, stdlib, /usr/lib64, /lib64]andexec = [code, stdlib]. - Regression test. The new
additions.test.tscase resolves every grant of every CodeZip template against the measured Runtime values (/usr,/usr/lib/python3.14,/var/task). It asserts that no grant is a Box Core system root and that no loader folder appears outsideread. It also checks that stdlib is in bothreadandexec, which covers thefailed to map segmentproblem in lib-dynload. With the old prefix grant this test fails, which is what we want. /usr/lib64(libssl, libffi). The grant was already there as a read-only loader folder and is unchanged. The PR body reports a live CodeZip deploy where box started, gave 3x 200 and loggedbox_started, plus an AL2023 arm64 container repro.- Fixture. All 204 changed lines in
old-renderer-tables.jsonare a purepython.prefixtopython.stdlibsubstitution. Every removed line pairs with an added line once that string is replaced. - Leftover references. None. Outside tests and the fixture, nothing in the repo mentions
python.prefix,BOX_PYTHON_PREFIX,pythonPrefixorbase_prefix. - Test hygiene. The
start.pyharness test skips without Python (describe.skipIf(PYTHON === undefined), and thespawnSyncprobe is guarded with try/catch). It stubsos.execveand points all folders at a mkdtemp folder. It builds no host paths by hand. - Must-survive fixes. The diff touches only these 12 files, and the #2866 derivation,
.venv/bin/python,boxStartFolder, the ffi shim removal,BOX_OTHER_MACHINE_HINTand the preview.10 pin are all untouched. - CI. Linux, Windows, CodeQL/Analyze and CodeBuild e2e pass.
verify / macOSwas still pending when I posted this.
Non-blocking
- Back-merge to
box-integration-development. That branch still has"BOX_PYTHON_PREFIX": sys.base_prefixinstart.pyandinputs.python.prefixin its fixture. A later squash from dev would bring the/usrrefusal back, so it is worth tracking the follow-up PR explicitly. - Old
start.pyin existing projects. As the body says, a CodeZip project created before this change still exports onlyBOX_PYTHON_PREFIX, so box fails with an unresolvedpython.stdlibinput. That failure is loud, and those projects never started on the Runtime anyway, so nothing that works today breaks. The suggested follow-up (build rejects astart.pythat exports the prefix, or prints how to fix it) would still be good. - Runtime versions other than PYTHON_3_14 were not probed.
start.pystill supports 3.10 (the tomllib fallback). If any Runtime Python uses alib64layout, the stdlib grant would be/usr/lib64/python3.X. That puts anexecgrant under a loader folder and nested inside the/usr/lib64read grant. I do not know whether Box Core accepts that. The Decisions section says sysconfig makes thelib64layout work, but that case has not been run, so the claim is untested.
| "BOX_CODE": CODE, | ||
| "BOX_PYTHON": box_python(sys.executable, sys.version_info), | ||
| "BOX_PYTHON_PREFIX": sys.base_prefix, | ||
| "BOX_PYTHON_STDLIB": sysconfig.get_path("stdlib"), |
There was a problem hiding this comment.
Non-blocking: this is right for the measured PYTHON_3_14 layout (/usr/lib/python3.14). On a lib64-layout Python (Fedora/RHEL style), stdlib becomes /usr/lib64/python3.X, which sits inside the /usr/lib64 loader-folder read grant and also lands in exec. Has a Runtime version other than 3.14 been probed, or do we know Box Core accepts exec on a subfolder of a loader folder? If not, a sentence in the PR body or a comment would set expectations.
| runtimeCode: WORKSPACE, | ||
| runtimePython: `${WORKSPACE}/.venv/bin/python`, | ||
| runtimePythonPrefix: "/usr/local", | ||
| runtimePythonStdlib: "/usr/local", |
There was a problem hiding this comment.
Nit: for a container, this field is the identity mapping of /usr/local (the local Pythons folder). It is not a standard library. With the rename to runtimePythonStdlib, a reader might think the container gets a stdlib grant. A short comment here, or keeping a neutral name such as runtimePythons, would avoid that.
| * {@link AGENTCORE_CODEZIP_PACK_PATH}. | ||
| */ | ||
| export function renderAgentCoreCodeZipPack(): string { | ||
| return `# The CodeZip Runtime's Python and code directory, read from the environment start.py exports. |
There was a problem hiding this comment.
Nit: the pack's header comment still says "The CodeZip Runtime's Python and code directory". It could mention the standard library too, to match the new inputs.python.stdlib and the updated JSDoc above.
|
I close this PR, because CodeZip box runtimes are not in the scope of our work. Our work is OTEL for box runtimes. The problem that this PR found stays open: on box-integration, a CodeZip box runtime does not start on AgentCore Runtime. Box refuses the grant of |
Description
Problem
A CodeZip box runtime does not start on AgentCore Runtime. The deploy stops with
CREATE_FAILED(
The runtime process exited unexpectedly), and box gives this error:[agent] filesystem entry "/usr" is refused: ... a system root is never a grantable treeA control run on 2026-11-20 gave the same result with each value of
instrumentation.enableOtel.A container runtime of the same project starts.
The Runtime profile moves the read grant and the exec grant of the uv Pythons of the local profile
to
${{ inputs.python.prefix }}. start.py sets this input tosys.base_prefix. On the Runtime,the value is
/usr. Box Core refuses a grant of/usras a system root, and each Box Core releaserefuses it.
Thus box did not start on a CodeZip Runtime before this change:
Runtime profile with the prefix grant. When box did not start, start.sh started the agent with no
box.
(in aws/agentcore-cli#2850) and
aws/agentcore-cli#2853 removed this fallback.
Since then, the Runtime stops the runtime.
the derivation, with the same value. Its fixture of the renderer that it replaced also has the prefix.
The branch box-integration-development has the same problem
(435b6ba1, with box 0.1.11-preview.10).
This PR does not change that branch.
The necessary grants for the Runtime Python
A probe deploy to a PYTHON_3_14 CodeZip Runtime gave these facts:
/usr/bin/python3.14, and itssys.base_prefixis/usr. Itslibpythonisstatic.
/usr/lib/python3.14, with the native modules inlib-dynload./usr/lib64, for examplelibssl.so.3andlibffi.so.8.box mounts a read grant that is not a loader folder with no exec. Thus a read grant of the
standard library is not sufficient: on the Runtime,
import socketstops withfailed to map segment from shared object.The change
With this change, the uv Pythons of the local profile go to the standard library of the Runtime
Python,
${{ inputs.python.stdlib }}, and not to its prefix.BOX_PYTHON_STDLIB(sysconfig.get_path("stdlib")). It does not exportBOX_PYTHON_PREFIX.@box/agentcore-codezippack has the inputpython.stdlib. It does not have the inputpython.prefix.${{ home }}/.local/share/uv/pythonto${{ inputs.python.stdlib }}.After this change, the Runtime profile of each CodeZip template has these grants:
The Python itself is the command of the agent, and box gives it exec.
Decisions
/usr/lib. A read grant of/usr/libalsolets the Python start, and an exec grant is not necessary. But box makes a loader folder
executable, thus that grant gives read and exec on all of
/usr/lib. The grant of the standardlibrary gives less access.
agentcore buildcan also make the path fromruntimeVersion, for example${{ inputs.python.prefix }}/lib/python3.14. That path is correct only for theliblayout ofthis Runtime. The value of
sysconfigis the path of the Python that operates, also for alib64layout.python.prefixgoes. The prefix grant was its only use./usr/lib64and/lib64stay. On the Runtime,/lib64is a link to/usr/lib64, and boxaccepts the two grants. This change does not change them.
Tests that CI cannot do
3.14.3 that has the paths of the Runtime Python. The zip of box-integration stops with the
/usrrefusal. The zip of this change starts the agent, and/pinggives 200. The zip ofbox-integration-development stops with the
/usrrefusal.change. box shows the grants
read /usr/lib/python3.14andexec /usr/lib/python3.14. Threeinvocations get status 200. The log group
-DEFAULTof the runtime has 54 box records: 48permitdecisions and the recordbox_started. The test removes all its resources.Known problems and subsequent work
agentcore createmade before this change has a start.py that exportsthe prefix. With this CLI, box stops on the Runtime, because the input
python.stdlibhas novalue. Put a copy of the start.py of this change into the code folder.
agentcore builddoes not examinestart.py. A subsequent change can make
agentcore buildstop for a start.py that exports theprefix.
that branch.
boxProfileResolver.test.tsshows an error when box 0.1.11-preview.10 is on PATH. This error isalso on box-integration without this change.
Related Issue
No issue is open for this problem.
Documentation PR
Not applicable. No document names the input
python.prefix.Type of Change
Testing
How have you tested the change?
bun test:5835 pass,6 skip,2 fail. This change does not cause the twoerrors.
boxProfileResolver.test.tsalso shows its error on box-integration.remove/index.test.tsstops at its timeout only when all tests operate together.bun run test:e2e, or explained why they are notapplicable (CI does not run them for you). Not done. The live test above is on AgentCore
Runtime.
bun run typecheckbun run lint:checkbun run format:checkbun run auditbun run secrets:checkbun run buildsrc/assets/, I updated affected snapshots withbun test <test-file> --update-snapshotsand committed them. No snapshot has start.py.by this PR. In
old-renderer-tables.json, only the 204 values of the prefix change.command.md(the release workflow regenerates it)Each new or changed test shows an error on the code before this change (115 tests). Five
mutations of the code each cause an error in these tests.
Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.