Skip to content

docs(box): Bring the local test guide up to date with box-integration-development - #2870

Merged
vvargu merged 3 commits into
box-integrationfrom
docs/box-local-testing-sync
Nov 20, 2026
Merged

vvargu merged 3 commits into
box-integrationfrom
docs/box-local-testing-sync

Conversation

@vvargu

@vvargu vvargu commented Nov 20, 2026 •

Copy link
Copy Markdown

Brings docs/box-local-testing.md on box-integration up to date with the guide on box-integration-development. It ports only what the box-integration code supports. I checked each part against the code of this branch.

Ported from box-integration-development:

  • Install box: download, check, and PATH steps for the pinned 0.1.11-preview.10 (chore(box): Pin box 0.1.11-preview.10 #2871), which has macOS (aarch64-apple-darwin) and Linux arm64 builds. Build, deploy, and dev read the profile with box profile show, so they need a local box.
  • Settings: absolute paths, the test -f "$CLI" && test -d "$L3" && echo ok check, and the $HOME note.
  • Step 6, test locally with agentcore dev: the box builds table, the checks before you start, the env:// key handoff, and the container and CodeZip notes. Build, deploy, verify, and tear down move to steps 7, 8, 9, and 11.
  • What build and deploy do: box profile show, the @box/ pack names, HTTP [mcp] server targets, the Build stopped: error format.
  • Step 9 live checks: the gateway ID from the stack output, and the provider <project>_default_github (credentials.ts names it ${project}_${target}_${credential}), in place of get --name github.
  • Step 10: box telemetry in the Runtime log group, logs runtime, the gateway APPLICATION_LOGS, and a custom cloudwatch_logs target with box-telemetry-policy-<runtime>.json.
  • Step 11 tear down: deploy --yes, and the resources that outlive the stack.
  • Troubleshooting: the entries for the local box, dev, and build.

Kept as box-integration has them:

  • The gateway <runtime>-box-egress, bound by boundRuntime. The step 9 query uses ApplicationGatewayAgentBoxEgressIdOutput.
  • --containment private-vpc on create.
  • The strict box.version pin and bun scripts/bump-box-release.ts --box <version>.
  • .box-dist/box-release.env for a container, and the .box-dist/profile.toml staging.
  • The build behavior for a rotated GITHUB_TOKEN.
  • The notice from agentcore create and agentcore add runtime.

Left out, because this branch does not have it: AGENTCORE_BOX_SECRETS_MANAGER.

Review fixes (eb333e8): the box --version wording, the box/<name> pack refusal, the additionalPolicies file warning, the dev stop condition, the CodeZip box run folder, the gateway logs with enableApplicationLogs / enableTraces, and a prerequisite for an L3 with aws/agentcore-l3-cdk-constructs#470.

Docs only. prettier --check docs/box-local-testing.md passes.

…-development

Port the sections of the box-integration-development guide that the
box-integration code supports: Install box with the pinned
0.1.11-mquang.preview.9 (now with a macOS build), the settings checks,
the dev step, the step 9 live checks by gateway ID and provider name,
the box telemetry and gateway logs step, and the full tear down.

Keep the names and behavior of box-integration: the gateway
<runtime>-box-egress bound by boundRuntime, --containment, the strict
box.version pin, the box-release.env of a container, and the build
behavior for a rotated GITHUB_TOKEN.
@vvargu
vvargu requested a review from a team November 20, 2026 17:53
@codecov-commenter

codecov-commenter commented Nov 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.80%. Comparing base (f98cfd6) to head (eb333e8).

Additional details and impacted files
@@               Coverage Diff                @@
##           box-integration    #2870   +/-   ##
================================================
  Coverage            96.80%   96.80%           
================================================
  Files                  766      766           
  Lines                66803    66803           
================================================
  Hits                 64666    64666           
  Misses                2137     2137           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@leo237 leo237 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plan mapping. This is DOCS-2 (phase 14) of the squash plan v2, the single README + testing-guide PR. It lands before phases 11-13 (MQ-5/MQ-6, CDK-SA-1, SA-5, SA-6). That is fine because it keeps to what box-integration has today, apart from the items below.

Verdict: APPROVE. CI is green: verify Linux/Windows/macOS, CodeQL, and the CodeBuild e2e all pass. The branch is up to date with box-integration (one commit on top) and merges cleanly. Nothing below breaks a reader's main path (install box, then create, build, deploy, verify, tear down). Each of the five items is either a wrong description of behavior or a step that a later planned PR enables. All are non-blocking, but please fix items 1-4 here or in a follow-up, because this guide is the reference.

I checked the guide against box-integration (CLI and L3) and against box-integration-development. These match box-integration:

  • the 0.1.11-mquang.preview.9 pin and its release assets (macOS + Linux arm64 + SHA256SUMS.txt, so the install steps work)
  • BOX_MINIMUM_VERSION = linux-preview.7
  • bump-box-release.ts --box, .box-dist/box-release.env, --containment private-vpc
  • Build stopped:/Deploy stopped:, [telemetry.agentcore]/agentcore_default, DISABLE_ADOT_OBSERVABILITY, agentcore/box-telemetry-policy-<runtime>.json with xray:PutSpans/PutTraceSegments
  • the dev port check, the missing-env:// warning text, BOX_RUN_FLAGS (SYS_ADMIN/unconfined), AGENTCORE_CONTAINER_TOOL, and the credential refresh
  • [project] access = "write" for CodeZip only
  • the ApplicationGatewayAgentBoxEgressIdOutput prefix and the AgentCore-<project>-default-agent-RuntimeId export
  • gateway get --id, identity api-key-credential-provider get/delete --name, logs runtime --name/--since/--query, remove all --yes, and the deploy --yes empty-stack confirmation
  • every troubleshooting string

Non-blocking (should fix):

  1. Step 10, gateway APPLICATION_LOGS (L479-485): nothing on box-integration creates this. No vended-logs delivery exists in L3 box-integration or in this CLI. So on box-integration today, aws logs tail /aws/vendedlogs/.../$GATEWAY_ID fails with ResourceNotFound. The planned CDK-SA-1 (L3 commit 1f8c3e0, now aws/agentcore-l3-cdk-constructs#470) adds it, with its CLI partner #2860, so this is non-blocking. But the wording will not match even after those merge: #470 makes enableApplicationLogs default to false and adds optional enableTraces for X-Ray, and #2860 writes enableApplicationLogs: true only for a new Box gateway. So "Deploy always sends" and "The gateway sends no traces to X-Ray" will be wrong. Either drop this block until #470/#2860 merge, or write it for that design.
  2. box/<name> pack names (L203-205): box-integration removed this stop in f40ebce ("Drop the old box/ pack-name stop", from review of #2867). Build no longer gives the @box/<name> names; box refuses the profile with its own error, under Box refused the profile …. The sentence comes from the dev branch, which still has the old code.
  3. Dev on a non-Box project (L288-290): dev does not detect a CodeZip start.py entrypoint or a Dockerfile that installs box. BoxDevRunner.run and the container runner decide only from runsInBox, and they stop with notBoxProjectMessage only when the runtime has box.version. L26-30 of this guide already says this ("The runtime's entry point and Dockerfile never decide it"), so the two passages contradict each other.
  4. CodeZip dev working folder (L366): on box-integration, dev runs box from boxStartFolder(runtime) (app/<agent> for CodeZip), not from the project folder. The profile argument is relative to that folder. This is the boxStartFolder change already on box-integration; the dev-branch text predates it.
  5. The PR description says the additionalPolicies warning was left out because this branch does not have it, but box-integration does have it. grantedActions in src/core/project/boxNetworkSource.ts warns for a listed .json file that does not exist, cannot be read, is not valid JSON, or is not a JSON object (see __fixtures__/box-network-build-warnings.golden.txt). The Warnings bullet (L223-225) can list it again.

Nits:

  • L103-105 says box --version "must print box 0.1.11-mquang.preview.9" and, in the next sentence, that linux-preview.7 or later is accepted. L319 says only "linux-preview.7 or later". Use one wording, for example "should print the pinned release; build, deploy and dev accept linux-preview.7 or later".
  • The CLI still says the pinned release has only a Linux arm64 build, in BOX_MINIMUM_RELEASE_PAGE's doc comment and in boxRelease.test.ts:289. This guide now says, correctly, that preview.9 has a macOS build. That is code and outside this PR; dev commit 233d3396 fixes the same text in the dev guide only.

New types: N/A (docs only). Tests: N/A. Every documented command maps to an existing handler or flag on box-integration, except item 1.

Comment thread docs/box-local-testing.md Resolved
Comment thread docs/box-local-testing.md Resolved
Comment thread docs/box-local-testing.md Resolved
Comment thread docs/box-local-testing.md Resolved
Comment thread docs/box-local-testing.md Resolved
Comment thread docs/box-local-testing.md Resolved
Resolve the conflict with #2871 (pin box 0.1.11-preview.10): keep the
pointer to Install box, and install the new pinned 0.1.11-preview.10,
which has macOS and Linux arm64 builds.

@leo237 leo237 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review at c2e1a58. Since my approval at bcc020b, the only PR change is the merge of box-integration (#2860 gateway telemetry fields, #2871 box pin 0.1.11-preview.10). I did not re-review those PRs.

Merge resolution: clean. I compared the PR's net diff to docs/box-local-testing.md at bcc020b (against its old base) with the net diff at c2e1a58 (against box-integration). The only differences are 0.1.11-mquang.preview.9 -> 0.1.11-preview.10 in the prerequisites, the Install box commands, and the box --version / xattr checks. One more difference comes from the base: the base intro line about which builds the pinned release has is now its preview.10 wording, and the PR still replaces that line with "install it as in [Install box]". No PR content was lost or garbled. No preview.9 or mquang.preview string is left. The mquang-amazon/box-releases repo name is still right (it is the repo in boxRelease.json), and v0.1.11-preview.10 has both aarch64-apple-darwin and aarch64-unknown-linux-gnu archives plus SHA256SUMS.txt, so the download commands resolve.

Earlier comments

  • Nit, the pointless linux-preview.7 fallback in the intro: addressed by the merge. The intro sentence is gone.
  • (1) Step 10 gateway logs: still open, and the merge makes it more pressing. See the inline comment. Non-blocking, because the plan's cross-repo pair already covers it ("#2860 is merged, so a CDK release with #470 and a CLI pin update must follow", plan v2, CDK-SA-1 / aws/agentcore-l3-cdk-constructs#470).
  • (2) L203-205, build stopping on a box/<name> pack: still open (refuseOldShippedNames is still gone on box-integration).
  • (3) L288-290, the dev stop condition: still open. Dev decides only from runsInBox, and it stops with notBoxProjectMessage only for a runtime with box.version (src/core/dev/box.ts:56, container.ts:378).
  • (4) L366, CodeZip dev runs box run from boxStartFolder(runtime) (app/<agent>): still open.
  • (5) The PR description still says the additionalPolicies file warning is left out. grantedActions in boxNetworkSource.ts still has it on box-integration: still open.
  • Nit, L103-105 vs L319 wording of the box --version requirement: still open.

New finding (non-blocking, planned): with #2860 merged, step 8 deploy fails on today's L3. boxGatewaySpec (src/projectSchemas/project.ts) now writes "enableApplicationLogs": true into the Box gateway of every new Box project. The generated CDK app runs transformAgentCoreJson, which runs AgentCoreProjectSpecSchema.safeParse. The gateway schema of the L3 there is .strict() and has no enableApplicationLogs on box-integration, box-integration-development, or v1.x. Only #470 adds it. So a reader who follows steps 2-8 with an L3 checkout of box-integration gets a ConfigValidationError (unrecognized key) at synth. Until #470 merges, suggest one line in Prerequisites or step 2: "Check out the L3 at aws/agentcore-l3-cdk-constructs#470 (or a box-integration that has it). The Box gateway that create writes has enableApplicationLogs, which an older L3 refuses." Another option is to drop that line from the Box gateway in agentcore.json by hand. This comes from the base, not from this PR, and #470 fixes it, so it does not block.

CI: CodeQL / Analyze / e2e CodeBuild pass; verify jobs: verify / Windows: SUCCESS, verify / Linux: SUCCESS, verify / macOS: QUEUED (Windows passed; any still-pending job is noted here as pending when this review was posted).

Approving: the merge is clean, and everything open is non-blocking or planned (#470).

Comment thread docs/box-local-testing.md Resolved
…m box-integration

- box --version: give the pinned version and the minimum in one sentence.
- A box/<name> pack: box refuses it; build no longer stops with a hint.
- Warnings: list the additionalPolicies .json file warning again.
- Dev: it stops only on a runtime with box.version, not on start.py or a
  Dockerfile.
- CodeZip dev: box run starts in app/<runtime>.
- Gateway logs: enableApplicationLogs (written by create and add runtime),
  enableTraces, and L3 #470.
- Prerequisites: the L3 must have #470.
@vvargu
vvargu requested a review from leo237 November 20, 2026 20:33

@leo237 leo237 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of eb333e8 (delta since our approval at c2e1a58). I checked each changed statement against box-integration (f98cfd6).

Addressed, and each fix is accurate:

  • Step 10 gateway logs: the new text matches the code. boxGatewaySpec writes enableApplicationLogs: true, create and add runtime give the notice (boxGatewayLogsNotice), and the schema leaves logs and enableTraces off by default. The X-Ray and Transaction Search note matches the schema comment. The new prerequisite for aws/agentcore-l3-cdk-constructs#470 is right too. That PR is now merged into the L3 box-integration branch, so the interim note is correct as written.
  • The box/<name> pack: build no longer claims to stop on it. Box refuses the profile itself.
  • Dev stop condition: it now keys on box.version with no "projectType": "box", as BoxDevRunner.run and notBoxProjectMessage do. This matches L26-30.
  • CodeZip box run: it now runs from app/<runtime> with the profile path relative to that folder. This matches boxStartFolder and posix.relative(startFolder, profileFile).
  • The additionalPolicies warning is now documented. The PR body no longer lists it as missing. It matches boxNetworkSource.ts.
  • Nit on box --version: the wording now matches L44-46. The pin is 0.1.11-preview.10, and BOX_MINIMUM_VERSION is 0.1.11-linux-preview.7.

Still open: none.

New findings: none. This commit introduces no new errors that I found.

CI: CodeQL, Analyze, and the e2e CodeBuild pass, and verify / Windows passes. verify / Linux and verify / macOS were still pending when I posted. This change is docs only.

@vvargu
vvargu merged commit e2ac9d7 into box-integration Nov 20, 2026
7 checks passed
@vvargu
vvargu deleted the docs/box-local-testing-sync branch November 20, 2026 20:53
SaiAbhishek98 pushed a commit to SaiAbhishek98/agentcore-cli that referenced this pull request Nov 20, 2026
`box-integration` has one new change,
[aws#2870](aws#2870).
It writes `docs/box-local-testing.md` again. Git merged the file with
no conflict, and the OTEL paragraphs of this branch stay in "What build
and deploy do with the box profile". The merge adds one paragraph to
step 10, which tells how to read box telemetry. That paragraph names the
default X-Ray target and the log group `aws/spans`.
SaiAbhishek98 pushed a commit to SaiAbhishek98/agentcore-cli that referenced this pull request Nov 21, 2026
`box-integration` has one new change,
[aws#2870](aws#2870).
It writes `docs/box-local-testing.md` again. Git merged the file with
no conflict, and the OTEL paragraphs of this branch stay in "What build
and deploy do with the box profile". The merge adds one paragraph to
step 10, which tells how to read box telemetry. That paragraph names the
default X-Ray target and the log group `aws/spans`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants