Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion .github/workflows/e2e-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,14 +92,15 @@ jobs:
|| matrix.name == 'Windows' &&
format('["codebuild-agentcore-e2e-{0}-{1}","image:windows-1.0","e2e-windows"]', github.run_id, github.run_attempt)
|| matrix.name == 'macOS' && '["macos-latest"]'
|| matrix.name == 'Linux-arm64' && '["ubuntu-24.04-arm"]'
) }}
permissions:
id-token: write
contents: read
strategy:
fail-fast: false
matrix:
name: ${{ fromJSON(inputs.linuxOnly && '["Linux"]' || '["Linux","Windows","macOS"]') }}
name: ${{ fromJSON(inputs.linuxOnly && '["Linux"]' || '["Linux","Linux-arm64","Windows","macOS"]') }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand All @@ -125,6 +126,36 @@ jobs:
role-to-assume: ${{ env.E2E_AWS_ROLE_ARN }}
aws-region: ${{ vars.E2E_REGION || 'us-east-1' }}
unset-current-credentials: true
# box ships only arm64 builds, so only the macOS (Apple silicon) and Linux arm64 runners can run
# the Box tests. They skip on a runner with no box on PATH.
- name: Install box
if: matrix.name == 'macOS' || matrix.name == 'Linux-arm64'
env:
# The build's key in boxRelease.json, which pins its archive and SHA-256.
BUILD: ${{ matrix.name == 'macOS' && 'darwin' || 'linux' }}
run: |
PIN=src/core/project/backends/boxRelease.json
REPO=$(jq -er .box.repo "$PIN")
VER=$(jq -er .box.version "$PIN")
ASSET=$(jq -er ".box.${BUILD}.asset" "$PIN")
SHA=$(jq -er ".box.${BUILD}.sha256" "$PIN")
WORK=$(mktemp -d)
curl -fsSL --retry 3 -o "${WORK}/${ASSET}" \
"https://github.com/${REPO}/releases/download/v${VER}/${ASSET}"
# Check against the pin in the repository, not a checksum file of the same release.
# Linux has sha256sum, and macOS may have only shasum.
SUM=$(command -v sha256sum || echo "shasum -a 256")
echo "${SHA} ${WORK}/${ASSET}" | $SUM -c -
# box reads its packs from resources/ beside it, so keep the release together.
sudo mkdir -p /opt/box
sudo tar --no-same-owner -xzf "${WORK}/${ASSET}" -C /opt/box
sudo ln -sf /opt/box/box /usr/local/bin/box
box --version
Comment thread
mquang-amazon marked this conversation as resolved.
Resolved
# Ubuntu 24.04 lets AppArmor stop an unprivileged process from making a user namespace, which
# `box run` needs for its sandbox. The runner is a throwaway VM.
- name: Allow user namespaces for box
if: matrix.name == 'Linux-arm64'
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- name: Build CLI
if: inputs.npmVersion == ''
run: bun run build
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,8 @@ is needed to download it. When that release has no build for your machine, use t
checks of `build`.

`src/core/project/backends/boxRelease.json` pins the Linux build of box that ships to AgentCore
Runtime, with its repository, version and SHA-256. `bun scripts/bump-box-release.ts --box <version>`
Runtime, with its repository, version and SHA-256. It also pins the macOS arm64 build, which the e2e
workflow installs on its macOS runner. `bun scripts/bump-box-release.ts --box <version>`
(with `--box-repo` to move repository) rewrites the file from the release's `SHA256SUMS.txt`.

How each command uses it:
Expand Down
14 changes: 9 additions & 5 deletions e2eTest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@ The `box` tests cover the Strands Box templates, which run their agent in a box.
invoke each one on AgentCore Runtime, and run `agentcore dev` with one local invoke for each. They
are optional and need a `box` on your PATH: `create` checks for one for a CodeZip box template, and
`build` and `deploy` run `box profile show` for every Box template, the container one too. Without
one, they skip. `create` also installs the agent's dependencies and the CDK app's, which `dev` and
`deploy` use, so the tests don't pass `--skip-install`.
one, they skip. box ships only arm64 builds, so in CI only the macOS runner (Apple silicon) and the
Linux arm64 runner (`ubuntu-24.04-arm`) have one: each installs the box that
`src/core/project/backends/boxRelease.json` pins. `create` also installs the agent's dependencies
and the CDK app's, which `dev` and `deploy` use, so the tests don't pass `--skip-install`.

The tests that deploy also need `AGENTCORE_E2E_BOX_CDK`: the pinned `@aws/agentcore-cdk` release
cannot deploy a Box project yet. Set it to an `@aws/agentcore-cdk` version or a `file:` tarball
Expand All @@ -37,8 +39,9 @@ that can, for example one packed from the `box-integration` branch of
`agentcore/cdk`. Without it, as in CI, they skip.

The `dev` tests also need a machine where `box run` can start its sandbox, and for the container
template a container tool. Box listens on the protocol's fixed port (8080, 8000 or 9000), so the
`dev` tests run one at a time and need those ports free:
template a container tool. The container template's `dev` test skips when no `docker`, `podman`
or `finch` runs from PATH, as on the CI macOS runner; the Linux arm64 runner runs it. Box listens on the protocol's fixed port (8080,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this line runs to 133 columns while the rest of the paragraph wraps at 100. Could you rewrap it? One caveat: docker --version passes even when the engine is down, so on a Mac where Docker Desktop is installed but stopped, this test runs and fails instead of skipping. That's probably fine for a local run.

8000 or 9000), so the `dev` tests run one at a time and need those ports free:

```sh
AGENTCORE_E2E_BOX_CDK=file:/path/to/aws-agentcore-cdk-1.1.0.tgz bun run test:e2e -- --tagsFilter='box'
Expand All @@ -52,7 +55,8 @@ pins a shim for `PYTHON_3_14` only, so they fail on any machine.

The `box-egress` test checks that box, under `agentcore dev`, lets the agent reach the host its
profile allows (`checkip.amazonaws.com`) and blocks any other (`example.com`). It reaches public
hosts, so the network can make it fail. It retries twice, and `--tagsFilter='box'` leaves it out:
hosts, so the network can make it fail. It retries twice, and `--tagsFilter='box'` leaves it out.
A full run includes it, so in CI it runs on the macOS and Linux arm64 runners:

```sh
bun run test:e2e -- --tagsFilter='box-egress'
Expand Down
12 changes: 11 additions & 1 deletion e2eTest/project/templates.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ const AGUI_RUN_PAYLOAD = {
* deploy of every Box template run `box profile show`, so the Box tests skip without one.
*/
const BOX_ON_PATH = spawnSync("box", ["--version"], { stdio: "ignore" }).status === 0;

/** Whether a container tool that `agentcore dev` uses (docker, podman or finch) runs from PATH. */
const CONTAINER_TOOL_ON_PATH = ["docker", "podman", "finch"].some(
(tool) => spawnSync(tool, ["--version"], { stdio: "ignore" }).status === 0,
);
/**
* The opt-in for the Box tests that deploy: an `@aws/agentcore-cdk` version or `file:` tarball
* that deploys Box projects, which the pinned release does not yet. It replaces the pin in each
Expand Down Expand Up @@ -420,8 +425,13 @@ describe.skipIf(!BOX_ON_PATH)(
{ tags: [TAGS.BOX], sequential: true },
() => {
const cli = new CliRunner();
// The container template's dev builds and runs its image, so it needs a container tool, which
// the CI macOS runner does not have (no nested virtualization).
const templates = BOX_TEMPLATES.filter(
(t) => CONTAINER_TOOL_ON_PATH || t.template !== "agent-python-strands-box-container",
);

test.each(BOX_TEMPLATES)(
test.each(templates)(
"$name",
{ timeout: TIMEOUT_MS.PROJECT_CREATE + TIMEOUT_MS.PROJECT_DEV + TIMEOUT_MS.PROJECT_INVOKE },
async (runtime) => {
Expand Down
20 changes: 13 additions & 7 deletions scripts/bump-box-release.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
#!/usr/bin/env bun

/**
Pins box to a published GitHub release, in src/core/project/backends/boxRelease.json. It reads
the release's SHA256SUMS.txt, so the file changes only to a version whose archives are published.
It imports nothing from src/.
Pins box to a published GitHub release, in src/core/project/backends/boxRelease.json: the Linux
arm64 build that ships to AgentCore Runtime, and the macOS arm64 build that the e2e workflow
installs on its macOS runner. It reads the release's SHA256SUMS.txt, so the file changes only to a
version whose archives are published. It imports nothing from src/.

bun scripts/bump-box-release.ts --box <version> [--box-repo <owner/repo>]
**/
Expand All @@ -20,6 +21,7 @@ const FILE = join(
"boxRelease.json",
);
const LINUX_TARGET = "aarch64-unknown-linux-gnu";
const DARWIN_TARGET = "aarch64-apple-darwin";

const { values } = parseArgs({
args: Bun.argv.slice(2),
Expand Down Expand Up @@ -50,10 +52,14 @@ const pinned = await Bun.file(FILE).json();
const repo = values["box-repo"] ?? pinned.box.repo;
const version = values.box;
const sums = await checksums(repo, version);
const asset = `box-v${version}-${LINUX_TARGET}.tar.gz`;
const linux = sums.get(asset);
if (!linux) throw new Error(`${repo} v${version} lists no ${asset}`);
/** The pinned archive of `target`, which the release must list. */
function pin(target: string): { target: string; asset: string; sha256: string } {
const asset = `box-v${version}-${target}.tar.gz`;
const sha256 = sums.get(asset);
if (!sha256) throw new Error(`${repo} v${version} lists no ${asset}`);
return { target, asset, sha256 };
}
console.log(`box: ${pinned.box.repo} ${pinned.box.version} → ${repo} ${version}`);
const box = { repo, version, linux: { target: LINUX_TARGET, asset, sha256: linux } };
const box = { repo, version, linux: pin(LINUX_TARGET), darwin: pin(DARWIN_TARGET) };

await Bun.write(FILE, `${JSON.stringify({ box }, null, 2)}\n`);
5 changes: 5 additions & 0 deletions src/core/project/backends/boxRelease.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@
"target": "aarch64-unknown-linux-gnu",
"asset": "box-v0.1.11-preview.10-aarch64-unknown-linux-gnu.tar.gz",
"sha256": "45740d3ea768783b9a8939111ea0a26efd2dacf27ffa86104a4634ce1c2dc583"
},
"darwin": {
"target": "aarch64-apple-darwin",
"asset": "box-v0.1.11-preview.10-aarch64-apple-darwin.tar.gz",
"sha256": "76a3909c5c99c435b6ead470be2f401a22c6155b5b60652263bfb1af3ef57e18"
}
}
}
5 changes: 5 additions & 0 deletions src/core/project/backends/boxRelease.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,11 @@ describe("the pinned releases", () => {
`box-v${BOX_RELEASE.version}-aarch64-unknown-linux-gnu.tar.gz`,
);
expect(BOX_RELEASE.linux.sha256).toMatch(/^[0-9a-f]{64}$/);
// The e2e workflow installs the macOS build on its macOS runner, from this pin.
expect(BOX_RELEASE.darwin?.asset).toBe(
`box-v${BOX_RELEASE.version}-aarch64-apple-darwin.tar.gz`,
);
expect(BOX_RELEASE.darwin?.sha256).toMatch(/^[0-9a-f]{64}$/);
expect(BOX_RELEASE_PAGE).toBe(
`https://github.com/mquang-amazon/box-releases/releases/tag/v${BOX_RELEASE.version}`,
);
Expand Down
4 changes: 3 additions & 1 deletion src/core/project/backends/boxRelease.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,14 @@ const TargetAssetSchema = AssetSchema.extend({ target: z.string().min(1) });

/**
* A box release: the Linux arm64 build that ships to AgentCore Runtime. Its archive holds box,
* its helpers, its packs and release.json.
* its helpers, its packs and release.json. `darwin`, the macOS arm64 build, is what the e2e
* workflow installs on its macOS runner; the CLI does not read it.
*/
const BoxReleaseSchema = z.object({
repo: z.string().min(1),
version: z.string().min(1),
linux: TargetAssetSchema,
darwin: TargetAssetSchema.optional(),
Comment thread
mquang-amazon marked this conversation as resolved.
Resolved
});

export type BoxRelease = z.infer<typeof BoxReleaseSchema>;
Expand Down
Loading