Repository navigation
fix: improve CLI examples and refresh API to v4.83.0 - #35
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
| "version": "4.83.0", | ||
| "ref": "v4.83.0", | ||
| "commit": "fe82bdc24d3bbaac919b20d8fe8708920a9a81a8", | ||
| "sha256": "cdaac03b07396353ffb7e5870285014f7aa3c35d342295987ebd7d81a2e36b19", |
There was a problem hiding this comment.
🔴 Self-hosted Langfuse 4.71.0–4.82.x users on --api-version auto now get the 4.35.0 contract, which sends single-object score bodies to servers that expect an array. The 4.71.0 entry was overwritten rather than kept. compatibleEntry() picks the newest bundled snapshot at or below the server version, and for these servers that is now 4.35.0. Fix: add 4.83.0 next to 4.71.0 instead of replacing it, as conformance:add-version does, so 4.71–4.82 servers still resolve to 4.71.0. If dropping snapshots is intended, document the gap and make auto warn when it falls back across it.
Why this was flagged
A self-hosted server at, say, 4.75.0 reports that version from /api/public/health. With --api-version auto, compatibleEntry() at src/contracts/loader.ts:100-112 keeps major-4 entries at or below 4.75.0. That set is now {4.10.0, 4.35.0}, so it picks 4.35.0. In the 4.35.0 spec, scores_create takes $ref: '#/components/schemas/CreateScoreRequest', a single object (openapi.yml:6752). The removed 4.71.0 spec used CreateScoresRequest with type: array, the same shape as 4.83.0. The request then goes out in a shape their server's contract no longer describes. On the base branch, these servers resolved to 4.71.0, whose contract matched. Scripts pinned to --api-version 4.71.0 now fail at src/contracts/loader.ts:157 with Unknown API version 4.71.0. The repo's own conformance:add-version tool (conformance/src/add-version.ts:80) appends new entries and never removes old ones, so it does not guard against this manual replacement.
Verification: The diff at conformance/catalog.json:51-54 swaps the 4.71.0 entry for 4.83.0 in place instead of adding a new one. compatibleEntry (src/contracts/loader.ts ~100-112) keeps same-major entries where compareVersion(entry.version, serverVersion) <= 0 and returns the newest of them. For a 4.75.0 server that is now 4.35.0. On the base it was 4.71.0.
| | Self-hosted | Your deployment URL, e.g. `http://localhost:3000` | | ||
|
|
||
| Use the deployment base URL, without `/api/public` or a project page path. | ||
| Load the file explicitly; the CLI does not automatically read `.env`: |
There was a problem hiding this comment.
🟡 (optional) Users following the recommended bunx --bun @ langfuse/cli invocation can still get .env credentials loaded silently, which contradicts the new docs. README.md:54 says "the CLI does not automatically read .env". The help line at src/cli.ts:207 says "no automatic .env loading". Both hold only under Node. Under the Bun runtime, Bun loads .env/.env.local from the current directory into process.env, and runtimeConfig then reads LANGFUSE_* from there. Fix: qualify both statements so they say Bun itself auto-loads .env from the working directory when run with --bun, or drop the absolute claim.
Why this was flagged
bin/langfuse.mjs uses a #!/usr/bin/env node shebang. bunx --bun (recommended at README.md:13, :31 and :59) forces the Bun runtime, and Bun loads .env, .env.local and similar files from the current directory before user code runs. runtimeConfig at src/cli.ts:126 builds env = { ...process.env, ...fileEnv } and takes LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_BASE_URL/LANGFUSE_HOST from it. Take a user who runs bunx --bun @ langfuse/cli api ... without --env in a directory that has a .env for another project or deployment. Their requests are sent with those keys to that host, while the new README text and the --env help line (src/cli.ts:207) both promise that nothing is loaded automatically. On the base branch the README made no such promise and did not recommend bunx --bun.
Verification: src/cli.ts:126 builds const env = { ...process.env, ...fileEnv };. The CLI's own code does no .env loading, but it trusts whatever is already in process.env. Bun loads .env, .env.<NODE_ENV> and .env.local from the current directory into process.env before user code runs. Nothing in the repo turns it off: there is no bunfig.toml. The runtime behavior itself is the same on the base branch.
Score creation help and missing-body errors now show a valid JSON example for the selected API snapshot. The README adds numeric, boolean, categorical, and file-based score examples, and clarifies deployment hosts, explicit
.envloading, configuration precedence, and the package name versus executable name.Replace the latest bundled snapshot,
4.71.0, with immutable upstream4.83.0, keeping seven snapshots. Refresh the command goldens and operation counts. The new snapshot adds four Topics read operations, blob-storage IAM role fields, and prompt-label removal.Validation (typecheck, tests, and conformance rerun in a clean worktree):
bun run typecheckbun test: 127 passing tests, including score examples against all seven source schemasbun run conformance:all: build and 716/716 mocked API operationsCompanion skill documentation: langfuse/skills#133.