diff --git a/CHANGELOG.md b/CHANGELOG.md index 33f0294..0fe0440 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,24 +1,15 @@ # Changelog -## Product candidate (not published) - -- New projects index and search without embedding or reranker models. Existing index modes are preserved; explicit mode switches rebuild fully. Transformers and LanceDB are optional peers with actionable preflight errors. -- Setup detects supported coding tools and configures the selected tools together. It preserves existing settings, saves version-pinned launches for the selected repository, and leaves indexing to the first search. -- Added `get_code_relationships`: directed import/re-export paths up to three hops, current resolution and source checks, confined endpoints, unresolved/stale diagnostics, and a 4,000-byte response cap. Resolution selects the root's `tsconfig.json` or, when absent, `tsconfig.base.json`, without inheriting an unrelated parent configuration. -- AST chunks preserve contiguous source and static property ownership, including concise descriptor methods, callback bodies and same-line siblings. Generated scope labels remain presentation metadata. Index format 4 rebuilds incompatible indexes even when sources are unchanged, preserves the selected embedding mode, and retains previous artifacts if rebuilding fails. -- Angular TypeScript eligibility uses parsed Angular imports; ordinary `resolve` or `validate` methods, comments, strings and imports from other packages no longer select the Angular analyzer. Other eligible analyzers retain the source and declaration context. The registry order and existing template/style routing are unchanged. -- Compact `signaturePreview` uses a contiguous window of up to three lines around the strongest query-term match in the returned snippet, falling back to its opening lines. Default and explicit exploration searches omit edit preflight while retaining search quality and incomplete-source warnings. Explicit `edit`, `refactor`, and `migrate` intents retain mutation guards. -- Search excludes ordinary stale/unavailable nominees without discarding verified current lower results or triggering an automatic full rebuild. Partial coverage, bounded exhaustion and failed semantic replenishment are explicit; corrupt metadata, build identity, paths and source ranges still fail closed. -- Symbol references scan same-build manifest sources in bounded read batches and skip parsing when a simple ASCII symbol occurs only inside longer identifiers. Unicode or arbitrary query spellings retain raw-text nomination. Coverage keeps raw-text absence separate from parser work and uncertain lexical fallback. Keyword artifact identity uses the shared index validator. Graph traversal verifies edges before consuming output caps, with bounded inspection and honest continuation. -- Watcher shutdown awaits handle closure and cancels pending callbacks. Connected stdio sessions have no default inactivity shutdown; explicit positive idle timeouts remain supported alongside handshake, EOF and parent-death guards. -- Chunk paths are persisted relative to the selected project root, including explicit-root indexing from another working directory. Previously incorrect relative paths can require a full reindex. -- Query advice is limited to grounded returned source, and abstained searches omit global pattern advice. Same-named symbols in different files remain eligible after file-level deduplication; TypeScript type arguments alone no longer select the JSX analyzer. -- The candidate requires Node 22+. CLI dispatch compares canonical entrypoint identity, including npm symlink launchers, while preserving the exported server's initialization on imports. -- Updated the runtime parser and watcher to remove the legacy braces dependency chain. Watcher exclusions use exact directory names; quoted import and export names retain their public symbol names. -- Known limits: model-free conceptual relevance is not guaranteed. Short requests can miss implementations when query words differ from source names; syntactic edges are not runtime calls; workspace/language-specific resolution can remain unresolved. Packed/protocol tests and scoped negative cases do not establish general agent-quality improvement or customer adoption. No npm publication is implied by this entry. - -## [2.3.0](https://github.com/PatrickSys/codebase-context/compare/v2.2.0...v2.3.0) (2026-04-30) +## [2.3.0](https://github.com/PatrickSys/codebase-context/compare/v2.2.0...v2.3.0) (unreleased — TBD) +### Release notes + +- **CLI:** Fix npm **2.2.0** regression where `npx codebase-context --version`, `status`, and `reindex` could exit with code 0 and no output (direct CLI subcommands lazy-load the MCP runtime; canonical entrypoint detection includes npm symlink launchers). Requires Node 22+. +- **Indexing:** Fresh project folders default to keyword-only indexing without downloading embedding or reranker models; existing index modes are preserved; opt in with `reindex --embeddings` and optional Transformers/LanceDB peers. Explicit mode switches rebuild fully. +- **Setup:** `init` wizard detects Claude Code, Codex, Cursor, and OpenCode, configures selected tools together, preserves existing settings, and leaves indexing to the first search. +- **Search & index pipeline:** AST-aligned chunks, index format metadata with atomic rebuilds, exploration vs edit/refactor/migrate preflight behavior, bounded symbol references, and source-validation hardening (see Features below). +- **Docs:** Position around local team conventions and edit preflight; cite Aider repo map, Repomix, and jCodeMunch as related tools; report **25.7%** vs **jCodeMunch 27.1%** expected-file recovery per [docs/benchmark.md](./docs/benchmark.md) without leadership claims; keep **2.4.0** features out of shipped language until published. +- **Thanks:** Nick Roth ([@rothnic](https://github.com/rothnic)) for the Intel Mac embedding crash analysis ([#68](https://github.com/PatrickSys/codebase-context/issues/68)) and custom OpenAI-compatible endpoint requirements ([#70](https://github.com/PatrickSys/codebase-context/issues/70)) that shaped workaround guidance in this release. ### Features diff --git a/MOTIVATION.md b/MOTIVATION.md index c83fcd8..916498b 100644 --- a/MOTIVATION.md +++ b/MOTIVATION.md @@ -24,6 +24,9 @@ The root cause isn't the model. It's the context layer. Agents get raw code from | ---------------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------- | | `.cursorrules` / `AGENTS.md` | Static rules the agent may or may not always follow | Can't express migration states ("use A for new code, B for old"). Goes stale. | | Semantic search / RAG | Finds similar code | Blind to quality. Can't tell the best example to follow from hacky workarounds. | +| **Aider repo map** | Compact structural map for the model | No team pattern frequencies, memory, or edit preflight card. | +| **Repomix** | Pack repo (or slices) into a prompt | One-shot context, not live search, conventions, or session memory. | +| **jCodeMunch** | Fast MCP code search | Strong retrieval in our fixed benchmark; less convention/memory/edit gating. | | Linters / formatters | Enforce static rules after writing | Don't guide before or during generation. Agent still writes wrong code first. | | Tool-heavy MCPs (50+ tools) | Expose everything | Agents don't know which tool to call. More tools = more confusion. | diff --git a/README.md b/README.md index d93d577..c1337f6 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,13 @@ [![npm version](https://img.shields.io/npm/v/codebase-context)](https://www.npmjs.com/package/codebase-context) [![license](https://img.shields.io/npm/l/codebase-context)](./LICENSE) [![node](https://img.shields.io/node/v/codebase-context)](https://github.com/PatrickSys/codebase-context/blob/master/package.json) -Codebase Context gives coding agents a map of your repository through MCP, for developers who want their agent to find relevant code and follow the team's patterns. Search, examples and project memory run locally by default. +Codebase Context is a local MCP server for **team conventions and edit preflight**: a bounded map of how your repo is organized, ranked search with pattern and memory context, and a decision card before risky edits. It is not a retrieval leaderboard; frozen benchmarks put it at **25.7%** expected-file recovery versus **jCodeMunch 27.1%** in the same adapter run ([details](https://github.com/PatrickSys/codebase-context/blob/master/docs/benchmark.md)). Search, examples, and project memory stay on your machine by default. + +**Colleagues, not competitors:** [Aider's repo map](https://aider.chat/docs/repomap.html) for structure, [Repomix](https://repomix.com/) for pack-and-prompt workflows, and jCodeMunch for fast MCP code search (see [benchmark comparators](https://github.com/PatrickSys/codebase-context/blob/master/docs/benchmark.md)). Codebase Context combines map-first routing, convention detection, scoped memory, and edit-intent gating in one local tool surface. ## Set up your AI client -Use Node.js 22 or newer. These commands use the current npm release of Codebase Context: +Use Node.js 22 or newer. Unpinned `npx` installs the **current npm release (2.2.0 today)**; **2.3.0** is the pending release from this repository (Patrick publishes manually). Pin `@2.3.0` after it lands. npm **2.2.0** has a regression where `npx codebase-context --version`, `status`, and `reindex` can exit silently; **2.3.0** fixes that. ```bash # Claude Code @@ -19,23 +21,22 @@ codex mcp add codebase-context -- npx -y codebase-context opencode mcp add codebase-context '--' npx -y codebase-context ``` -In the unreleased candidate, `--version` prints the installed package version and `map` builds a missing index before printing the map. A low-confidence search includes a hint that weak matches do not establish that code is absent. These additions require the built candidate; the unpinned commands above install published 2.2.0. - -This repository's `init` command detects Claude Code, Codex, Cursor, and OpenCode and writes a project config for each selected tool. The saved launch is the Node binary and the package file from the npx cache. That command is in this source and is not part of published 2.2.0. Until it is published, use the copy-paste lines above. The [client setup guide](./docs/client-setup.md) marks the unpublished command. +**Ships in 2.3.0 (not on npm until published):** `--version`, non-silent `status` / `reindex`, `init` for supported clients, fresh indexes without a model download (use `reindex --embeddings` when you want local semantic search), `map` that builds a missing index first, and low-confidence search hints that weak matches do not prove absence. Until 2.3.0 is on npm, use the manual MCP lines below or install from a built checkout / packed tarball. Start a new agent session in your project, then ask: > Use Codebase Context to find [feature] in this repository. Pass this repository's absolute path as project, wait for indexing if needed, and open the relevant source files. -Replace `[feature]` with something you want to find. The first run downloads a local model and indexes the repository; time depends on repository size, CPU and model. +Replace `[feature]` with something you want to find. The first index pass scans the repository locally; **2.3.0** uses keyword search by default (no embedding model download). npm **2.2.0** still downloads a local embedding model on first index unless you change providers. Duration depends on repository size and CPU; optional semantic search adds model download time when enabled. -### Planned for 2.4.0 (not yet published) +### Planned for 2.4.0 (not in 2.3.0) -The unreleased source candidate starts new indexes without a model download. Semantic embeddings are optional; existing indexes keep their recorded mode. Its setup wizard can configure Claude Code, Codex, Cursor and OpenCode using the Node binary and package file that ran setup. These changes are not in the current npm release. +Do not treat these as shipped until 2.4.0 is published: -- NestJS support from community contributor Aaron Olin. See the [analyzer details](./docs/capabilities.md#analyzers). -- Better React/Next.js detection and a fix for reranker passage scoring. -- Transformers.js v4 for local models. +- NestJS analyzer from community contributor Aaron Olin ([analyzer details](./docs/capabilities.md#analyzers)). +- React/Next.js detection improvements and reranker passage-scoring fix. +- Transformers.js v4 for optional local embeddings. +- Per-project `parsing.maxChunks` in config and packaged `codebase-context-review` polish called out in [review-context](./docs/review-context.md). Known limits: [Intel Mac indexing crash (#68)](https://github.com/PatrickSys/codebase-context/issues/68), [custom OpenAI endpoints (#70)](https://github.com/PatrickSys/codebase-context/issues/70) and [Ollama (#71)](https://github.com/PatrickSys/codebase-context/issues/71) remain open. @@ -76,7 +77,7 @@ The default connection is `stdio` (standard input/output): your client starts th ### Team patterns and examples -`get_team_patterns` shows the approaches used in the repository and points to representative files. Dedicated analyzers for Angular, React, and Next.js ship in published 2.2.0; NestJS support (along with React and Next.js detection improvements and a reranker fix) is planned for unpublished 2.4.0. +`get_team_patterns` shows the approaches used in the repository and points to representative files. Angular, React, and Next.js analyzers ship today on npm 2.2.0 and remain in 2.3.0. NestJS and the 2.4.0 analyzer/reranker work above are not published yet. ### Project memory @@ -88,11 +89,11 @@ The `codebase-context-review` CLI creates a bounded context packet from a commit ## How it works -1. **Index locally.** Codebase Context scans the project, builds a keyword index, and creates local semantic embeddings - numeric representations used to match code by meaning as well as exact words. +1. **Index locally.** Codebase Context scans the project and builds a keyword index; opt into local semantic embeddings with `reindex --embeddings` when you need meaning-based search (npm **2.2.0** still pulled embeddings on first index). 2. **Understand the repository.** The agent can request a compact codebase map with structure, patterns, and representative files. 3. **Find the code for the task.** Search returns ranked files and symbols; the agent reads the selected files before editing. -The same information is available from the terminal. Run these commands from your project root. The first run downloads a local model and indexes the repository; time depends on repository size, CPU and model. Once it is ready, inspect the map and search for the code you need: +The same information is available from the terminal. Run these commands from your project root. **2.3.0** can index without a model download; semantic search is optional. Once the index is ready, inspect the map and search: ```bash # Build or refresh the local index @@ -161,8 +162,8 @@ The method and failures are documented so the measurements can be inspected with - Retrieval measurements describe expected-file coverage, file precision, and reported `peakPrivateGb`, not patch correctness or end-to-end coding quality. - The paired token observation covers two frozen investigation tasks and records observed agent behavior; it is not a universal token or time guarantee. - Setup checks differ by client. The detailed guide distinguishes a written config, a config recognized by the client, a local connection, and instructions checked only against official docs. -- Dedicated Angular, React, and Next.js analyzers ship in published 2.2.0; NestJS support is planned for unpublished 2.4.0. Other projects use the generic analyzer and the language parsers available for that stack. -- The default searchable-chunk limit is 5,000 per project. Customizing this via `projects[].parsing.maxChunks` in `~/.codebase-context/config.json` requires 2.4.0 (not yet published). +- Angular, React, and Next.js analyzers are on npm 2.2.0; NestJS lands in 2.4.0. Other stacks use the generic analyzer and available parsers. +- Default searchable-chunk cap is 5,000 per project. `projects[].parsing.maxChunks` in `~/.codebase-context/config.json` requires 2.4.0 (not yet published). - The agent must identify its repository in tool calls when no default or unambiguous client root is available. Concurrent HTTP client isolation is not established by the stdio routing checks. ## Reference diff --git a/docs/capabilities.md b/docs/capabilities.md index 49fc4f9..0e52321 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -1,6 +1,6 @@ # Capabilities Reference -Unpinned `npx codebase-context` commands install the published 2.2.0 package. Sections marked as unreleased describe the source candidate and require a built checkout or a local candidate installation. +Unpinned `npx codebase-context` installs **npm 2.2.0 today**. **2.3.0** is the pending release from this repo; **2.4.0** is not published. Sections marked **2.4.0** or **planned** describe future work only. Start with the repository map, then search for relevant code. For an overview, see [README.md](../README.md). Features marked as planned for 2.4.0 are not yet available from npm. @@ -65,7 +65,7 @@ npx -y codebase-context reindex --incremental ## Tool Surface -10 MCP tools + active/project-scoped context resources. +10 MCP tools plus project-scoped context resources in published 2.2.0/2.3.0. **`get_code_relationships`** and related navigation improvements are documented below for the in-repo build; treat them as **2.4.0** until that release is on npm unless you install from source. Shared selector inputs: @@ -317,6 +317,6 @@ Reproducible evaluation is shipped as a CLI entrypoint backed by shared scoring - **Symbol refs are not a call-graph.** `get_symbol_references` counts identifier-node occurrences in the AST (comments/strings excluded via Tree-sitter). It does not distinguish call sites from type annotations, variable assignments, or imports. Full call-site-specific analysis (`call_expression` nodes only) is a roadmap item. - **Impact is 2-hop max.** `computeImpactCandidates` walks direct importers then their importers. Full BFS reachability is on the roadmap. -- **Planned for unpublished 2.4.0:** NestJS analyzer support, React and Next.js detection improvements, and reranker fixes. Dedicated Angular, React, and Next.js analyzers already ship in published 2.2.0. Other languages use the Generic analyzer for indexing and retrieval; available symbol extraction depends on parser support. File import resolution and multi-hop navigation currently support TypeScript and JavaScript. Python and Go import navigation are not supported. -- **The unreleased candidate defaults new indexes to model-free mode. Published 2.2.0 enables embeddings by default.** When semantic mode is enabled, its default model is `bge-small-en-v1.5` (512-token context). Granite (8192 context) is opt-in via `EMBEDDING_MODEL`. OpenAI is opt-in via `EMBEDDING_PROVIDER=openai` — sends code externally. +- **Planned for 2.4.0 (not on npm):** NestJS analyzer, React/Next.js detection improvements, reranker fixes, optional `maxChunks` config, and full `get_code_relationships` / review-CLI publication called out in [review-context](./review-context.md). Angular, React, and Next.js analyzers ship on npm 2.2.0 and remain in 2.3.0. +- **Indexing defaults in 2.3.0:** new project folders index keyword-only until you run `reindex --embeddings` (optional `@huggingface/transformers` + `@lancedb/lancedb` peers). npm **2.2.0** still pulled embeddings on first index. When semantic mode is enabled, default model is `bge-small-en-v1.5` (512-token context). Granite (8192 context) is opt-in via `EMBEDDING_MODEL`. OpenAI is opt-in via `EMBEDDING_PROVIDER=openai` — sends code externally. - **Patterns are file-level frequency counts.** Not semantic clustering. Rising/Declining trend is derived from git commit recency for files using each pattern, not from usage semantics. diff --git a/docs/cli.md b/docs/cli.md index 9ae65f6..7d7f443 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -46,11 +46,11 @@ CODEBASE_CONTEXT_ASCII=1 npx -y codebase-context patterns - `reindex` — rebuild index (full or incremental) - `style-guide` — find style guide sections in docs - `memory list|add|remove` — manage team memory (stored in `.codebase-context/memory.json`) -- MCP client registration — use the recipes in the [client setup guide](./client-setup.md). Published 2.2.0 has the older interactive HTTP wizard; the four-client wizard below requires the unreleased candidate. +- MCP client registration — use the [client setup guide](./client-setup.md). npm **2.2.0** ships the older HTTP-only wizard path; the multi-client `init` flow ships in **2.3.0** (pending npm publish). -## `init` in the unreleased candidate +## `init` (2.3.0 — pending npm publish) -`init` detects Claude Code, Codex, Cursor and OpenCode, then configures the selected tools together. In a terminal, detected tools start selected; use Space to change the selection and Enter to apply it. Without a terminal, it configures all detected tools. +`init` detects Claude Code, Codex, Cursor, and OpenCode, then configures the selected tools together. In a terminal, detected tools start selected; use Space to change the selection and Enter to apply it. Without a terminal, it configures all detected tools. Not available in npm **2.2.0**. ```powershell codebase-context init --help @@ -59,15 +59,15 @@ codebase-context init --yes codebase-context init --client codex --client cursor --root "C:\path\to\project" ``` -Use repeatable `--client` options to choose tools explicitly. `--yes` configures every detected tool when no clients are named. `--root` selects another repository; otherwise `CODEBASE_ROOT` or the current directory is used. Setup writes client registrations but does not build an index or change agent instructions. The first search creates the index. The multi-tool flow is not in published npm `2.2.0` yet; see the [main setup guide](../README.md#set-up-your-ai-client) for its publication status and the [client setup guide](./client-setup.md) for current manual recipes. +Use repeatable `--client` options to choose tools explicitly. `--yes` configures every detected tool when no clients are named. `--root` selects another repository; otherwise `CODEBASE_ROOT` or the current directory is used. Setup writes client registrations but does not build an index or change agent instructions. The first search creates the index. Until **2.3.0** is on npm, use the manual recipes in the [client setup guide](./client-setup.md). -The built candidate also supports `--version` and builds a missing index when you run `map`. New indexes use keyword search without a model download; `reindex --embeddings` explicitly enables local semantic search. Preview, reindex, and map apply the same project exclusions and analyzer hints. A low-confidence search result means the matches are weak; it does not establish that code is absent. +**Also in 2.3.0:** `--version`; non-silent `status` / `reindex` (fixes npm 2.2.0 silent exit); fresh indexes without a model download; `map` builds a missing index first; low-confidence search hints. Use `reindex --embeddings` for local semantic search. --- ## First use -Run these CLI commands from the project root. Build or refresh the local index first; the first run downloads a local model and indexes the repository; time depends on repository size, CPU and model. Then inspect the conventions map and search for the code you need: +Run these CLI commands from the project root. **2.3.0** indexes keyword-only on first run (no model download); add `reindex --embeddings` when you want local semantic search. npm **2.2.0** still downloaded a model on first index. ```bash npx -y codebase-context reindex diff --git a/docs/client-setup.md b/docs/client-setup.md index f43ab54..c404d56 100644 --- a/docs/client-setup.md +++ b/docs/client-setup.md @@ -1,6 +1,6 @@ # Client Setup -These commands use the current npm release because the package name is unpinned. The current published version is 2.2.0; 2.4.0 is not yet published. +These commands use the current npm release (**2.2.0** on npm today; **2.3.0** pending). **2.4.0** is not published. Historical verification notes below reference **2.2.0** unless another version is named. Codebase Context runs as a local MCP server: a tool your AI editor or command-line client can call while it works. Register it without a fixed folder, then let the agent supply its current repository through the `project` tool parameter. @@ -27,7 +27,7 @@ After registration, start a new agent session in your project and ask: > Use Codebase Context to find [feature] in this repository. Pass this repository's absolute path as project when checking get_indexing_status and searching. Wait for indexing if needed, read codebase://context, then search_codebase and open a returned source file. Show me the relevant files. -The first run downloads a local model and indexes the repository; time depends on repository size, CPU and model. You can prepare it separately with `npx -y codebase-context reindex` from the project folder. Index readiness, client connection and useful agent behavior are separate checks. +The first index pass scans the repository locally. Pending **2.3.0** uses keyword search by default (no model download); npm **2.2.0** still downloads embedding models on first index. You can prepare the index separately with `npx -y codebase-context reindex` (add `--no-embeddings` on 2.3.0, or `--embeddings` when you want semantic search). Index readiness, client connection, and useful agent behavior are separate checks. ## Claude Code diff --git a/docs/comparison-table.md b/docs/comparison-table.md index 1b55f11..96b11ba 100644 --- a/docs/comparison-table.md +++ b/docs/comparison-table.md @@ -1,6 +1,6 @@ # Corrected retrieval comparison -This table summarizes the corrected retrieval-only family across five local code-context tools. In this fixed setup, Codebase Context recovered 25.7% of expected gold files at 11.5% file precision; this does not show better completed coding tasks. Expected-file coverage means expected gold files recovered divided by expected gold files. File precision means expected gold files recovered divided by all files returned. The reported `peakPrivateGb` field is copied from retained telemetry; its underlying unit was not independently verified. +This table summarizes the corrected retrieval-only family across five local code-context tools. **It is not a product ranking:** Codebase Context optimizes for team conventions and edit preflight; jCodeMunch leads this fixed expected-file metric at 27.1% versus Codebase Context at 25.7%. Expected-file coverage means expected gold files recovered divided by expected gold files. File precision means expected gold files recovered divided by all files returned. The reported `peakPrivateGb` field is copied from retained telemetry; its underlying unit was not independently verified. | Tool | Completed | Failed | Expected-file coverage | File precision | Reported `peakPrivateGb` | | --- | ---: | ---: | ---: | ---: | ---: |