[Kuang-xianxin]

Closes #713
Closes #714

On RESP3 connections, migration key enumeration reads byte-keyed FT.INFO maps as string-keyed dictionaries, treating failed or incomplete indexes as ready. Healthy indexes then hit a KeyError when the aggregate cursor's result map is sliced as a RESP2 row list.

The missing prefix also has a data-safety consequence: with RESP3 and decode_responses=False, if FT.AGGREGATE raises ResponseError, the SCAN fallback can use * instead of the source index's prefix. A field-rename migration can then rewrite a same-named field on unrelated applications' keys. This cross-prefix rewrite was reproduced on Redis 8.2.7 during review.

Changes:

• Normalize FT.INFO before checking readiness and extracting SCAN prefixes, keeping fallback enumeration scoped to the source index.
• Preserve a real zero for percent_indexed in enumeration, both executors' apply() guards, and the planner. Schema-only migrations of zero-progress indexes enumerate the stored keys and pass their count to validation; both sync and async planning emit the still-building warning.
• Share aggregate key extraction between sync and async executors, handling RESP2 rows and RESP3 result maps while retaining cursor pagination and cleanup.

Validation (current follow-up):

• Replayed the retained 22 enumeration cases against unmodified main 31d2d0a: 12 failed and 10 passed, covering RESP3 parsing, byte-keyed prefixes, zero progress and cursor cleanup. The new helper import/test was omitted only for this baseline run because main has no helper.
• The async-planner warning regression fails at zero progress on the previous PR head 0866fa8, while its fully indexed control passes. Both pass with this follow-up.
• On this branch, 124 related enumeration, executor, planner, cluster-scan and batch-migration mock unit tests pass with Python 3.13.14 and redis-py 8.1.0. Run with pytest --confcutdir=tests/unit to exclude the root autouse Docker fixture.
• Full source mypy passes (120 files); source/test Black and isort checks and scoped codespell pass.
• The full pre-commit attempt could not run its Bash/Make hook on this Windows environment; its component commands were run directly. Repository-wide codespell also flagged an unchanged comment in redisvl/redis/connection.py; that unrelated auto-edit is excluded.
• Redis-backed integration and cluster execution were not run locally because Docker is unavailable. The Redis 8.2.7 reproduction linked above is the reviewer's evidence.

Release Notes: Fix RESP3 migration enumeration and preserve source-index prefixes during SCAN fallback; unfinished indexes with zero progress remain on the complete-enumeration path.

Suggested release label: auto:patch.

---

Note

High Risk
Changes core migration enumeration and SCAN scoping; incorrect readiness or prefix handling can skip documents or mutate unrelated keys during renames/quantization.

Overview
Fixes migration document enumeration on RESP3 connections where byte-keyed FT.INFO and aggregate cursor payloads were misread, which could treat unfinished indexes as ready, crash on aggregate parsing, or widen SCAN fallback to * and touch keys outside the source index prefix.

Normalization and parsing: FT.INFO is passed through convert_bytes before readiness checks and prefix extraction; parse_percent_indexed centralizes percent_indexed handling so real 0 progress (RESP2 string or RESP3 float) still triggers complete enumeration and planner/executor warnings, while only a missing value defaults to fully indexed.

Shared aggregate extraction: Sync and async executors use _extract_aggregate_keys for RESP2 row lists and RESP3 result maps, with cursor pagination and cleanup unchanged.

Planning parity: The async planner now emits the same “still building” warning as the sync planner when percent_indexed < 1.0.

Unit tests cover wire-format enumeration, incomplete-index SCAN fallback, and async planner warnings.

^{Reviewed by Cursor Bugbot for commit 52e2370. Bugbot is set up for automated code reviews on this repo. Configure here.}

Reviewer Brief

Start here: redisvl/migration/executor.py, _enumerate_indexed_keys and _extract_aggregate_keys: readiness, source-prefix fallback and RESP2/RESP3 cursor parsing determine which documents are migrated; the async executor shares the parsing helpers.

Where it could fail

• Verify the six parse_percent_indexed call sites preserve numeric zero while defaulting only missing/None values, including warnings in both planners; FT.INFO defines 1 as 100%.

Not verified: live Redis Search migrations, Redis Cluster execution, and older Python/redis-py versions in these follow-ups. Full pre-commit has the Windows Bash/Make and baseline spelling limits noted above.

Beyond CI: on Oct 28, the retained enumeration regression suite failed 12 cases on main 31d2d0a. The new async-planner regression fails on previous head 0866fa8 at zero progress; its fully indexed control passes.

_{Reviewed 52e2370 · Codex (GPT-6) · lenses: correctness, security, design, tests, docs, Redis behaviour}

[vishal-bala]

Thanks for your contribution! This looks good, but it looks like there are a couple more spots where the fix should be applied and a couple details that could be clarified.

[Kuang-xianxin]

Hi @vishal-bala, following up on affa15a: both apply guards and the planner now preserve numeric zero progress, and the remaining description, terminology, and test-cleanup points are addressed in the inline replies. The September 11 validation recorded 162 related passing tests on each of redis-py 8.1.0 and 6.3.0.

The head is unchanged, and Lint and Test Suite still require workflow approval. Could you take another look at the revisions, and could a maintainer approve the pending runs? Thanks!

[vishal-bala]

Thanks for the thorough follow-up, this all looks right! One last thing to keep the tests lean, and then this is good to go.

[vishal-bala]

Also, can you rebase on main and follow the latest pre-PR review process a well?

[Kuang-xianxin]

Updated through main 31d2d0a and pushed 0866fa8. I merged main to preserve the existing review history rather than force-pushing a rebase. The helper/test simplification is addressed in the inline threads.

I also completed the current pre-PR review process and added the Reviewer Brief. Validation: 122 related mock unit tests, full source mypy, Black/isort and scoped codespell pass. The PR description records the Windows Bash/Make hook limitation, the unrelated baseline spelling issue, and the live Redis Search/cluster coverage that was not run locally.

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 0866fa8. Configure here.}

[vishal-bala]

LGTM!

[applied-ai-release-bot]

🚀 PR was released in v0.29.0 🚀