Skip to content

feat(router): add async aroute and aroute_many to SemanticRouter - #784

Open
Yarmoluk wants to merge 2 commits into
redis:mainfrom
Yarmoluk:feat/router-async-query
Open

Yarmoluk wants to merge 2 commits into
redis:mainfrom
Yarmoluk:feat/router-async-query

Conversation

@Yarmoluk

@Yarmoluk Yarmoluk commented Oct 26, 2026 •

Copy link
Copy Markdown

Motivation

Closes #783.

SemanticRouter had no async query path, so routing from async apps (LangGraph nodes, FastAPI handlers, MCP servers) blocked the event loop twice per call: on the embedding and on the FT.AGGREGATE round trip. SemanticCache already has async lookups; the router had none.

Changes

aroute() and aroute_many()

Async counterparts of router(...) and route_many(...). They embed with vectorizer.aembed and run the same aggregate request on an AsyncSearchIndex over the same index, so they return exactly what the sync methods return. Everything after vector is keyword-only, so a positional distance_threshold (deprecated in route_many) can't be read as an aggregation method.

  • _get_route_matches is split into request building (_route_matches_request) and row processing (_route_matches_from_result), shared by the sync and async paths so the two cannot drift. _build_aggregate_request is untouched, so this does not collide with fix(router): apply per-route thresholds before max_k truncation #766.
  • The async index is created on first use (same pattern as SemanticCache), derived from the sync SearchIndex: when the sync index owns its client and has a URL, the async index connects from that URL, connection kwargs and lib_name; otherwise the caller's client is converted with RedisConnectionFactory.sync_to_async_redis. from_existing(redis_url=...) builds an owned client from the URL, so it takes the URL branch too.
  • adisconnect() closes the async connection; aindex exposes the async index (None before first use).
  • Mutating methods (add_route, remove_route, ...) stay sync to keep the change small.

Release Notes

SemanticRouter gains aroute(), aroute_many(), adisconnect() and an aindex property. The async index is created on first use. Routers given an SSL, unix-socket, Sentinel or RedisCluster client can't convert it to async; pass redis_url to the router for async routing in those setups. Vectorizers without a native aembed (e.g. the default HFTextVectorizer) still embed synchronously; the Redis call is non-blocking either way. The async client is bound to the event loop that made the first call; call adisconnect() before switching loops.

Reviewer Brief

Start here: redisvl/extensions/router/semantic.py, _get_async_index. It decides how the async index reaches Redis, and every aroute / aroute_many call goes through it.

Where it could fail

  • redisvl/extensions/router/semantic.py:557: the URL branch is taken only when the sync index owns its client and has a URL. Check that a router built with redis_client= plus redis_url= (both given) converts the client rather than connecting from the URL, since the client is what the sync path uses.
  • redisvl/extensions/router/semantic.py:567: the client-conversion branch raises on the first async call for SSL, unix-socket, Sentinel and Cluster clients (sync_to_async_redis limitation, pre-existing). The docstrings say to pass redis_url instead; nothing prevents the mismatch at construction.

Not verified: cluster mode (async path unsupported for RedisCluster clients); non-blocking embedding with a native-async vectorizer (the tests use HFTextVectorizer, which embeds synchronously, so only the Redis call is exercised async); Redis < 7 error mapping (shared code, unreachable on the test container).

Beyond CI: test_async_routing_matches_sync (tests/integration/test_semantic_router.py:822) fails with AssertionError: the async path made a blocking call when aroute is changed to call the sync embed, and passes on this branch; test_aroute_from_existing_url_connects_from_url passes with the async index derived from the sync one, so the earlier from_existing patch is no longer needed.

Since the last brief: 8a70416 applies the review: the async index is derived from self._index (no router-side copies of URL/kwargs/lib_name, no from_existing change), keyword-only arguments after vector, the three docstring notes, and the six async tests replaced by one that blocks the sync paths.

Reviewed 8a70416 · Claude Code (claude-fable-5-1) · lenses: correctness, tests. Ran: pytest tests/integration/test_semantic_router.py (42 passed, 1 skipped; Redis via the repo's testcontainers compose), pytest tests/unit -k "router or create_index" (50 passed), make check-types (120 files, no issues), make format.

🤖 Generated with Claude Code


Note

Low Risk
New async API reuses the existing FT.AGGREGATE flow and SemanticCache-style connection setup; main caveats are documented (cluster client conversion, event-loop binding), not new security or data paths.

Overview
Adds non-blocking routing for async apps via aroute() and aroute_many(), mirroring router() / route_many() by using vectorizer.aembed and a lazily created AsyncSearchIndex on the same Redis index.

The sync lookup path is refactored so request building (_route_matches_request), Redis error handling (_raise_aggregate_error), and result parsing (_route_matches_from_result) are shared with _aget_route_matches, reducing drift between sync and async. adisconnect() and an aindex property manage the optional async client (URL-based when the router owns its connection, otherwise sync_to_async_redis).

Integration tests assert async results match sync, that the async path does not call blocking embed/aggregate, and that routers loaded with from_existing(redis_url=...) wire async Redis from the URL.

Reviewed by Cursor Bugbot for commit 8a70416. Bugbot is set up for automated code reviews on this repo. Configure here.

@vishal-bala vishal-bala self-assigned this Oct 27, 2026
@vishal-bala
vishal-bala self-requested a review October 27, 2026 14:36
@vishal-bala vishal-bala added the auto:minor Increment the minor version when merged label Oct 27, 2026

@vishal-bala vishal-bala left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! That's a useful addition, glad to have it 😄
Just a few minor comments and we should be good to go

Comment thread redisvl/extensions/router/semantic.py Outdated
Comment thread redisvl/extensions/router/semantic.py Outdated
Comment thread redisvl/extensions/router/semantic.py
Comment thread tests/integration/test_semantic_router.py Outdated
Routing from async code (LangGraph nodes, FastAPI handlers, MCP servers)
blocked the event loop twice per call: once to embed, once on the
FT.AGGREGATE round trip. SemanticCache already has async lookups; the
router had none.

aroute() and aroute_many() embed with the vectorizer's aembed and run the
same aggregate request on a lazily created AsyncSearchIndex over the same
index, so they return what router() and route_many() return. Request
building and row processing are shared with the sync path. adisconnect()
closes the async connection. Mutating methods stay sync.
Address review on redis#784:
- _get_async_index reads the URL, connection kwargs, lib_name and client
  ownership from self._index instead of keeping copies on the router,
  which also makes the from_existing patch unnecessary.
- aroute/aroute_many take everything after vector keyword-only, so a
  positional distance_threshold can't be read as an aggregation method.
- Docstrings note that SSL, unix-socket, Sentinel and Cluster clients
  should pass redis_url for async routing, that vectorizers without a
  native aembed still block on embedding, and that adisconnect() must be
  called before switching event loops.
- Replace the six async tests with one that blocks the sync embed and
  sync aggregate paths, so it fails if aroute takes either.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto:minor Increment the minor version when merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add async routing methods to SemanticRouter (aroute / aroute_many)

2 participants