Skip to content

fix(vectorize): use Cohere's async client for aembed and aembed_many - #785

Open
Yarmoluk wants to merge 2 commits into
redis:mainfrom
Yarmoluk:fix/cohere-async-client
Open

Yarmoluk wants to merge 2 commits into
redis:mainfrom
Yarmoluk:fix/cohere-async-client

Conversation

@Yarmoluk

@Yarmoluk Yarmoluk commented Oct 27, 2026 •

Copy link
Copy Markdown

Motivation

CohereTextVectorizer only creates a sync cohere.Client, so aembed and aembed_many go through BaseVectorizer's fallback (base.py:350-364): they log "Falling back to sync" and make a blocking HTTP call on the event loop. Any async caller using Cohere, such as SemanticCache.acheck / astore or SemanticRouter's async path, stalls the loop for the length of each embedding request.

CohereReranker already creates a cohere.AsyncClient next to its sync client (rerank/cohere.py:84,99); this brings the vectorizer in line.

Changes

  • _initialize_client creates self._aclient = AsyncClient(...) alongside the sync client, with the same API key, client_name="redisvl" and kwargs.
  • New _aembed and _aembed_many call the async client. _aembed_many keeps the sync method's batching and @retry policy.
  • Request preparation (the input_type check, the embedding_types warning, mapping dtype to Cohere embedding types) and response parsing move into _prepare_request and _extract_embeddings. The sync and async paths share them, so the four methods can't drift apart.
  • A user-supplied httpx_client is synchronous and can't be awaited. When one is passed, the async client isn't created, and aembed / aembed_many keep using the sync fallback exactly as they do today. A new optional async_httpx_client kwarg gives the async client its own transport.
  • New tests/unit/test_cohere_vectorizer.py, using a stand-in cohere module (the same approach as test_ollama_vectorizer.py), so it runs in CI without an API key.

Notes

  • CohereReranker passes the same kwargs to both of its clients, so a sync httpx_client breaks its arank the same way. That's left out of this PR to keep it to the vectorizer; it's a small follow-up if you want it.
  • _aembed_many retries the whole batch loop on any non-TypeError error, re-sending batches that already succeeded. That matches the existing _embed_many, so changing the retry policy is left to a separate change that can cover both.

Release Notes

CohereTextVectorizer.aembed and aembed_many now make non-blocking calls through Cohere's async client instead of falling back to the sync client. If you pass your own httpx_client, async calls still use it through the sync fallback; pass async_httpx_client as well to make them non-blocking.

Reviewer Brief

Start here: redisvl/utils/vectorize/text/cohere.py, _initialize_client. It decides whether an async client exists, and that decides which path every aembed / aembed_many call takes.

Where it could fail

  • redisvl/utils/vectorize/text/cohere.py:156: the async client is skipped whenever httpx_client is passed without async_httpx_client. Check that this keeps today's behaviour for sync-transport users and doesn't silently drop transport settings, such as a proxy, on the async path.
  • redisvl/utils/vectorize/text/cohere.py:423: _aembed_many_with is the only async method wrapped in @retry. Check that TypeError from input validation still surfaces at once instead of being retried.

Not verified: live Cohere API calls. The async paths are exercised against a stand-in client, and tests/integration/test_vectorizers.py (test_vectorizer_aembed, test_vectorizer_aembed_many) needs COHERE_API_KEY, so it doesn't run in CI. Only cohere 7.0.0 was installed, so the cohere>=4.44 floor wasn't checked; CohereReranker already relies on AsyncClient(client_name=...) at the same floor.

Beyond CI: 7 of the 10 new unit tests fail against origin/main. The other 3 pin behaviour that already exists: the two input checks, and the sync fallback when httpx_client is passed. Against the real cohere 7.0.0 SDK, AsyncClient(httpx_client=httpx.Client()) fails on await with TypeError: object Response can't be used in 'await' expression. That's the case the httpx_client handling avoids, and the async client is confirmed absent in that configuration.

Reviewed df8d324, fixes in 5bb127a · Claude Code (claude-opus-5-5) · lenses: correctness, tests, design, docs

🤖 Generated with Claude Code


Note

Medium Risk
Touches embedding I/O used by async cache/router paths; custom httpx_client users must opt into async_httpx_client for non-blocking behavior, but default callers gain true async without API surface changes beyond the new kwarg.

Overview
CohereTextVectorizer now wires aembed / aembed_many to Cohere's AsyncClient instead of BaseVectorizer's sync fallback that blocked the event loop.

Initialization creates self._aclient alongside the sync client (same API key and kwargs). If callers pass only a sync httpx_client, the async client is skipped and async methods keep the existing sync fallback; optional async_httpx_client supplies a dedicated async transport.

Shared embed logic moves into _prepare_request, _extract_embeddings, and _validate_contents so sync and async paths stay aligned. tests/unit/test_cohere_vectorizer.py adds fake-cohere coverage for client setup, async routing, batching, dtype mapping, and httpx kwargs.

Reviewed by Cursor Bugbot for commit 5bb127a. Bugbot is set up for automated code reviews on this repo. Configure here.

CohereTextVectorizer only created a sync Client, so aembed and
aembed_many fell back to the blocking sync call (and logged a
fallback warning), stalling the event loop in async callers such
as SemanticCache.acheck. Create an AsyncClient alongside the sync
one, as CohereReranker already does, and implement _aembed and
_aembed_many on it. Shared request preparation and response
parsing move into helpers so the four paths stay in step.
…client

A user-supplied httpx_client is synchronous; handing it to AsyncClient
made aembed fail on await where it used to work through the sync
fallback. Only build the async client when no sync httpx_client is
given, or when async_httpx_client is passed for it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant