Repository navigation
Conversation
CohereTextVectorizer only created a sync Client, so aembed and aembed_many fell back to the blocking sync call (and logged a fallback warning), stalling the event loop in async callers such as SemanticCache.acheck. Create an AsyncClient alongside the sync one, as CohereReranker already does, and implement _aembed and _aembed_many on it. Shared request preparation and response parsing move into helpers so the four paths stay in step.
…client A user-supplied httpx_client is synchronous; handing it to AsyncClient made aembed fail on await where it used to work through the sync fallback. Only build the async client when no sync httpx_client is given, or when async_httpx_client is passed for it.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
CohereTextVectorizeronly creates a synccohere.Client, soaembedandaembed_manygo throughBaseVectorizer's fallback (base.py:350-364): they log "Falling back to sync" and make a blocking HTTP call on the event loop. Any async caller using Cohere, such asSemanticCache.acheck/astoreorSemanticRouter's async path, stalls the loop for the length of each embedding request.CohereRerankeralready creates acohere.AsyncClientnext to its sync client (rerank/cohere.py:84,99); this brings the vectorizer in line.Changes
_initialize_clientcreatesself._aclient = AsyncClient(...)alongside the sync client, with the same API key,client_name="redisvl"and kwargs._aembedand_aembed_manycall the async client._aembed_manykeeps the sync method's batching and@retrypolicy.input_typecheck, theembedding_typeswarning, mappingdtypeto Cohere embedding types) and response parsing move into_prepare_requestand_extract_embeddings. The sync and async paths share them, so the four methods can't drift apart.httpx_clientis synchronous and can't be awaited. When one is passed, the async client isn't created, andaembed/aembed_manykeep using the sync fallback exactly as they do today. A new optionalasync_httpx_clientkwarg gives the async client its own transport.tests/unit/test_cohere_vectorizer.py, using a stand-incoheremodule (the same approach astest_ollama_vectorizer.py), so it runs in CI without an API key.Notes
CohereRerankerpasses the same kwargs to both of its clients, so a synchttpx_clientbreaks itsarankthe same way. That's left out of this PR to keep it to the vectorizer; it's a small follow-up if you want it._aembed_manyretries the whole batch loop on any non-TypeErrorerror, re-sending batches that already succeeded. That matches the existing_embed_many, so changing the retry policy is left to a separate change that can cover both.Release Notes
CohereTextVectorizer.aembedandaembed_manynow make non-blocking calls through Cohere's async client instead of falling back to the sync client. If you pass your ownhttpx_client, async calls still use it through the sync fallback; passasync_httpx_clientas well to make them non-blocking.Reviewer Brief
Start here:
redisvl/utils/vectorize/text/cohere.py,_initialize_client. It decides whether an async client exists, and that decides which path everyaembed/aembed_manycall takes.Where it could fail
redisvl/utils/vectorize/text/cohere.py:156: the async client is skipped wheneverhttpx_clientis passed withoutasync_httpx_client. Check that this keeps today's behaviour for sync-transport users and doesn't silently drop transport settings, such as a proxy, on the async path.redisvl/utils/vectorize/text/cohere.py:423:_aembed_many_withis the only async method wrapped in@retry. Check thatTypeErrorfrom input validation still surfaces at once instead of being retried.Not verified: live Cohere API calls. The async paths are exercised against a stand-in client, and
tests/integration/test_vectorizers.py(test_vectorizer_aembed,test_vectorizer_aembed_many) needsCOHERE_API_KEY, so it doesn't run in CI. Only cohere 7.0.0 was installed, so thecohere>=4.44floor wasn't checked;CohereRerankeralready relies onAsyncClient(client_name=...)at the same floor.Beyond CI: 7 of the 10 new unit tests fail against
origin/main. The other 3 pin behaviour that already exists: the two input checks, and the sync fallback whenhttpx_clientis passed. Against the real cohere 7.0.0 SDK,AsyncClient(httpx_client=httpx.Client())fails onawaitwithTypeError: object Response can't be used in 'await' expression. That's the case thehttpx_clienthandling avoids, and the async client is confirmed absent in that configuration.Reviewed
df8d324, fixes in5bb127a· Claude Code (claude-opus-5-5) · lenses: correctness, tests, design, docs🤖 Generated with Claude Code
Note
Medium Risk
Touches embedding I/O used by async cache/router paths; custom
httpx_clientusers must opt intoasync_httpx_clientfor non-blocking behavior, but default callers gain true async without API surface changes beyond the new kwarg.Overview
CohereTextVectorizernow wiresaembed/aembed_manyto Cohere'sAsyncClientinstead ofBaseVectorizer's sync fallback that blocked the event loop.Initialization creates
self._aclientalongside the sync client (same API key and kwargs). If callers pass only a synchttpx_client, the async client is skipped and async methods keep the existing sync fallback; optionalasync_httpx_clientsupplies a dedicated async transport.Shared embed logic moves into
_prepare_request,_extract_embeddings, and_validate_contentsso sync and async paths stay aligned.tests/unit/test_cohere_vectorizer.pyadds fake-coherecoverage for client setup, async routing, batching, dtype mapping, and httpx kwargs.Reviewed by Cursor Bugbot for commit 5bb127a. Bugbot is set up for automated code reviews on this repo. Configure here.