Repository navigation
feat(search): add Keenable as an opt-in keyless search provider - #347
Open
ilya-bogin-keenable wants to merge 3 commits into
Open
ilya-bogin-keenable wants to merge 3 commits into
ilya-bogin-keenable wants to merge 3 commits into
Conversation
Add a KeenableSearchProvider that is registered only when SEARCH_PROVIDER=keenable or KEENABLE_API_KEY is set. Without a key it calls the keyless /v1/search/public endpoint; a key switches to /v1/search and only raises rate limits. Default provider and the existing fallback order are unchanged. Signed-off-by: Ilya Bogin <ilya.bogin@keenable.ai>
Signed-off-by: Ilya Bogin <ilya.bogin@keenable.ai>
Signed-off-by: Ilya Bogin <ilya.bogin@keenable.ai>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Adds Keenable as an optional web search provider in
backend_api_python/app/services/search.py, next to GDELT and DuckDuckGo as a free option. It needs no API key: setSEARCH_PROVIDER=keenable(it is added to the setting's options inroutes/settings.py) and AI analysis and the research agent get web results with page text snippets and publish dates. An optionalKEENABLE_API_KEYonly raises the rate limits. Tavily stays the default and the existing fallback order is unchanged.Related issue
requestslike the other REST providers), no route, schema or migration change.Changes
KeenableSearchProvider(BaseSearchProvider):POST https://api.keenable.ai/v1/search/publicwithout a key,POST https://api.keenable.ai/v1/searchwithX-API-Keywhen a key is set. Results maptitle,url,snippet(whitespace collapsed, capped at 500 characters like Tavily/SerpAPI/SearXNG) andpublished_at.days <= 30becomespublished_after(Google CSE appliesdateRestrictup to the same 30-day cutoff); longer windows such assearch_researchsend no date filter. HTTP errors and empty results returnsuccess=False, so the chain moves on to the next provider.SearchService._init_providers: Keenable is registered only whenSEARCH_PROVIDER=keenableorKEENABLE_API_KEYis set. When selected it goes first through the existingpreferredlogic; with only a key it is appended before DuckDuckGo. No other provider moves.provider_status()and_search_provider_note(): a Keenable row and a setup note.routes/settings.py:keenablein theSEARCH_PROVIDERoptions and an optionalKEENABLE_API_KEYpassword field (Advanced tab, next toSERPAPI_KEYS).utils/config_loader.py,.env.example,backend_api_python/env.example:KEENABLE_API_KEYmapping and an empty entry.tests/test_keenable_search.py: 7 offline tests (keyless and keyed request shape, snippet mapping and cap, date window, error and empty fall-through, not registered by default, selected provider first, key-only placement).What is sent to Keenable: the search query,
max_results, andpublished_afterfor short windows, with anX-Keenable-Title: QuantDingerheader (required by the keyless endpoint), plus the key only if one is set.Test plan
Tested locally with
docker compose up -d --build(not run; the search service was exercised directly, see below)Backend logs show no errors (server not started)
Relevant pytest tests pass
python -m pytest -m "not integration and not stress"on Python 3.12 (requirements.lock+requirements-dev.txt), macOS: 3688 passed, 286 failed. The same 286 fail onmainhere (3681 passed); 233 of them arestrategy sandbox memory limit could not be applied, which macOS does not support. The 7 new tests pass.ruff check app scripts tests: passed.Live run without a key through
get_search_service()withSEARCH_PROVIDER=keenable(provider orderKeenable -> GDELT -> DuckDuckGo):search_stock_newsfor NVDA and BTC,search_with_fallback(7 days),search_stock_events(30 days) andsearch_researcheach returned 5 results from Keenable in 0.4 to 1.9 s. Every result of the four news calls carried apublished_atinside the requested window.API documentation (if routes/schemas changed)
Disclosure: I work at Keenable, which runs this API.