Skip to content

feat: add Keenable as a configurable internet search backend - #2072

Open
ilya-bogin-keenable wants to merge 19 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search
Open

feat: add Keenable as a configurable internet search backend#2072
ilya-bogin-keenable wants to merge 19 commits into
MemTensor:mainfrom
keenableai:feat/keenable-web-search

Conversation

@ilya-bogin-keenable

Copy link
Copy Markdown

Summary

Adds Keenable as a new internet search backend alongside the existing Bocha / Tavily / Google / Bing / Xinyu retrievers, following the Tavily backend pattern (#1357). Additive and opt-in via INTERNET_SEARCH_BACKEND=keenable; existing backends are untouched.

Keenable is a web search API built for AI agents. Unlike the key-required backends it is keyless by default: with no key it calls the public endpoint (rate-limited), and an optional KEENABLE_API_KEY only lifts the cap.

Files changed

  • src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (new): InternetKeenableRetriever. No SDK dependency, a thin requests call. Keyless requests hit /v1/search/public; a configured key switches to /v1/search with an X-API-Key header. Attribution via X-Keenable-Title. Results map into TextualMemoryItem the same way the Tavily retriever does.
  • src/memos/configs/internet_retriever.py: KeenableSearchConfig (API key optional) + registration in InternetRetrieverConfigFactory.
  • src/memos/memories/textual/tree_text_memory/retrieve/internet_retriever_factory.py: register keenable + constructor branch.
  • src/memos/api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY optional).

Testing

  • python -m py_compile on all changed files: passes.
  • ruff check on all changed files: passes.

Bryunyon and others added 2 commits July 7, 2026 16:16
Add Keenable alongside the existing bocha / tavily / google / bing / xinyu
internet retrievers, following the Tavily backend pattern.

- retrieve/keenablesearch.py: InternetKeenableRetriever. Keyless by default
  (no SDK, a thin requests call): with no key it hits /v1/search/public
  (rate-limited); a key switches to /v1/search with an X-API-Key header.
  Attribution via X-Keenable-Title. Results map into TextualMemoryItem
  exactly like the Tavily retriever.
- configs/internet_retriever.py: KeenableSearchConfig (api_key optional) and
  registration in InternetRetrieverConfigFactory.
- retrieve/internet_retriever_factory.py: register "keenable" + constructor.
- api/config.py: INTERNET_SEARCH_BACKEND=keenable branch (KEENABLE_API_KEY
  optional, keyless by default).

py_compile and ruff pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Memtensor-AI Memtensor-AI added area:memory 记忆存储、检索、更新、召回逻辑 area:api 云服务 / FastAPI / OpenAPI / MCP labels Jul 8, 2026
@Memtensor-AI
Memtensor-AI requested a review from bittergreen July 8, 2026 11:43
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Hey @bittergreen could you please take a look? Thanks!

@Memtensor-AI Memtensor-AI removed the area:api 云服务 / FastAPI / OpenAPI / MCP label Jul 13, 2026
@Memtensor-AI Memtensor-AI added area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee July 31, 2026 19:09
@Memtensor-AI

Memtensor-AI commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2072
Task: ffe1c10a605ed5dd
Base: main
Head: feat/keenable-web-search

🔍 OpenCodeReview found 2 issue(s) in this PR.


1. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L195-L199)

The broad except Exception block silently swallows all errors — connection timeouts, authentication failures (401/403), and malformed responses are all handled identically. The caller receives an empty list with no signal about the failure type or severity, making it impossible to distinguish a transient network hiccup from a misconfigured API key.

Consider at minimum separating the HTTP call from the JSON parsing, catching specific exceptions (requests.exceptions.HTTPError, requests.exceptions.Timeout, requests.exceptions.ConnectionError), and re-raising or logging distinct messages for each.

💡 Suggested Change

Before:

        except Exception:
            import traceback

            logger.error(f"Keenable search error: {traceback.format_exc()}")
            return []

After:

        try:
            resp = requests.post(
                f"{KEENABLE_BASE_URL}{path}",
                json={"query": query, "mode": "pro"},
                headers=headers,
                timeout=self.timeout,
            )
            resp.raise_for_status()
        except requests.exceptions.Timeout:
            logger.error("Keenable search timed out")
            return []
        except requests.exceptions.HTTPError as e:
            logger.error(f"Keenable search HTTP error {e.response.status_code}: {e}")
            return []
        except requests.exceptions.RequestException as e:
            logger.error(f"Keenable search request failed: {e}")
            return []
        try:
            raw_results = resp.json().get("results", [])[:limit]
        except Exception as e:
            logger.error(f"Keenable search response parse error: {e}")
            return []

2. src/memos/memories/textual/tree_text_memory/retrieve/keenablesearch.py (L268)

_extract_tags accepts a parsed_goal parameter (used to extend tags from parsed_goal.tags), but it is never passed here — the only call site in this file. The parsed_goal variable is available in _process_result's scope, so the omission silently ignores goal-derived tags for non-Chinese content. Either pass parsed_goal here (consistent with the Bocha peer) or remove the parameter from _extract_tags if it's intentionally unused.

💡 Suggested Change

Before:

            else self._extract_tags(title, content, summary)[:3]

After:

            else self._extract_tags(title, content, summary, parsed_goal)[:3]

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-33586be3c2b3f89c-20260801031509: 107/107 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Jul 31, 2026
- gate jieba behind require_python_package, like the bocha retriever, so an
  English-only install no longer fails at construction
- skip embedding when the content is empty
- lowercase GDP/AI in the keyword lists; the text is lowercased before matching
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 1, 2026
@Memtensor-AI
Memtensor-AI requested a review from WeiminLee August 1, 2026 05:56
@ilya-bogin-keenable

Copy link
Copy Markdown
Author

Аixed 4 of the 5: jieba is now gated behind require_python_package like the bocha retriever (an English-only install used to fail at construction), empty content is no longer embedded, and GDP/AI are lowercased so they can actually match.

Will not fix for #2. Forwarding the local mode to the API would break the default path: the search endpoint returns 400 for mode: "fast", and fast is the default the caller passes. The API's mode and this method's mode are different things, so I added a comment saying so instead.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-b3c50e54d556e024-20260804132254: 86/87 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added the status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 label Aug 4, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 17, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 5s

Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-cebeff6b3ed746cf-20260818124755: 139/140 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 18, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 19, 2026
Keenable returns both fields on every result: snippet carries the page text
and description is the page's meta description, which is empty for most
pages. Reading description alone stored memories with a title and a URL but
no text.
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-d929796b8f2bd83f-20260819141815: 136/136 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/0bb2eb3c8566c1ed/repo
Cloning into '/data/test-workspaces/0bb2eb3c8566c1ed/repo'...
Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

⚠️ Automated Test Results: ENV ISSUE

The test environment encountered an issue that requires manual attention.

Details: Executor error: Command failed: git clone --depth 1 --branch feat/keenable-web-search git@github.com:keenableai/MemOS.git /data/test-workspaces/1a3c8d0e9d69e77a/repo
Cloning into '/data/test-workspaces/1a3c8d0e9d69e77a/repo'...
kex_exchange_identification: Connection closed by remote host
Connection closed by UNKNOWN port 65535
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Branch: feat/keenable-web-search

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 5s [advisory, non-gating] AI-generated tests on branch test/auto-gen-6b2de16b7c284ae6-20260828124956: 123/125 passed, 2 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 28, 2026
@Memtensor-AI Memtensor-AI added status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 and removed status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 labels Aug 29, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (5/5 executed). memos_python_core/changed-python-source: 5/5. Duration: 4s [advisory, non-gating] AI-generated tests on branch test/auto-gen-ffe1c10a605ed5dd-20260829140808: 106/107 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually.

Branch: feat/keenable-web-search

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:api 云服务 / FastAPI / OpenAPI / MCP area:core MOS 编排层 / 框架底座 / 跨模块问题 area:memory 记忆存储、检索、更新、召回逻辑 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.