Drop SearXNG references: use built-in web_search
This commit is contained in:
1 parent
16c843966e
commit
d1c87572ee
7 files changed
+19
-18
No files matched your search
@@ -11,9 +11,10 @@ Read this file before answering any question about knowledge, memory, or researc
|
|||||||
|
|
||||||
1. **Your memory comes FIRST.** Before searching the web, call `memory_get_context` or
|
1. **Your memory comes FIRST.** Before searching the web, call `memory_get_context` or
|
||||||
`memory_search`. You may already know the answer.
|
`memory_search`. You may already know the answer.
|
||||||
2. **NEVER search your own knowledge base on the web.** `web_search` and `mcp__searxng__*`
|
2. **NEVER search your own knowledge base on the web.** `web_search` queries the OUTSIDE
|
||||||
query the OUTSIDE internet only. Questions about *what you know*, *how much you know*,
|
internet only. Questions about *what you know*, *how much you know*, *what gaps exist* are
|
||||||
*what gaps exist* are answered **only** by the `memory_*` tools — never by `web_search`.
|
answered **only** by the `memory_*` tools — never by `web_search`. Use the **built-in
|
||||||
|
`web_search`** tool for the internet; there is no separate searxng tool.
|
||||||
3. **Save typed atoms, not paragraphs.** One atom = one atomic statement (a fact, a definition,
|
3. **Save typed atoms, not paragraphs.** One atom = one atomic statement (a fact, a definition,
|
||||||
a rule). No fluff, no marketing text.
|
a rule). No fluff, no marketing text.
|
||||||
4. **Verify before trusting.** `memory_verify` + cross-confirmation from 2 or more independent
|
4. **Verify before trusting.** `memory_verify` + cross-confirmation from 2 or more independent
|
||||||
@@ -42,7 +43,7 @@ Read this file before answering any question about knowledge, memory, or researc
|
|||||||
| "Run logical inference" | `memory_infer` |
|
| "Run logical inference" | `memory_infer` |
|
||||||
|
|
||||||
**Ambiguity rule:** if the question is about *knowledge content* → memory tools.
|
**Ambiguity rule:** if the question is about *knowledge content* → memory tools.
|
||||||
If it is about *the outside world right now* → `web_search` / `searxng`.
|
If it is about *the outside world right now* → `web_search` (built-in).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -52,7 +53,7 @@ If it is about *the outside world right now* → `web_search` / `searxng`.
|
|||||||
1. DETECT memory_stats / memory_list_gaps → what is missing?
|
1. DETECT memory_stats / memory_list_gaps → what is missing?
|
||||||
2. CHECK memory_search → do I already know it?
|
2. CHECK memory_search → do I already know it?
|
||||||
3. PLAN memory_add_gap(question) → register the gap
|
3. PLAN memory_add_gap(question) → register the gap
|
||||||
4. ACQUIRE searxng search / read URL → find sources
|
4. ACQUIRE web_search / web_fetch / read URL → find sources
|
||||||
5. EXTRACT memory_ingest(text|url|path) → distill into atoms
|
5. EXTRACT memory_ingest(text|url|path) → distill into atoms
|
||||||
6. VERIFY memory_verify → candidate → verified/contested
|
6. VERIFY memory_verify → candidate → verified/contested
|
||||||
7. COMMIT (automatic: ingest writes to PostgreSQL → Qdrant → Memgraph)
|
7. COMMIT (automatic: ingest writes to PostgreSQL → Qdrant → Memgraph)
|
||||||
|
|||||||
+2
-2
@@ -96,7 +96,7 @@ agent's experience, atoms are curated facts.
|
|||||||
| `outbox_sync.py` | CDC: PostgreSQL outbox → Qdrant and Memgraph |
|
| `outbox_sync.py` | CDC: PostgreSQL outbox → Qdrant and Memgraph |
|
||||||
| `dedup.py` | semantic deduplication by embedding cosine (threshold 0.90) |
|
| `dedup.py` | semantic deduplication by embedding cosine (threshold 0.90) |
|
||||||
| `verify.py` | self-critique (LLM verdict) + cross-confirmation (multiple sources) |
|
| `verify.py` | self-critique (LLM verdict) + cross-confirmation (multiple sources) |
|
||||||
| `research.py` | gap-filling loop: SearXNG search → ingest → commit |
|
| `research.py` | gap-filling loop: web search → ingest → commit |
|
||||||
| `code_symbols.py` | tree-sitter symbol extraction |
|
| `code_symbols.py` | tree-sitter symbol extraction |
|
||||||
| `code_graph.py` | tree-sitter dependency extraction (calls/imports/inherits) |
|
| `code_graph.py` | tree-sitter dependency extraction (calls/imports/inherits) |
|
||||||
| `inference.py` | forward-chaining logical rules over the graph until fixpoint |
|
| `inference.py` | forward-chaining logical rules over the graph until fixpoint |
|
||||||
@@ -124,7 +124,7 @@ source (text|pdf|epub|web)
|
|||||||
```
|
```
|
||||||
memory_stats / memory_list_gaps → detect gap
|
memory_stats / memory_list_gaps → detect gap
|
||||||
memory_add_gap(question) → register
|
memory_add_gap(question) → register
|
||||||
SearXNG search → URLs
|
web_search → URLs
|
||||||
memory_ingest(url) → atoms (full page; snippet fallback if blocked)
|
memory_ingest(url) → atoms (full page; snippet fallback if blocked)
|
||||||
memory_verify → cross-confirm → verified
|
memory_verify → cross-confirm → verified
|
||||||
```
|
```
|
||||||
|
|||||||
+1
-1
@@ -92,7 +92,7 @@ If nothing is found → say "not in the knowledge base" and offer to research it
|
|||||||
2. memory_add_gap("How does HNSW work?",
|
2. memory_add_gap("How does HNSW work?",
|
||||||
reason = "low_coverage", priority = 0.8)
|
reason = "low_coverage", priority = 0.8)
|
||||||
3. memory_research("How does HNSW indexing work?")
|
3. memory_research("How does HNSW indexing work?")
|
||||||
→ searches SearXNG, ingests sources,
|
→ searches the web, ingests sources,
|
||||||
distills atoms, saves them
|
distills atoms, saves them
|
||||||
4. memory_verify() → cross-confirm the new atoms
|
4. memory_verify() → cross-confirm the new atoms
|
||||||
5. memory_search("HNSW") → now returns verified hits
|
5. memory_search("HNSW") → now returns verified hits
|
||||||
|
|||||||
@@ -38,7 +38,7 @@ schema.sql — DDL PostgreSQL
|
|||||||
|
|
||||||
## Подключение к DeepSeek Harness (dsh)
|
## Подключение к DeepSeek Harness (dsh)
|
||||||
|
|
||||||
Память подключается как **MCP-сервер** (паттерн dsh, как SearXNG/Gitea/Outline/Playwright):
|
Память подключается как **MCP-сервер** (паттерн dsh, как Gitea/Outline/Playwright):
|
||||||
|
|
||||||
- **Сервер**: [mcp_server.py](./mcp_server.py) — чистый stdlib, JSON-RPC 2.0, stdio.
|
- **Сервер**: [mcp_server.py](./mcp_server.py) — чистый stdlib, JSON-RPC 2.0, stdio.
|
||||||
- **Инструменты** (видны модели как `mcp__pegas-memory__*`):
|
- **Инструменты** (видны модели как `mcp__pegas-memory__*`):
|
||||||
@@ -67,7 +67,7 @@ schema.sql — DDL PostgreSQL
|
|||||||
Cloudflare anti-bot (neo4j.com — 403). Поведение research-цикла:
|
Cloudflare anti-bot (neo4j.com — 403). Поведение research-цикла:
|
||||||
|
|
||||||
1. Полная загрузка URL (httpx, таймаут 30s).
|
1. Полная загрузка URL (httpx, таймаут 30s).
|
||||||
2. При блокировке — fallback на сниппет из SearXNG (локальный инстанс).
|
2. При блокировке — fallback на сниппет из веб-поиска.
|
||||||
|
|
||||||
Для заблокированных сайтов можно включить SOCKS5/HTTP-прокси:
|
Для заблокированных сайтов можно включить SOCKS5/HTTP-прокси:
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
+1
-1
@@ -64,7 +64,7 @@ outbox — CDC-журнал для синка в Qdrant + Memgraph
|
|||||||
### Фаза 2 — Исследование пробелов ✅ (реализовано, snippet-fallback)
|
### Фаза 2 — Исследование пробелов ✅ (реализовано, snippet-fallback)
|
||||||
- [x] Детект пробелов (research_gap + add_gap/list_gaps)
|
- [x] Детект пробелов (research_gap + add_gap/list_gaps)
|
||||||
- [x] Планирование (приоритизированная очередь пробелов)
|
- [x] Планирование (приоритизированная очередь пробелов)
|
||||||
- [x] Acquire: SearXNG web-поиск (3 источника) + pdf/epub/text
|
- [x] Acquire: web-поиск (3 источника) + pdf/epub/text
|
||||||
- [x] Оценка ценности (verify_atom + min_confidence)
|
- [x] Оценка ценности (verify_atom + min_confidence)
|
||||||
- [~] Верификация (verify_atom готов; полный автономный прогон требует прокси для зарубежных сайтов)
|
- [~] Верификация (verify_atom готов; полный автономный прогон требует прокси для зарубежных сайтов)
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -312,8 +312,8 @@ TOOLS = [
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "memory_research",
|
"name": "memory_research",
|
||||||
"description": "Полный цикл исследования вопроса: поиск в интернете (SearXNG), "
|
"description": "Полный цикл исследования вопроса: веб-поиск, загрузка найденных источников, "
|
||||||
"загрузка найденных источников, дистилляция в атомы, сохранение.",
|
"дистилляция в атомы, сохранение. Для ручного поиска используй встроенный web_search.",
|
||||||
"inputSchema": {
|
"inputSchema": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"properties": {
|
"properties": {
|
||||||
|
|||||||
@@ -23,12 +23,12 @@ def add_gap(conn, question: str, reason: str = "user_query", priority: float = 0
|
|||||||
)
|
)
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
||||||
def web_search(query: str, engine: str = "searxng") -> list[dict]:
|
def web_search(query: str, engine: str = "local") -> list[dict]:
|
||||||
"""Поиск через SearXNG. Возвращает [{title, url, snippet}]."""
|
"""Веб-поиск (локальный инстанс). Возвращает [{title, url, snippet}]."""
|
||||||
# локальный SearXNG (MCP mcp__searxng__search); здесь — прямой HTTP JSON API
|
# прямого MCP-инструмента searxng больше нет; internal HTTP JSON API остаётся
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
q = urllib.parse.quote(query)
|
q = urllib.parse.quote(query)
|
||||||
# типичный endpoint локального SearXNG
|
# endpoint локального поискового инстанса
|
||||||
for base in ("http://127.0.0.1:8889", "http://127.0.0.1:8080"):
|
for base in ("http://127.0.0.1:8889", "http://127.0.0.1:8080"):
|
||||||
try:
|
try:
|
||||||
r = _client.get(f"{base}/search", params={"q": query, "format": "json"}, timeout=15)
|
r = _client.get(f"{base}/search", params={"q": query, "format": "json"}, timeout=15)
|
||||||
@@ -43,7 +43,7 @@ def web_search(query: str, engine: str = "searxng") -> list[dict]:
|
|||||||
def research_question(conn, question: str, *, top_k: int = 5, model: str | None = None) -> dict:
|
def research_question(conn, question: str, *, top_k: int = 5, model: str | None = None) -> dict:
|
||||||
"""Полный цикл исследования одного вопроса.
|
"""Полный цикл исследования одного вопроса.
|
||||||
|
|
||||||
1. web-поиск (SearXNG);
|
1. веб-поиск;
|
||||||
2. для каждого источника: полная загрузка URL; при недоступности (гео-блок,
|
2. для каждого источника: полная загрузка URL; при недоступности (гео-блок,
|
||||||
нет прокси) — дистилляция сниппета из поиска;
|
нет прокси) — дистилляция сниппета из поиска;
|
||||||
3. пометка вопроса resolved.
|
3. пометка вопроса resolved.
|
||||||
|
|||||||
Reference in new issue
Block a user