115 lines
4.8 KiB
Markdown
115 lines
4.8 KiB
Markdown
# Cyborg Architecture v1.0
|
|
|
|
**Status:** Architecture Design — Phase 1 (Planning)
|
|
**Type:** Local Knowledge Infrastructure
|
|
**Hardware:** 128 GB RAM Server (no GPU), 90 GB DDR5 Desktop
|
|
|
|
## Vision
|
|
|
|
Build a deterministic, proactive knowledge layer that transforms a reactive LLM agent into a **cyborg-like system** with:
|
|
|
|
- **Proactive initiative** — predicts next steps before being asked
|
|
- **Accurate foresight** — risk analysis based on graph dependencies and vector memory
|
|
- **Deterministic reliability** — no emotional noise, pure signal + structured knowledge
|
|
- **Self-improving memory** — learns from feedback, updates facts, prunes noise
|
|
|
|
## Core Problem
|
|
|
|
Standard LLM architecture is a "glass sphere" — it only sees what's in the current context. When a session ends, it "forgets" unless manually saved to memory. The system is reactive, not proactive.
|
|
|
|
## Architecture Layers
|
|
|
|
### Layer 1: Memgraph (Graph Database)
|
|
- **Role:** Semantic relationships and dependencies
|
|
- **Why Memgraph over Neo4j:** Full Cypher compatibility, native C++ (no JVM overhead), ~200-400 MB RAM vs 2-4 GB, faster on single-core
|
|
- **Use cases:**
|
|
- File-to-file dependency graphs
|
|
- "What breaks if I change X?" queries
|
|
- Project knowledge graph (functions → files → modules → projects)
|
|
- Risk path analysis (find all code paths affected by a change)
|
|
|
|
### Layer 2: Qdrant (Vector Database)
|
|
- **Role:** Semantic search and pattern matching
|
|
- **Why Qdrant over Milvus:** Rust-based (no GC pauses), single binary, lightweight (~150-300 MB RAM), powerful filters, full RAM access for fast lookups
|
|
- **Use cases:**
|
|
- "How did I solve this before?" — semantic code search
|
|
- Personal coding patterns memory
|
|
- Feedback and correction history
|
|
- Cross-project pattern extraction
|
|
|
|
### Layer 3: Mem0 (Agent Memory Manager)
|
|
- **Role:** The "synapses" between agent and databases
|
|
- **Functions:**
|
|
- Noise filtering — stores only significant facts
|
|
- Memory updates — overwrites old facts when corrected
|
|
- Confidence scoring — rates memory reliability
|
|
- Automatic deduplication
|
|
- **Integration:** Native providers for Qdrant + Neo4j-compatible (Memgraph)
|
|
|
|
### Layer 4: Ollama + Qwen2.5-7B (Local Router)
|
|
- **Role:** Fast dispatch and classification (not full reasoning)
|
|
- **Why 7B not 30B+:**
|
|
- 30B Q4 on CPU → ~5-8 tokens/sec (DDR5 memory bandwidth bottleneck)
|
|
- 7B Q4 on CPU → ~15-30 tokens/sec (acceptable for routing)
|
|
- 7B in ~5-6 GB RAM vs ~18 GB for 30B
|
|
- **Use cases:**
|
|
- Intent classification
|
|
- Entity extraction from messages
|
|
- Routing to correct database/knowledge path
|
|
- Plan validation and decomposition
|
|
- **Optimization:** `mlock`, `--threads` = physical cores, `--ctx-size 4096`, Q4_K_M
|
|
|
|
### Layer 5: Async Event Bus (Python + asyncio)
|
|
- **Role:** Decouple agent from inference latency
|
|
- **Pattern:** Event-driven workers, not blocking LLM calls
|
|
- **Flow:**
|
|
1. Agent action triggers event (tool call, git hook, cron tick)
|
|
2. Event pushed to async queue
|
|
3. Worker picks up, routes through local LLM
|
|
4. Results written to Memgraph/Qdrant
|
|
5. Agent reads ready results on demand
|
|
- **Effect:** Inference latency completely hidden; system reacts instantly
|
|
|
|
## Deployment
|
|
|
|
### Container Orchestration
|
|
- **Tool:** `podman-compose` (no root, no systemd, no CRI issues)
|
|
- **Containers:**
|
|
- `memgraph` — graph database
|
|
- `qdrant` — vector database
|
|
- `ollama` — local LLM router
|
|
- `worker` — Python async workers (Mem0, ingestion, routing)
|
|
- `redis-lite` (optional) — event bus for inter-service communication
|
|
|
|
### Memory Limits
|
|
- `memgraph`: 1 GB
|
|
- `qdrant`: 2 GB
|
|
- `ollama`: 8 GB (Qwen2.5-7B Q4 + context)
|
|
- `worker`: 4 GB
|
|
- Total overhead: ~15 GB (leaving 113 GB for agent + system)
|
|
|
|
## Key Design Principles
|
|
|
|
1. **No single point of failure** — each database can function independently
|
|
2. **Deterministic first, LLM second** — use scripts/tools before calling LLM
|
|
3. **Async by default** — no blocking on inference
|
|
4. **Minimal footprint** — every component chosen for low resource usage
|
|
5. **Podman-native** — no root, no systemd dependency
|
|
6. **Scalable** — can add more databases or models without architectural changes
|
|
|
|
## Future Phases
|
|
|
|
- **Phase 1 (Current):** Infrastructure setup, data ingestion pipeline
|
|
- **Phase 2:** Dependency-aware code analysis ("what breaks when")
|
|
- **Phase 3:** Proactive risk detection (auto-scan for anti-patterns)
|
|
- **Phase 4:** Self-improving knowledge graph (auto-merge, dedup, confidence scoring)
|
|
- **Phase 5:** Multi-agent coordination (specialist routers for different domains)
|
|
|
|
## Files
|
|
|
|
- `ARCHITECTURE.md` — Full architecture specification
|
|
- `podman-compose.yml` — Container orchestration
|
|
- `scripts/ingest.py` — Knowledge ingestion script (WIP)
|
|
- `scripts/router.py` — Local LLM routing script (WIP)
|
|
- `scripts/mem0-bridge.py` — Mem0 integration bridge (WIP)
|