In-Process Local Tools (MCP Host)

Empowering local LLMs with safe, read-only tools gated by explicit user confirmation.

⚒️ tool_call: search_local_kb ✓ Awaiting Approval
// Human-in-the-loop review card rendered directly in chat:
{
  "query": "quarterly revenue breakdown",
  "top_k": 3
}
Reject ✓ Approve & Execute

1. Plain-Language Definition

The In-Process Local Tool Host (Phase 2) allows the running AI model to request specific actions—such as listing active workspace documents, performing hybrid search over personal archives, or fetching exact citation chunks—while requiring your manual approval before any action takes place.

2. Why It Exists

Standard LLM chats are passive: they can only respond using whatever initial text was supplied in the prompt. When an AI needs to clarify which files exist in your workspace or wants to retrieve a cited passage to verify its facts, tool calling gives it an active, structured way to request that information without guessing.

3. What Breaks Without Safe Tool Calling

Without an in-process execution gate, tool calling is either completely disabled (forcing the model to hallucinate details it cannot retrieve) or dangerously unmonitored (executing system commands or external network queries without user awareness or consent).

4. How StarkLLM Implements It

StarkLLM provides a strictly scoped, local-first tool system engineered for research privacy:

  • In-Process FastAPI Host: Tool registration and execution occur entirely inside the local backend container. No external RPC networks or third-party servers are contacted.
  • Three Built-in Read-Only Tools:
    • list_workspace_files: Tenant-isolated document inspection (names, sizes, chunk counts).
    • search_local_kb: Hybrid vector and BM25 search across your personal Knowledge Base.
    • fetch_cited_chunk: Retrieves the exact stored passage text for a citation locator from Chroma collections.
  • Server-Side Execution Gate: Direct invocation of POST /api/v1/mcp/execute requires a valid single-use token registered when the model emits a tool call. Direct calls without pending tokens are rejected with HTTP 403 Forbidden.
  • Human-in-the-Loop Cards: When a tool call is generated, execution pauses. An interactive card appears in the UI displaying the tool name and arguments. You can edit the arguments, approve execution, or cancel the call without breaking the chat session.
  • RAG Prompt Injection Defense: Vector search excerpts containing embedded tool delimiters (```tool_call) are sanitized before prompt formatting so malicious document passages cannot trick the UI into triggering tool executions.

5. User-Visible Benefit

You maintain 100% control over model actions. You can inspect exactly what query the model intends to run, correct any misspelled parameters in real time, and verify citations against genuine vector passages.

6. Explicit Non-Goals & Safety Boundaries

StarkLLM strictly rejects arbitrary shell execution, file system writes, and external web access. Your machine's OS remains safe and isolated.