1. What StarkLLM is

StarkLLM is a private, local-first AI assistant designed for Windows. It provides a secure interface to chat with open-weight Large Language Models (LLMs) running directly on your machine. Through Retrieval-Augmented Generation (RAG), it allows you to upload documents (PDF, DOCX, TXT, Markdown (.md), CSV, JSON, HTML, and images (PNG, JPG, with local OCR for scans and images)), storing them in a local vector database to provide source-grounded answers to your questions.

2. Who it is for

StarkLLM is built for researchers, analysts, developers, and privacy-conscious professionals who need to interrogate sensitive documents without exposing them to third-party cloud providers. If you work under strict data governance policies or simply value data privacy, this tool is designed for you.

3. What it is not for

StarkLLM is not a cloud chatbot (like ChatGPT or Claude). It does not rely on remote server farms. It is also not a zero-configuration consumer app; the initial setup requires installing underlying dependencies like Docker and Ollama. If you want instant, low-effort AI and do not mind cloud data transmission, a cloud subscription will be easier and faster.

4. Required hardware

• Basic / Laptop Tier: Modern CPU, 8 GB RAM. Recommended for lightweight 3B models (e.g. qwen2.5:3b). Usable for document summaries, but slower on large context.
• Standard Tier (Recommended): Intel i7 / Ryzen 7, 16–32 GB RAM, NVIDIA RTX 3060 / 4060 (8GB+ VRAM). Offers fast, fluid inference for 7B models (e.g. qwen2.5:7b). 14B models need more VRAM or will run slowly on this tier.
• Power User Tier: Intel i9 / Ryzen 9, 32–64 GB RAM, NVIDIA RTX 3090 / 4090 (16–24GB VRAM). Enables running larger 14B–32B models with extensive context windows.
• Storage: Fast NVMe or SATA SSD with at least 30–50GB of free space for Docker containers and local model weights.

5. Required software

Because StarkLLM runs entirely locally, it relies on standard containerization and inference engines:

• Windows 10 or 11 with WSL2 (Windows Subsystem for Linux) enabled.
• Docker Desktop: Used to orchestrate the vector database, backend API, and frontend UI.
• Ollama: The inference engine responsible for running LLM models locally on your host machine.

6. Recommended models

• Recommended Default (qwen2.5:7b): High-accuracy reasoning model with strong document comprehension. Balanced reasoning for 8–16 GB RAM and dedicated 8GB+ VRAM.
• Lighter Alternative (qwen2.5:3b): Fast, low-resource model for laptops with 8 GB RAM or CPU-only setups.
• Power User Option (qwen2.5:14b or larger): Best with 16 GB+ VRAM or 32 GB+ system RAM. Slower on lesser hardware.
• Default Embedding Model (nomic-embed-text): Fast, lightweight embedding model optimized for local RAG retrieval.

7. Installation overview

Installation involves preparing your host machine and then running the launcher:

1. Ensure Docker Desktop is installed and running.
2. Install Ollama from ollama.com.
3. Pull the default models in your terminal:
   ollama pull qwen2.5:7b
   ollama pull nomic-embed-text
4. Run StarkLLM-Setup.exe (Windows Installer) or extract the Portable ZIP.

8. Launching StarkLLM (v2.0.0 Native Desktop)

StarkLLM v2.0.0 runs as an independent, standalone desktop application window (similar to Cursor and Claude Desktop). Choose your preferred launch option:

Option 1 — Windows Desktop Installer (Recommended)

Run StarkLLM-v2.0.0-Setup.exe to install StarkLLM as a native Windows application. It creates desktop and start menu shortcuts and runs in a sleek, isolated native window without opening web browser tabs.

Option 2 — Portable Standalone Application

Run StarkLLM-v2.0.0-Portable.exe directly from any directory or USB drive without system installation.

Option 3 — start.exe / start.cmd Launcher

StarkLLM includes a standalone start.exe launcher that automatically checks Docker Desktop status, checks Ollama availability, starts containers, and launches the native desktop window.

1. Open the extracted folder in File Explorer.
2. Double-click start.exe, or run in Command Prompt: start.exe (PowerShell: .\start.exe).

Option 4 — Direct Docker Compose

For headless or server environments, start containers directly from terminal:

docker compose up --build -d

Option 5 — Run unblock.ps1 (If Execution Blocked)

If Windows SmartScreen or execution policies prevent script execution, open PowerShell in the extracted folder and run:

powershell -ExecutionPolicy Bypass -File unblock.ps1

After running unblock.ps1, run start.exe or start.cmd (or start.bat).

9. First-week usage path

To get the most out of StarkLLM, we recommend this progression during your first week:

1. Day 1: Start small. Create a single workspace, upload 2-3 standard documents (like research papers or manuals), and practice asking questions. Observe how the citations work.
2. Day 2: Experiment with model switching. Download a different model via Ollama (e.g., try qwen2.5:3b vs qwen2.5:7b or llama3.2) and compare how they answer the same document questions.
3. Day 3: Upload a complex multi-page report. Notice how semantic chunking breaks it down into logical blocks.
4. Day 4: Organize. Create distinct workspaces for different projects (e.g., "Finances", "Codebase", "Legal").
5. Day 5+: Set up your Global Knowledge Base. Point StarkLLM to your permanent notes folder so you can query it across all chat sessions.

10. Creating a workspace

Workspaces isolate your data. In the left sidebar, click "New Workspace" to create a fresh context. For example, you might create a "Legal Review" workspace and a separate "Market Research" workspace. Documents and chats in one workspace cannot cross-contaminate another.

11. Uploading documents & Best Practices

Inside a workspace, click the "Upload Document" button or drag-and-drop supported files (PDF, DOCX, TXT, MD, HTML, and images) into the designated area. StarkLLM processes the document in the background—extracting text, generating semantic chunks, and embedding vectors into local ChromaDB.

12. Using chat with documents

Once a document is processed, you can ask questions in the chat interface. StarkLLM performs hybrid search to find the most relevant chunks from your uploaded documents, feeds those chunks to your local LLM, and generates an answer with verifiable citations.

13. Using Knowledge Base & Best Practices

The Knowledge Base feature allows you to map a local folder on your Windows host. StarkLLM performs a one-way copy of supported files (PDF, DOCX, TXT, MD, HTML, and images) into the container and indexes them into a global vector collection. Unlike workspace-specific documents, the Knowledge Base acts as a global reference library that you can query from any chat session. The system polls the mapped folder approximately every minute and re-indexes new or modified files automatically. Source files on your host are never deleted or modified.

How to add a Knowledge Base folder

Register a folder using the launcher from your terminal:

• Interactive: After start.exe finishes launching, press [A] in the launcher console and paste the full Windows path when prompted (e.g. C:\Users\me\research).
• CLI flag: start.exe --add-folder --folder="C:\Users\me\research"

14. Workspace vs Knowledge Base

StarkLLM offers two ways to contextualize the AI. Understanding the difference is critical to organizing your intelligence:

15. Switching models

If you have downloaded multiple models via Ollama (e.g., qwen2.5:7b, llama3.2, or mistral), you can switch between them using the dropdown menu in the StarkLLM chat interface. Changing models retains chat history while using the selected model for new replies.

16. Voice features

StarkLLM supports local Speech-to-Text (STT) for dictating messages and Text-to-Speech (TTS) for reading answers aloud. Ensure browser microphone permissions are enabled. Voice processing is handled locally without sending audio to external clouds.

17. Using In-Process Local Tools (MCP Host)

Phase 2 adds safe, in-process read-only tool calling. When an open-weight model determines that it needs more information to fulfill your request, it issues a structured tool call. StarkLLM provides three built-in tools:

• list_workspace_files: Discovers document names, file sizes, chunk counts, and upload timestamps in your current workspace without filesystem escapes.
• search_local_kb: Executes a hybrid vector and BM25 search over your global Knowledge Base to find matching passages.
• fetch_cited_chunk: Retrieves the exact stored passage text for any citation locator directly from Chroma collections.

Human-in-the-Loop Confirmation

StarkLLM requires manual confirmation for every tool execution. When the model invokes a tool:

1. A styled confirmation card appears in the chat stream showing the tool name and arguments.
2. You can review the arguments, click into parameter fields to adjust values (e.g. refining a search query or increasing top_k), and click Approve & Execute.
3. If you prefer the model to proceed without running the tool, click Reject. The pending call token is cancelled, and your chat session continues uninterrupted.

18. Customizing Themes & Appearance

StarkLLM includes eight hand-crafted color palettes designed for comfort across day and night sessions:

• Cyberpunk Rose: High-energy pink, rose, and magenta neon highlights.
• OLED Pitch Black: Absolute #000000 true black backgrounds optimized for OLED screens.
• Liquid Glass: Translucent glassmorphism with frosted blur layers and glowing borders.
• Matrix Emerald: Terminal-inspired cyberpunk emerald green.
• Sunset Amber: Warm volcanic amber and orange gradient accents.
• Nordic Frost: Cool cyan and arctic blue minimal tones.
• Midnight Indigo: Deep cosmic purple and violet (default).
• Daylight Minimal: Clean, crisp light mode with high daylight contrast.

How to Change Themes

You can switch themes in two ways:

• Quick Switcher: Click the Palette icon (🎨) at the bottom of the left sidebar. A popover opens immediately; click any theme to apply it instantly.
• Settings Gallery: Open Settings from the sidebar, navigate to Appearance, and browse all 8 theme cards with live swatch dots, gradient bars, and descriptions.

Your active theme is automatically remembered across browser tabs and system restarts via client-side storage.

19. Honest Citations & Locators

Hallucinated citations are a significant issue in many RAG systems. StarkLLM eliminates guesswork with verifiable, honest locators flowing from ingestion to display:

• PDF Documents: Citations display the 1-indexed source page number: report.pdf (Page 4).
• DOCX Documents: Citations group content by structural headings: contract.docx (Section: 'Termination' Chunk 2).
• Text & Markdown: Citations reference the precise semantic partition: notes.md (Chunk 5).
• Images & Vision: Scanned documents and images carry an Image locator.

Headers are clearly formatted above AI responses, allowing you to cross-examine any claim against the original source file on your machine.

20. Chat Folders & Organization

As your research expands, StarkLLM keeps your workspace clean and organized with sidebar conversation folders:

• Create Folders: Click the + New Folder icon in the sidebar conversation list to group sessions by project, client, or topic (e.g., "Financial Audits", "Legal Briefs", "Code Reviews").
• Drag or Assign: Assign any active conversation into a folder via the three-dot context menu or folder selection dropdown.
• Collapsible Tree: Expand or collapse folders to keep focus on your active stream without visual clutter.
• Folder Filtering: Search conversations within a specific folder or across the entire workspace in real time.

21. Multi-Format Conversation Export

Never lose your insights. StarkLLM provides one-click export into four standardized formats directly from the chat header:

• Markdown (.md): Formatted with full headings, code fences, source citations, timestamps, word count badges, and generation speed statistics.
• Clean JSON (.json): Structured machine-readable export including message metadata, prompt parameters, citations, and model details for data pipelines.
• Self-Contained HTML (.html): Standalone styled document with embedded dark mode aesthetics, perfect for sharing or browser-based "Print to PDF".
• Plain Text (.txt): Stripped unformatted text for immediate copying into emails, notes, or scratchpads.

22. Internationalization & Language Roadmap

StarkLLM's internationalization architecture is prioritized according to global internet user demographics to serve users worldwide systematically:

• Default Language: English (en) — 1,500M+ users worldwide. Fully active and verified across all features and document parsers.
• Tier 1 Roadmap: Chinese (zh, 1,100M+ users) and Spanish (es, 550M+ users). Core localization in development.
• Tier 2 Roadmap: French (fr, 320M+ users) and German (de, 180M+ users). High-demand enterprise European localizations.
• Tier 3 Roadmap: Arabic (ar, 270M+ users) and Persian / Farsi (fa, 80M+ users). Bidirectional (RTL) layout support and specialized font rendering.
• Tier 4 Roadmap: Portuguese (pt, 260M+ users), Russian (ru, 250M+ users), and Japanese (ja, 125M+ users).

All interface controls, settings, and documentation remain completely in English, with zero regional script clutter until an explicit language selection is enabled by the user.

23. Hardware Telemetry & VRAM Monitor

Local AI requires clear visibility into system resources. The StarkLLM Dashboard provides real-time hardware telemetry:

• VRAM & GPU Utilization: Monitor allocated video memory to ensure your chosen model (3B, 7B, 14B) fits comfortably within your GPU VRAM without spilling into system RAM.
• System Health: Instant status indicators for the FastAPI backend, ChromaDB vector store, and Ollama inference daemon.
• Workspace Metrics: Total indexed chunks, active documents, total conversations, and rapid-access links to your recent chat sessions.

24. Troubleshooting

• "Cannot connect to Ollama": Ensure Ollama is running in your Windows system tray. Ollama runs natively on the host, and the Docker container communicates via host.docker.internal:11434. Verify you pulled the required model (ollama pull qwen2.5:7b).
• Docker fails to start: Ensure Docker Desktop is running with WSL2 integration. If containers fail, try running docker compose down followed by docker compose up --build -d.
• Slow responses: Ensure Ollama is utilizing your dedicated NVIDIA GPU rather than defaulting to CPU inference.

25. Limitations

• Initial setup requires installing Docker Desktop and Ollama.
• Response speed and throughput depend on your local hardware.
• Document retrieval quality depends on the formatting and clarity of the input text.
• StarkLLM is provided "as is" as an open-source Research Beta under Apache 2.0.

26. Frequently Asked Questions