How keyword RAG works
The agent's search_guides tool fetches a remote design manual, splits it into paragraphs, ranks them by keyword overlap, and returns the top 3 matches — no embeddings, no vector DB.
Fetch — ingest remote markdown
The pipeline starts with a plain HTTP fetch of the raw GitHub file. No preprocessing — just the full markdown text in memory.
GET https://raw.githubusercontent.com/RayFernando1337/llm-cursor-rules/main/fire-your-design-team.mdChunk — split by double newline
Markdown is split on \n\n+ (paragraph boundaries). Short fragments like --- are dropped, but markdown headings (e.g. ## Typography System) are kept — they label sections and help topic matching.
Kept chunks (sample)
# shadcn/ui with Tailwind v4 Design System Guidelines
This document outlines design principles and implementation guidelines for applications using shadcn/ui with Tailwind v4. These guidelines ensure consistency, a…
## Core Design Principles
### 1. Typography System: 4 Sizes, 2 Weights - **4 Font Sizes Only**: - Size 1: Large headings - Size 2: Subheadings/Important content - Size 3: Body text…
### 2. 8pt Grid System - **All spacing values must be divisible by 8 or 4** - **Examples**: - Instead of 25px padding → Use 24px (divisible by 8) - Instead …
### 3. 60/30/10 Color Rule - **60%**: Neutral color (white/light gray) - **30%**: Complementary color (dark gray/black) - **10%**: Main brand/accent color (e.g.…
Dropped as noise (if any)
Visual: each blank line pair in the source becomes a split point. Short fragments between splits never reach scoring.
Tokenize — normalize query words
The query string is lowercased and split on non-alphanumeric characters. Duplicates in the query count once when scoring.
"typography font sizes weights"
Auto-Layout → auto, layout · flex-wrap: nowrap → flex, wrap, nowrap
Score — keyword intersection
Each chunk gets a score equal to how many unique query tokens appear in that chunk. More overlapping keywords = higher rank.
### 1. Typography System: 4 Sizes, 2 Weights - **4 Font Sizes Only**: - Size 1: Large headings - Size 2: Subheadings/Important content - Siz…
### Core Design Principles - [ ] Typography: Uses only 4 font sizes and 2 font weights (Semibold, Regular) - [ ] Spacing: All spacing values…
### Font Sizes & Weights - **Strictly limit to 4 distinct sizes**: - Size 1: Large headings (largest) - Size 2: Subheadings - Size 3: Body t…
### Common Issues to Flag - [ ] Too many font sizes (more than 4) - [ ] Inconsistent spacing values (not divisible by 8 or 4) - [ ] Overuse …
## Typography System
### Typography Implementation - **Reference shadcn's typography primitives** for consistent text styling - **Use monospace variant** for num…
Retrieve — top 3 joined context
Matching chunks are sorted by score, filtered by the minimum threshold, limited to 3, then joined with \n\n---\n\n. This string is injected into the agent prompt as grounded context.
### 1. Typography System: 4 Sizes, 2 Weights - **4 Font Sizes Only**: - Size 1: Large headings - Size 2: Subheadings/Important content - Size 3: Body text - Size 4: Small text/labels - **2 Font Weights Only**: - Semibold: For headings and emphasis - Regular: For body text and general content - **Consistent Hierarchy**: Maintain clear visual hierarchy with limited options --- ### Core Design Principles - [ ] Typography: Uses only 4 font sizes and 2 font weights (Semibold, Regular) - [ ] Spacing: All spacing values are divisible by 8 or 4 - [ ] Colors: Follows 60/30/10 color distribution (60% neutral, 30% complementary, 10% accent) - [ ] Structure: Elements are logically grouped with consistent spacing --- ### Font Sizes & Weights - **Strictly limit to 4 distinct sizes**: - Size 1: Large headings (largest) - Size 2: Subheadings - Size 3: Body text - Size 4: Small text/labels (smallest) - **Only use 2 font weights**: - Semibold: For headings and emphasis - Regular: For body text and most UI elements - **Common mistakes to avoid**: - Using more than 4 font sizes - Introducing additional font weights - Inconsistent size application
Short query (≤ 4 tokens)
minScore = 1 — any single keyword match qualifies.
Long query (≥ 5 tokens)
minScore = 2— prevents one common word (e.g. "primary") from returning unrelated sections.
Why this approach?
- Zero costNo embedding API calls or vector database — just string matching.
- DeterministicSame query always returns the same chunks. Easy to debug and demo.
- GroundedAgent recommendations cite real paragraphs from the design manual.