An end-to-end AI system that transforms static PDFs into a structured knowledge graph and enables intelligent question answering using GraphRAG and LLM routing.
The platform extracts knowledge from documents, builds a Neo4j knowledge graph, and dynamically routes user queries to the most suitable reasoning pipeline to generate precise and context-aware answers.
-
📄 PDF Knowledge Extraction
- Convert unstructured PDF documents into structured JSON knowledge.
-
🧠 Knowledge Graph Construction
- Automatically transform extracted knowledge into a Neo4j graph database.
-
🔍 GraphRAG Retrieval
- Retrieve relevant information through graph traversal instead of traditional vector search.
-
🤖 LLM Router
- Classifies user queries and routes them to the appropriate reasoning pipeline.
-
💬 Natural Language Q&A
- Users can ask questions in natural language and receive context-aware answers.
-
PDF Ingestion
- PDF documents are parsed and converted into structured JSON.
-
Knowledge Extraction
- Important entities and relationships are extracted using LLM-based parsing.
-
Knowledge Graph Construction
- Extracted knowledge is converted into nodes and relationships inside Neo4j.
-
LLM Router
- Incoming user queries are classified to determine the best retrieval strategy.
-
GraphRAG Querying
- The system retrieves relevant graph context and feeds it into the LLM.
-
Answer Generation
- The LLM generates responses using retrieved graph knowledge.
- PDF ingestion pipeline
- PDF → Markdown knowledge extraction
- Knowledge graph construction with Neo4j
- GraphRAG-based retrieval
- Fine-tuned LLM Router for query classification
- Intelligent routing between query pipelines
- Natural language question-answering API
- LLM Router evaluation framework
- GraphRAG retrieval evaluation
- Async ingestion pipeline for large document processing
- Logging and system observability
To minimize system latency and slash API costs, I fine-tuned a Llama-3-8B-Instruct model using LoRA (Low-Rank Adaptation). This model serves as the "intelligent gatekeeper," dynamically routing queries to the most efficient reasoning pipeline.
- Base Model: Llama-3-8B-Instruct.
- Method: LoRA Fine-tuning (balancing parameter updates with limited compute).
- Final Training Loss: 0.09.
- Core Objective: Binary classification. The router identifies "Easy" tasks for the EFFICIENT (Local) pipeline and "Complex" tasks for the CLOUD pipeline.
A key part of this project was engineering a solution within strict hardware bottlenecks. Training and running an 8B model on an NVIDIA RTX 3060 Ti (8GB VRAM) frequently triggered Out-of-Memory (OOM) errors during initial tests.
How I resolved the constraints:
- 4-bit Quantization: Integrated
BitsAndBytesto compress the model into 4-bit NormalFloat (NF4), reducing the memory footprint by roughly 75%. - Precision Capping: I strictly limited
max_new_tokens(10 for routing, 250 for local inference) and tuned training steps to stay within the 8GB VRAM safety zone. - Resource Management: Leveraged SSD-backed swap to handle peak loads during large-scale evaluation, preventing system crashes during long benchmark runs.
I built a custom benchmarking framework using the routellm/gpt4_dataset to validate the router's performance across 500 samples:
| Metric | Result |
|---|---|
| Easy Task Routing Accuracy | 95.0% |
| Overall System Accuracy | 96.6% |
| Net Cloud Token Reduction | ~38% |
Developer Note: Achieving 95% accuracy on easy tasks means the vast majority of daily interactions are handled locally and instantly. This doesn't just save money—it drastically improves the user experience by providing near-zero latency responses for common queries.
Documents with high image density (charts, diagrams, and tables) often cause issues for standard RAG pipelines. When a PDF is "naively" chunked, critical visual information is frequently lost or decoupled from its context.
To solve this, I am using MinerU to convert complex PDFs into Markdown. This preserves the structural relationship between text and images. This project serves as a benchmark to compare three distinct approaches:
Native PDF RAG: Standard chunking directly from the source file.
Markdown-Enhanced RAG: Utilizing MinerU to extract structured text and image data before indexing.
GraphRAG: (Upcoming phase) Building a knowledge graph to connect entities across the document.
🧪 Methodology & Comparison My goal is to evaluate these methods based on two primary dimensions:
Accuracy: How well the system retrieves and interprets information stored within images and tables.
By comparing Native PDF vs. MinerU-Markdown, I can quantify the accuracy gains versus the extra processing time. Following this, I will integrate GraphRAG into the benchmark to see if the global context it provides justifies the higher resource cost.
To ensure reliability and answer quality, a structured evaluation pipeline will be introduced.
The evaluation framework will focus on measuring both retrieval quality and LLM response quality.
Planned metrics include:
-
Retrieval Accuracy
Measures whether the GraphRAG pipeline retrieves the correct graph context. -
Answer Relevance
Evaluates how well the generated answer addresses the user's query. -
Faithfulness (Hallucination Detection)
Ensures the generated response is grounded in the retrieved knowledge graph.
Future work will include automated evaluation scripts and benchmark datasets to continuously assess system performance.
| Component | Technology |
|---|---|
| Backend API | FastAPI |
| Knowledge Graph | Neo4j |
| Document Processing | Python |
| LLM Integration | OpenAI API |
| Retrieval | GraphRAG |
| Orchestration | Python |
This project is actively under development.
The roadmap below highlights the current progress and planned improvements toward a production-ready AI knowledge system.