A complete Python template for building Model Context Protocol (MCP) servers with documentation scraping, vector search, and flexible deployment options.
- 🔍 Dual Search Modes: Semantic search (ML-powered) or keyword search (lightweight)
- 🤖 Multi-Provider Embeddings: HuggingFace (local), OpenAI, or Azure OpenAI
- 📚 Web Scraping: Configurable documentation crawler with depth control
- 💾 Vector Storage: ChromaDB with persistent storage
- ⏰ Auto-Updates: Scheduled documentation refresh
- 🔌 Dual Transport: stdio (local MCP) or HTTP/SSE (remote access)
- 🐳 Docker Ready: Containerized deployment with profiles
- ☸️ Kubernetes Ready: Production deployment with autoscaling
For: VS Code, Claude Desktop integration on your local machine
-
Configure environment:
cp .env.example .env # Edit .env and set your DOCS_URLS # Keep TRANSPORT=stdio (or leave it out, stdio is default)
-
Start container:
docker-compose up -d
This starts
mcp-server-templatecontainer (no ports exposed). -
Configure MCP client (VS Code or Claude Desktop):
{ "mcpServers": { "docs-scraper": { "command": "docker", "args": ["exec", "-i", "mcp-server-template", "python", "-m", "mcp_server_template.server"] } } }
For: Testing API, remote access, or preparing for Kubernetes deployment
-
Configure environment for HTTP:
cp .env.example .env # Edit .env and set: # TRANSPORT=http # DOCS_URLS=https://your-docs-site.com/
-
Start container with HTTP profile:
docker-compose --profile http up -d
This starts
mcp-server-template-httpcontainer on port 3003. -
Test the server:
# Check health curl http://localhost:3003/health # MCP clients connect to: # http://localhost:3003/sse
Stop current mode:
docker-compose downStart stdio mode:
docker-compose up -dStart HTTP mode:
docker-compose --profile http up -dView logs:
# stdio mode
docker logs mcp-server-template -f
# HTTP mode
docker logs mcp-server-template-http -fKey environment variables (see .env.example for full list):
TRANSPORT: Transport mode -stdio(local MCP) orhttp(remote/Kubernetes)DOCS_URLS: Documentation URLs to scrape (comma-separated for multiple sites)- Single:
DOCS_URLS=https://docs.example.com/ - Multiple:
DOCS_URLS=https://docs.example.com/,https://api.example.com/docs/,https://guides.example.com/
- Single:
SWAGGER_URLS: Swagger/OpenAPI JSON URLs (comma-separated, optional)USE_EMBEDDINGS: Enable semantic search (true) or keyword search (false)EMBEDDING_PROVIDER:huggingface,openai, orazureCRAWL_MAX_DEPTH: Maximum crawl depth (0-3, recommended 2)AUTO_UPDATE_ENABLED: Enable scheduled documentation updates (true/false)HTTP_PORT: Port for HTTP mode (default: 3000, mapped to 3003 on host)
Best for: Local development, VS Code, Claude Desktop
This server uses stdio transport for direct MCP client communication via stdin/stdout pipes.
VS Code (settings.json):
{
"mcp.servers": {
"docs-scraper": {
"command": "docker",
"args": ["exec", "-i", "mcp-server-template", "python", "-m", "mcp_server_template.server"]
}
}
}Claude Desktop (claude_desktop_config.json):
{
"mcpServers": {
"docs-scraper": {
"command": "docker",
"args": ["exec", "-i", "mcp-server-template", "python", "-m", "mcp_server_template.server"]
}
}
}Best for: Remote access, multi-user, Kubernetes deployments
Set TRANSPORT=http in .env and use the HTTP profile:
docker-compose --profile http up -dEndpoints:
GET /health- Health checkGET /sse- MCP Server-Sent Events endpoint
MCP Client Configuration (for remote HTTP clients):
{
"mcpServers": {
"docs-scraper": {
"url": "http://localhost:3003/sse",
"transport": "sse"
}
}
}See k8s-deployment.yaml for production Kubernetes deployment with:
- Horizontal Pod Autoscaling
- Persistent Volume Claims for ChromaDB
- Ingress configuration
- Health checks and readiness probes
This is a template - fork and customize for your documentation sources:
- Update
DOCS_URLSin.env - Customize scraping in
src/mcp_server_template/documentation_scraper.py - Add custom tools in
src/mcp_server_template/server.py - Adjust chunking and search parameters as needed
- Python 3.12+
- Docker & Docker Compose (recommended)
- 500MB+ memory for semantic search (50MB for keyword-only)