WhisperDocs is a voice-first AI platform that lets you upload PDFs and have natural, real-time conversations with their content. It combines document processing, secure storage, and voice AI to turn static documents into interactive conversations.
- 📄 PDF Upload & Processing: Upload PDFs and extract their content directly in the browser.
- 🎙️ Voice Conversations: Ask questions about your documents using your voice and receive spoken responses.
- 🔎 Document-Grounded Answers: Retrieve relevant document content to provide context-aware responses.
- 🔐 Secure Authentication: User authentication and session management with Clerk.
- ☁️ Cloud File Storage: Store PDFs and cover images using Vercel Blob.
- 🗄️ Persistent Document Data: Store document metadata, chunks, and voice sessions with MongoDB and Mongoose.
- 🔒 Secure Document Access: HMAC-signed document tokens protect Vapi tool calls from unauthorized document access.
- 🧹 Failure Recovery: Rollback and orphan-file cleanup prevent inconsistent database and storage state.
- 📱 Responsive Interface: Built with Next.js, React, and Tailwind CSS for a responsive document experience.
- Next.js 16
- React 19
- TypeScript
- Tailwind CSS
- pdfjs-dist
- Next.js Server Actions & API Routes
- Node.js
- MongoDB
- Mongoose
- Clerk
- Vercel Blob
- Vapi
- ElevenLabs
- Vapi Web SDK
Follow the steps below to set up WhisperDocs locally.
git clone https://github.com/your-username/whisperdocs.git cd whisperdocs
Install the project dependencies using your preferred package manager:
npm install
# or
yarn install
# or
pnpm install
# or
bun install
Create a .env.local file in the root directory of the project and add the required environment variables:
# Authentication
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=
CLERK_SECRET_KEY=
# Database
MONGODB_URI=
# File Storage
BLOB_READ_WRITE_TOKEN=
# Voice AI
NEXT_PUBLIC_ASSISTANT_ID=
NEXT_PUBLIC_VAPI_API_KEY=
VAPI_SERVER_SECRET=
VAPI_DOCUMENT_TOKEN_SECRET=WhisperDocs uses Vapi for its voice AI functionality.
Before running the application, configure your Vapi assistant from the Vapi Dashboard:
- Create an account or log in to the Vapi Dashboard.
- Create a new assistant.
- Configure the assistant with the voice, model, and other settings required by WhisperDocs.
- Add the document search/tool configuration used by the application.
- Copy the Assistant ID and add it to:
NEXT_PUBLIC_ASSISTANT_ID=your_assistant_id - Get your Vapi API key and add it to:
NEXT_PUBLIC_VAPI_API_KEY=your_vapi_api_key - Configure the required server-side Vapi credentials:
VAPI_SERVER_SECRET=your_vapi_server_secret
VAPI_DOCUMENT_TOKEN_SECRET=your_document_token_secret
Important: Never expose server-side secrets such as VAPI_SERVER_SECRET or VAPI_DOCUMENT_TOKEN_SECRET in client-side code or commit them to Git.
Start the Next.js development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Once the development server is running, open the application in your browser:
You should now have WhisperDocs running locally.
Detailed documentation covering the system architecture, design trade-offs, security, testing, and planned scalability improvements.
Read the WhisperDocs Architecture Documentation
